> ## Documentation Index > Fetch the complete documentation index at: https://docs.monostate.ai/llms.txt > Use this file to discover all available pages before exploring further. # Entropy & Uncertainty > Understanding the mathematical roots of model behavior # Entropy, Uncertainty, and Why Models Hallucinate When training AI models, understanding entropy and uncertainty helps you build more reliable systems. These concepts explain why models sometimes confidently state complete nonsense. For a deeper technical dive into this topic, read our research post: [Hallucinations in LLMs: The Entropy Problem and Current Solutions](https://monostate.com/blog/hallucinations-entropy-llms) ## What is Entropy? Entropy measures disorder or unpredictability in a system. In AI models, it tells us how "spread out" the probability distribution is over possible next words or tokens. ### Simple Example Imagine the model is completing: "The capital of France is..." **Low Entropy (Certain)**: * Paris: 95% probability * Lyon: 2% probability * Marseille: 1% probability * Other: 2% probability **High Entropy (Uncertain)**: * Paris: 25% probability * London: 20% probability * Berlin: 20% probability * Rome: 20% probability * Other: 15% probability Low entropy means the model is confident. High entropy means it's considering many options as equally likely. ## Entropy vs Uncertainty These terms are related but not identical: | Concept | What It Measures | Example | | ------------------------- | ------------------------------------------------- | -------------------------------------- | | **Entropy** | Mathematical disorder in probability distribution | How spread out token probabilities are | | **Uncertainty** | Any lack of knowledge or predictability | Model doesn't know it doesn't know | | **Epistemic Uncertainty** | What the model doesn't know | Facts outside training data | | **Aleatoric Uncertainty** | Inherent randomness | Multiple valid answers exist | ## The Entropy Paradox You might think high entropy equals more hallucinations. Not always true. **Counter-intuitive Finding** Models often hallucinate with extremely LOW entropy - they're confidently wrong. ### Why This Happens **Scenario**: Multiple valid options "The weather might be..." * sunny (30%) * cloudy (25%) * rainy (25%) * snowy (20%) Model correctly represents uncertainty **Scenario**: Confident hallucination "The Treaty of Versailles was signed in..." * 1923 (98%) * 1919 (1%) * Other (1%) Model is wrong but very confident ## Types of Uncertainty in Models ### 1. Token-Level Uncertainty The model is unsure about the exact next word: ```python theme={null} # High token uncertainty "The best programming language is..." # Could be: Python, JavaScript, Rust, etc. # Low token uncertainty "Water freezes at..." # Almost certainly: 0, zero, 32°F ``` ### 2. Semantic Uncertainty The model is unsure about meaning, not just words: ```python theme={null} # Same meaning, different tokens "The capital of France" = "Paris" = "City of Light" # Traditional entropy sees 3 options # Semantic entropy sees 1 meaning ``` ### 3. Factual Uncertainty The model doesn't know if information is true: ```python theme={null} # Model might generate: "The population of Tokyo is 37.4 million" # But has no way to verify this number # Could be confidently wrong ``` ## Measuring Uncertainty ### Traditional Entropy Shannon entropy formula: ``` H(X) = -Σ p(x) log p(x) ``` Where p(x) is the probability of each token. **Problems**: * Treats all tokens as independent * Can't distinguish between word choice and meaning * Doesn't capture epistemic uncertainty ### Semantic Entropy A better approach that measures uncertainty over meanings: 1. Generate multiple outputs 2. Group by semantic equivalence 3. Calculate entropy over meaning clusters ```python theme={null} # Traditional entropy sees 4 different outputs: outputs = [ "It's raining", "It is raining", "Rain is falling", "It's pouring" ] # High entropy # Semantic entropy groups them: meanings = { "precipitation": ["It's raining", "It is raining", "Rain is falling", "It's pouring"] } # Low entropy - one meaning ``` ### Confidence Calibration Measuring if confidence matches correctness: | Model Says | Confidence | Actually Correct | Calibration | | ---------------------------- | ---------- | ---------------- | ----------- | | "Paris is capital of France" | 99% | Yes | Good | | "Treaty signed in 1923" | 98% | No | Bad | | "Might be Python or Java" | 50% | Uncertain | Good | | "Definitely JavaScript" | 95% | No | Bad | ## Entropy in Training ### During Training Entropy changes as models learn: **High entropy everywhere** * Model knows nothing * Random predictions * Uniform distributions **Mixed entropy** * Learning patterns * Some confident predictions * Some uncertainty remains **Risk of overconfidence** * Low entropy on training data * May hallucinate confidently * Needs regularization ### Loss Functions and Entropy Cross-entropy loss directly relates to entropy: ```python theme={null} # Cross-entropy loss loss = -Σ y_true * log(y_pred) # Encourages: # - Low entropy when answer is clear # - High entropy when unsure ``` But this doesn't prevent confident hallucinations! ## Controlling Entropy ### Temperature Scaling Temperature controls randomness in generation: ```python theme={null} # Low temperature (0.1): More deterministic logits = logits / 0.1 # Sharp distribution, low entropy # High temperature (2.0): More random logits = logits / 2.0 # Flat distribution, high entropy ``` **Effects**: * T \< 0.5: Conservative, repetitive * T = 1.0: Default balance * T > 1.5: Creative, risky ### Top-k and Top-p Sampling Limit which tokens to consider: ```python theme={null} # Top-k: Only consider k most likely tokens top_k = 50 # Reduces effective entropy # Top-p: Consider tokens until cumulative prob = p top_p = 0.9 # Dynamic entropy reduction ``` ### Entropy Regularization Add entropy constraints during training: ```python theme={null} # Penalize too-confident predictions loss = cross_entropy + lambda * entropy_penalty # Where entropy_penalty encourages uncertainty # when appropriate ``` ## Advanced Techniques ### Entropy-Aware Training Our approach at Monostate modifies attention mechanisms: ``` Attention_controlled = softmax(QK^T/√dk - λH(p))V ``` Where λ controls entropy suppression strength. ### Layer-wise Entropy Budgets Different layers get different entropy allowances: | Layer Type | Entropy Budget | Purpose | | ---------- | -------------- | --------------------- | | Early | High | Explore possibilities | | Middle | Medium | Reason about options | | Final | Low | Commit to answer | ### Semantic Entropy Probes Extract semantic uncertainty from internal states: * 5-10x faster than generating multiple outputs * Can detect hallucinations before they happen * Works with any transformer model ## Practical Implications ### For Training * Monitor entropy during training * Use temperature scaling * Implement entropy regularization * Track confidence calibration * Assume low entropy = correct * Ignore semantic uncertainty * Over-regularize entropy * Trust confidence scores blindly ### For Inference **High-Stakes Applications**: * Use lower temperature (0.3-0.7) * Implement semantic entropy checks * Require multiple consistent outputs * Add confidence thresholds **Creative Applications**: * Use higher temperature (1.0-1.5) * Allow more entropy * Embrace variation * Filter bad outputs later ## Connection to Hallucinations Entropy relates to hallucinations in complex ways: 1. **Not Just High Entropy**: Hallucinations occur at all entropy levels 2. **Confidence ≠ Correctness**: Low entropy can mean confident nonsense 3. **Semantic vs Syntactic**: Word uncertainty doesn't equal meaning uncertainty 4. **Knowledge Gaps**: No entropy signal for "I don't know this" ## Current Research ### Semantic Entropy (Oxford/MIT) Measures uncertainty over meanings rather than tokens. Shows 2-3x better hallucination detection than traditional methods. ### Conformal Prediction Provides statistical guarantees: * "With 95% confidence, the answer is in this set" * If set is too large or empty, admits uncertainty * Mathematically rigorous approach ### Multi-Agent Consensus Our fuzzy logic approach: * Multiple agents evaluate options * Consensus through fuzzy scoring * Disagreement triggers re-evaluation * Natural uncertainty handling ## Best Practices ### During Training 1. **Track Multiple Metrics**: * Token entropy * Semantic entropy * Confidence calibration * Factual accuracy 2. **Use Appropriate Regularization**: * Dropout for uncertainty * Label smoothing * Entropy penalties * Early stopping 3. **Validate Uncertainty**: * Test on out-of-distribution data * Check confidence on known errors * Measure calibration curves ### During Inference 1. **Set Appropriate Temperature**: * Match to task requirements * Lower for facts * Higher for creativity 2. **Implement Safeguards**: * Semantic entropy checks * Multiple generation consensus * Confidence thresholds * Human-in-the-loop for critical decisions 3. **Design for Uncertainty**: * Allow "I don't know" responses * Show confidence levels * Provide alternatives * Flag potential hallucinations ## The Bottom Line Entropy and uncertainty are fundamental to how models work, but they're not simple predictors of reliability. Understanding these concepts helps you: * Build more reliable models * Detect potential hallucinations * Set appropriate generation parameters * Design better training procedures Remember: A confident model isn't necessarily a correct model. The key is building systems that know what they don't know. ## Watch: The RL Overfitting Problem Models trained with reinforcement learning can become overconfident and lose the ability to say "I don't know." This video explains why RL-trained models sometimes become worse at handling uncertainty.