🌡️ Temperature

The Dial Analogy

Imagine a dial on a writing machine:

• Turn it all the way left → Predictable output. Newsletter, legal document.
• Turn it all the way right → Wild, creative output. Experimental poetry, brainstorming.

Temperature is that dial for AI.

It controls how "creative" or "random" the AI's responses are. Same prompt, different temperature = different results.

What Temperature Actually Does

When AI generates text, it predicts "what word comes next?"

But it doesn't just pick ONE word. It calculates probabilities for EVERY possible word:

Prompt: "The capital of France is ___"

AI's internal probabilities:
- One option is very likely
- A few options are plausible
- Many other options are very unlikely

Temperature controls how the AI picks from these probabilities:

Low Temperature

Heavily favors the highest-probability tokens. With greedy decoding (and no other randomness), outputs are often deterministic.

"Paris" wins every time
Same prompt → Same answer

High Temperature

More willing to pick lower-probability words.

"Paris" wins most times
But sometimes "a" or other surprises
Same prompt → Different answers each time

Temperature Scale (Intuition)

• Very low: most consistent, least varied
• Medium: balanced
• High: more varied and surprising

Same Prompt, Different Temperatures

Prompt: "Write a tagline for a coffee shop"

Temperature 0:

"A great cup, every time."
(Run it 10 times → same tagline 10 times)

Medium temperature:

Run 1: "Wake up to extraordinary."
Run 2: "Where every sip tells a story."
Run 3: "Fuel your day with flavor."
(Each run produces something different)

Higher temperature:

Run 1: "Coffee chaos, embrace the roast rebellion."
Run 2: "Beans dreaming in ceramic kingdoms."
Run 3: "Liquid sunrise for curious souls."
(More unusual, sometimes weird)

When to Use What

Use LOW Temperature When:

Use HIGH Temperature When:

Temperature vs Top-P

There's another similar setting called top_p (nucleus sampling):

top_p means: "Consider the most likely words until you reach about 90% cumulative probability, and sample from that set."

If top_p is set:

Case: "Paris" has 95% probability
- "Paris" alone already exceeds 90% cumulative probability → considered
- Most other tokens are excluded in this case

Note: top_p limits *which tokens are eligible*; temperature still affects how you sample *within* the eligible set.

General advice: Adjust ONE, not both. Temperature is more intuitive for most users.

Common Mistakes

Using High Temperature for Factual Tasks

High temperature + "What's the capital of France?"
→ "Paris" (usually)
→ But sometimes: "France's vibrant heart beats in..." (hallucination)

For facts, use low temperature.

Using Low Temperature for Creative Tasks

Temperature 0 + "Generate 10 unique product names"
→ "Product One, Product One, Product One..."
(Same top choice repeated)

For creativity, raise the temperature.

Setting Temperature Too High

Very high temperature:
"The quick brown fox quantum synthesizes rainbow paradigms..."
(Incoherent word salad)

Very high temperatures can drift into incoherence.

The Technical Details

For those curious, here's what happens mathematically:

Normal probabilities:
  "Paris" is most likely
  "Berlin" is less likely
  "London" is even less likely

Apply temperature T:
  new_probability = probability^(1/T)

Lower temperature (makes differences more extreme):
  top options dominate more

Higher temperature (makes differences less extreme):
  more options stay in the running

Lower temperature = sharper differences = more deterministic. Higher temperature = flatter distribution = more random.

FAQ

Q: What temperature does ChatGPT use by default?

Defaults vary by model/product and can change over time. If you care about reproducibility, explicitly set temperature (and, if available, a random seed) in your API call.

Q: Does temperature affect accuracy?

Often. Lower temperature tends to be more consistent, while higher temperature tends to be more varied (and can drift).

Q: Should I use temperature 0 for all technical work?

It's a reasonable starting point. For coding, a small amount of randomness can sometimes help explore alternatives.

Q: What is temperature in stable diffusion / image AI?

Similar concept! Controls how "creative" the image generation is. Higher = more unusual interpretations.

Q: Can temperature ever be negative?

Typically no. Most APIs use a non-negative temperature where 0 is the most deterministic setting; supported ranges vary by system.

Q: Does changing temperature cost more?

No. It's just a parameter change. Doesn't affect compute costs or token limits.

Summary

Temperature controls how random or deterministic AI outputs are. It's your creativity dial - turn it down for factual work, up for creative exploration.

Key Takeaways:

• Temperature = randomness setting (lower = more consistent, higher = more varied)
• Low: factual work, code, reliable answers
• Medium: general use
• High: brainstorming and creative writing
• Works by adjusting word probability selection
• Same prompt + different temperature = different results
• Use temperature OR top_p, not both

Think of temperature as choosing between a careful editor (low) and a spontaneous artist (high)!