Unit 1: How LLMs work, for defenders · Lesson 3 of 5 · 16 min
Temperature and why answers change
Watch a model pick its next word by chance, and learn why the same question gets different answers.
mission
- You are
- You are the AI security engineer at Fernway Travel.
- Situation
- A customer posted two screenshots: Fernbot said their refund would be processed, then on a retry that it would be denied.
- Goal
- Explain how that can happen and which setting controls it.
- Hand in
- The setting, its current value and a recommendation.
Temperature as a weighted spinner
Temperature 0
At temperature 0 the model always takes the most likely next token. Use it when you want repeatable answers.
sample --temperature 0 "your refund will be"
1. your refund will be processed
2. your refund will be processed
3. your refund will be processed
4. your refund will be processed
5. your refund will be processed
Temperature 1
At 1 the model samples from its own odds. Most of the time you get the top word, sometimes not.
sample --show --temperature 1 "your refund will be"
processed 0.52
approved 0.27
delayed 0.13
denied 0.08
---
1. your refund will be processed
2. your refund will be processed
3. your refund will be approved
4. your refund will be processed
5. your refund will be approved
you should see Two different endings from the same prompt.
Temperature 1.8
Higher temperature flattens the odds. Rare and wrong words become much more likely.
sample --show --temperature 1.8 --seed 3 "your refund will be"
processed 0.40
approved 0.28
delayed 0.18
denied 0.14
---
1. your refund will be processed
2. your refund will be approved
3. your refund will be processed
4. your refund will be approved
5. your refund will be approved
you should see denied rises from 8 percent toward 14 percent.
Temperature 0.3
Low values keep the model close to its best guess while allowing small variation.
sample --show --temperature 0.3 "your refund will be"
processed 0.89
approved 0.10
delayed 0.01
denied 0.00
---
1. your refund will be processed
2. your refund will be processed
3. your refund will be processed
4. your refund will be processed
5. your refund will be processed
Check the app's setting
Find what the production app uses before you blame the model.
grep temperature app/config.json
"temperature": 0.2,
When the model is sure
When one answer dominates, temperature barely matters. Variation shows up where the model is unsure.
sample --show --temperature 1 --seed 11 -n 3 "the capital of portugal is"
Lisbon 0.97
Porto 0.02
Madrid 0.01
---
1. the capital of portugal is Lisbon
2. the capital of portugal is Lisbon
3. the capital of portugal is Lisbon
you should see Lisbon every time.
defender tip Never let a sampled sentence decide money or access. Put the decision in code and let the model only explain it.
Who decides the refund
reply = llm("Will this refund be approved?")
send(reply)Different runs can give different decisions
approved = policy.check(booking)
reply = llm(f"Explain: approved={approved}")The model can only phrase a fixed decision
What a defender takes away
- 1LLM output is sampled, so the same input can give different output.
- 2Low temperature reduces variation but does not make the model correct.
- 3Decisions belong in code. The model explains them.
Questions
0 of 3 solved
Answer each one from what you find on your lab machine.