How do you update on evidence—and choose what to ask next?

First remove the cases that do not fit the evidence

Imagine a thousand cards. Ten represent people with a condition. A test marks nine of those ten red, but also marks 89 of the other 990 red. Given a red card, look for those nine among the 98 red cards, not among the ten cards representing the condition. Both 9/10 and 9/98 are correct calculations. They answer questions pointing in opposite directions. These are hypothetical teaching figures, not the performance of a particular test.

Another talk replaces the cards with areas. The prior sets the initial size of each region. Once you see the description, compare only the parts consistent with it. Use the pictures to check each other: if the area left in your drawing does not represent every red card, a group is missing from your denominator.

Why is the calculation different before the colors appear?

Before a Wordle guess you do not know which pattern will light up. Judging a word by its luckiest possible feedback is misleading. Look at how much uncertainty each result would remove and average by the chance of getting it. Updating after seeing the colors is the counterpart of keeping only the red cards. The first calculation designs a question; the second interprets an answer.

Try four candidate words with prior probabilities of 70%, 10%, 10% and 10%. A guess that splits them into two groups of two still splits the probability 80–20. Ask instead whether it is the most common word. One word versus three gives a more balanced 70–30 probability split. Of these two yes-or-no questions, the second has higher expected information. In the talk, a group containing 32 unlikely words is about as probable as one containing just eight. Even after weighting the groups, specify the goal: maximizing expected information and winning on this guess are different objectives.

Why can the same light or sound prompt a different judgment?

Lotto’s tiles send the same light to the eye while the surround suggests shadow on one tile. Seth supplies a speech cue and replays a distorted sound that now seems intelligible. The examples change expectations about illumination and sound source respectively. Hoffman’s desktop-icon analogy asks a further question: does usefulness for acting imply resemblance to what a representation stands for? The demonstrations test particular changes in perception. His broader evolutionary claim requires a separate argument.

A missing signal and a box of repeating molecules

Loeb lists the missing expected tail of ʻOumuamua as something an explanation must account for. That challenges a particular comet model without selecting an artificial origin on its own. Cronin starts from a different observation, combining an assembly measure with the number of identical molecules. Draw two columns: what does the proposed explanation predict, and what could an alternative also produce? A missing signal or many copies only distinguish explanations insofar as the predictions differ.

Ten heads: out of which attempts?

Ten heads in one specified round are unlikely. Stop each failed round at its first tail and keep filming until successful, and the existence of a successful clip answers a different question. The presenter accounts for failed rounds, which lets a viewer reconstruct the stopping rule. Ask where those rounds went in a collection showing only successes.

What lies between rated watts and seconds to a boil?

Dividing the water’s required joules by rated watts gives a testable time to boil. To explain a longer measurement, account for the vessel heating up and for a gap between delivered and nominal power. As with reinforced sand, treating the object as one lump simplifies the first calculation. Tracing energy or load through its parts helps explain the residual.

Does a 4,500-watt rating predict the bill?

A power rating gives a rate. Energy use also depends on the duration at each level. A water heater adds another trap: elements at different heights can take turns drawing the same power budget. Adding both ratings confuses two installed elements with simultaneous full load. Compare appliances over a defined task and accumulate their draw over time.

On which side of the building did the saving occur?

An awning intercepts sunlight before it passes through glass; cooling equipment removes heat after it enters. A streetlight retrofit adds spectral effects on astronomical observations alongside energy use. List the required work before and after, then check lighting, comfort, energy and spillover separately. One percentage cannot silently carry every outcome.

Did the appliance transfer work to its owner?

A chest freezer can limit cold-air loss on opening, while a manual-defrost model requires downtime and intervention. Define the comparison interval: an hour at the outlet and a full cycle including frost buildup, emptying, defrosting and recooling answer different questions.

Move an hour into the evening and which loads change?

Shifting daylight can change lighting and cooling loads in opposite directions, and household equipment changes across eras. Comparing adjacent bills mixes those effects with weather and season. Weekend Wednesday poses a related measurement problem: count total work time, longest uninterrupted run of work and overlap with colleagues separately before declaring the arrangement better.

Does a balance sheet give the savings from a policy change?

Grey subtracts expenses from estate receipts in one historical arrangement, then considers what could be retained if the arrangement changed. The first result comes from a ledger. The second requires an account of asset control, receipts and maintenance in the alternative. Crediting both scenarios with receipts they cannot both retain would distort the comparison.

Before the result arrives, what would change your mind?

Muller explains how reviewing hypotheses and methods in advance and committing to publish regardless of the result if the study follows the agreed protocol can limit the temptation to report only significant results after seeing the data. Olson recounts a shareholder vote. In his telling, a community expected the vote count to supply evidence and then began explaining why that evidence had not appeared. The two cases suggest two checks. Before seeing the result, did anyone specify what would weaken the claim? After the explanation changes, what new prediction can still be tested?