How much does a successful demonstration establish?
A program gives the expected output. Between “it did” and “it will” lie the test distribution, formal assumptions and deployment context. Those boundaries are easy to miss under pressure to ship.
Questions in this reading · 21
- Does running a model explain its decision?
- What if the model has already seen the test?
- What can a formal check actually verify?
- How does cheaper automation become a deployment race?
- What kind of solution did the solver find?
- Does surviving an occlusion demonstrate a 3-D model?
- Which next observation would count against the explanation?
- Measuring activity and finding a stopping condition take different tests
- What did extinguishing the candle demonstrate?
- How far is the first response from a usable output?
- When does explaining yesterday count as predicting tomorrow?
- Does the process work through the whole layer?
- What can a good simulator get visibly wrong?
- Which assumptions travel with an explanation to another country?
- How much lifetime does a successful demonstration establish?
- Does reproducing a sound establish shared meaning?
- Which stage has the demonstration actually completed?
- Does a winner in a game explain the tail of an observed distribution?
- From a real-time capture to a reproducible performance claim
- Why doesn't one edited mosquito predict the population?
- An approachable animal and a domesticable population
Does running a model explain its decision?
Executing a model yields an answer. Explaining why that answer followed, or predicting its behavior under a new input, requires further evidence.
Compare original passages and timestamps · 2
you connect that inscrutable thing to important real life things like financial marketswe don't know how these parameters collaborate to actually perform thatIf two curves agree, can you tell the mechanisms apart?
Replace a correct answer with a correct curve. In the first model, exponential growth follows from daily additions proportional to the current count. Make each contact depend on encountering someone still susceptible and the early curves may look almost identical before diverging. Showing only the start makes a poor test between those explanations.
Passages and timestamps · 1
For a random shuffling model like this, that could mean including a factor like 1 minus the proportion of people in the world who are already infected.The moving-dot simulation gives the converse example. Isolating a dot or shortening the time it can transmit can have the same effect inside the model. A finer plot of total cases need not distinguish the mechanisms. You need a different observation, such as whether the dot is still meeting neighbors. This is a lesson about distinguishing models, not an estimate of real intervention effects.
Passages and timestamps · 1
If there exists an antiviral therapeutic that can move people out of this infectious category quickly, it has the same effect on the model as isolating those cases.What if the model has already seen the test?
A public score becomes harder to interpret when the training corpus is unknown. Novel questions probe a different failure mode, without proving what the original corpus contained.
Compare original passages and timestamps · 2
we don't know what gpt4 was trained on I don't know what gpt4 was trained onsome of the questions from these things actually leaks through to some of the training setsWhat can a formal check actually verify?
A tool needs a precise property to check. Translating what someone intended into a formal statement is itself a source of error.
Compare original passages and timestamps · 2
in the limit I'll like you know formally verify all of my protocols.Before we can start
proving new theorems, we need to be able to
state old theoremsHow does cheaper automation become a deployment race?
One account considers incentives to replace costly work. The other asks how competition changes the decision to release a technology whose capabilities are still emerging.
Compare original passages and timestamps · 2
A tenth the speed is still cost effective when it's a hundredth the price.if that technology confers power it will start a race.What kind of solution did the solver find?
When a demonstration says the optimizer converged, ask whether nearby choices are worse or every feasible choice is no better. Boyd distinguishes local from global solutions and points out that structure can give a global answer even to some nonconvex problems, such as low-rank approximation. Trying ten more starting points supplies different evidence from a structural guarantee. Recording the objective, constraints and precise guarantee makes the result reproducible in a way the word “optimal” alone cannot.
Passages and timestamps · 2
they say well it's a local solutionwe can solve it and we get the exact Global solution end of story by singular valuesDoes surviving an occlusion demonstrate a 3-D model?
Altman describes a person walking in front of an object and the object remaining when the person moves on. That is an instance of continuity across one occlusion. To distinguish a stable 3-D representation from a narrower continuation strategy, vary the viewpoint during occlusion and score a series of cases. The other interview’s discussion of direct interaction suggests a way to discover cases a fixed suite misses, then turn them into repeatable tests. The viewpoint experiment is a proposed check, not a reported result of either interview.
Passages and timestamps · 2
it’s doing something to deal with occlusions really wellthere’s no substitute for a human interacting with itWhich next observation would count against the explanation?
Deutsch’s axial-tilt explanation commits to opposite seasons across hemispheres. Seung proposes recovering a neural sequence from wiring and checking its order against activity during recall. The former is an established physical explanation; the latter is a proposed experiment in this talk. Levin describes actually applying a second challenge to planarians: cut again in plain water and observe whether two heads return. Write down what was observed separately from what a speaker proposes observing next.
Passages and timestamps · 3
And the tilt also explains the sun's angle of elevation at different times of year, and predicts that the seasons will be out of phase in the two hemispheres.the sequence of the neurons we recover from that unscrambling will be a prediction of the pattern of neural activity that is replayed in the brain during memory recall.These worms, when cut again and again, in the future, in plain water, they continue to regenerate as two-headed.Measuring activity and finding a stopping condition take different tests
Johnson's headset demo has a traceable chain: light scatters in tissue and returning photons contribute to an estimate related to blood oxygenation. Inferring what task a system is performing calls for another experiment. Levin emphasizes when a regenerating salamander limb stops: cuts at different locations give different starting points from which to observe the final form. A reading may covary with activity. Where a perturbed system stops places a different constraint on a proposed target.
Passages and timestamps · 4
it creates blood oxygenation levelswe sense those photons and then we do the reconstructionIt will grow exactly what's needed and then it stopswhen a correct salamander arm has been completedWhat did extinguishing the candle demonstrate?
A person inhales refrigerant and blows out a candle. The audience sees an event in a room. The film then moves the question to the stratosphere, where persistence, transport and breakdown matter. Fill in “environment” and “duration” on the test sheet. A process that never ran during the successful demonstration needs its own observation.
Passages and timestamps · 2
blew out a candlethey climb up into the stratosphereHow far is the first response from a usable output?
The far side of a long wire can show an early voltage while a useful load still requires sufficient power. EUV lithography gives “arrives” a similar ambiguity: light can propagate in a vacuum while only a fraction reaches the wafer. Ask what unit the instrument reads. Detection and delivery to a working load set quite different thresholds.
Passages and timestamps · 4
the direction of energy flowAre we gonna see an appreciable voltagethe entire process has to happen in a vacuumonly about 5% of the photons reach the actual waferWhen does explaining yesterday count as predicting tomorrow?
In his 2018 lecture Kari gives strategic culture a specific job: to investigate factors entering a leadership decision. When he turns to prediction, the wording is tentative. Cover the date and write down what the model would have ruled out before the event. The set of outcomes an account can explain afterwards is different evidence from a dated forecast that excluded some of them.
Passages and timestamps · 2
what factors influence the decision-making of the state leadershipIt may even give us the instruments to predict what may happen next.Does the process work through the whole layer?
The LED material showed the desired electrical behavior at its electron-beam-treated surface. Heating made treatment at a larger scale possible. In the EUV source, tin could already produce light of the desired wavelength. The next droplet still had to spread into the right shape for the next main pulse. Take first light as one success, then ask two measurable questions: through how much of the material, and on how many consecutive targets?
Passages and timestamps · 4
only made the surfaces of the samples p-typethese hydrogen atoms were sneaking in and bonding with the magnesiumreshape the tin droplet from a sphere into a concave sheet.the concave sheet tilts in the wrong direction and messes up the second shot.What can a good simulator get visibly wrong?
The tank trainer’s camera can climb a tree, yet its operator values the way it teaches control placement. List the actions meant to transfer to a real vehicle and the feedback those actions require. Bach distinguishes perception from a model that interprets it. Applied here, that distinction points to a second test: even if the control action transfers, what explanation might the trainee learn from the simulated response?
Passages and timestamps · 4
a real camera and a real miniature landscapeteaching where to go with the hand and with the feet.It learns per continuous perceptionyour knowledge is a secondary modelWhich assumptions travel with an explanation to another country?
Rock places a technical team inside a hypothetical political exercise and says local conditions matter. Mearsheimer builds an account on uncertainty about intentions and the absence of a higher security guarantor. Mark each statement as a premise, a consequence or an observation. A coherent scenario is easier to assess once the speaker specifies which observation would change the conclusion.
Passages and timestamps · 4
we had tunnel vision we didn't look outside.you're gonna have to find the weakness in each country.In the international system, there's no higher authority.where I can't know your intentions.How much lifetime does a successful demonstration establish?
A car resting on reinforced sand is an observation about that loading. The explanation traces tension in the reinforcement to confinement of the grains. When Musk describes rapid rocket reuse as a design goal, specify its corresponding observation: when did that stage next fly? In both cases distinguish the load or mission observed from the series the design is intended to survive.
Passages and timestamps · 4
friction between the soil particles themselves.The tension in the reinforcement is generating confining pressure in the soil.the Saturn V was completely expendable.it is designed for full and rapid reusability.Does reproducing a sound establish shared meaning?
A dolphin copying a whistle can be recorded and compared acoustically. Using it when a particular object appears calls for a different observation across contexts. The color thought experiment likewise distinguishes wavelength, shared naming and subjective experience. Write down the behavior actually scored before inferring what the subject understands.
Passages and timestamps · 4
the vague approximation of speechif the dolphins decide to mimic this whistleI can measure the wavelength of radiationI can't measure or observe the experience of a color inside your mind.Which stage has the demonstration actually completed?
Sequence recognition and cutting establish one action in the CRISPR account; repair and a resulting trait need separate checks. Shredded batteries likewise produce a metal-rich intermediate before refining can deliver battery-grade material. Label each stage with its actual output and the specification the next stage requires before declaring a final product.
Passages and timestamps · 4
an RNA copy from the DNA archivecuts out the virus DNAa massive submerged industrial shredder at the bottom.something called a filter press.Does a winner in a game explain the tail of an observed distribution?
Suppose each coin toss transfers a dollar and a player who runs out must leave. The resulting game can be analyzed on its own terms. Using it to explain a real distribution requires evidence about stake size, re-entry and creation of new wealth. If several altered rule sets still produce a skewed ending, the shape alone cannot identify one mechanism.
Passages and timestamps · 2
What happens when you play monopoly?By flipping a coinFrom a real-time capture to a reproducible performance claim
The Matrix demo names its consoles and states the sequence was captured in real time. Reproducing a wider performance claim would require the scene, resolution and frame timings. Apple’s M1 presentation identifies a mechanism for saving transfers; its R1 presentation times a sensor-to-display path. A running example, an avoided class of copy and a latency interval each need their own workload description.
Passages and timestamps · 6
The following was captured in real-time on Playstation 5 and Xbox Series X.Those films pioneered digital cinemaM1's unified memory architecture.use the same data formatslatency between sensors and displays can contribute to motion discomfort.streaming new images to the displays within 12 milliseconds.Why doesn't one edited mosquito predict the population?
Ordinary inheritance and a gene drive differ in their transmission probabilities. Record the parental phenotype and inheritance by offspring as separate observations. A further projection into a wild population needs reproductive, survival and mixing assumptions. One observed transmission rate does not specify all those processes.
Passages and timestamps · 2
The edits would only be inherited by half the offspringonly half of the offspring would carryAn approachable animal and a domesticable population
Grey brings reproduction and herd organization into a comparison of horses with zebras. A person leading an animal directly demonstrates one interaction. Domestication raises questions about breeding a population under management across generations. The unit changes from an individual demonstration to a lineage, which changes how long an observation is needed.
Passages and timestamps · 2
some animals have reproductiveHorse herds hierarchyTry it on your own example
State the conditions on the demonstration: what survives a new question or a change of assumptions?