HackerWhat improved when a game agent got better?
A rising score might reflect a better network, deeper search or a loophole. Trace what the agent observes, what it tries and why the resulting move earns a reward.
Questions in this reading · 8
- Can a network evolve its own wiring?
- Reward and fitness both give feedback. What changes?
- A better network or a longer look ahead?
- Did the game agent see too little or optimize the wrong thing?
- What cannot be recovered from the current observation?
- Can a technique learned on arcade buttons transfer to a small pad?
- Does a known final round change the game?
- Why act differently before the other player leaves?
Can a network evolve its own wiring?
First follow activations through a fixed network. In MarI/O, selection changes the connections themselves. Following a signal and searching for a useful topology are different operations.
Compare original passages and timestamps · 2
the heart of the network as an information processing mechanismwithout presupposing the best structure for the neurons and their connectionsReward and fitness both give feedback. What changes?
In reinforcement learning, rewards inform action choices. MarI/O uses a fitness score to select entire networks across generations.
Compare original passages and timestamps · 2
there's no supervisor there's just this reward signalOnly the neural networks that produce the highest fitness are selectedA better network or a longer look ahead?
Evolving a network changes its structure. Looking farther ahead changes how an agent evaluates future moves. Each addresses a different limitation.
Compare original passages and timestamps · 2
without presupposing the best structure for the neurons and their connectionsthis algorithm does not think very far aheadDid the game agent see too little or optimize the wrong thing?
A limited input can hide distinctions needed for a decision. A misleading objective can reward a bad move even when the state is visible.
Compare original passages and timestamps · 2
the representation is ambiguous and of course when it gets it wrongplaying Tetris well requires some thinking ahead, and this algorithm does not think very far ahead.What cannot be recovered from the current observation?
Ask which distinctions the input representation discards. Earlier observations, feedback or prior knowledge may change what can be inferred.
Compare original passages and timestamps · 3
this has to have some mistakes since the representation is ambiguousthe future is independent of the past given the presentThe white squares stand for blocks the player can stand onCan a technique learned on arcade buttons transfer to a small pad?
Large arcade buttons leave room for a finger and supporting hand in a way the NES pad does not. Copying a gesture can therefore trigger the wrong game action. Repeated input is the transferable objective, while button geometry and grip change the solution. Measure registered inputs, positions reached and game scores separately when evaluating a new technique.
Passages and timestamps · 2
arcade cabinets have very large buttonsthe buttons on a Nintendo controller are tiny.Does a known final round change the game?
Against an opponent that defects once, retaliating for one round and refusing all future cooperation have different long-run costs. Veritasium then changes the setup: if both players know the final round, future retaliation cannot constrain that last decision. Evaluate the same pair twice, announcing a fixed endpoint in one run and only a continuation probability in the other. Record the actions round by round to separate a change in policy from a change in the game.
Passages and timestamps · 2
A forgiving strategy is one that can retaliate but it doesn't hold a grudge.when the last round is, then there's no reason to cooperate in that round.Why act differently before the other player leaves?
Grey’s succession model changes one condition in a repeated game: the person allocating rewards may be replaced. Even with present payoffs fixed, expectations about the successor can affect whom supporters back. Unlike the final round of a tournament, future interaction may continue, just with a different counterparty. Explaining a particular historical succession also requires evidence about the actual participants.
Passages and timestamps · 2
forces your keys to plan for your replacementfamilies are so popular among rulers and their key supporters.Try it on your own example
To test your explanation, would you first change the observations, the search depth or the reward?