HackerWhere does learning get a signal without an answer key?
Start with an unlabeled patch of an image, then switch to a game that requires a sequence of actions. Neither has an answer key, but finding structure and choosing a move need different signals.
Questions in this reading · 9
- What can images teach without labels?
- Where does the learning signal come from?
- Why can’t a policy classify each move in isolation?
- Where does AlphaGo get experience?
- Where do strategies absent from demonstrations come from?
- How do game agents improve beyond their starting point?
- What must be recovered behind the mask?
- Which two kinds of replay share a not-guilty label?
- What can score a decryption without the plaintext?
What can images teach without labels?
Unsupervised learning looks for structure in observations. Learning features from image patches shows what that structure can look like before labels enter the task.
Compare original passages and timestamps · 2
I'm not going to tell you what the right answer isfind a better way to represent images than the raw input pixelsWhere does the learning signal come from?
Finding patterns in unlabeled data and choosing actions from rewards both dispense with a teacher giving the right answer. Only the latter must account for the consequences of a decision.
Compare original passages and timestamps · 2
I'm not going to tell you what the right answer isthere's no supervisor there's just this reward signalWhy can’t a policy classify each move in isolation?
An action changes the next situation and thus the choices and rewards still available. The dependence is temporal even when the delay is short.
Compare original passages and timestamps · 2
the consequence of trashing it may not happen until much laterthe agent is actually influencing the data that it seesWhere does AlphaGo get experience?
Describe the task, experience and performance measure before examining the algorithm. AlphaGo provides distinct examples of experience from human games and self-play.
Compare original passages and timestamps · 2
a computer program is set to learn from an experiencethrough self-playing, reinforcement learning, it plays against different versions of itselfWhere do strategies absent from demonstrations come from?
Compare self-play in Go with exploration in language-model problem solving. Trial and error can find new strategies when feedback distinguishes useful attempts.
Compare original passages and timestamps · 2
through self-playing,
reinforcement learning, it plays against different versions
of itself many millions of times,we're going to check them and we're going to do more of what workedHow do game agents improve beyond their starting point?
AlphaGo imitates games and then plays against itself. MarI/O selects and reproduces networks according to game progress.
Compare original passages and timestamps · 2
through self-playing, reinforcement learning, it plays against different versions of itselfOnly the neural networks that produce the highest fitness are selectedWhat must be recovered behind the mask?
Mask part of a tree. Recovering every pixel and recovering an encoding define different training tasks. LeCun uses this distinction to explain JEPA: the full observation is encoded too, and its representation supplies the prediction target. A predictor need not chase every leaf’s incidental motion, but accuracy cannot restore distinctions discarded by the representation itself. Identify the space in which the error is computed.
Passages and timestamps · 2
you’re only trying to predict an abstract representation of the inputsyou’re not going to have to predict every single pixel of every leafWhich two kinds of replay share a not-guilty label?
Counter-Strike’s jurors see eight rounds and are instructed to convict only beyond reasonable doubt. A skilled player and a subtle cheater can both leave insufficient evidence and receive the same label. Those verdicts let a team ask whether a model finds cases jurors would convict. To ask how much cheating remains undetected, it needs observations capable of finding the blind spot shared by its labels and its evaluation.
Passages and timestamps · 2
Only convict them if you believe there is evidence beyond a reasonable doubt.Overwatch doesn't catch subtle cheaters. Because we ask players, like, don't convict if you're not sureWhat can score a decryption without the plaintext?
After trying an Enigma setting, sample two positions in the candidate plaintext. Their chance of matching depends on its letter distribution. The demonstration ranks rotor settings with this statistic, exploiting a trace that can remain when only part of a setting is correct. That supplies a search signal without a labelled plaintext. In a short message a few chance repetitions can change the ranking. Readable text or another independent clue must still corroborate the proposed key.
Passages and timestamps · 2
the probability that when you pick two letters at random they'll be the samefor short messages where these fitness functions start to break downTry it on your own example
If you change one action but leave the current picture unchanged, which tasks can still have a different outcome?