HackerWe Were Right! Real Inner Misalignment
Robert Miles AI Safety YouTube ↗
Further reading 1
Caption notes
In the training mazes the cheese and a fixed goal location always coincide. Moving the cheese makes two learned policies predict different routes: seek cheese or head for the old location. Repeated success on the training distribution cannot readily separate them.
Archived captions (transcription method unspecified) ↗Original passages · 2
during training, the cheese was always in the same place.the cheese was placed in a random location in the maze.