HackerTube 中文?
Representation, intuition and feedback →

What does a matrix do, and which outputs can disappear?

Read a column as an arrow, then a matrix as an action on that arrow. Solving an equation becomes a question about reachable targets, giving collapse, eigenvectors and rotations a geometric interpretation.

Questions in this reading · 11
  1. What do the numbers in a vector describe?
  2. What does multiplying a vector mean?
  3. What is the same in a diagram and a column calculation?
  4. How can columns explain a matrix equation?
  5. Which equations survive a collapse in dimension?
  6. Which outputs disappear when a transformation collapses space?
  7. Why shift a matrix before looking for an eigenvector?
  8. How do rotations change in four dimensions?
  9. Which distance survives when the higher-dimensional picture fails?
  10. When do two reflections rotate a figure, and when do they translate it?
  11. A circular representation looks right; does the model actually use it for addition?

What do the numbers in a vector describe?

Translate between an arrow and a list of coordinates. Each representation makes a different aspect of the same object easier to inspect.

Compare original passages and timestamps · 2
the geometric understanding is what lets you judge what tools to use
Essence of linear algebra preview · ▶ 1:08–1:24
a language to describe space and the manipulation of space using numbers that can be crunched and run through a computer.
Vectors | Chapter 1, Essence of linear algebra · ▶ 9:00–9:11

What does multiplying a vector mean?

Scaling one vector and combining two vectors are different operations. Check the operands and the type of result before interpreting the notation.

Compare original passages and timestamps · 2
these two fundamental operations, vector addition and scalar multiplication.
Vectors | Chapter 1, Essence of linear algebra · ▶ 8:14–8:19
multiplying two vectors together is the dot product plus the wedge product
Why can't you multiply vectors? · ▶ 38:28–38:37

What is the same in a diagram and a column calculation?

Switch between geometric motion and coordinate arithmetic to see one linear operation from two angles. Strang’s column combination is a worked example; neither representation is best for every task.

Compare original passages and timestamps · 3
the ability to translate back and forth between them
Vectors | Chapter 1, Essence of linear algebra · ▶ 8:45–8:49
the geometric understanding is what lets you judge what tools to use
Essence of linear algebra preview · ▶ 1:08–1:24
A times x is a combination of the columns of A.
Lec 1 | MIT 18.06 Linear Algebra, Spring 2005 · ▶ 38:24–38:36

How can columns explain a matrix equation?

Translate coordinates into a geometric picture. The columns of a matrix then show which combinations of vectors can reach the desired output.

Compare original passages and timestamps · 2
the ability to translate back and forth between them
Vectors | Chapter 1, Essence of linear algebra · ▶ 8:45–8:49
linear combinations of the columns fill three dimensional space
Lec 1 | MIT 18.06 Linear Algebra, Spring 2005 · ▶ 28:43–29:03

Which equations survive a collapse in dimension?

Once a transformation flattens space, reachable outputs lie in its image. A target inside that image and one outside it lead to different answers.

Compare original passages and timestamps · 2
A determinant of 0 would mean that all of space is squished onto something with 0 volume
The determinant | Chapter 6, Essence of linear algebra · ▶ 6:32–6:42
If these three columns all lie in the same plane, then their combinations will lie in that same plane.
Lec 1 | MIT 18.06 Linear Algebra, Spring 2005 · ▶ 31:06–31:18

Which outputs disappear when a transformation collapses space?

Some target vectors fall outside the image after a collapse. In the eigenvalue example it is A minus lambda times the identity that becomes singular; this does not make every system unsolvable.

Compare original passages and timestamps · 3
A determinant of 0 would mean that all of space is squished onto something with 0 volume
The determinant | Chapter 6, Essence of linear algebra · ▶ 6:32–6:42
If these three columns all lie in the same plane, then their combinations will lie in that same plane.
Lec 1 | MIT 18.06 Linear Algebra, Spring 2005 · ▶ 31:06–31:18
make this determinant zero, meaning the tweaked transformation squishes space into a lower dimension
Eigenvectors and eigenvalues | Chapter 14, Essence of linear algebra · ▶ 7:58–8:07

Why shift a matrix before looking for an eigenvector?

A zero determinant signals collapse into a lower dimension. Applying that test to the shifted matrix explains how a nonzero vector can lie in its null space.

Compare original passages and timestamps · 2
A determinant of 0 would mean that all of space is squished onto something with 0 volume
The determinant | Chapter 6, Essence of linear algebra · ▶ 6:32–6:42
make this determinant zero, meaning the tweaked transformation squishes space into a lower dimension
Eigenvectors and eigenvalues | Chapter 14, Essence of linear algebra · ▶ 7:58–8:07

How do rotations change in four dimensions?

Compare a fixed axis in three dimensions with planes of rotation in four. The familiar axis description does not carry over unchanged.

Compare original passages and timestamps · 2
what you have found is the axis of rotation
Eigenvectors and eigenvalues | Chapter 14, Essence of linear algebra · ▶ 4:11–4:20
two separate 2D rotations, happening perpendicular to and in sync with each other
Visualizing the 4d numbers Quaternions · ▶ 23:19–23:34

Why does an exponential trace a circle?

Return to the plane and separate position from the rule for moving. If velocity equals position multiplied by i, it is perpendicular to the radius and has the radius’s length. Starting at one gives motion around the unit circle at unit angular speed. This account explains how a local rule determines a trajectory.

Passages and timestamps · 1
the velocity at that time will be a 90 degree rotation of that position.
e^(iπ) in 3.14 minutes, using dynamics | DE5 · ▶ 2:21–2:33

Now ask whether moving through angle a and then b is the same as moving through a+b. The group-theoretic account starts with composition: adding inputs corresponds to multiplying outputs. One account gives the velocity; the other lets you check how two actions combine. A single angle works beautifully here. It does not follow that one angle specifies every rotation in four dimensions.

Passages and timestamps · 1
adding the inputs corresponds with multiplying the outputs
Euler's formula with introductory group theory · ▶ 16:54–17:05

Which distance survives when the higher-dimensional picture fails?

Write each corner center as coordinates ±1. Its distance from the origin is √d. With corner spheres of radius one, a central sphere tangent to them has radius √d−1. The nearest outer face stays two units from the origin. In ten dimensions those two numbers explain how the central sphere reaches outside the outer box. A three-dimensional drawing suggests a guess; the length formula and tangency condition carry the argument. Check which distances scale before taking a picture into another dimension.

Passages and timestamps · 2
the radius of the inner circle is square root of 2 minus 1
Thinking outside the 10-dimensional box · ▶ 9:58–10:10
the face of the box is always two units away from the origin
Thinking outside the 10-dimensional box · ▶ 21:50–21:59

When do two reflections rotate a figure, and when do they translate it?

In the example described at SIGGRAPH 2019, reflecting successively across two intersecting lines produces a rotation about their intersection. Reflecting across two distinct parallel lines produces a translation. The talk also considers those parallel lines in the projective model, where their intersection is a point at infinity representing their shared direction. Start by identifying the two lines used in the operation. Then ask which point stays fixed or how far the resulting motion moves every point in the same direction.

Passages and timestamps · 2
rotation is actually just a composition of two reflections of lines that intersect
Siggraph2019 Geometric Algebra · ▶ 73:04–73:11
you will get an infinite point exactly in the right direction
Siggraph2019 Geometric Algebra · ▶ 35:35–35:43

A circular representation looks right; does the model actually use it for addition?

This reading begins by translating between vector coordinates and geometric actions. The addition paper brings that reading habit inside a model: integer representations contain both linear variation and periodic structure, fitted as a generalized helix. Projecting numbers into circles is not enough. The authors replace internal activations to test whether the fitted representation restores the tendency toward the originally correct answer, and use ablations to investigate the components’ role.

Distinguish explaining a representation geometrically from explaining every computational step. The paper supplies intervention evidence for its tested models and two-digit, single-token addition regime, but does not isolate the exact construction of the answer helix from the two inputs. Drawing structure proposes a question; predicting and testing how a change affects the output makes the picture carry more explanatory weight.

Passages and timestamps · 2
all patching experiments only use prompts the model can successfully complete.
Language Models Use Trigonometry to Do Addition · paper · § 4.4 Evaluating the Quality of the Helical Fit · PDF p. 4
we do not know the exact mechanism
Language Models Use Trigonometry to Do Addition · paper · § 5.5 Limitations of Our Understanding · PDF p. 7

Try it on your own example

Choose a matrix that flattens the plane. Draw one reachable and one unreachable target, then check both by combining its columns.