Skip to content

Commit 24eb7c8

Browse files
committed
Merge branch 'main' of github.com:mlberkeley/nmep
2 parents 4312058 + 5d5d6bd commit 24eb7c8

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

docs/homeworks/hw4.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -359,7 +359,7 @@ Word to embed: “hello”
359359
## `nn.Embedding` Simplification
360360
361361
If we look closely at the matrix multiplication, we can notice that for each token,
362-
the “multiplication” is just choosing the column in W corresponding to that token!
362+
the “multiplication” is just choosing the row in W corresponding to that token!
363363
364364
So, this “linear transformation” is just a lookup table, where we have
365365
$$V$$ vectors (V being the vocab size), and we look up the vector for each token and pile them together in a $$T \times C$$ matrix.

0 commit comments

Comments
 (0)