Change of Basis The major oversight of the book was what I thought a lack of focus on change of basis. This is the best way to understand matrix similarity in general and what PCA, SVD, and Product Quantization[https://www.pinecone.io/learn/series/faiss/product-quantization/] do in particular. I’ll go over the idea of basis in the abstract and then give a concrete example. Basis to Representation in the Abstract Not all vector spaces represent numbers along a set of dimensions, nor operators are represent coefficient multiplication on the underlying variables....

See https://github.com/CLARKBENHAM/sep_finetune_llm/blob/main/README.md for updated results Seperator Tokens Feb 27,2024 Inserting separator tokens is the process of adding a fixed default token after every encoded token in a string. The string “Hello, world!” has the token encoding [9906, 11, 1917, 0]. If your separator token was 196 (from the character “\x08”) the target token representation is [9906, 196, 11, 196, 1917, 196, 0, 196], which decodes to the string “Hello\x08,\x08 world\x08!\x08”. This string is what would be sent....