Swayam’s Scripts
Swayam’s Scripts
A collection of my original writings
Categories
All
(3)
LLM
(1)
NLP
(1)
NumPy
(1)
Open-Source
(1)
Transformers
(1)
Understanding Perplexity
A New Perspective on Model Uncertainty
LLM
Recently, I was reading the Chapter 5 (Pretraining) of the book
“Build a Large Language Model (From Scratch)”
by Sebastian Raschka. I stumbled upon an intriguing…
Oct 10, 2024
Swayam Singh
3 min
595 words
Numpy QuadDType: Quadruple Precision for Everyone
Quad Precision for All: Simplifying High-Accuracy Computing with numpy_quaddtype
NumPy
Open-Source
Sep 30, 2024
Swayam Singh
1 min
5 words
Self-Attention Mimicking Gradient Descent
NLP
Transformers
This section of paper
Uncovering mesa-optimization algorithms in Transformers
presents a theoretical construction where a linear self-attention layer in a Transformer…
Oct 14, 2023
Swayam Singh
9 min
1,671 words
No matching items
