Skip to content
Navigation Menu
Toggle navigation
Sign in
Appearance settings
Platform
AI CODE CREATION
GitHub Copilot
Write better code with AI
GitHub Spark
Build and deploy intelligent apps
GitHub Models
Manage and compare prompts
MCP Registry
New
Integrate external tools
DEVELOPER WORKFLOWS
Actions
Automate any workflow
Codespaces
Instant dev environments
Issues
Plan and track work
Code Review
Manage code changes
APPLICATION SECURITY
GitHub Advanced Security
Find and fix vulnerabilities
Code security
Secure your code as you build
Secret protection
Stop leaks before they start
EXPLORE
Why GitHub
Documentation
Blog
Changelog
Marketplace
View all features
Solutions
BY COMPANY SIZE
Enterprises
Small and medium teams
Startups
Nonprofits
BY USE CASE
App Modernization
DevSecOps
DevOps
CI/CD
View all use cases
BY INDUSTRY
Healthcare
Financial services
Manufacturing
Government
View all industries
View all solutions
Resources
EXPLORE BY TOPIC
AI
Software Development
DevOps
Security
View all topics
EXPLORE BY TYPE
Customer stories
Events & webinars
Ebooks & reports
Business insights
GitHub Skills
SUPPORT & SERVICES
Documentation
Customer support
Community forum
Trust center
Partners
Open Source
COMMUNITY
GitHub Sponsors
Fund open source developers
PROGRAMS
Security Lab
Maintainer Community
Accelerator
Archive Program
REPOSITORIES
Topics
Trending
Collections
Enterprise
ENTERPRISE SOLUTIONS
Enterprise platform
AI-powered developer platform
AVAILABLE ADD-ONS
GitHub Advanced Security
Enterprise-grade security features
Copilot for Business
Enterprise-grade AI features
Premium Support
Enterprise-grade 24/7 support
Pricing
Search or jump to...
Search code, repositories, users, issues, pull requests...
Search syntax tips
Provide feedback
Saved searches
Use saved searches to filter your results more quickly
Sign in
Sign up
Appearance settings
Resetting focus
You signed in with another tab or window.
Reload
to refresh your session.
You signed out in another tab or window.
Reload
to refresh your session.
You switched accounts on another tab or window.
Reload
to refresh your session.
Dismiss alert
{{ message }}
rasbt
/
LLMs-from-scratch
Public
Notifications
You must be signed in to change notification settings
Fork
12.3k
Star
82.1k
Code
Issues
0
Pull requests
0
Discussions
Actions
Security
Uh oh!
There was an error while loading.
Please reload this page
.
Insights
Additional navigation options
Code
Issues
Pull requests
Discussions
Actions
Security
Insights
Commits
Branch selector
main
User selector
All users
Datepicker
All time
Commit History
Commits on Dec 27, 2025
Cover Python 3.12 (#933)
rasbt
authored
d85ba93
Copy full SHA for d85ba93
Commits on Dec 21, 2025
upload saved nb
rasbt
committed
1cea30f
Copy full SHA for 1cea30f
Update memory efficient loading nb
rasbt
committed
2b9a67c
Copy full SHA for 2b9a67c
Commits on Dec 20, 2025
update submodule
rasbt
committed
695ecb6
Copy full SHA for 695ecb6
Commits on Dec 19, 2025
Add some appendix E runtimes (#927)
Show description for 1c9f49c
rasbt
authored
1c9f49c
Copy full SHA for 1c9f49c
Gated DeltaNet updates (#926)
rasbt
authored
57430d2
Copy full SHA for 57430d2
Commits on Dec 16, 2025
Sliding window KV Cache bug fix (#925)
Show description for d7f178d
talentJay-ux
authored
d7f178d
Copy full SHA for d7f178d
Commits on Nov 25, 2025
Remove persistent flag from cache buffers (#916)
rasbt
authored
a11965f
Copy full SHA for a11965f
Commits on Nov 23, 2025
Add Olmo 3 README (#915)
Show description for c195338
rasbt
authored
c195338
Copy full SHA for c195338
Olmo 3 from scratch (#914)
Show description for bc6f335
rasbt
authored
bc6f335
Copy full SHA for bc6f335
Commits on Nov 17, 2025
RoPE decay plot (#910)
Show description for 398b079
rasbt
authored
398b079
Copy full SHA for 398b079
Update README wrt multi-query attention
Show description for 28a8408
rasbt
authored
28a8408
Copy full SHA for 28a8408
Commits on Nov 13, 2025
Write-up on how to get the most out of this book (#909)
rasbt
authored
a409447
Copy full SHA for a409447
Commits on Nov 9, 2025
fix(GatedDeltaNet): Init param A from log of a uniform distrib (#906)
casinca
authored
7d92267
Copy full SHA for 7d92267
Commits on Nov 6, 2025
Use consistent title case
rasbt
committed
35354fa
Copy full SHA for 35354fa
Fix empty device issue (#904)
rasbt
authored
58f45ae
Copy full SHA for 58f45ae
n_heads × d_head -> d_head × d_head in DeltaNet (#903)
Show description for bcc73f7
rasbt
authored
bcc73f7
Copy full SHA for bcc73f7
Commits on Nov 3, 2025
Image resizing
rasbt
authored
488bef7
Copy full SHA for 488bef7
Gated DeltaNet write-up (#901)
Show description for c6b8332
rasbt
authored
c6b8332
Copy full SHA for c6b8332
Commits on Nov 1, 2025
Training on MPS in PyTorch 2.9 (#900)
Show description for d6c3990
rasbt
authored
d6c3990
Copy full SHA for d6c3990
Fix MHAEinsum weight dimension bug when d_in != d_out (#857) (#893)
Show description for 27d52d6
aviralgarg05
and
rasbt
authored
27d52d6
Copy full SHA for 27d52d6
simplify uv command (#898)
rasbt
authored
b1db33b
Copy full SHA for b1db33b
Commits on Oct 29, 2025
Add bonus dependencies to pyproject (#897)
Show description for 760f4c9
rasbt
authored
760f4c9
Copy full SHA for 760f4c9
Commits on Oct 22, 2025
Fix ffn link (#892)
Show description for 0adb5b8
rasbt
authored
0adb5b8
Copy full SHA for 0adb5b8
Make quote style consistent (#891)
rasbt
authored
7ca7c47
Copy full SHA for 7ca7c47
Commits on Oct 21, 2025
- docs(moe): correct arXiv link for DeepSeekMoE (#890)
Show description for 9276edb
casinca
authored
9276edb
Copy full SHA for 9276edb
Commits on Oct 20, 2025
Mixture-of-Experts intro (#888)
rasbt
authored
218221a
Copy full SHA for 218221a
Commits on Oct 17, 2025
Make it easier to toggle between thinking and instruct variants (#887)
rasbt
authored
27b6dfa
Copy full SHA for 27b6dfa
Commits on Oct 14, 2025
Update the compression rate comment in MLA (#883)
Show description for 7fe4874
rasbt
authored
7fe4874
Copy full SHA for 7fe4874
Commits on Oct 13, 2025
Use figure numbers in ch05-7 (#881)
rasbt
authored
b969b3e
Copy full SHA for b969b3e
Add alternative attention structure (#880)
rasbt
authored
bf039ff
Copy full SHA for bf039ff
sliding window attention (#879)
rasbt
authored
6eb6adf
Copy full SHA for 6eb6adf
Add other appendices for completeness (#878)
Show description for 21f0617
rasbt
authored
21f0617
Copy full SHA for 21f0617
Commits on Oct 12, 2025
rm plot
rasbt
committed
44eda53
Copy full SHA for 44eda53
Multi-Head Latent Attention (#876)
Show description for 9b95866
rasbt
authored
9b95866
Copy full SHA for 9b95866
Pagination
Previous
Next
You can’t perform that action at this time.