This repository contains implementation of core LLM architectures like GPT-2 built from scratch using PyTorch. The main purpose of this project is to understand how modern transformer-based language models work internally by manually implementing key components such as multi-head self-attention, feed-forward layers, positional embeddings (RoPE), GQA, and KV caching.
GPT-2 Implementation
Token Embeddings
Sinusoidal Positional Encoding
Multi-Head Self Attention
Feed Forward Neural Network (FFN)
Residual Connections + Layer Normalization
Transformer Blocks Stacking
Final Vocabulary Projection Layer
Output Logits Generation ([batch, seq_len, vocab_size])