Back to Articles NLP & LLMs • 18 min read

The Rise of Large Language Models (LLMs) & Transformers

I printed out "Attention Is All You Need" on a Saturday morning, poured a large coffee, and told myself I'd have it understood by Sunday evening. I did not. What I did have by Monday was a notebook full of confused margin scrawlings, three failed Python scripts, and a genuine, hard-won appreciation for why this eight-page paper changed everything in NLP. What follows is what I wish someone had told me before I started.

---

1. The QKV Problem — Or: Why I Stared at Three Letters for Two Hours

Before 2017, the standard approach to sequence modelling was RNNs and LSTMs. They worked, but they had a structural flaw: they processed tokens sequentially, one at a time. To connect the word "bank" at position 1 to the word "river" at position 47, the model had to thread that information through 46 intermediate hidden states. Gradients vanished over long distances. Context degraded. And because processing was sequential, you couldn't parallelise it across modern GPU hardware — meaning you couldn't scale.

Vaswani et al. solved this with self-attention. On first read, their explanation seemed clear enough: every token gets projected into three vectors — a Query, a Key, and a Value. The model computes attention by comparing Queries against Keys, then uses the resulting weights to blend the Values. Simple. I nodded along and kept reading.

Then I tried to actually implement it. That's when I realised I had no real intuition for what Q, K, and V meant. I kept confusing myself. Is the Query what a token is looking for? Is the Key what it's offering? Is the Value just... a third copy of the same embedding? I spent a genuinely embarrassing amount of time rereading the same three paragraphs.

The mental model that finally unstuck me was a library retrieval system. Every book in the library has a label on the spine — that's its Key. It has actual content inside — that's its Value. When you walk in with a question, that question is your Query. You scan every spine label, figure out which books are most relevant to your question, then read those books in proportion to their relevance. That's self-attention. The model isn't reading tokens in order — it's simultaneously querying every token against every other token and constructing each output as a relevance-weighted blend. Once that image was in my head, the formula clicked.

The Formula, and Why the Scaling Term Is Not Optional

Here's the attention equation:

That d_k in the denominator is the dimensionality of the Key vectors. I skipped it on my first implementation attempt. That was a mistake I will not be making again.

---

2. Implementing Attention in Raw Python — and the Bug That Broke Everything

I decided to implement scaled dot-product attention using only NumPy — no PyTorch, no Einops, no shortcuts. My goal was to see every tensor operation with my own eyes. Here is the version that actually works, with the comments I wish I had the first time:

import numpy as np

def scaled_dot_product_attention(Q, K, V):
    """
    Q, K, V: matrices of shape (seq_len, d_k).
    This is single-head attention. Multi-head just runs this
    in parallel across h separate learned projections.
    """
    d_k = Q.shape[-1]  # dimensionality of keys and queries

    # Step 1: raw attention scores — how much should each query
    # "attend to" each key? Result shape: (seq_len, seq_len)
    scores = Q @ K.T

    # Step 2: scale by sqrt(d_k).
    # Without this, at d_k=512 the dot products can reach magnitudes
    # of ~500+, which pushes softmax into saturation — gradients
    # become near-zero and training stalls. I learned this the hard way.
    scores = scores / np.sqrt(d_k)

    # Step 3: softmax row-wise so each query's weights sum to 1.
    # Subtracting the row max first is a standard numerical stability trick
    # — it prevents exp() overflow without changing the output.
    scores -= np.max(scores, axis=-1, keepdims=True)
    exp_scores = np.exp(scores)
    attention_weights = exp_scores / np.sum(exp_scores, axis=-1, keepdims=True)

    # Step 4: weighted sum of value vectors.
    # Each output token is now a blend of ALL value vectors,
    # weighted by how relevant each one was to that particular query.
    output = attention_weights @ V
    return output, attention_weights


# --- Quick sanity check ---
np.random.seed(42)
seq_len, d_k = 5, 16  # 5-token sequence, 16-dimensional keys

# In a real transformer these come from learned linear projections W_Q, W_K, W_V.
# Here I'm using random values just to verify shapes and weight row-sums.
Q = np.random.randn(seq_len, d_k)
K = np.random.randn(seq_len, d_k)
V = np.random.randn(seq_len, d_k)

out, weights = scaled_dot_product_attention(Q, K, V)

print(f"Output shape:  {out.shape}")       # expect (5, 16)
print(f"Weights shape: {weights.shape}")   # expect (5, 5)
print(f"Row sums (must all be 1.0): {weights.sum(axis=-1).round(6)}")

When I ran that, the shapes were right and the row sums were all exactly 1.0. Satisfying — for about thirty seconds. Because I had a very clear memory of what happened on my first attempt, before that / np.sqrt(d_k) line existed.

The Debugging Moment: Scores That Went to Infinity

My original version omitted the scaling step entirely. I had half-read the paper's explanation, convinced myself it was just a "nice to have" numerical nicety, and skipped it. The code ran fine at small dimensions. Then I tried d_k = 512 — the exact size used in the original paper's base model — and ran it again.

The terminal output looked like this:

Row sums: [nan nan nan nan nan]

attention.py:14: RuntimeWarning: invalid value encountered in true_divide
  attention_weights = exp_scores / exp_scores.sum(axis=-1, keepdims=True)

nan. Every single weight. I assumed I had a shape mismatch somewhere. I printed every intermediate tensor. Everything looked fine — until I printed the raw scores before softmax and saw values like 847.3, -912.1, 1203.8. When you call np.exp(1203), you get a number larger than a 64-bit float can represent. NumPy writes inf. Then the division inf / inf produces nan. The entire weight matrix collapses.

Adding / np.sqrt(512) — which is roughly dividing by 22.6 — brought every score back into a range where softmax could operate sensibly. The paper actually explains this directly: dot products grow in magnitude with d_k because you're summing d_k individual multiplications, each with variance 1. The expected magnitude of the sum scales as sqrt(d_k), so you divide it back out. I had read that sentence. I had not understood it until I watched my own code produce a terminal full of nan.

---

3. Tokenization: What the Model Actually Sees

Before any of this matrix math happens, raw text needs to become numbers. I used to treat tokenization as a boring preprocessing detail. It isn't. The vocabulary construction method has real consequences for what the model can and can't represent.

Tokenization Strategy	Pros	Cons
Character-Level	Tiny vocabulary; handles any input without OOV errors.	Very long sequences; semantic patterns are hard to learn.
Word-Level	Intuitive; maps closely to human language structure.	Enormous vocabularies; fails completely on typos or new words.
Sub-word (BPE/WordPiece)	Balanced length; handles morphology and prefixes well.	Sometimes creates non-intuitive token boundaries.

GPT-style models use Byte-Pair Encoding (BPE): start by treating every character as its own token, then greedily merge the most frequent adjacent pair, repeat until you hit a target vocabulary size. It's why "unbelievable" might tokenise as ["un", "bel", "iev", "able"]. The splits look odd to us, but the model doesn't care — the embeddings learn to assemble meaning from sub-word fragments just fine. What matters is that the vocabulary is large enough to be expressive but small enough that the final softmax layer stays tractable.

---

4. The Positional Encoding Experiment That Surprised Me

Here's something I only really understood once I broke it deliberately. Pure self-attention, as I described above, is completely position-blind. The library analogy is useful again: if you shuffle every book's spine label to a random position, the retrieval mechanism doesn't care — it still computes dot products, still produces weights, still returns output. But that output will be wrong, because the model has no idea that "the cat sat on the mat" and "the mat sat on the cat" are different sequences.

To test this, after I had a working attention function, I fed it two sequences that were permutations of each other — same five token embeddings, different order — and compared the outputs. They were identical to six decimal places. The model had zero awareness that the order had changed.

That's what positional encodings are there to fix. The paper injects a sinusoidal signal into each token's embedding vector before attention runs, giving every position a unique mathematical fingerprint. When I added even a crude version of this — just a small position-dependent offset to each embedding — the two permuted sequences immediately diverged in output. It was one of those moments where a piece of maths shifts from abstract understanding to visceral intuition: I had just seen what positional encoding is actually doing, and why without it the whole system is broken.

The original paper uses sine and cosine functions at different frequencies across embedding dimensions. The sinusoidal choice lets the model generalise to sequence lengths it never saw in training, because sinusoidal relationships between positions are smooth and continuous. Later work — RoPE, ALiBi — has improved on this for long-context scenarios, but the core problem remains the same: attention alone is blind to position, so position has to be injected separately.

---

5. Training Pipelines: Pre-training and What Comes After

Getting attention to work in a toy example is satisfying. Understanding how a real model like GPT-4 or Llama is actually produced is a different matter entirely. Training happens in structured phases:

• Pre-training (Causal Language Modelling): The model ingests hundreds of billions of tokens from books, code, and the web. Its only objective is predicting the next token — no curated labels, no task-specific supervision. Just next-token prediction at massive scale. This phase is where the model absorbs grammar, factual knowledge, reasoning structure, and coding syntax. It requires thousands of GPUs running for months and is responsible for the bulk of what the model "knows".
• Supervised Fine-Tuning (SFT): The pre-trained model is then trained on a smaller, hand-curated dataset of prompt-response pairs. This teaches it to answer questions in a useful format rather than just continuing text in a document-completion style. The base model already has the knowledge — SFT is largely about teaching it how to express that knowledge helpfully.
• Reinforcement Learning from Human Feedback (RLHF): Human raters compare pairs of responses and select the better one. These preferences train a separate reward model, which is then used to fine-tune the LLM via PPO — nudging it towards outputs that score well on the reward model. This is the stage that produces the safety-aware, instruction-following behaviour people associate with modern chatbots. It also introduces the alignment tradeoffs that ML researchers are still actively debating.

---

6. Token Sampling: Where Temperature Actually Lives

The last piece I needed to understand was how a model actually selects a token at inference time. The final transformer layer outputs a vector of raw scores — one per vocabulary token. Softmax converts those into probabilities. What I hadn't appreciated was how dramatically the temperature parameter shifts the behaviour:

import numpy as np

def softmax_with_temperature(logits, temperature=1.0):
    # temperature < 1.0 → sharper, more confident distribution (repetitive)
    # temperature > 1.0 → flatter, more diffuse distribution (creative/random)
    logits = np.array(logits, dtype=float) / temperature

    # Subtract max before exponentiation for numerical stability
    logits -= np.max(logits)
    exp_logits = np.exp(logits)
    return exp_logits / np.sum(exp_logits)


vocab = ["robot", "human", "code", "learning"]
sample_logits = [2.5, 0.1, 4.2, 1.8]

print("=== Temperature = 1.0 (default) ===")
for word, prob in zip(vocab, softmax_with_temperature(sample_logits, temperature=1.0)):
    print(f"  {word:<10}  {prob:.4f}")

print("\n=== Temperature = 0.3 (very sharp) ===")
for word, prob in zip(vocab, softmax_with_temperature(sample_logits, temperature=0.3)):
    print(f"  {word:<10}  {prob:.4f}")
# At 0.3, 'code' captures >99% of probability mass — the model almost always
# picks it. At 1.0, 'robot' and 'learning' both get meaningful probability.
# That difference is the entire creative vs deterministic axis in practice.

When I actually ran this comparison, the gap between the two temperature settings was starker than I expected. At 0.3, "code" took 99.1% of the probability mass. At 1.0, it was around 67% — still dominant, but with real probability on "robot" and "learning". Production systems layer Top-P nucleus sampling on top of this to cut off the long tail of improbable tokens, but the softmax temperature is where the fundamental character of a model's output is set.

After that weekend, I had working toy implementations of scaled dot-product attention and temperature-controlled sampling, a much better intuition for the architecture, and one firm rule: read the footnotes in academic papers. The sqrt(d_k) scaling lived in a footnote. It cost me an entire afternoon of debugging. Read the footnotes.