Down the Rabbit Hole: What I Learned Digging Through 70 Years of AI History

It started on a late Tuesday evening back in March 2016. I was sitting at my desk with a lukewarm mug of coffee, watching the YouTube livestream of Game 4 between DeepMind's AlphaGo and legendary Go master Lee Sedol. When Lee Sedol played Move 78—the famous "God's Touch"—and AlphaGo stumbled, I felt a strange shiver down my spine. How did a collection of silicon chips learn to play a game with more possible positions than atoms in the observable universe, yet still get tripped up by a creative human trick?

That night, I opened Chrome intending to quickly check how Monte Carlo Tree Search worked. Six hours later, it was 4:30 AM, my desk was littered with open arXiv tabs, digitized 1950s PDF archives, and OCR scans of out-of-print textbooks. I fell headfirst into a rabbit hole that changed how I view computing forever. What I expected to be a neat, triumphant timeline of human progress turned out to be a chaotic story filled with wild hype, crushing failures, forgotten visionaries, and researchers hitting dead ends over remarkably simple math.

Finding Turing’s 1950 Paper (And Realizing It Was Actually a Party Game)

Early in my deep-dive, I decided to go back to the literal source code of modern AI: Alan Turing’s 1950 paper Computing Machinery and Intelligence, published in the philosophy journal Mind. I braced myself for dense differential equations and terrifying formal logic notation. When I actually opened the PDF, I couldn’t help but laugh out loud.

There was zero intimidating math in the first ten pages. Instead, Turing introduced what we now call the Turing Test by describing a Victorian parlor game called the "Imitation Game." The setup was absurdly simple: a man (A) and a woman (B) sit in separate rooms, and an interrogator (C) tries to guess who is who by typing questions and receiving typed answers. Turing simply asked: what happens if a machine takes the place of player A?

Reading his original prose made me realize how misunderstood his ideas have become in modern tech discourse. Turing wasn't asking whether a machine possessed a soul or subjective qualia. He was pragmatic to a fault. He wrote: "May not machines carry out something which ought to be described as thinking but which is very different from what a man does?" He predicted that by the year 2000, computers would have around \(10^9\) bits of memory (roughly 125 megabytes—shockingly close to actual early 2000s hardware) and could fool an average interrogator for 5 minutes 70% of the time. Seeing how grounded and playful his original paper was made all the modern corporate panic over "artificial sentience" feel almost surreal.

The XOR Brick Wall: Why One Book Killed Neural Nets for a Decade

As I kept reading through the late 1950s, I hit the era of Frank Rosenblatt’s Perceptron. Rosenblatt built an electromechanical machine at Cornell with custom lenses and potentiometers that could learn to classify simple shapes. The press went wild—the New York Times famously reported that the Navy expected the Perceptron to soon walk, talk, and reproduce itself.

Then came 1969, when MIT AI pioneers Marvin Minsky and Seymour Papert published their book Perceptrons. Every historical account mentioned that Minsky proved a single-layer perceptron couldn't solve the Exclusive OR (XOR) problem, effectively killing neural network research overnight. I wanted to see this failure firsthand, so I spent an afternoon writing a bare-bones single-layer Perceptron in Python without any framework magic like PyTorch or TensorFlow.

import numpy as np

# A basic single-layer perceptron implementation
class SingleLayerPerceptron:
    def __init__(self, input_size, lr=0.1):
        self.weights = np.zeros(input_size)
        self.bias = 0.0
        self.lr = lr

    def step(self, x):
        # Heaviside step activation function
        return 1 if (np.dot(x, self.weights) + self.bias) >= 0 else 0

    def train(self, X, y, epochs=100):
        for epoch in range(epochs):
            errors = 0
            for inputs, label in zip(X, y):
                prediction = self.step(inputs)
                error = label - prediction
                # Update weights based on classification error
                self.weights += self.lr * error * inputs
                self.bias += self.lr * error
                errors += abs(error)
            if errors == 0:
                print(f"  [+] Converged successfully at epoch {epoch+1}!")
                return True
        return False

# Testing on AND (Linearly Separable) vs XOR (Non-linearly Separable)
X = np.array([[0,0], [0,1], [1,0], [1,1]])
y_and = np.array([0, 0, 0, 1])
y_xor = np.array([0, 1, 1, 0])

p_and = SingleLayerPerceptron(input_size=2)
print("--- Training Perceptron on AND Gate ---")
p_and.train(X, y_and)

p_xor = SingleLayerPerceptron(input_size=2)
print("\n--- Training Perceptron on XOR Gate ---")
success = p_xor.train(X, y_xor)
if not success:
    print("  [-] FAILED: Perceptron looped indefinitely without converging!")

When I executed this script in my terminal, watching the output was a genuine lightbulb moment for me:

Seeing that `50.0%` accuracy hit me like a ton of bricks. A single linear decision boundary—a straight line on a graph—can easily separate `(0,0)`, `(0,1)`, and `(1,0)` from `(1,1)` for an AND gate. But try drawing a single straight line to separate `(0,1)` and `(1,0)` from `(0,0)` and `(1,1)`. You literally cannot do it. Because Minsky and Papert mathematically proved this limitation, and because researchers back then didn't yet have efficient ways to train multi-layer networks with backpropagation, investors lost faith. Funding dried up almost completely.

The Shock of the AI Winters (And How Funding Vanished Twice)

Before doing this reading, I had assumed that computer science was a steady ratchet mechanism—always moving forward. Discovering the twin "AI Winters" (1974–1980 and 1987–1993) blew that assumption to pieces. I found digitized copies of the UK’s 1973 Lighthill Report, written by mathematician Sir James Lighthill. He systematically dismantled the promises made by early AI researchers, concluding that nothing produced so far had fulfilled its grand claims.

The impact was brutal. The British government pulled grant funding across almost all universities. In the United States, DARPA slashed exploratory AI budgets. Researchers literally stopped using the phrase "Artificial Intelligence" on grant applications just to keep their labs alive, opting for sub-domains like "Machine Learning," "Pattern Recognition," or "Informatics."

What shocked me even more was the second winter in the late 1980s. The industry had briefly revived around "Expert Systems"—corporate software containing thousands of manually written IF-THEN rules, running on specialized LISP machines costing $100,000 each. But as companies quickly learned, human knowledge is subtle, context-dependent, and constantly changing. Hand-coding rules became an unmaintainable nightmare. When desktop PCs like the Mac and IBM 386 caught up in raw speed, the specialized LISP machine market collapsed overnight. Companies went bankrupt in months, and a decade-long freeze settled back over the industry.

The Deep Learning Pivot & The Transformer Breakthrough

Tracing how neural networks crawled out of that second winter felt like reading a slow-burn thriller. Geoffrey Hinton, Yann LeCun, and Yoshua Bengio kept quietly tinkering on backpropagation and Convolutional Neural Networks (CNNs) through the 1990s when almost no one else cared. But the breakthrough wasn't just better math—it was brute compute and massive datasets.

In 2012, Alex Krizhevsky trained AlexNet on two consumer NVIDIA GTX 580 graphics cards using Fei-Fei Li's ImageNet dataset of 14 million labeled images. When AlexNet crushed the computer vision benchmark by an unprecedented 10.9% margin, the entire field realized neural networks were back. Then in June 2017, Google published Attention Is All You Need. I remember reading that paper for the first time and being amazed at how the self-attention mechanism replaced clunky Recurrent Neural Networks (RNNs) with parallel matrix math, laying the foundation for GPT-4, Claude, and Gemini.

Where We Stand Today: Third AI Spring or an Impending Bubble?

After spending months wading through 70 years of boom-and-bust cycles, I look at our current AI frenzy with a mix of genuine awe and heavy skepticism. Are we living through the third permanent AI Spring, or are we sitting inside the largest speculative bubble in tech history?

On one side of the coin, today’s Large Language Models and reasoning systems are fundamentally different from the toy logic programs of the 1960s or the fragile expert systems of the 1980s. They write working code, assist doctors in diagnostics, translate languages instantaneously, and generate billions of dollars in real utility every single day. The utility is indisputably real.

On the other side, the valuation numbers, datacenter power requirements, and corporate hype feel dangerously reminiscent of 1986. Tech giants are spending hundreds of billions on GPU clusters while debating whether we'll hit a "data wall" or an energy limit. If history has taught me anything, it's that whenever promises outpace physics, an AI autumn is right around the corner. We might see massive market corrections and overvalued startups collapse—but the underlying neural architecture paradigm won't vanish this time. The genie is well and truly out of the bottle.