Back to Articles Prompt Engineering • 14 min read

Confessions of a Prompt Engineer: What Worked, What Failed, and What I Learned

I used to think prompt engineering was a meme title people on Twitter invented to charge $200 for PDF ebooks. Then I spent three weeks trying to build an automated documentation extractor for my team's Python codebase using GPT-4, and I realized how wrong I was. The first time I ran my automation script on our 40,000-line repository, the LLM hallucinated functions that didn't exist, ignored half my output constraints, and cost me $42 in API credits in twenty minutes.

That failure forced me to stop treating language models like smart search engines and start treating them like non-deterministic execution runtimes. Here is what I learned from testing thousands of prompt variations in production.

---

1. My Worst Prompt vs. The Fix

When I started out, my prompts looked like lazy Slack messages. I would write:

# BAD PROMPT (What I originally wrote):
"Extract all the API endpoints from this Python file and summarize them in JSON format."
            

The model returned formatted text inside markdown blocks, added conversational filler ("Here is the JSON you requested!"), omitted parameters it deemed 'unimportant', and occasionally outputted malformed JSON with trailing commas that broke my `json.loads()` parser.

After a week of debugging parser crashes, I rewrote the prompt using a strict zero-shot system schema. Here is the exact Python wrapper I ended up using in my production pipeline:

# PRODUCTION-GRADE PROMPT WRAPPER (What actually worked):
import openai
import json

def extract_endpoints_safely(python_code_str):
    system_prompt = (
        "You are an automated AST parsing assistant. Your task is to extract API endpoints.\n"
        "STRICT CONSTRAINTS:\n"
        "1. Output ONLY valid raw JSON matching the target schema.\n"
        "2. Do NOT wrap output in markdown code blocks like ```json.\n"
        "3. Do NOT include greetings, intro, or concluding remarks.\n"
        "4. If no endpoints exist, return {\"endpoints\": []}."
    )
    
    user_prompt = f"Analyze this source code:\n\n{python_code_str}"
    
    response = openai.chat.completions.create(
        model="gpt-4o-mini",
        messages=[
            {"role": "system", "content": system_prompt},
            {"role": "user", "content": user_prompt}
        ],
        temperature=0.0 # Deterministic decoding was key to stopping hallucinations
    )
    
    raw_output = response.choices[0].message.content.strip()
    return json.loads(raw_output)

# When I set temperature to 0.0 and banned markdown block wrappers, my parser error rate fell from 34% to 0.2%.
            

---

2. The Few-Shot Discovery

Zero-shot prompting works for simple tasks, but when I needed the model to translate legacy SQL queries into modern Pydantic schemas, explanations were useless. I spent hours writing paragraphs of rules explaining how to map data types. The model still got edge cases wrong.

Then I tried Few-Shot Prompting—providing three concrete input/output pairs right inside the system prompt. The difference was night and day. Instead of reading rules, the model pattern-matched the attention weights against my examples. My schema translation accuracy instantly jumped from 62% to 98%.

# FEW-SHOT PATTERN THAT FIXED MY SQL MAPPINGS:
EXAMPLE 1:
Input SQL: CREATE TABLE users (id INT PRIMARY KEY, email VARCHAR(255));
Output Pydantic: 
class User(BaseModel):
    id: int
    email: str

EXAMPLE 2:
Input SQL: CREATE TABLE logs (timestamp TIMESTAMP, event TEXT);
Output Pydantic:
class Log(BaseModel):
    timestamp: datetime
    event: str

# Now process the target input...
            

---

3. Chain-of-Thought: Forcing Intermediate Reasoning

I ran into another wall when I asked the model to solve complex business logic calculations (like calculating prorated subscription refunds). If I asked for the final number immediately, it got the math wrong nearly 40% of the time.

When I added a single line to the prompt: "Think step-by-step. First write out the daily rate, then count the active days, then calculate the total," the error rate vanished. Forcing the model to output intermediate tokens allows the transformer layers to compute context before generating the final answer.

---

My Real-World Prompting Rules

Technique	What I Used It For	My Empirical Finding
Temperature = 0.0	JSON extraction, code gen, data cleaning	Drastically reduces hallucinated field names.
Few-Shot Examples	Custom data format conversion	Beats long text instructions 10 times out of 10.
Chain-of-Thought	Multi-step math & logic validation	Prevents premature token sampling errors.

Prompt engineering isn't magic or tricking the AI—it's learning how to constrain the output distribution so the model gives you deterministic, production-ready results every single time.