LLM Foundations

What are Large Language Models?

Large Language Models (LLMs) are AI systems trained on massive amounts of text data to understand and generate human-like language. They power applications like ChatGPT, Claude, and countless AI tools that have transformed how we work with technology.

At their core, LLMs are prediction machines - they predict the most likely next word (or token) given a sequence of previous words. But through scale and sophisticated training, they've developed remarkable abilities: answering questions, writing code, summarizing documents, and reasoning through complex problems.

Why Do LLMs Exist?

Before LLMs, AI systems needed to be built for specific tasks - one system for translation, another for summarization, another for Q&A. LLMs changed this by being general-purpose:

• One model, many tasks: The same model can translate, summarize, code, and chat
• No task-specific training: You describe what you want in natural language
• Emergent abilities: Large models develop capabilities not explicitly programmed
• Context understanding: They grasp nuance, tone, and implicit meaning

The Transformer Architecture

All modern LLMs are built on the Transformer architecture, introduced in the landmark 2017 paper "Attention Is All You Need." Understanding Transformers helps you work with LLMs effectively.

Key Components

• Tokenization: Text is split into tokens (words or subwords). "programming" might become ["program", "ming"]
• Embeddings: Each token is converted to a numerical vector
• Attention Mechanism: Allows the model to focus on relevant parts of the input
• Feed-Forward Networks: Process the information at each position
• Output Layer: Predicts probability of each possible next token

The Attention Mechanism

Attention is what makes Transformers powerful. For each word, the model asks: "Which other words in this context should I pay attention to?"

For example, in "The cat sat on the mat because it was tired," attention helps the model understand that "it" refers to "cat," not "mat."

# Simplified attention intuition
# For each word, compute relevance scores with all other words
# "it" in our example:
attention_scores = {
    "The": 0.02,
    "cat": 0.85,      # High attention - "it" refers to "cat"
    "sat": 0.03,
    "on": 0.01,
    "the": 0.01,
    "mat": 0.05,      # Some attention - also a candidate
    "because": 0.02,
    "was": 0.01,
}

How LLMs are Trained

LLM training happens in stages:

1. Pre-training

The model learns from massive text datasets (books, websites, code) by predicting the next word millions of times. This teaches:

• Grammar and language structure
• Facts and knowledge
• Reasoning patterns
• Different writing styles

2. Fine-tuning

The pre-trained model is trained on specific data to improve performance on particular tasks or domains.

3. RLHF (Reinforcement Learning from Human Feedback)

Human reviewers rank model outputs, and the model learns to produce responses humans prefer. This makes models:

• More helpful and relevant
• Safer and more aligned with human values
• Better at following instructions

Major LLM Providers

Key Concepts You Need to Know

Tokens

LLMs process text as tokens, not characters or words. A token is typically 3-4 characters. Understanding tokens matters for:

• Cost: You pay per token (input + output)
• Context limits: Models have maximum token limits
• Speed: More tokens = slower responses

# Rough estimates:
# 1 token ≈ 4 characters ≈ 0.75 words
# 100 tokens ≈ 75 words
# 1000 tokens ≈ 750 words ≈ 1.5 pages

# Example tokenization:
"Hello, how are you?"
# → ["Hello", ",", " how", " are", " you", "?"]
# → 6 tokens

Context Window

The maximum number of tokens (input + output) the model can handle at once:

• GPT-4 Turbo: 128K tokens (~300 pages)
• Claude 3: 200K tokens (~500 pages)
• Gemini 1.5: 1M tokens (~2,500 pages)

Temperature

Controls randomness in outputs:

• 0.0: Deterministic, always picks the most likely token
• 0.7: Balanced creativity (common default)
• 1.0+: More creative but potentially incoherent

from openai import OpenAI

client = OpenAI()

# Factual, consistent responses
response = client.chat.completions.create(
    model="gpt-4",
    messages=[{"role": "user", "content": "What is 2+2?"}],
    temperature=0  # Always says "4"
)

# Creative writing
response = client.chat.completions.create(
    model="gpt-4",
    messages=[{"role": "user", "content": "Write a poem about coding"}],
    temperature=0.9  # More varied, creative outputs
)

Working with LLM APIs

Most LLMs are accessed through APIs. Here's how to get started:

OpenAI

from openai import OpenAI

client = OpenAI()  # Uses OPENAI_API_KEY env variable

response = client.chat.completions.create(
    model="gpt-4",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Explain Python decorators simply."}
    ]
)

print(response.choices[0].message.content)

Anthropic (Claude)

from anthropic import Anthropic

client = Anthropic()  # Uses ANTHROPIC_API_KEY env variable

response = client.messages.create(
    model="claude-3-sonnet-20240229",
    max_tokens=1024,
    messages=[
        {"role": "user", "content": "Explain Python decorators simply."}
    ]
)

print(response.content[0].text)

Using LangChain (Unified Interface)

from langchain_openai import ChatOpenAI
from langchain_anthropic import ChatAnthropic

# Same interface for different providers
gpt4 = ChatOpenAI(model="gpt-4")
claude = ChatAnthropic(model="claude-3-sonnet-20240229")

# Switch providers easily
response = gpt4.invoke("Explain Python decorators")
# or
response = claude.invoke("Explain Python decorators")

Choosing the Right Model

LLM Limitations

Understanding limitations helps you build better applications:

• Hallucinations: LLMs can confidently state false information. Always verify critical facts.
• Knowledge cutoff: Training data has a cutoff date; models don't know recent events.
• No true understanding: LLMs predict likely text; they don't "understand" in the human sense.
• Context limits: Long conversations may lose early context.
• Consistency: Same prompt can give different answers (unless temperature=0).
• Math and logic: Complex calculations can be unreliable.

Best Practices

• Start with prompting: Good prompts often beat complex solutions
• Use system messages: Set context and constraints clearly
• Iterate on prompts: Test and refine based on outputs
• Handle errors: APIs fail; implement retries and fallbacks
• Monitor costs: Track token usage, especially in production
• Validate outputs: Don't trust LLM outputs blindly for critical decisions

Related Articles