AI Foundations¶

This section builds the core mental models needed to design and reason about AI applications before moving into RAG, MCP, agentic skills, agentic loops, memory, and larger agent architectures.

The goal is not to memorize model internals. The goal is to understand where an LLM is useful, where deterministic software should take over, and how the surrounding application makes AI behavior reliable.

Detailed topics¶

1. LLM Mental Model ¶

Understand what an LLM does at runtime, what it learned during training, why its output is probabilistic, and why hallucination is a natural failure mode. Separate model capability from application capability.

2. Tokens and Context Window ¶

Understand tokenization, input and output tokens, context limits, context budgets, and why more context is not automatically better. Learn to think about relevance, prioritization, and context pollution.

3. Model Inputs and Structured Outputs ¶

Understand the main forms of model input and output: instructions, conversation context, retrieved information, tool results, free text, JSON, schema-constrained output, extraction, classification, and tool calls.

4. Prompt Engineering ¶

Treat prompts as task specifications rather than magic wording. Focus on goals, constraints, inputs, examples, decomposition, explicit output contracts, and the boundary between prompts and deterministic application logic.

5. Context Engineering ¶

Learn how to decide what information the model should see at the moment it performs a task. Cover conversation history, retrieved documents, project state, tool results, memory, compression, relevance, and authorization.

6. Sampling and Model Behaviour ¶

Understand why repeated calls may produce different results, how temperature and top-p influence output, and why deterministic business rules should stay in deterministic code.

7. Model Selection and Routing ¶

Choose models according to task capability, latency, cost, context size, multimodality, structured output, and tool-use requirements. Cover static selection, dynamic routing, escalation, and fallbacks.

8. Embeddings ¶

Understand vector representations, semantic similarity, vector search, chunking, metadata, hybrid retrieval, and why embeddings are one retrieval signal rather than a source of truth.

9. Tool Calling ¶

Understand how a model requests external capabilities through structured tool calls while the application remains responsible for validation, authorization, execution, side effects, error handling, and auditability.

10. Reliability and Deterministic Boundaries ¶

Learn how to combine probabilistic model behavior with deterministic contracts, business rules, retries, idempotency, stop conditions, fallbacks, validation, and observability.

11. Evaluation Basics ¶

Measure AI changes using representative datasets, golden examples, deterministic checks, semantic evaluation, human review, LLM-as-a-judge, regression testing, component-level evaluation, and baselines.

12. Cost and Latency ¶

Understand token economics, model-call latency, time to first token, streaming, routing, caching, retries, agent-loop multiplication, cost budgets, latency budgets, and end-to-end optimization.

13. Safety and Trust Boundaries ¶

Treat model output and external content as untrusted. Cover prompt injection, authorization, least privilege, safe tools, secrets, tenant isolation, memory poisoning, sandboxing, side-effect controls, and auditability.

14. AI Application Mental Model ¶

Assemble the previous concepts into a software architecture mental model where the LLM is one probabilistic component surrounded by deterministic application logic, retrieval, tools, state, validation, security, observability, and evaluation.

Learning order¶

Foundation status¶

Complete at first-draft depth.

The individual documents are expected to evolve as questions arise, but the complete foundation map now has detailed coverage.

Possible next areas¶

RAG and retrieval architecture
MCP
AI / agent skills
agentic loops
agent state and memory
workflow engines and durable execution
agent architecture
deeper production evaluation and observability