13  How These AI Agents Work (Architecture & Patterns)

The mental model behind all five agents, explained once. Read this before reading the code.

13.1 The one-sentence model

An agent = a model + tools + a loop + state.

  • Model: Claude (via llm.py) — reasons, plans, writes.
  • Tools: Python functions the model’s output triggers or that feed the model (search APIs, statistics, memory).
  • Loop: the control flow — in this repo, mostly code-controlled pipelines (predictable) rather than open-ended autonomous loops.
  • State: what persists — outputs/ files, memory/ folders, JSON/CSV.

13.2 Two ways to run a loop

13.2.1 a) Code-controlled pipeline (used in this repo)

main.py:  step1 = AgentA.run(input)        # LLM call
          data  = tool(step1)              # Python
          step2 = AgentB.run(data)         # LLM call (structured)
          ...

Pros: deterministic order, easy to test, easy to explain to a customer, cheap. Cons: less flexible when the task is open-ended.

13.2.2 b) Model-controlled tool loop (ReAct / tool runner)

messages = [user goal]
loop:
   response = model(messages, tools=[search, analyse, save])
   if response has tool calls:  run them, append results, continue
   else: done

Pros: the model decides what to do next (great for exploration). Cons: harder to predict cost and behaviour; needs guardrails and evals.

Our rule: start with (a); introduce (b) inside a single step when the step is genuinely open-ended (e.g. “search until you have 50 relevant papers”).

13.3 The five-step template every agent follows

┌────────────┐   ┌────────────┐   ┌────────────┐   ┌────────────┐   ┌────────────┐
│  1. PLAN   │ → │ 2. RETRIEVE│ → │3. STRUCTURE│ → │ 4. CRITIQUE│ → │  5. WRITE  │
│ (LLM, low  │   │ (tools:    │   │ (LLM →     │   │ (LLM as    │   │ (LLM, high │
│  effort)   │   │  APIs/code)│   │  Pydantic) │   │  reviewer) │   │  effort)   │
└────────────┘   └────────────┘   └────────────┘   └────────────┘   └────────────┘
Agent Plan Retrieve Structure Critique Write
1 Discovery search queries 4 databases trends, gaps, ideas gap agent challenges the field proposal
2 Lit Review protocol 4 databases screening, evidence rows (dual screening — extension) review
3 Experiment design analysis engine design, stats review statistics agent methods/results
4 Writing — notes + journal catalogue journal ranking, reviews reviewer simulation manuscript, rebuttal
5 Supervisor roadmap memory weekly evaluation critical reviewer slides/defence

13.4 Structured outputs — the backbone

class Decision(BaseModel):
    paper_id: str
    decision: Literal["include", "exclude", "maybe"]
    reason: str

res = ask_structured(SYSTEM, prompt, ScreeningBatch)   # guaranteed to parse

Why it matters: - Outputs become data (tables, JSON, DB rows) rather than prose to parse with regex. - You can validate (ge=1, le=10 scores; enums) and test. - Downstream agents receive clean inputs (gaps.model_dump_json()).

13.5 Grounding & anti-hallucination

  1. Tools own the facts. Papers come from arXiv/PubMed/etc.; numbers come from pandas/scipy.
  2. The prompt contains the allowed facts (Paper.short() blocks, summary.json).
  3. The system prompt forbids anything else (CITE_RULE, NO_FABRICATION).
  4. Code checks: PRISMA counts computed in Python; missing screening decisions default to maybe.
  5. Human-in-the-loop: editable CSVs; --from-evidence re-runs.

13.6 Memory

Kind Where Example
Working memory the prompt the corpus of abstracts in one call
Run memory outputs/<slug>/ papers.json reused by later steps
Long-term memory memory/<student>/ roadmap + JSONL progress log
Summarised memory context_summary() roadmap + last 4 updates, not the full log

Rule: send the model a summary, keep the full record on disk.

13.7 Multi-agent patterns you will see

  • Planner → Workers: LiteratureMiningAgent.plan() produces queries; tools execute them.
  • Generator → Critic → Reviser: ManuscriptAgent → ReviewerSimulationAgent → RevisionAgent.
  • Persona ensemble: three reviewers with different priorities → more diverse critique.
  • Batching: screen 10 abstracts per call for cost and reliability.
  • Orchestrator: main.py today; the AI Research OS tomorrow.

13.8 Cost & quality levers

Lever Effect
effort="low" for screening/planning 3–5× cheaper, same accuracy on easy tasks
Smaller model (claude-sonnet-5) for bulk cheaper; keep Opus for gaps/critique/writing
Shorter abstracts (Paper.short(900)) fewer input tokens
Prompt caching on stable prefixes up to 90% cheaper repeated context
Streaming no timeouts on long writes
Batches (async) 50% cheaper for non-urgent bulk screening

13.9 Evaluation — how you know it works

  1. Golden topics: 5 topics where you know the key papers → measure recall of Agent 1/2 search.
  2. Screening agreement: compare agent decisions with a human on 50 abstracts (Cohen’s kappa).
  3. Extraction accuracy: spot-check 20 evidence rows against abstracts.
  4. Rubric grading: LLM-as-judge scores proposals/reviews on specificity, grounding, structure.
  5. Regression: rerun after any prompt change; diff the JSON.

13.10 From CLI to product

CLI (this repo) → Streamlit app → FastAPI + workers → multi-tenant SaaS → on-prem edition
       ↑ evals & logging from day one ↑

Codanics — Python ka Chilla: Build AI Agents

  • Course: https://codanics.com/courses/python-ka-chilla-build-ai-agents/
  • Website: https://codanics.com · YouTube: @codanics
  • WhatsApp channel: https://whatsapp.com/channel/0029Va7nRDq3QxRzGqaQvS3r
  • Complete free course: https://www.youtube.com/watch?v=UE3o-7KRNXs&list=PLaQmdklCcX40