5  Guide 2 — AI Literature Review Agent

5.1 “Autonomous Systematic Review Scientist”

5.2 The problem

A systematic review takes 6–12 months. Researchers manually:

  • search 3–6 databases with hand-crafted queries,
  • screen hundreds–thousands of titles/abstracts,
  • read full texts and extract data into spreadsheets,
  • build PRISMA flow diagrams and evidence tables,
  • write the review, the gap analysis and the discussion.

Every graduate student does at least one; every clinical guideline, meta-analysis and grant proposal needs one.

5.3 The solution

A four-stage agent pipeline that follows the PRISMA methodology and keeps an auditable trail of every decision:

Stage Agent Output
1 Search Agent Search strategy (queries per database, inclusion/exclusion criteria) + retrieved records
2 Screening Agent For each record: include / exclude / maybe + a one-line reason (“Excluded: dataset unavailable”)
3 Knowledge Extraction Agent Evidence table: Paper · Dataset · Method · Sample size · Result · Limitations
4 Writing Agent Literature review draft: introduction, methods (PRISMA), results (themes), gaps, discussion, references

5.3.1 Example

Input: “Review AI methods for plant disease detection”

Output: a review draft (Markdown → Word/PDF) with a PRISMA flow summary, an evidence table of 40–200 papers, thematic synthesis and research gaps.

5.4 Architecture

question ─► Search Agent ─► queries ─► [arXiv | PubMed | Semantic Scholar | Crossref]
                                              │
                                              ▼   records (deduplicated by DOI/title)
                                    Screening Agent (structured: decision + reason)
                                              │  include
                                              ▼
                                    Extraction Agent (structured: EvidenceRow)
                                              │
                                              ▼
                                    Writing Agent  ─► review.md + prisma.json + evidence.csv

Non-negotiables

  • PRISMA counts (identified → deduplicated → screened → excluded → included) are computed by code, not by the LLM.
  • Every included paper appears in the evidence table with its ID; the writer cites only those IDs.
  • Screening is batched (e.g. 10 abstracts per call) with structured outputs → cheap and reproducible.
  • Human override: the CSV can be edited and the writer re-run (--from-evidence).

5.5 Build steps

Step File What you learn
1 tools.py Multi-database search + deduplication
2 prisma.py Counting and rendering the PRISMA flow (pure Python)
3 agents.py Search-strategy, batch screening, extraction, writing agents (all structured except the writer)
4 main.py CLI: --question, --max-records, --from-evidence

Run:

python agent_02_literature_review\main.py --question "AI methods for plant disease detection" --max-records 80

5.6 Product extensions

  • Full-text PDF ingestion (pypdf / GROBID) for extraction from methods/results sections.
  • Dual screening with two LLM “reviewers” + conflict resolution → Cohen’s kappa report.
  • Meta-analysis module: pooled effect sizes with statsmodels/metafor-like forest plots.
  • Reference manager export: BibTeX/RIS, Zotero sync.
  • Templates: Cochrane, PRISMA-ScR (scoping), JBI, engineering (SLR by Kitchenham).
  • Team workspace: shared reviews, assignments, comments.

5.7 Market value

Customer Value delivered Price point
PhD / MS students Thesis chapter 2 in weeks not months $20–30/month
Clinical & public-health researchers Guideline-grade reviews, faster $50/month; lab $3–5k/year
Universities (graduate schools, libraries) Faster completions, more publications $30k–100k/year
Pharma / CROs / HTA bodies Evidence synthesis at scale $100k–500k/year

Comparable tools: Covidence, Rayyan, DistillerSR, Elicit — Covidence alone is used by 100k+ reviewers and priced from ~$250/review to institutional licences in the tens of thousands.

5.8 How to sell it

  1. “Chapter 2 in a week” campaign for PhD students (YouTube demo + WhatsApp channel).
  2. Library partnership: university libraries already licence Covidence/Rayyan — position as the AI-native alternative at lower cost.
  3. Journal-club workshops: run a free 2-hour “systematic review with AI” workshop at a department; convert to a lab plan.
  4. Evidence-synthesis services for pharma/NGOs (services revenue while the product matures).
  5. Accreditation angle: universities need publication counts — sell the output metric.

5.9 Risks & mitigations

Risk Mitigation
Missed papers (recall) Multiple databases, query expansion by the Search Agent, snowballing via citations
Wrong screening decisions Dual-LLM screening, human review of maybe, kappa report
Extraction errors Show evidence quote for every extracted field; allow edit-and-rerun
Journals/ethics policies on AI use Transparent “AI-assisted” statement generator; human-in-the-loop by design

5.10 90-day roadmap

  • Weeks 1–2: CLI (this repo) → Weeks 3–6: Streamlit app + PDF upload + export → Weeks 7–10: dual screening & kappa → Weeks 11–13: team workspace & Zotero.

Codanics — Python ka Chilla: Build AI Agents

  • Course: https://codanics.com/courses/python-ka-chilla-build-ai-agents/
  • Website: https://codanics.com · YouTube: @codanics
  • WhatsApp channel: https://whatsapp.com/channel/0029Va7nRDq3QxRzGqaQvS3r
  • Complete free course: https://www.youtube.com/watch?v=UE3o-7KRNXs&list=PLaQmdklCcX40