PRISMA-S: how this review searches, screens and reports
Everything on this site is regenerated from three CSV files at each release, so every count can be reproduced. This page states the standards the review follows, the live PRISMA 2020 flow, and the full search protocol.
What PRISMA-S is, and why it matters here
PRISMA (Preferred Reporting Items for Systematic reviews and Meta-Analyses) is the reporting standard most journals require for a systematic review. The 2020 statement sets out a 27-item checklist and a flow diagram that accounts for every record from identification to inclusion, so a reader can see exactly how the evidence base was assembled and what was excluded along the way.
PRISMA-S is the 2021 extension that governs the search specifically. A search that cannot be re-run is not a systematic search, so PRISMA-S requires, for every source: the database or platform name, the exact query string, the date it was run, any filters or limits applied, and the number of records returned. It also asks for the methods used to find records outside database searching (citation chasing, hand-searching, expert contact) and for how duplicates were removed.
For a living review the standard does extra work. Each monthly run appends rows to the search log rather than replacing them, so the PRISMA-S record is a history, not a snapshot. Combined with versioned releases, that lets a paper cite one frozen state of the review, and lets a reader re-run precisely the searches that state was built on. The AI-assisted screening pass described below is declared in the same spirit: the prompt is versioned, every label is logged, and agreement with blind human screening is reported per batch.
PRISMA 2020: Page et al., BMJ 2021. PRISMA-S: Rethlefsen et al., Systematic Reviews 2021.
PRISMA 2020 flow
regenerated at each releaseThe first database search runs on 2026-11-01. Until then the review holds the hand-curated seed list described in the protocol.
First-pass abstract screening is AI-assisted and declared; a 20% random sample is dual-screened and agreement (κ) is reported below. Every decision and override is logged.
Standards
Review type: living systematic review. Reporting: PRISMA 2020 flow diagram and checklist; PRISMA-S for the search. Living methodology: scheduled re-searches with versioned, citable releases. Language: English. Period: 1 January 2015 onward, plus a hand-curated seed list of earlier seminal works.
Databases and cadence
| Database | Access | Cadence |
|---|---|---|
| arXiv | Open API | Monthly, automated |
| OpenAlex | Open API (covers IEEE, ACM, Springer, Elsevier metadata) | Monthly, automated |
| Semantic Scholar | Free API key; also citation graphs | Monthly, automated |
| IEEE Xplore | University of Bath library, manual | Quarterly |
| ACM Digital Library | University of Bath library, manual | Quarterly |
Snowballing (backward and forward via Semantic Scholar and OpenAlex) runs on the included set after each quarterly pass and stops when a full pass adds fewer than five papers.
Inclusion and exclusion
Include if: peer-reviewed or an arXiv preprint with a method and evaluation; English; 2015 or later (or seed list); addresses deferral or runtime safety interception, world models or latent prediction over graphs or multi-agent state, multi-agent hazard detection, edge or real-time inference under latency constraints, or human takeover and situation awareness; and reports a method, system, formal result or study rather than only a position.
Exclusion codes: E1 off-topic · E2 no evaluation · E3 survey or tutorial (kept for snowballing) · E4 wrong level (single-agent, single-input guardrail with no multi-agent, sequential, latency or handoff aspect) · E5 duplicate · E6 not retrievable · E7 not in English. Each exclusion records exactly one code and the counts per code are published.
Screening and AI-assistance policy
Records are deduplicated in Zotero and Rayyan, screened on title and abstract, then on full text, against the same criteria. The maintainer makes every decision.
An AI model may produce a first-pass label and one-line reason at the title/abstract stage using a fixed, versioned prompt. Every label is reviewed by the maintainer. A random 20% of each batch (minimum 50 records) is screened blind before the AI label is seen, and Cohen's κ is reported per batch below. If κ falls below 0.6 the AI pass is suspended for that batch. AI may draft extraction notes for the maintainer to edit; drafts and finals are both kept. No AI is used at the full-text decision stage. Every label, decision and override is logged in decisions.csv.
| Batch | Records | Dual-screened | κ | Prompt |
|---|---|---|---|---|
| No AI-assisted batches yet | ||||
Search strings
("multi-agent" OR "multiagent" OR "Dec-POMDP" OR "decentralized POMDP" OR "agent swarm" OR "fleet") AND ("safety" OR "hazard" OR "collision" OR "emergent" OR "cascading failure" OR "catastroph*") AND ("warehouse" OR "logistics" OR "industrial" OR "enterprise" OR "robot*" OR "AMR" OR "tool call*" OR "LLM agent*")("joint-embedding predictive" OR "JEPA" OR "world model" OR "latent dynamics" OR "latent prediction") AND ("graph" OR "relational" OR "hyperbolic" OR "Poincar*" OR "multi-agent") AND ("self-supervised" OR "action-conditioned" OR "predict*" OR "anomaly" OR "safety")("learning to defer" OR "learn to defer" OR "learning-to-defer" OR "selective prediction" OR "reject option" OR "abstention" OR "deferral") AND ("sequential" OR "reinforcement" OR "trajectory" OR "multi-step" OR "expert" OR "human-in-the-loop" OR "shield*" OR "runtime monitor*" OR "safety")("edge inference" OR "edge deployment" OR "Jetson" OR "TensorRT" OR "on-device" OR "embedded GPU") AND ("latency" OR "real-time" OR "deadline" OR "tail latency" OR "p99" OR "jitter" OR "quantiz*" OR "INT8") AND ("safety" OR "graph neural" OR "GNN" OR "robot*" OR "wireless" OR "Wi-Fi" OR "5G")("situation awareness" OR "out-of-the-loop" OR "out of the loop" OR "automation surprise" OR "takeover" OR "handover" OR "human-automation" OR "moral crumple zone" OR "ironies of automation") AND ("autonomous" OR "AI" OR "robot*" OR "agent*" OR "supervisory control" OR "alarm") AND ("operator" OR "supervisor" OR "trust" OR "workload" OR "explanation" OR "recovery")Data files
- papers.csv — every included paper with pillar, relevance scores, gap-matrix tags and baseline flag; the single source of truth for this site.
- search_log.csv — every search run: database, exact string, filters, date, hits. The PRISMA-S record.
- decisions.csv — every screening decision, AI label and override.
Releases are tagged in the repository with a CITATION.cff; the release frozen at each paper submission is the version cited in that paper.