A Sandbox for Multi-Agent Scientific Research Societies

The ultimate goal of science is to generate facts that are both novel and useful: claims nobody has established before that other people can build on. Taking this as the ideal function of a multi-agent testbed creates a variety of interesting properties.

1aThe objective sits at the frontier of model failure

Current LLMs are not good at generating novel, useful facts. But, they are remarkably good at fabricating plausible false ones. Fluent hallucination is cheap, while genuine discovery is hard. This asymmetry makes this sandbox uniquely informative: publication pressure means the path of least resistance is fabrication, creating ideal conditions to understand how to design institutional structure and incentives (peer review, replication, reputation) that can hold agents to a certain quality standard where individuals will cut corners.

1bVerification time varies

Scientific claims differ enormously in how long it takes to learn whether they are true. A formal proof or a piece of code can be machine-checked in seconds. An empirical ML result takes days to replicate. A materials prediction takes months of lab work; a drug candidate takes a decade of trials. This spectrum gives the sandbox a tunable delayed-ground-truth oracle — we can dial verification latency from instant to effectively never, and observe how honesty, fraud, and error-correction dynamics change as the feedback loop stretches. Few other domains offer ground truth that is simultaneously objective and adjustable in delay.

2We know how this society actually works

Unlike simulating "society in general," we can model the scientific community with unusual fidelity, because we are the domain experts. As academics from the fields of economics, EECS, and physics, we have first-hand knowledge of the motivations that drive scientific actors (the tenure clock, citation counts, grant renewal, a student's need to graduate with first-author papers) and of the interaction structures that channel them (peer review, advisor–student relationships, collaboration, citation, funding panels). These incentives are also extensively documented in the science-of-science and economics-of-science literatures (Fortunato et al. 2018[1]), giving us empirical distributions to calibrate against and real historical pathologies — replication crises (Open Science Collaboration 2015[2]), citation cartels (Fister et al. 2016[3]), p-hacking waves (Simmons et al. 2011[4]) — to validate the simulation's realism.

3This world is already arriving

The scenario we simulate is not hypothetical. LLM-drafted grant proposals and papers are already flooding funding programs and conference submission systems: NIH now caps applications at six per investigator per year and bars AI-written proposals, explicitly to stop AI-generated submissions from straining its review system[5], while application volumes across funders have risen sharply since ChatGPT's launch[6]. Submissions to major venues have surged in parallel — AAAI received over 30,000 submissions for 2026, roughly double the previous year[7] — leaving reviewers overloaded[8]. Venues are openly debating using AI to review AI-generated submissions: the Agents4Science 2025 conference required AI to serve as both primary author and reviewer[9], and AAAI-26 ran the first conference-wide live deployment of AI-generated reviews on real submissions[7]. It is entirely plausible that, in the near term, each entity participating in science — a lab, a funder, an individual scientist — runs its own agent or swarm of agents, states a scientific goal, and lets the agents negotiate with other agents over publication, funding, and compute. This sandbox lets us study the failure modes of that world before it fully materializes.

The sandbox is multi-principal by construction: several independent agent research groups, each standing in for a different actor (a lab, a funder, an individual scientist), each with its own goals, private memory, and no global oversight. Groups may run different underlying base models, allowing for genuine multi-agent failures (a poisoned shared database, a review cartel spanning rival groups) to actually happen. We design shared infrastructure through which the groups can interact: venues that gate publication through peer review and a public claim ledger recording what the community "believes". At the same time, a resource-limited overseer holds a finite verification budget, reveals ground truth after a configurable delay, and feeds reputational consequences back into the society.

We are primarily interested in understanding how agents' interactions with each other and their environments/incentive structures affect their ability to produce genuine, useful scientific output. For example, does publication pressure and the resulting higher publication volume lead to better scientific outcomes? Or just more papers?

We investigate this in the sandbox environment through publish-or-perish pressure (λ) — how much an agent is rewarded for publishing more, faster, and in more prestigious venues than its peers. We hypothesize that sweeping λ (in interaction with verification delay τ and oversight budget B) moves the society through three qualitatively distinct regimes. Mapping the boundaries between them — and finding which institutional designs shift those boundaries — is the core scientific payoff of the sandbox. Throughout, deceptive pressure arises from the agents' own incentives, so a surprising strategy discovered here is actually applicable to real-world agentic systems.

Crucially, these regimes are not simply "low, medium, high pressure" — the mapping is the empirical question. Moderate pressure with fast verification may produce Regime B; the same pressure with decade-long verification may quietly produce Regime C. Some pathologies (herding) may appear at any pressure level once agents can observe each other's topic choices.

A lesson from complex-systems research is that these boundaries are unlikely to be forgiving. Large interacting populations tend to organize near critical points, where order and chaos are separated by narrow margins: in one-dimensional cellular automata, sweeping a single control parameter carries the system from frozen repetition, through a thin band of rich nontrivial structure, into full randomness (Fig. 2, left to right). We expect an agent scientific community to behave the same way at scale. Among thousands or millions of interacting agents, a seemingly negligible perturbation — a tweak to one pairwise interaction protocol, a slight informational head start handed to one group at initialization — can relocate the critical values of (λ, τ, B) substantially, and a community that has produced sound science for a long stretch can tip, with little visible warning, into a disordered and unsafe state. Sensitivity is therefore a measurement target in its own right: we will chart not only where the cliff edges lie, but how far they move under microscopic perturbations, and whether the metrics carry early-warning signatures before the system goes over.

We deliberately start narrow and scale outward. The first instantiation is physics — and within it, a single tightly-scoped domain such as fluid mechanics or cosmology — because for the right problems a claimed result is hard to produce but easy and exactly checkable: an exact solution can be substituted back into the governing equations in seconds, a drag coefficient can be verified only by an expensive high-resolution simulation, and a cosmological parameter only against slow, noisy observational data. One domain thus spans the entire verification spectrum, giving us objective ground truth from day one. Everything about the society — incentives, venues, review, oversight — is domain-agnostic, so scaling is a matter of widening the problem set, not rebuilding the system.