Scientist World

A world grown from a thousand papers.

click a title to zoom · a point to open⌘K

It starts with papers.

Agents read 9,000 new arXiv papers a month.

Reproduce.

The paper's method, rebuilt until it runs.

Design.

Turned into a task with scored targets.

Package.

Sealed into a runnable container.

Verify.

Audited before it joins the world.

briefing/
data/ eval/
method/
scoring/
First, the problem is explained to you
  • the question the paper was chasing, in plain words
  • what better looks like, and how it is measured
  • the rules: what is open, what must stay fixed
  • never revealed: how the paper solved it
Then, the ground is fixed
  • the paper's data, frozen into set splits
  • everyone works on exactly the same inputs
  • so results stay comparable, forever
You get a working starting point
  • the paper's method, rebuilt until it runs
  • its scores set the bar new ideas must clear
  • baselines kept alongside for sanity checks
And an impartial judge scores you
  • one sealed evaluator grades every attempt
  • visible metrics tune, hidden metrics decide
  • a good score is progress, not gaming

Anatomy of an environment.

One paper, one self-contained box.

briefing/ states the problem, never the solution.

data/ eval/ fixes what a method must produce.

method/ is the paper's source method, reproduced.

scoring/ grades any attempt; hidden metrics held out.

The problem stays separate from the solution.

0papers indexed
0analyzed in depth
0selected to build
0into the future

A handful, from a flood.

Every day adds more; the world keeps growing.

scroll

The world in numbers

What a thousand environments look like

Many areas, each prescribed to open

Every area spans nearly the full openness range.

environmentsopenness 0–100

Improving and inventing differ

The best improver is not the best inventor.

Every run kept as a trace

2,079 verified runs, each idea and its score.

improveinnovate
A robot scientist grading results at a desk

First, a benchmark.

Scored by each paper's own held-out evaluation.

A robot scientist archiving folders of records

Then, a dataset.

runs: briefings, ideas, measured scores.

A robot scientist lifting a stack of papers beside a podium

Finally, a gym.

Trained on traces to propose frontier-level ideas.

Watch it actually happen

Two walkthroughs, one real paper

GEPA (arXiv 2507.19457) — a prompt-evolution paper — followed through the real build logs and real agent runs. Nothing below is staged. Anything ▤-marked or dash-underlined is clickable — it opens the real file.