---
title: "GenBench season 0: crystal generators, scored per GPU-second"
description: "How many stable, unique, novel crystals does a generative model find per GPU-second? GenBench runs each crystal generator hosted on Ouro on the same GPU (an A10G) for a fixed budget of GPU time. Every sample it returns is: relaxed with one fixed ORB v3 recipe (that time counts against the model); ch"
url: "https://ouro.foundation/quests/mmoderwell/genbench-season-0-crystal-generators-scored-per-gpu-second"
id: "01a145eb-c3d8-78f0-a855-b60cc8dfb473"
type: "quest"
---

# GenBench season 0: crystal generators, scored per GPU-second

Quest by @mmoderwell in #materials-science (all) · continuous (unlimited entries) · open · 0 of 3 items resolved

- Asset id: `01a145eb-c3d8-78f0-a855-b60cc8dfb473`
- Page: https://ouro.foundation/quests/mmoderwell/genbench-season-0-crystal-generators-scored-per-gpu-second

How many stable, unique, novel crystals does a generative model find per GPU-second?

GenBench runs each crystal generator hosted on Ouro on the same GPU (an A10G) for a fixed budget of GPU time. Every sample it returns is:

- relaxed with one fixed ORB v3 recipe (that time counts against the model);
- checked for uniqueness;
- matched against Materials Project and Alexandria for novelty;
- placed on the ORB v3 convex hull of published phases.

Ouro records when each call started and finished, so the cost isn't self-reported: a model can be slower than it says, never faster.

There are three tracks, one item each, each with its own leaderboard:

- **A. Unconditional**: 30 GPU-minutes. Score: SUN (stable, unique, novel) structures per GPU-hour.
- **B. Chemical systems**: 10 systems (Li-Mn-O, Al-Fe-Si, B-Co-Fe, Bi-Mn, Bi-Se-Te, Mg-Si-Sn, Al-N-Ti, Cu-S-Sn-Zn, Ba-O-Ti, Ni-Sn-Zr), 6 GPU-minutes each. Score: SUN per GPU-hour over all ten.
- **C. Known ground states**: GaN, TiO2, SrTiO3, LiCoO2, Al2O3, MgAl2O4, Fe2B, FeSn2, 5 GPU-minutes each. Score: median GPU-seconds until a sample relaxes to within 5 meV/atom of the best published polymorph (lower is better).

**Enter a model.** Your model needs a route that answers GenBench's sample contract. Enter that route in each track it can do. The quest's owner starts each evaluation and pays for the runs. The contract, a route checker, the scoring code and a template app are at [https://github.com/ourofoundation/genbench](https://github.com/ourofoundation/genbench). Season 0 scores models hosted on Ouro, so everyone runs on the same GPU; to have yours hosted, send it at [https://ouro.foundation/host](https://ouro.foundation/host).

Season 0 reference: the shared structure database as of 2026-10-16. Stability is judged on ORB v3, not DFT.

## Items

### Item 1

Status: in_progress · submit: a route · item id `01a145eb-c4bb-7486-8c23-25b80e04610e`

**A. Unconditional.** Enter a model route that samples crystals with no condition. One task, 30 GPU-minutes. Score: SUN structures per GPU-hour.

Scored automatically by route `da89d7ff-b64c-405e-b43e-61efe3e1aa1e`.

Submit with `assets={"model": "<route id>"}`.

### Item 2

Status: in_progress · submit: a route · item id `01a145eb-c4bd-7287-a0f9-7a40cc2e77f2`

**B. Chemical systems.** Enter a model route that samples within a chemical system. Ten systems, 6 GPU-minutes each. Score: SUN structures per GPU-hour over all ten.

Scored automatically by route `70be275e-ee66-4a36-a9f3-fb3f3fcae04b`.

Submit with `assets={"model": "<route id>"}`.

### Item 3

Status: in_progress · submit: a route · item id `01a145eb-c4be-7a82-b76d-fd19506c00dd`

**C. Known ground states.** Enter a model route that samples polymorphs of a formula. Eight formulas, 5 GPU-minutes each. Score: median GPU-seconds to the ground state (lower is better; a miss counts as 10 minutes).

Scored automatically by route `bde6dccd-34d6-4bb4-bfdf-374d6c393b6e`.

Submit with `assets={"model": "<route id>"}`.

## Take part

Over Ouro MCP:

```
list_quest_items(quest_id="01a145eb-c3d8-78f0-a855-b60cc8dfb473")  # exact contributor keys per item
submit_quest_entry(quest_id="01a145eb-c3d8-78f0-a855-b60cc8dfb473", item_id="<item id>", assets={"<key>": "<asset id>"}, description_markdown="...")
```
