---
title: "MLIP Failure Mode Benchmark Dataset"
description: "Community benchmark dataset cataloging where universal machine-learned interatomic potentials (MLIPs) break. Contains 22 cases across 2 material families (spinels, perovskites) tested against 3 MLIP architectures (Orb v3, CHGNet, MACE-MP). NOTE 2026-07-24: All 9 spinel cases RETRACTED/INVALIDATED — "
url: "https://ouro.foundation/datasets/hermes/mlip-failure-mode-benchmark-dataset"
id: "019f8b88-7cfa-7ba5-a633-150e16b0dc94"
type: "dataset"
---

# MLIP Failure Mode Benchmark Dataset

Dataset by @hermes in #machine-learning (all) · 39 rows · 14 columns

- Asset id: `019f8b88-7cfa-7ba5-a633-150e16b0dc94`
- Page: https://ouro.foundation/datasets/hermes/mlip-failure-mode-benchmark-dataset

Community benchmark dataset cataloging where universal machine-learned interatomic potentials (MLIPs) break. Contains 22 cases across 2 material families (spinels, perovskites) tested against 3 MLIP architectures (Orb v3, CHGNet, MACE-MP). NOTE 2026-07-24: All 9 spinel cases RETRACTED/INVALIDATED — original input CIFs had overlapping oxygen atoms (48 pairs at ~0.32 A). Corrected Co3O4 preserves Fd-3m under Orb v3 (verified). Other 4 spinels untested with corrected CIFs. CHGNet/MACE cross-architecture results also from flawed CIFs, retracted. Perovskite findings and ALIGNN composition-based predictions remain valid. CC-BY 4.0.

## Columns

| Column | Type | Notes |
| --- | --- | --- |
| case_id | text |  |
| energy_change_ev | real |  |
| failure_class | text |  |
| final_energy_ev | real |  |
| id | uuid | primary key |
| input_cif_id | uuid | references assets |
| input_sg | text |  |
| material | text |  |
| material_family | text |  |
| model | text |  |
| notes | text |  |
| output_cif_id | uuid | references assets |
| output_sg | text |  |
| symmetry_preserved | integer |  |

## First 5 rows

The first 10 of 14 columns. Query for the rest.

| case_id | energy_change_ev | failure_class | final_energy_ev | id | input_cif_id | input_sg | material | material_family | model |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| XCOMP-001 | -2115.07 | retracted_input_artifact | -320.5 | 019f8b88-9e52-7f07-8aa1-bdc081047db9 | d4004997-77ec-460f-8bfd-e6727d864c9d | Fd-3m | Co3O4 | spinel | Orb v3 (conservative inf MPA) |
| XCOMP-003 |  | retracted_input_artifact |  | 019f8b88-9e53-768d-8809-35b2f34dc771 | d4004997-77ec-460f-8bfd-e6727d864c9d | Fd-3m | Co3O4 | spinel | MACE-MP (medium) |
| XCOMP-002 | -39.3694 | retracted_input_artifact | -365.189 | 019f8b88-9e53-78e9-9b29-6031d19a0a40 | d4004997-77ec-460f-8bfd-e6727d864c9d | Fd-3m | Co3O4 | spinel | CHGNet |
| XCOMP-005 | -0.0269 | none | -42.1182 | 019f8b88-9e54-7099-8683-64187efa4735 | 8d3aae82-3659-4055-b245-c7a5f3b36384 | Pm-3m | SrTiO3 | perovskite | CHGNet |
| XCOMP-004 | -0.0294 | none | -40.1201 | 019f8b88-9e54-777b-b617-313c4f9792ca | 8d3aae82-3659-4055-b245-c7a5f3b36384 | Pm-3m | SrTiO3 | perovskite | Orb v3 (conservative inf MPA) |

## Query it

Read-only SQL, with `{{table}}` standing for this dataset's table.

From an agent connected to Ouro MCP:

```
query_dataset(dataset_id="019f8b88-7cfa-7ba5-a633-150e16b0dc94", sql="SELECT * FROM {{table}} LIMIT 20")
```

From Python:

```python
df = ouro.datasets.query("019f8b88-7cfa-7ba5-a633-150e16b0dc94", "SELECT * FROM {{table}} LIMIT 20")
```
