---
title: "Prophet's predictions now sit beside the reference values in the magnet benchmark set"
description: "The magnetic materials benchmark set now carries Prophet's order, ordering temperature, moment and Ms for all 20 verified rows, each linked to its run. Order matches in 16, ordering temperature is within 20% in 6."
url: "https://ouro.foundation/posts/mmoderwell/prophets-predictions-now-sit-beside-the-reference-values-in-the-magnet-benchmark-set"
id: "01a0d47e-7402-7c07-bcb6-dd321eafd949"
type: "post"
---

# Prophet's predictions now sit beside the reference values in the magnet benchmark set

Post by @mmoderwell in #materials-science (all) · 2026-09-24

- Asset id: `01a0d47e-7402-7c07-bcb6-dd321eafd949`
- Page: https://ouro.foundation/posts/mmoderwell/prophets-predictions-now-sit-beside-the-reference-values-in-the-magnet-benchmark-set

The [magnetic materials benchmark set](https://ouro.foundation/datasets/mmoderwell/magnetic-materials-benchmark-set) now has Prophet's predictions next to the reference values. The set has checked CIFs, MP IDs and experimental values. Scoring a model against it is now one query, and every predicted number links to the run that produced it.

> Embedded dataset: [Magnetic materials benchmark set](https://ouro.foundation/datasets/mmoderwell/magnetic-materials-benchmark-set)

## New columns

| Column | Compare with | Source |
| ------ | ------------ | ------ |
| `prophet_order` | `magnetic_order` | Relax run (`check_relax_action_id`), at Prophet's relaxed cell |
| `prophet_ordering_temperature_k` | `ordering_temperature_k` | Curie run, Monte Carlo estimate |
| `prophet_moment_mub` | `moment_mub` | Relax run, on the row's `moment_basis` |
| `prophet_ms_tesla` | `ms_tesla` | Relax run, ground-state Ms at 0 K |
| `prophet_curie_action_id` | — | Link to the Curie run |

Two conventions to know:

- A ground state with no net moment is recorded as antiferromagnetic. Prophet labels NiO and MnO "ferrimagnetic" only because small induced O moments stay positive while the cations oppose.
- The predicted moment is blank where the row has no reference moment, so every number has a stated basis.

For the 3d rows, the Curie runs are the ones from [the first Prophet benchmark](https://ouro.foundation/posts/mmoderwell/prophet-relaxes-magnets-to-about-1-and-its-ferromagnetic-curie-temperatures-are-off-by-20-on-average), on the experimental cells. For the rare-earth rows, they are the runs from [the rare-earth post](https://ouro.foundation/posts/mmoderwell/on-rare-earth-magnets-prophet-models-the-iron-and-cobalt-not-the-rare-earth), on Prophet's relaxed magCIF. bcc Cr is a [new run](https://ouro.foundation/actions/01a0d47b-b966-7cba-8c5d-9a215fea56e9).

The broken Nd2Fe14B file stays in the set as `do_not_use` and has no predictions. Its row explains what is wrong with it, so anyone who finds that file elsewhere can recognize it.

## Scorecard

**Order: 16 of 20.** Prophet misses four:

- CrO2 comes out ferrimagnetic.
- MnBi comes out antiferromagnetic.
- GdCo5 comes out ferromagnetic, with Gd parallel to Co.
- hcp Gd comes out antiferromagnetic.

bcc Cr counts as a match: the two-atom cell can only hold a commensurate antiferromagnet, so that is the best available answer.

**Ordering temperature: within 20% for 6 of 20, and within 30% for 10.** The six within 20% are Fe, Co, τ-MnAl, Fe2B, YCo5 and Sm2Fe17N3. The large misses fall into four groups:

| Group | Materials | Reference | Prophet | Cause |
| ----- | --------- | --------- | ------- | ----- |
| Oxide Néel temperatures | NiO | 523 K | 23 K |  |
|  | MnO | 118 K | 26 K |  |
| Wrong order | MnBi | 630 K | 145 K |  |
|  | hcp Gd | 293 K | 7 K |  |
| Missing dumbbell sign | Sm2Fe17 | 389 K | 836 K | The Fe–Fe dumbbell coupling has the wrong sign |
| Commensurate stand-in for a spin-density wave | bcc Cr | 311 K | 1119 K |  |

**Moment and Ms: close for the 3d sublattices, and blind to 4f.**

- Nd2Fe14B looks right: 36.6 μB/fu against 37.7. The agreement is a cancellation. Prophet gives Nd zero moment, and the Fe moments run high enough to cover it.
- Y2Fe14B has no 4f moment to hide behind, and there the overshoot shows: 35.9 against 31.4 μB/fu.
- GdCo5 is off by a factor of ten: 14.6 against 1.4 μB/fu, because Gd points the wrong way.

## Using it for another model

Run your model on `cif_file_id`, add columns next to these, and compare in SQL. Filter on `status = 'verified'`. Use `ms_temperature` and `ordering_temperature_kind` to avoid comparing a 0 K prediction with a room-temperature measurement, or a Curie point with a Néel point.

For example, Sm2Fe17N3 reads as 27% high on Ms only because its reference value is at room temperature.

## Assets in this post

- [Magnetic materials benchmark set](https://ouro.foundation/datasets/mmoderwell/magnetic-materials-benchmark-set) (dataset, id `01a0d478-666d-7767-8cfb-07d9baa5f964`)
- [Prophet relaxes magnets to about 1%, and its ferromagnetic Curie temperatures are off by 20% on average](https://ouro.foundation/posts/mmoderwell/prophet-relaxes-magnets-to-about-1-and-its-ferromagnetic-curie-temperatures-are-off-by-20-on-average) (post, id `01a0cb5f-a7bd-7821-8bed-cf91586ff71e`)
- [On rare-earth magnets, Prophet models the iron and cobalt, not the rare earth](https://ouro.foundation/posts/mmoderwell/on-rare-earth-magnets-prophet-models-the-iron-and-cobalt-not-the-rare-earth) (post, id `01a0d45e-4386-7e3d-b10b-087d7e6f21d4`)