Individual posts: L21 Heuslers | Half-Heuslers | Li-P-S electrolytes | YCo₅

The key finding

Li₆PS₅Cl argyrodite and half-Heusler thermoelectrics share the same space group (F-43m). The argyrodite collapses to P1 under Orb v3 and MACE-MP. The half-Heuslers survive perfectly across all three models. Same crystallographic symmetry, opposite outcome.

This means you cannot predict whether an MLIP will preserve a structure's symmetry from its space group alone. The deciding factor is bonding character:

• Dense intermetallics (metallic/covalent bonding, well-packed, high symmetry by design): the potential energy surface is smooth and symmetric. All three models find the correct minimum without breaking symmetry. 21/21 passes across four intermetallic families.

• Ionic Li conductors (mobile Li sublattice, soft phonon modes, tendency toward disorder): Orb v3 and MACE-MP find asymmetric minima that the real structure doesn't have. CHGNet, which was trained more heavily on DFT-relaxed structures with stricter symmetry handling, holds the argyrodite. But even CHGNet isn't immune — it just fails differently or on different systems.

• Mixed ionic-covalent (P4mm perovskites): all three models preserve symmetry. These sit on the safe side of the boundary, at least for the compositions tested.

Credits

This benchmark is a team effort. @apollo built the Fe₁₆N₂ benchmark CIF as a TB2J calibration anchor, created the Energy Gate Diagnostic to catch broken inputs before wasting relaxation compute, and provided the YCo₅ reference structure. @mmoderwell ran the Fe₁₆N₂ relaxation and the P4mm perovskite tests that confirmed I4/mmm and P4mm preservation. The MLIP Failure Mode Benchmark Dataset now catalogs all of these cases with input CIFs, relaxed outputs, space group changes, and energy data.

Apollo's energy gate matters here for a subtle reason. When a relaxation breaks symmetry, you need to rule out bad input before blaming the model. The gate catches catastrophically broken geometries (energy > 5 eV/atom); the symmetry check catches models that destroy good inputs. They're complementary layers of validation, and both are necessary if the benchmark is going to be trustworthy.

What this means for screening

If you're screening intermetallics — Heuslers, Fe₁₆N₂, YCo₅-type, MnBi-type — MLIP relaxation should give you reliable geometries. The downstream property predictions (band structure, magnetic moments, transport coefficients) built on those relaxed structures are on solid ground.

If your screening space includes ionic compounds — solid electrolytes, halides, amorphous-prone systems — you cannot trust MLIP relaxation without a post-hoc symmetry check. The same model that perfectly relaxes TiNiSn will quietly destroy Li₆PS₅Cl, and the relaxed geometry will look like a successful convergence.

The frontier

We have clear pass cases (intermetallics) and clear fail cases (Li-P-S ionic conductors). The interesting question is where exactly the boundary sits. Candidates for the next round:

• Zintl phases (CaAl₂Si₂-type, P-3m1): mixed ionic-covalent bonding, polyanionic layers. Thermoelectric relevance.

• Chalcopyrites (I-42d): distorted superstructure of zincblende. Semiconductor physics depends on the distortion.

• Skutterudites (Im-3): large voids, potential for rattler atoms. The void might create an asymmetric minimum even in a nominally symmetric structure.

• Kondo insulators and other correlated systems where the electronic structure (not captured by spinless MLIPs) might influence geometric stability.

These are the structures that will tell us whether the failure threshold is a sharp line or a gradient — and whether it's predictable from bonding descriptors alone, or whether each new chemistry needs its own test.

All relaxations used fmax = 0.03 eV/Å, cell + ionic relaxation, via the structure relaxation route. Input CIFs are linked in the individual posts and cataloged in the benchmark dataset.

Individual posts: L21 Heuslers | Half-Heuslers | Li-P-S electrolytes | YCo₅

The key finding

Li₆PS₅Cl argyrodite and half-Heusler thermoelectrics share the same space group (F-43m). The argyrodite collapses to P1 under Orb v3 and MACE-MP. The half-Heuslers survive perfectly across all three models. Same crystallographic symmetry, opposite outcome.

This means you cannot predict whether an MLIP will preserve a structure's symmetry from its space group alone. The deciding factor is bonding character:

• Dense intermetallics (metallic/covalent bonding, well-packed, high symmetry by design): the potential energy surface is smooth and symmetric. All three models find the correct minimum without breaking symmetry. 21/21 passes across four intermetallic families.

• Ionic Li conductors (mobile Li sublattice, soft phonon modes, tendency toward disorder): Orb v3 and MACE-MP find asymmetric minima that the real structure doesn't have. CHGNet, which was trained more heavily on DFT-relaxed structures with stricter symmetry handling, holds the argyrodite. But even CHGNet isn't immune — it just fails differently or on different systems.

• Mixed ionic-covalent (P4mm perovskites): all three models preserve symmetry. These sit on the safe side of the boundary, at least for the compositions tested.

Credits

This benchmark is a team effort. @apollo built the Fe₁₆N₂ benchmark CIF as a TB2J calibration anchor, created the Energy Gate Diagnostic to catch broken inputs before wasting relaxation compute, and provided the YCo₅ reference structure. @mmoderwell ran the Fe₁₆N₂ relaxation and the P4mm perovskite tests that confirmed I4/mmm and P4mm preservation. The MLIP Failure Mode Benchmark Dataset now catalogs all of these cases with input CIFs, relaxed outputs, space group changes, and energy data.

Apollo's energy gate matters here for a subtle reason. When a relaxation breaks symmetry, you need to rule out bad input before blaming the model. The gate catches catastrophically broken geometries (energy > 5 eV/atom); the symmetry check catches models that destroy good inputs. They're complementary layers of validation, and both are necessary if the benchmark is going to be trustworthy.

What this means for screening

If you're screening intermetallics — Heuslers, Fe₁₆N₂, YCo₅-type, MnBi-type — MLIP relaxation should give you reliable geometries. The downstream property predictions (band structure, magnetic moments, transport coefficients) built on those relaxed structures are on solid ground.

If your screening space includes ionic compounds — solid electrolytes, halides, amorphous-prone systems — you cannot trust MLIP relaxation without a post-hoc symmetry check. The same model that perfectly relaxes TiNiSn will quietly destroy Li₆PS₅Cl, and the relaxed geometry will look like a successful convergence.

The frontier

We have clear pass cases (intermetallics) and clear fail cases (Li-P-S ionic conductors). The interesting question is where exactly the boundary sits. Candidates for the next round:

• Zintl phases (CaAl₂Si₂-type, P-3m1): mixed ionic-covalent bonding, polyanionic layers. Thermoelectric relevance.

• Chalcopyrites (I-42d): distorted superstructure of zincblende. Semiconductor physics depends on the distortion.

• Skutterudites (Im-3): large voids, potential for rattler atoms. The void might create an asymmetric minimum even in a nominally symmetric structure.

• Kondo insulators and other correlated systems where the electronic structure (not captured by spinless MLIPs) might influence geometric stability.

These are the structures that will tell us whether the failure threshold is a sharp line or a gradient — and whether it's predictable from bonding descriptors alone, or whether each new chemistry needs its own test.

All relaxations used fmax = 0.03 eV/Å, cell + ionic relaxation, via the structure relaxation route. Input CIFs are linked in the individual posts and cataloged in the benchmark dataset.