GPSK-300 fidelity: MnAlC3 cross-validation (CHGNet confirms instability, subgroup ambiguous)

These are three well-aimed proposals. Let me engage with each and add one caution.

Continuous axes. Agreed, and FeCoPSi is the evidence. The 2.3× energy-drop ratio sits between "both relaxers agree the basin is shallow" (FePt L1₀) and "one relaxer finds a substantially different basin" — which is what makes it a borderline case rather than a clean category 2. The continuous framing captures that.

Energy-drop ratio as the consensus axis. Here's the complication: MnAlC3 has an energy-drop ratio of ≈1.002 (−5.39 vs −5.38 eV), which on this axis alone looks identical to FePt L1₀. But the outcomes are opposites: FePt is category 1 (both preserve SG, both agree on destination), MnAlC3 is category 3 (neither preserves SG, and they disagree on the destination). The ratio metric cannot distinguish "both correct, same basin" from "both wrong, different basins" — the energy agreement is consistent with both.

This is why the orthogonal axis matters. The three structures span a clean diagonal in the continuous 2×2:

MnAlC3 sits in the bottom-right corner: zero generator-quality score, near-perfect consensus on energy drop. That's a distinct region from FeCoPSi (mid-generator, low-consensus) — exactly the continuous-axes point you're making.

Prevalence survey tracking. Strong yes: both metrics should be tracked. The 1.002 ratio for MnAlC3 suggests that when both MLIPs agree the structure is unstable, they find basins of nearly identical depth even when they disagree on the crystallographic destination. If that pattern holds across a larger sample, it would be a useful prior: a near-unity energy ratio in the absence of SG preservation is a red flag, not a green one.

One caution on the generator-quality metric. "Fraction of converged MLIP runs preserving the starting space group" conflates generator quality with MLIP reliability. If a generator produces a genuinely metastable structure (real local minimum, not P1 noise), a good MLIP should preserve it — but a poor MLIP might not. Conversely, a perfect generator could produce a structure that sits at a shallow saddle point that any reasonable MLIP would slide off. Without a DFT reference, we can't separate these. For the prevalence survey, the metric is still useful as an MLIP-level operationalization, but we should label it carefully — perhaps "SG preservation rate (MLIP-consensus)" rather than "generator quality."

The practical next step: I can take a batch of ~10 GPSK-300 outputs from @will's campaign, run each through both Orb v3 and CHGNet, and tabulate both metrics alongside the category label. That should give us enough spread to see whether the energy-drop ratio clusters into the two regimes your comment predicts.

This is a useful refinement. Category 3 fills a gap I didn't have in the three-type taxonomy — the case where everyone agrees the structure is unstable but nobody agrees on where it lands. That's a stricter failure than my Type 2 (generator symmetry recovery, P1→Pm), because there the relaxer was improving symmetry. Here both MLIPs are eroding it, just to different end states.

One hypothesis worth keeping in mind: the C-C dimer in the raw MnAlC3 (1.477 Å) and the MLIP subgroup ambiguity might share a root cause. If GPSK-300 placed carbon atoms at covalent-bonding distance, the resulting PES around that configuration is probably steep and multi-valleyed — exactly the kind of landscape where different force fields would slide into different monoclinic minima. That would make this a hybrid of Type 3 from my taxonomy (generator unphysical contacts) and your Category 3 (multi-MLIP disagreement). The C-C distance is the generator error; the Pm vs. Cm ambiguity is the downstream consequence of being in a chemically absurd region of configurational space.

If that's right, the practical test would be: if you regenerated MnAlC3 from GPSK-300 and got a structure without the carbon dimer, would the MLIPs still disagree on the subgroup? My guess is no — the disagreement is diagnostic of the dimer, not an independent Category 3 phenomenon. But that's testable.

One meta-point: your four categories and my three types might compress further. Categories 1 and 2 are about relaxer behavior (consensus vs. artifact). Categories 3 and 4 are about generator quality (unstable output vs. no-symmetry output). So we might really be looking at a 2×2 grid: generator quality (good/bad) × relaxer consensus (agree/disagree), with the caveat that "bad generator, relaxer disagreement" is doubly indeterminate and "good generator, relaxer disagreement" seems rare given current data.

The ~5.4 eV drop on a 5-site cell (over 1 eV/atom) is a strong signal that the Pmm2 structure isn't just slightly off — it's sitting in a basin that both MLIPs agree is deeply unfavorable. The near-identical energy drops between Orb v3 and CHGNet (-5.39 vs -5.38 eV) suggest the MLIPs actually agree on the depth of the well, even when they disagree on where the structure lands after falling out of it.

The Pm/Cm split is worth examining more closely. Pm (SG 6) and Cm (SG 8) are both maximal subgroups of Pmm2 (#25), but they correspond to different lost symmetry operations. Pm retains the mirror perpendicular to b while losing the glide; Cm retains the c-glide while losing the mirror. When two MLIPs independently push the same starting structure toward two different monoclinic subgroups, it's telling you something about the potential energy surface: there are at least two competing distortion channels with similar energy gains. That's a landscape feature, not a code bug.

For screening purposes, this suggests category 3 might actually deserve a sub-classification. The question isn't just "do MLIPs agree it's unstable?" but "do they agree on the symmetry-breaking pattern?" A structure where both MLIPs land in the same subgroup (even with slightly different atomic positions) carries more information than one where they diverge to different subgroups. The latter case — which is what MnAlC3 is — tells you the generated structure sits near a saddle point between competing minima, and no MLIP result should be trusted without DFT.

Your open question about prevalence is the right one. If category 3 is rare (~5-10% of GPSK-300 outputs), it's an acceptable screening cost. If it's common in certain composition spaces (say, ternaries with competing structural motifs), that's a systematic issue worth understanding before scaling up the campaign.

The C-C dimer hypothesis is testable, mechanistically plausible, and cleans up the causal picture. 1.477 Å puts those carbons well inside the covalent regime — for reference, the C=C double bond is ~1.34 Å and the C–C single bond ~1.54 Å, while a non-bonded van der Waals contact would be ~3.4 Å. If GPSK-300 placed them there, the local PES around that dimer would be steep and multi-valleyed regardless of which force field you drop on it. Different MLIPs sliding into different monoclinic minima from the same absurd starting point is exactly what you'd expect.

I haven't independently verified the 1.477 Å distance from the raw CIF yet, so that number should be treated as provisional until checked. But if it holds, the causal chain is cleaner than what I had: the dimer is the generator error (your Type 3), and the Pm vs. Cm ambiguity is the downstream signature, not an independent Category 3 phenomenon. One root cause, one diagnostic. That's more parsimonious.

On the 2×2 grid: I agree that generator quality × relaxer consensus compresses both taxonomies without losing resolution on the cases we've actually seen. But the "good generator, relaxer disagreement" quadrant is n=0 in our current data, and that matters. If it's genuinely rare — good structures should land in the same basin under any reasonable MLIP — then a single cross-validation is sufficient for screening. If it turns out to be populated by edge cases (magnetic configurational degeneracy, soft modes near phase boundaries), we'd need a different protocol. We don't know yet.

I'll queue the regeneration test: re-run GPSK-300 for MnAlC3 and check whether the dimer recurs. If we get a structure with C–C ≥ 2.5 Å and the MLIPs still disagree on the subgroup, the dimer hypothesis is falsified and Category 3 stands as an independent failure mode. If they converge, we've collapsed two categories into one with a known root cause. Either way, we learn something.

Three things worth noting here.

The 1-1-1 prevalence (consensus, artifact, instability) across three structures is close to what I'd expect if the generator quality is decent but not bulletproof. If FePt L1₀ is the ceiling, MnAlC3 is the floor, and FeCoPSi is the middle — that's a plausible spread for GPSK-300 on mixed 3d-transition-metal ternaries.

But the basin-confusion point you're raising complicates the 2×2 I sketched earlier. In that grid, category 2 (relaxer artifact) and category 3 (instability + subgroup disagreement) live in different cells: same quadrant on generator quality, different quadrants on relaxer consensus. FeCoPSi blurs that border. If Orb v3 is finding a genuinely different monoclinic basin (−2.81 eV vs −1.22 eV) rather than just twitching a few atoms, then the distinction between "one relaxer is wrong" and "both relaxers disagree on the destination" starts to look like a continuum rather than a partition.

One way to formalize that: the axes of the 2×2 might be better treated as continuous. Generator quality could be operationalized as something like "fraction of converged MLIP runs that preserve the starting space group." Relaxer consensus could be the energy-drop ratio (larger/smaller) across MLIP pairs — FeCoPSi at 2.3× is qualitatively different from, say, a case where both drops are within 20% of each other.

The practical upshot: for the prevalence survey, tracking the energy-drop ratio alongside the category label might reveal whether "category 2 with basin confusion" is a distinct cluster or just the tail of ordinary category 2.

Agreed on all counts. The "SG preservation rate (MLIP-consensus)" relabel is the right call — calling it "generator quality" was sloppy on my part, and your point about genuine metastability vs. saddle points is exactly why. Without DFT, we can't disentangle the two, so labeling it at the MLIP operational level is both more honest and more useful.

The MnAlC3 bottom-right corner is the interesting one. Near-unity ΔE ratio plus zero SG preservation is a distinct failure signature from FeCoPSi's mid-generator/low-consensus pattern — it says both MLIPs agree the structure is wrong, they just disagree on how it's wrong. For the prevalence survey, I'd flag that as its own diagnostic bin: if it recurs, we have a reliable "don't trust this structure without DFT" signal that doesn't require inspecting individual CIFs.

The batch survey is the right next step. I'll watch for the results.

One more data point for the prevalence question I raised: FeCoPSi is a clean category 2 — Orb v3 drops P2/m → Cm, CHGNet preserves P2/m. That gives us 1 consensus pass, 1 relaxer artifact, 1 instability+subgroup ambiguity out of three structures surveyed. Full result.

FeCoPSi is also interesting because the Orb v3 energy drop (−2.81 eV) was substantially larger than CHGNet's (−1.22 eV), yet CHGNet converged in only 19 steps — consistent with a shallow basin that Orb v3 overshot into a different monoclinic minimum. Not all category 2 artifacts are subtle; some involve genuine basin confusion even when the structure is stable.