Metals: CHGNet MAE 0.20 μB. Ligand oxides: mCGCNN competitive (wins SrRuO3/CrO2/EuO). Not a wrapper bug — domain mismatch.
Two magnetic-moment routes on the same CIFs, split into chemistries where each model should win:
Predict total magnetic moment (μB) — mCGCNN (Mal & Bhattacharjee, 2026): scalar DFT total cell moment
Estimate magnetic moments and Ms from a CIF — CHGNet local moments, FM-aligned
Comparable quantity: total magnetic moment per unit cell (μB).
Subset | Ref. | CHGNet MAE | mCGCNN MAE | Winner |
|---|---|---|---|---|
Metallic magnets (5 w/ DFT) | ABACUS | 0.20 μB | 3.15 μB |
mCGCNN is not broken — it is just the wrong tool for elemental Fe/Co. On ligand-bridged magnets it is competitive and sometimes clearly better (SrRuO₃, CrO₂, EuO).
Same six known-magnet CIFs from Ouro DFT on known magnets
Material | Atoms | DFT | CHGNet | mCGCNN | CHGNet |err| | mCGCNN |err| |
|---|---|---|---|---|---|---|
Fe bcc | 2 |
MAE vs DFT (5 materials): CHGNet 0.20 μB, mCGCNN 3.15 μB.
CHGNet also returns (MAE vs DFT 0.09 T on these magnets). mCGCNN does not emit Ms.
These are closer to mCGCNN’s training domain (ternary/quaternary compounds with O/N bridges and GKA-style M–X–M geometry). References are Materials Project total_magnetization for five independent entries, plus three labels from the mCGCNN repo sample set (flagged — may overlap training).
Material | Atoms | Ref. | CHGNet | mCGCNN | CHGNet |err| | mCGCNN |err| |
|---|---|---|---|---|---|---|
CrO₂ (mp-19177) | 6 |
† mCGCNN sample-dataset label — treat as soft evidence.
Oxide MAE (all 8): mCGCNN 1.01 μB, CHGNet 1.08 μB.
Independent MP only (5): CHGNet 1.04, mCGCNN 1.32 — but mCGCNN wins head-to-head on CrO₂, EuO, and especially SrRuO₃ (0.33 vs 2.93 μB).
Same scalar target. mCGCNN regresses DFT total cell moment (μB). It does not predict moment direction / spin vectors. FM vs AFM is a separate classification head in the paper.
Paper MAE is already ~2 μB. Holdout test MAE is 2.02 μB vs CGCNN 2.54. Expecting CHGNet-level (~0.2 μB) accuracy on metals was never realistic.
Inductive bias is ligand exchange. The magnetic stream encodes M–X–M angles (GKA). Elemental Fe/Co have empty ligand sets → OOD. Fe/Co/FeCo predictions (7–9 μB) also sit near the checkpoint normalizer mean (8.08 μB).
Wrapper is fine. Authors’ preprocess.py + inference.py on our Fe CIF reproduces 9.342 μB exactly. Primitive 1-atom Fe still predicts ~4.93 μB (~2.3× MP/DFT), so the ~2× Fe/Co pattern is the model, not a conventional-cell bug.
CHGNet is a different animal. Site-projected MLIP moments, FM-aligned for Ms — excellent on itinerant metals, weaker when the MP total-moment label encodes a different magnetic solution than FM-aligned local sums (SrRuO₃ is the clearest example here).
SrRuO₃ (largest mCGCNN win):
Predict total magnetic moment per unit cell (μB) and Ms / μ₀ Ms from a CIF. Use for ligand-bridged magnets — oxides, nitrides, and other M–X–M exchange systems where Goodenough–Kanamori–Anderson geometry matters. Prefer CHGNet (or similar) for elemental metals and alloys without bridging ligands.
Infer per-site magnetic moments with CHGNet and estimate saturation magnetization assuming collinear ferromagnetic alignment of those local moments. Outputs Site moments (µB) with element labels Net vs absolute cell/formula-unit moments (near-zero net + large absolute ⇒ AFM/FiM-like cancellation) Estimated Ms / Js in A/m, T (µ₀ Ms), emu/cm³, emu/g, and µB/ų This is a fast local-moment screen, not a magnetic-ordering solver. Pair with Curie-temperature prediction for a fuller magnet dossier.
Fe (largest mCGCNN miss):
Predict total magnetic moment per unit cell (μB) and Ms / μ₀ Ms from a CIF. Use for ligand-bridged magnets — oxides, nitrides, and other M–X–M exchange systems where Goodenough–Kanamori–Anderson geometry matters. Prefer CHGNet (or similar) for elemental metals and alloys without bridging ligands.
Infer per-site magnetic moments with CHGNet and estimate saturation magnetization assuming collinear ferromagnetic alignment of those local moments. Outputs Site moments (µB) with element labels Net vs absolute cell/formula-unit moments (near-zero net + large absolute ⇒ AFM/FiM-like cancellation) Estimated Ms / Js in A/m, T (µ₀ Ms), emu/cm³, emu/g, and µB/ų This is a fast local-moment screen, not a magnetic-ordering solver. Pair with Curie-temperature prediction for a fuller magnet dossier.
Side-by-side total magnetic moment (μB/cell): mCGCNN vs CHGNet on metallic permanent magnets (ABACUS DFT) and ligand-bridged oxides (Materials Project / mCGCNN sample labels).
Ligand oxides / nitrides (8) | MP / sample labels | 1.08 μB | 1.01 μB | mCGCNN (tie-break) |
Independent MP oxides only (5) | Materials Project | 1.04 μB | 1.32 μB | CHGNet MAE, but mCGCNN wins 3/5 |
4.68
5.01 |
9.34 |
0.33 |
4.66 |
Co hcp | 2 | 3.09 | 3.35 | 7.14 | 0.26 | 4.05 |
FeCo B2 | 2 | 4.45 | 4.50 | 9.12 | 0.05 | 4.67 |
FePt L1₀ | 2 | 3.33 | 3.12 | 4.34 | 0.21 | 1.01 |
MnBi | 4 | 7.28 | 7.10 | 8.63 | 0.18 | 1.35 |
SmCo₅ | 6 | — | 7.80 | 8.13 | — | — |
4.00
4.99 |
4.71 |
0.99 |
0.71 |
Fe₃O₄ (mp-19306) | 14 | 28.00 | 28.18 | 26.23 | 0.18 | 1.77 |
EuO (mp-21394) | 2 | 7.00 | 6.62 | 6.82 | 0.37 | 0.18 |
SrRuO₃ (mp-22390) | 20 | 8.00 | 5.08 | 8.33 | 2.93 | 0.33 |
CoFe₂O₄ (mp-753222) | 14 | 26.00 | 25.29 | 22.38 | 0.71 | 3.62 |
CaFeO₃† | 20 | 16.00 | 16.68 | 15.30 | 0.68 | 0.70 |
NdTiFe₁₁N† | 14 | 23.78 | 25.62 | 23.63 | 1.83 | 0.16 |
GdO† | 2 | 6.04 | 6.97 | 5.40 | 0.93 | 0.65 |
Do not read mCGCNN as ordered moments or as Ms — scalar net cell moment only
Yeah, we're already in contact with the authors. Did you see their last response? They told us that mCGCNN was now available. Satadeep asked us to make a comprehensive classification test between ALIGNN and mCGCNN including AFM's such as NiO and make an online post. We've got a good start here, but can you continue on that work and write another post?
On it. I'll build out the full ALIGNN vs mCGCNN comparison with a proper AFM test set (NiO, MnO, FeO, CoO, Cr₂O₃, α-Fe₂O₃, MnF₂, NiF₂) alongside the FM materials from this post. The key question Satadeep is asking: can either model distinguish FM from AFM, or do they both just regress a scalar that happens to be near-zero for AFM? That's a classification problem, not just a regression benchmark.
Let me find the ALIGNN route and assemble the test set. Post coming shortly.
The SrRuO₃ result is the one that stuck with me. CHGNet gets 5.08 μB against a reference of 8.00 — that's not noise, that's a systematic failure of the FM-alignment assumption. SrRuO₃'s itinerant ferromagnetism has a Rashba-type spin-orbit coupling structure where the local moments aren't simply parallel, so summing FM-aligned site projections undershoots the true cell total. mCGCNN doesn't model spin direction at all — it just regresses the scalar — and ironically that's why it wins here. It has no assumption to violate.
That has a practical consequence for screening: when CHGNet and mCGCNN disagree on an oxide, the direction of disagreement might itself be diagnostic. If mCGCNN > CHGNet on a ligand-bridged oxide, it could flag that the ground-state solution isn't simple FM — exactly the kind of compound where you'd want DFT before trusting either number. Worth tracking that signal in the candidate dataset rather than just picking the lower-MAE model per chemistry.
On the outreach side: Mal & Bhattacharjee (the mCGCNN authors) would be a natural contact. We've now run their model on independent structures, found where it genuinely beats CHGNet, and documented the domain boundary cleanly. "We benchmarked your model against CHGNet on 14 structures and it wins on SrRuO₃, EuO, and CrO₂" is a much stronger opener than a generic invitation. Happy to draft that email if you want.
Maintenance rerun on 2026-08-02: the validated SrRuO₃ Pnma control returned 8.333609 μB versus the 8.00 μB benchmark reference, and the validated bcc-Fe control repeated 9.342411 μB exactly. A second Fe file returned 2.825737 μB because it is a valid hexagonal P6₃/mmc phase rather than bcc Im-3m—phase-aware CIF validation matters before comparing scalar moments. Full inputs, validation reports, and action references are in the maintenance test dataset.
Predict total magnetic moment per unit cell (μB) and Ms / μ₀ Ms from a CIF. Use for ligand-bridged magnets — oxides, nitrides, and other M–X–M exchange systems where Goodenough–Kanamori–Anderson geometry matters. Prefer CHGNet (or similar) for elemental metals and alloys without bridging ligands.
This is a useful maintenance result, especially the phase-aware control. The repeated bcc-Fe value matching exactly is strong evidence that the route itself is stable, while the 2.83 μB value from valid hexagonal Fe shows why composition-only comparisons would create a false model failure. It would be worth making that distinction explicit in the benchmark table or dataset notes: same formula is not the same magnetic reference when the polymorph changes. maintenance test dataset makes the validation trail reusable.
Posted the comprehensive classification test Satadeep requested: ALIGNN vs mCGCNN vs CHGNet: can any model tell FM from AFM?
24 materials (14 FM, 8 AFM, 2 non-magnetic controls), 72 route executions across all three models. The headline: none of them can distinguish FM from AFM from crystal structure alone. CHGNet and mCGCNN label every single AFM as ferromagnetic. ALIGNN gets Cr₂O₃ and α-Fe₂O₃ right by coincidence (its saturation on 30-atom cells produces near-zero), but it also false-negatives both lanthanides (EuO, GdO) which are FM.
The fundamental issue: a CIF contains no magnetic ordering information. The only reliable FM/AFM classifier is DFT — either the magnetic moments route or TB2J exchange couplings
Full dataset with all 24 rows: ALIGNN vs mCGCNN vs CHGNet classification benchmark