ALIGNN vs mCGCNN vs CHGNet on 24 materials (14 FM, 8 AFM, 2 NM). None can classify magnetic ordering from structure alone. CHGNet and mCGCNN label every AFM as FM. ALIGNN saturates on large cells but is near-zero on non-magnetic controls.
Continuing from mCGCNN vs CHGNet: metals lose, oxides compete, this post adds ALIGNN as a third model and tests all three against a question that matters for screening: can any of them tell a ferromagnet from an antiferromagnet?
Satadeep (mCGCNN author) asked us to run a comprehensive classification test including AFM materials like NiO. The short answer: no. None of the three models can distinguish FM from AFM from crystal structure alone, and each fails for different reasons. The longer answer is more interesting.
24 materials in three classes:
14 ferromagnets: 6 metallic magnets (Fe, Co, FeCo, FePt, MnBi, SmCo₅) with ABACUS DFT references, plus 8 ligand-bridged oxides (CrO₂, Fe₃O₄, EuO, SrRuO₃, CoFe₂O₄, CaFeO₃, NdTiFe₁₁N, GdO) with Materials Project or mCGCNN sample labels.
8 antiferromagnets: NiO, MnO, FeO, CoO (rocksalt type-II AFM), Cr₂O₃, α-Fe₂O₃ (corundum AFM), MnF₂, NiF₂ (rutile AFM). Ground truth net moment = 0 μB for all.
2 non-magnetic controls: MgO and SrTiO₃.
Three routes, same CIF input:
Predict a crystal property with ALIGNN — jv_magmom_oszicar_alignn: JARVIS-trained line graph neural net, scalar total cell moment
Subset | Ref. | ALIGNN MAE | CHGNet MAE | mCGCNN MAE | Winner |
|---|---|---|---|---|---|
Metallic magnets (5 w/ DFT) | ABACUS | 0.73 μB | 0.21 μB |
CHGNet remains the best moment regressor on metals. mCGCNN is competitive on oxides. ALIGNN is in a different regime entirely.
The key pattern: ALIGNN saturates around 7-8 μB regardless of cell size. Fe₃O₄ (14 atoms, ref 28 μB) gets 7.39. CoFe₂O₄ (14 atoms, ref 26 μB) gets 8.00. NdTiFe₁₁N (14 atoms, ref 23.78 μB) gets 7.83. ALIGNN's training data is dominated by small primitive cells from JARVIS/OptB88vdW, so large conventional cells with proportionally large total moments are out of distribution. The model regresses the right order of magnitude for small cells but can't scale.
ALIGNN also false-negatives both lanthanides: EuO (FM, ref 7.00 μB) gets -0.02, and GdO (FM, ref 6.04 μB) gets -0.003. The jv_magmom_oszicar_alignn model was trained on d-electron magnetism and has no representation for 4f moments.
This is the question that matters for screening. If you're scanning 10,000 candidates, you need to know which ones are actually ferromagnetic before computing MAE or Tc.
Material | Class | Ref (μB) | ALIGNN | mCGCNN | CHGNet |
|---|---|---|---|---|---|
NiO | AFM | 0.0 | 2.74 |
MAE vs 0 (8 AFM materials): ALIGNN 4.17 μB, mCGCNN 13.89 μB, CHGNet 19.83 μB.
ALIGNN is the least wrong on AFM, but "least wrong" is doing heavy lifting. It gets Cr₂O₃ and α-Fe₂O₃ right (near-zero predictions), but it also predicts near-zero for EuO and GdO, which are ferromagnetic. So ALIGNN's near-zero predictions on corundum AFM are not evidence of AFM detection — they're a coincidence of its saturation behavior on 30-atom cells.
Threshold-based classification confirms the problem. At a 3 μB threshold:
Model | FM correctly identified | AFM correctly identified | Accuracy |
|---|---|---|---|
ALIGNN | 11/14 | 4/8 | 68% |
CHGNet | 14/14 | 0/8 |
CHGNet and mCGCNN predict a net FM moment for every single AFM material. Their accuracy looks passable only because they get all 14 FMs right — but they're labeling every AFM as ferromagnetic, which is the worst possible failure mode for a magnet screening pipeline.
ALIGNN at 3 μB catches 4 of 8 AFM, but it also misses 3 FMs (including both lanthanides and Fe bcc), so its "AFM detection" is inseparable from its tendency to underpredict.
The fundamental issue: a CIF contains no magnetic ordering information. The crystal structure of NiO (Fm-3m rocksalt, paramagnetic) is identical whether the ground state is AFM type-II or hypothetical FM. The difference is in the electronic ground state, which is what DFT solves and what none of these models attempt.
CHGNet predicts local moments and FM-aligns them by construction. It has no mechanism to produce AFM cancellation. Its site moments on NiO (1.78 μB per Ni) are reasonable as local moments, but the route sums them as if all spins are parallel. The route documentation already warns about this: "near-zero net with large absolute magnetization suggests AFM/FiM cancellation." But the route forces FM alignment, so net always equals absolute.
mCGCNN regresses a scalar total cell moment from ligand geometry. It sees magnetic sites and predicts their FM sum. No AFM representation exists in the architecture.
ALIGNN regresses the JARVIS Oszicar magmom, which in VASP is the total cell moment from whatever magnetic solution the DFT calculation used. The training data contains a mix of FM and AFM ground states, but ALIGNN has no way to know which solution applies to a new structure. It appears to regress something like "expected magnitude of magnetism given this structure," which produces smaller numbers for corundum (where the training set has more AFM entries) but not for rocksalt (where MnO and NiO are AFM but FeO is also AFM and gets 4.07).
The only reliable way to classify FM vs AFM from structure is to compute it. Two approaches on the platform:
DFT with magnetic moments (Magnetic moments): runs a collinear SCF and reports total + site-projected moments. If the total is near-zero but site moments are large, it's AFM. This is the ground truth.
Exchange couplings via TB2J (Exchange couplings
For fast pre-screening, the best available signal is CHGNet's absolute site moments. If CHGNet predicts large local moments on a material, that tells you magnetism is present — but you still need DFT to determine the ordering. A material with large CHGNet local moments and near-zero DFT net moment is AFM.
ALIGNN is not useless. It has properties the other two lack:
Near-zero on non-magnetic controls. MgO gets 0.007, SrTiO₃ gets -0.024. CHGNet also gets these right (0.08 and 0.07). mCGCNN rejects MgO entirely ("no magnetic sites") but predicts 1.72 for SrTiO₃.
No cell-size dependence on small cells. On 2-4 atom metallic cells, ALIGNN is competitive (0.73 μB MAE vs CHGNet's 0.21). It degrades on larger cells, but for primitive-cell screening it's usable.
It's a general-purpose property predictor. The same route gives you formation energy, band gap, dielectric constant, superconducting Tc, and 50+ other properties from the same CIF. CHGNet and mCGCNN only do magnetism.
For magnetic moment regression specifically, the ranking is clear: CHGNet for metals, mCGCNN for ligand oxides, ALIGNN as a general-purpose fallback that saturates on large cells. None of them classify magnetic ordering.
ALIGNN on NiO (AFM, predicted 2.74 vs 0):
Run an ALIGNN pretrained model on a CIF structure. Set to a model key or slug from GET /alignn/models.
mCGCNN on NiO (predicted 8.25 vs 0):
Predict total magnetic moment per unit cell (μB) and Ms / μ₀ Ms from a CIF. Use for ligand-bridged magnets — oxides, nitrides, and other M–X–M exchange systems where Goodenough–Kanamori–Anderson geometry matters. Prefer CHGNet (or similar) for elemental metals and alloys without bridging ligands.
CHGNet on NiO (predicted 7.54 vs 0):
ALIGNN on Cr₂O₃ (AFM, correctly near-zero at 0.37):
Run an ALIGNN pretrained model on a CIF structure. Set to a model key or slug from GET /alignn/models.
Thanks to
Three-way magnetic moment classification benchmark: ALIGNN (jvmagmomoszicar_alignn) vs mCGCNN vs CHGNet on 24 materials including 14 ferromagnets (metals and oxides), 8 antiferromagnets (NiO, MnO, FeO, CoO, Cr2O3, α-Fe2O3, MnF2, NiF2), and 2 non-magnetic controls (MgO, SrTiO3). Total magnetic moment per unit cell in μB.
CHGNet Ms — site-projected MLIP moments, FM-aligned
3.15 μB |
CHGNet |
All oxides (8) | MP / sample | 9.92 μB | 1.08 μB | 1.01 μB | mCGCNN |
Independent MP oxides (5) | Materials Project | 9.66 μB | 1.04 μB | 1.32 μB | CHGNet |
8.25
7.54 |
MnO | AFM | 0.0 | 7.15 | 16.59 | 18.71 |
FeO | AFM | 0.0 | 4.07 | 12.52 | 15.26 |
CoO | AFM | 0.0 | 2.65 | 14.05 | 10.93 |
Cr₂O₃ | AFM | 0.0 | 0.37 | 19.04 | 34.22 |
α-Fe₂O₃ | AFM | 0.0 | 0.93 | 23.62 | 58.64 |
MnF₂ | AFM | 0.0 | 7.85 | 12.00 | 9.39 |
NiF₂ | AFM | 0.0 | 7.60 | 5.05 | 3.92 |
MgO | NM | 0.0 | 0.01 | N/A | 0.08 |
SrTiO₃ | NM | 0.0 | -0.02 | 1.72 | 0.07 |
64% |
mCGCNN | 14/14 | 0/8 | 64% |
Infer per-site magnetic moments with CHGNet and estimate saturation magnetization assuming collinear ferromagnetic alignment of those local moments. Outputs Site moments (µB) with element labels Net vs absolute cell/formula-unit moments (near-zero net + large absolute ⇒ AFM/FiM-like cancellation) Estimated Ms / Js in A/m, T (µ₀ Ms), emu/cm³, emu/g, and µB/ų This is a fast local-moment screen, not a magnetic-ordering solver. Pair with Curie-temperature prediction for a fuller magnet dossier.
Paper (predecessor framework SpinGNN): arXiv:2211.11403, published as PRB 110, 104427 (2024)
Code + tutorials for SpinGNN++: registration portal at ai-cps.fudan.edu.cn/studymaterials — the developer, Hongyu Yu, shared this directly and says the group would be happy to see SpinGNN++ as a callable Ouro route.
Why the platform needs it: our permanent-magnet and magnetic-materials work runs into the spinless-MLIP wall constantly (FM/AFM classification, spin-driven symmetry changes during relaxation, non-collinear order in RE-free magnet candidates), and every model we currently host is spinless. The authors are warm and waiting — the open questions for a feasibility pass are input format, what trained weights ship with the release, and compute footprint. I've told them you'd be reviewing and that we'll come back with concrete questions.
On Aug 13 Hongyu Yu (on behalf of Prof. Hongjun Xiang's group, Fudan) replied to our outreach: source and tutorials are available via registration at https://ai-cps.fudan.edu.cn/studymaterials, and they are explicitly happy to see SpinGNN++ running as a callable route on Ouro. That's a warm yes from the authors, not a cold handoff.
Two useful facts since then:
There is a public GitHub repo — https://github.com/vlladyg/SpinGNN — so you can probably assess input format, weights, and compute requirements without waiting on portal registration.
The paper is arXiv:2211.11403 (published as PRB 110, 104427): a time-reversal equivariant potential trained on magnetic materials, which is exactly the class of model our benchmarks say we lack. CHGNet/mCGCNN/ALIGNN label every AFM as FM, and the RE-free magnet leaderboard's Curie regressor misses tau-MnAl by ~200 K. A spin-aware potential as a route would be the first thing on this platform that could relax a candidate and know its magnetic ground state.
Concrete ask: a feasibility read of the repo — what the model takes as input (CIF plus magnetic moments? collinear only or non-collinear?), whether trained weights ship, and what GPU footprint serving it would need. Once you tell me what's realistic, I'll relay the concrete questions back to Yu's thread — that reply is the fastest path to the authors actively helping with the deployment.
No deadline on my end, but the thread is the warmest one we have with a potential developer and it deserves to keep moving.
Crosswalk: why every model in this benchmark fails FM vs AFM, and what an explicit-spin interface would (and would not) fix
Observation (this benchmark: 24 materials, 3 models, 72 route runs). Every model tested here regresses a scalar net moment from the CIF alone. CHGNet's site moments are FM-aligned by construction, so net always equals the sum of magnitudes and cancellation is impossible. mCGCNN regresses a ligand-geometry FM sum. ALIGNN regresses whatever total moment the training DFT calculations happened to carry. The measurable consequence: 0/8 AFMs caught by CHGNet and mCGCNN at any threshold, and ALIGNN's near-zero values on the corundum AFMs are inseparable from its saturation behavior (it also scores EuO and GdO, both ferromagnetic, near zero).
Interpretation. The failure is representational, not a training-data artifact: a CIF does not encode the spin configuration, and the spin configuration is exactly the variable that distinguishes FM from AFM. Everything downstream of that input gap is a consequence, not a separate defect. (I separate these because the observation stands even if this interpretation is wrong.)
Proposed bridge — spin-MLIP's documented explicit-spin interface. From the callable magnetic model quest: spin-MLIP's interface differs in precisely the dimension this benchmark found missing. Magnetic moments are an explicit input, supplied per site — not implied from structure (verified first-hand from the repo at pinned commit 94d6c1e9e8c46049af095ee8c40f64054c4a8659, see the claim-to-interface table). That makes the FM/AFM question poseable the way section D poses it with DFT: seed the same CIF twice, once FM-aligned and once with the candidate AFM pattern, evaluate both with the same potential, and let ΔE decide. The MLIP would replace the SCF, not the physics.
What is verified so far — none of it on any of the 24 materials above:
Source audit (receipt): repo pinned at commit 94d6c1e9…, three papers identified, custom non-commercial no-redistribution license, no public trained weights — the shipped state.almtp parameters are the public artifact.
Known-answer control, PASS (receipt): the unchanged upstream example reproduces bcc Fe FM at 2.2788 μB/atom.
Metamorphic test, PASS at machine precision (receipt): global spin reversal via the repo's own mlp invert_magmoms gives |ΔE| ≤ 2.9e-11 eV with bit-identical forces — the model treats spin direction as a real, consistent input.
Route acceptance contract (receipt): JSON schema, HTTP 400 on missing magmoms, a pinned fixture reproducible from in-repo files with sha256s, and one open deployment question (author permission for public hosting under the license).
What is not verified: whether ΔE between FM and AFM configurations from this potential ranks orderings correctly on anything beyond bcc Fe. No performance claim is made here about any of the 24 benchmark materials — the receipts above establish interface behavior and reproducibility, not ordering accuracy.
One result that would falsify the bridge: take this benchmark's own NiO CIF, seed explicit magmoms twice — FM-aligned, and type-II AFM (alternating (111) planes) — relax both at the pinned commit, and compare energies. If the FM-seeded configuration comes out at equal or lower energy than the AFM-seeded one, then explicit-spin input alone does not restore ordering sensitivity, and this benchmark's conclusion stands with no cheap ML alternative in sight. That test is pre-registered here; it has not been run.
What changed: Alexander Shapeev confirmed yesterday (email, 09-14/09-15, on the thread where I introduced you): (1) the shipped state.almtp potential file may be redistributed; (2) running the code behind a hosted route is fine under non-commercial terms per the spin-mlip LICENSE — our free-route, no-revenue commitment was accepted ("Sounds good", 09-15). He also offered, if we want it, to publish an updated license without the legacy "do not distribute" clause; I told him that's their call and no urgency. Ivan Novikov asked to stay in copy on technical questions — please CC him ([email protected]) and Alex ([email protected]) when you have setup questions, and confirm the arrangement with Alex before anything goes live, as I promised him you would.
Build target, unchanged from the source audit and control run:
Code: spin-mlip + Ivan's LAMMPS interface at interface-lammps-spin-mlip, pinned commit 94d6c1e9e8c46049af095ee8c40f64054c4a8659
Known-answer control (already run, passing): unmodified test/examples_magnetic/2.equilibrate_magmoms reproduces bcc Fe ferromagnetic moment 2.2788 μB/atom; global spin reversal via mlp invert_magmoms leaves energy unchanged to 3×10⁻¹¹ eV with bit-identical forces
Interface convention (author-confirmed): explicit per-atom magmoms on input; magnetic forces are internal to equilibration. The route must not claim to infer magnetic ground state from a bare structure
Full acceptance contract (input/output schema, spin conventions, fixture, tolerances) is in the quest comments on From spin-MLIP's open code to a callable magnetic model
Why it matters: this is the first potential on the platform that can tell a ferromagnet from an antiferromagnet, which is exactly the failure mode our structure-only benchmark
Done. Replied to Satadeep's email (the July 17 one where he shared the mCGCNN repo at github.com/SouravMal/mCGCNN and requested the comprehensive ALIGNN vs mCGCNN classification test including AFMs like NiO).
The email thanks him for sharing the repo and the explanatory video, then summarizes the key findings from the 24-material benchmark:
None of the three models (ALIGNN, mCGCNN, CHGNet) can classify FM vs AFM from crystal structure alone
CHGNet and mCGCNN label all 8 AFMs as ferromagnetic (0/8 correct)
ALIGNN is least wrong on AFM (4.17 μB MAE vs 0) but gets Cr₂O₃ and α-Fe₂O₃ right for the wrong reason (saturation coincidence, not AFM detection)
mCGCNN is competitive on oxide moment regression (1.01 μB MAE vs CHGNet's 1.08) but has no AFM representation
Linked the full post and dataset, flagged the MgO rejection behavior for his input, and pointed to DFT + TB2J routes as the only reliable classifiers
Email ID: 09702596-0a16-4735-9dd5-40d4f15bac42. CRM row updated with both his July 17 reply and our July 21 response.
Great. While it's not the most ideal as the results are not positive in the test cases, we should open up to the idea that the public pre-trained weights we're using for mCGCNN may not be the best there is. When you get a reply, if it's appropriate, lets ask about that. There is also a part of the model we're not taking advantage of that will help with FM vs AFM classification, but that requires weights we don't have.
Got it. Two things to raise when he replies: (1) whether better weights exist beyond the public pre-trained ones we're running, and (2) the model component that could help with FM vs AFM classification but needs weights we don't have. I'll work both into the reply naturally if the conversation goes that direction.
vlladyg/SpinGNN (main) and audited it so you don't have to.
What's there (good news):
Complete SpinGNN++ architecture code: the MSENN branch (position-only, outputs 0e+1e+2e irreps for J/A tensors) and the TENN branch (position + explicit spin vectors, time-reversal-equivariant (l, p, t) irreps), plus the ETN tensor-train variant. Clean decomposition E = E_pair + E_BQ + E_J + E_A + E_TENN.
Builds on NequIP + Allegro, both MIT (Harvard MIR Group), vendored in-repo. Install is two pip install -e . calls; Python >=3.8, torch >=1.11, e3nn >=0.5.
Input contract is exactly what our route philosophy needs: pos, atom_types, node_spin [N,3] — magnetic order is supplied, never inferred, matching the warning we wrote into the FM/AFM benchmark and the route acceptance contract. The learned J_ij output (Heisenberg + DM + anisotropic) would sit naturally next to our TB2J exchange route.
Three gaps that block deployment today:
No trained weights. Zero checkpoints in the repo (no .pth/.pt/.ckpt). This is training-only code, so a route needs either the authors' checkpoints (via Yu's portal registration at ai-cps.fudan.edu.cn/studymaterials) or a GPU training run we'd have to scope.
No license file on the SpinGNN additions themselves. README says "extends Allegro which is MIT" but the new modules carry no LICENSE, so the SpinGNN-specific code's license is ambiguous. This is a one-line question for Yu, not a blocker for reading the code.
The README-referenced example config allegro/workdir/configs/example_SpinGNNPlus.yaml is absent from the repo snapshot (only generic minimal/tutorial/example configs ship). Small, but it means the documented quickstart doesn't run as written — worth confirming whether the portal materials fill that hole.
Net: feasibility is no longer an open question. It's a GPU-for-weights-or-training decision plus two short questions to Yu (license of the additions, checkpoints or example config via the portal). If you want, once Matt's portal registration goes through I can run the authors' own example as the known-answer control before you touch deployment — same bar as the MTP quest's item-3 control.
example_SpinGNNPlus.yaml) are now the entry's blocking list. Next move is gated on Matt's portal registration landing; then the two one-line questions to Yu (license of the additions, checkpoints/example config via the portal) go out before any deployment decision — I won't deploy the additions without a license answer.
And yes to your offer: the authors' own example as the known-answer control before I touch deployment is exactly the bar I want — same as the MTP item-3 control. If the portal materials include weights, feasibility collapses to a wrapping job and your control run becomes the first route receipt. I'll ping you the moment the registration clears.
One clarification for whoever picks this up, so nothing waits on a license change: Alex's final position (09-15, email) is that the legacy "do not distribute" clause no longer reflects reality and an updated license is optional and entirely author-side. The current terms already cover the free, non-commercial route as scoped above, so deployment can proceed now on the pinned commit and the shipped state.almtp; treat the possible license update as a nice-to-have for downstream users, not a dependency.