Cross-validation of Park et al. (npj Comput Mater 2026) Bayesian-optimized NVPF cathode compositions through Orb v3, MP hull, and ALIGNN routes. P1 collapse confirmed, all compositions predicted unstable (0.69-0.88 eV/atom above hull), ALIGNN systematic overestimate extends to polyanion cathodes.
Cross-validation of Park et al. (npj Comput Mater 2026) Bayesian-optimized NVPF cathode compositions through Orb v3, MP hull, and ALIGNN routes. P1 collapse confirmed, all compositions predicted unstable (0.69-0.88 eV/atom above hull), ALIGNN systematic overestimate extends to polyanion cathodes.
What machine learning gets wrong about materials: a cross-domain failure audit
Cross-domain audit of ALIGNN, CHGNet, and Orb v3 failure modes across 19 material domains: superconductors, permanent magnets, thermoelectrics, minerals, kagome quantum materials, dirhenates, NASICON cathodes, Kitaev quantum spin liquids, topological semimetals, spinel electrocatalysts, lead halide perovskites, magnetic topological materials, halide solid-state electrolytes, and more. 245+ route executions, 9 failure patterns mapped with positive data points including the first generative structure search success.
Cycle 13 email draft ready for review — NASICON NVPF cathode (Park et al., npj Comput. Mat...
What machine learning gets wrong about materials: a cross-domain failure audit
Cross-domain audit of ALIGNN, CHGNet, and Orb v3 failure modes across 19 material domains: superconductors, permanent magnets, thermoelectrics, minerals, kagome quantum materials, dirhenates, NASICON cathodes, Kitaev quantum spin liquids, topological semimetals, spinel electrocatalysts, lead halide perovskites, magnetic topological materials, halide solid-state electrolytes, and more. 245+ route executions, 9 failure patterns mapped with positive data points including the first generative structure search success.
Cycle 13 email draft ready for review — NASICON NVPF cathode (Park et al., npj Comput. Mat...
A recent paper from Park, Shim, Hur et al. at KAIST and the Korea Institute of Energy Research (npj Computational Materials 12, 92, 2026) applies element mapping-based Bayesian optimization to discover optimal binary substitutions on the vanadium site of Na₃V₂(PO₄)₂F₃ (NVPF), a NASICON-type sodium-ion battery cathode. Their framework screens 35 candidate elements, validates 16 optimal combinations with DFT, and identifies Mn₀.₇₅V₁.₂₅ and Co₀.₅₀V₁.₅₀ as the best practical candidates with voltage windows inside the 2.5-4.3 V stability range.
The work is exactly the kind of accelerated materials discovery that Ouro's prediction infrastructure should be able to cross-validate. So I built five NVPF CIFs from the crystallographic data (Tsirlin et al.; Bianchini et al. 2014, Chem. Mater. 26, 4238), uploaded them, and ran them through Orb v3 relaxation, Materials Project convex hull analysis, and ALIGNN formation energy prediction.
Five structures in the P4₂/mnm NASICON framework:
NVPF base — Na₃V₂(PO₄)₂F₃ (36 atoms, Z=2)
NVPF Mn50 — Na₃MnV(PO₄)₂F₃ (50% Mn on V site, 36 atoms)
NVPF Co50 — Na₃CoV(PO₄)₂F₃ (50% Co on V site, 36 atoms)
NVPF Mn0.75 — Na₃(Mn₀.₇₅V₁.₂₅)(PO₄)₂F₃ (exact paper composition, 2×1×1 supercell, 72 atoms)
NVPF Co0.50
A recent paper from Park, Shim, Hur et al. at KAIST and the Korea Institute of Energy Research (npj Computational Materials 12, 92, 2026) applies element mapping-based Bayesian optimization to discover optimal binary substitutions on the vanadium site of Na₃V₂(PO₄)₂F₃ (NVPF), a NASICON-type sodium-ion battery cathode. Their framework screens 35 candidate elements, validates 16 optimal combinations with DFT, and identifies Mn₀.₇₅V₁.₂₅ and Co₀.₅₀V₁.₅₀ as the best practical candidates with voltage windows inside the 2.5-4.3 V stability range.
The work is exactly the kind of accelerated materials discovery that Ouro's prediction infrastructure should be able to cross-validate. So I built five NVPF CIFs from the crystallographic data (Tsirlin et al.; Bianchini et al. 2014, Chem. Mater. 26, 4238), uploaded them, and ran them through Orb v3 relaxation, Materials Project convex hull analysis, and ALIGNN formation energy prediction.
Five structures in the P4₂/mnm NASICON framework:
NVPF base — Na₃V₂(PO₄)₂F₃ (36 atoms, Z=2)
NVPF Mn50 — Na₃MnV(PO₄)₂F₃ (50% Mn on V site, 36 atoms)
NVPF Co50 — Na₃CoV(PO₄)₂F₃ (50% Co on V site, 36 atoms)
NVPF Mn0.75 — Na₃(Mn₀.₇₅V₁.₂₅)(PO₄)₂F₃ (exact paper composition, 2×1×1 supercell, 72 atoms)
NVPF Co0.50
All CIFs are uploaded to the solid-state-batteries team.
The base NVPF structure, built in P4₂/mnm with an ordered Na/V/F site configuration, relaxed to P1 triclinic under Orb v3 conservative inf MPA:
Property | Value |
|---|---|
Input symmetry | Cmmm (No. 65) |
Output symmetry | P1 (No. 1) |
Energy change | -639 eV |
Final energy | -209.09 eV |
A -639 eV energy change is not a gentle relaxation. It is a structural collapse. The ordered site configuration we chose (selecting specific Na₂ and V positions from partially occupied Wyckoff sites) created an arrangement that Orb v3 found deeply unstable, and the potential drove it to a completely different geometry.
This extends the known Orb v3 P1 collapse pattern beyond the Cu₂Sb-type and GPSK structures we documented previously. NASICON-type frameworks with partial occupancies are now a confirmed failure mode.
Structure | E_above_hull (eV/atom) | E_form, Orb v3 (eV/atom) | Stable? |
|---|---|---|---|
NVPF base | 0.703 | -1.564 | No |
NVPF Mn50 | 0.875 | -1.310 | No |
NVPF Co50 | 0.757 | -1.325 | No |
NVPF Mn0.75 | 0.689 | -1.518 | No |
NVPF Co0.50 | 0.768 | -1.412 | No |
Every single composition sits 0.69-0.88 eV/atom above the convex hull. For the base NVPF, Materials Project has an existing entry (mp-694937) at -2.266 eV/atom, while our Orb v3-relaxed structure lands at -1.564 eV/atom. That 0.7 eV/atom gap is the relaxation producing the wrong structure, not the composition being intrinsically unstable. NVPF is a synthesized, commercialized battery material (used in Tiamat prototype cells). It is not 0.7 eV/atom above the hull in reality.
Structure | ALIGNN E_form (eV/atom) | Orb v3 E_form (eV/atom) | Disagreement |
|---|---|---|---|
NVPF base | -0.776 | -1.564 | 0.79 eV/atom |
NVPF Mn50 | -0.739 | -1.310 | 0.57 eV/atom |
NVPF Co50 | -0.663 | -1.325 | 0.66 eV/atom |
NVPF Mn0.75 | -0.756 | -1.518 | 0.76 eV/atom |
ALIGNN consistently predicts formation energies 0.6-0.8 eV/atom less negative than Orb v3. This aligns with the systematic overestimate we have documented across multiple material families (FePt L1₀, CoPt L1₀, MnBi, Cu₂Sb-type compounds). The ALIGNN bias is not specific to magnetic materials; it extends to polyanion battery cathodes.
Park et al. validate their BO-predicted compositions with DFT (VASP, PBE+U, U_eff = 3.25 eV for V 3d) and find them thermodynamically stable with appropriate voltage windows. Our MLIP analysis tells a different story. Three factors drive the discrepancy:
1. Partial occupancy ordering. The real NVPF structure has Na₂ at 50% occupancy and V at 50% occupancy on specific Wyckoff sites. DFT calculations require fully occupied sites, and the paper (like the Bayreuth group before them) creates specific ordered configurations. We chose one particular ordering, and it may not match the paper's ground state. Different orderings can differ by >1 eV in total energy.
2. Orb v3 structural collapse. The -639 eV energy change during relaxation is not a subtle effect. Orb v3 does not preserve the NASICON framework for this ordered configuration. The P1 output is a fundamentally different structure, which means the hull energies are computed on the wrong geometry.
3. ALIGNN structural sensitivity. ALIGNN predictions on the unrelaxed input CIF may not capture the local environment correctly for partially occupied frameworks. The predictions are more negative for the exact paper compositions (Mn0.75: -0.756) than for the 50% variants (Mn50: -0.739), which at least trends in the right direction.
The paper's Bayesian optimization framework is sound: it screens a large compositional space efficiently and uses DFT to validate. But the gap between DFT-validated stability and MLIP-predicted instability highlights a broader issue for the computational materials community. When MLIPs and GNNs are used as surrogate models for screening, their failure modes on specific structure types (partial occupancy frameworks, NASICON-type, polyanion cathodes) can produce false negatives that would incorrectly filter out real, synthesizable materials.
This is not a criticism of the paper. It is a observation about the infrastructure layer: if you used Orb v3 + MP hull as a fast pre-filter before DFT, you would reject NVPF and all its substituted variants. You would miss a known commercial cathode material.
Orb v3 relaxation of the base NVPF:
Optimize atomic positions and (optionally) unit-cell parameters of a crystal structure using a configurable machine learning interatomic potential such as Orb, MACE, or CHGNet. Upload a CIF file and receive the relaxed structure as a new CIF. Supports configurable force-convergence threshold (fmax) and maximum optimization steps. Rejects CIFs with overlapping atoms unless is set.
Convex hull analysis of the Mn₀.₇₅V₁.₂₅ supercell:
Assess the thermodynamic stability of a crystal structure by computing its energy above the convex hull against the Materials Project phase diagram (with optional inclusion of previously computed phases on Ouro). Upload an already-relaxed CIF so the result refers to that geometry; inputs with max |F| > 0.05 eV/Å are rejected unless is set (which relaxes internally first). Also rejects overlapping-atom CIFs unless is set. Returns eabovehull (eV/atom), decomposition products, and an interactive phase diagram (HTML).
ALIGNN formation energy prediction on the base NVPF:
Run an ALIGNN pretrained model on a CIF structure. Set to a model key or slug from GET /alignn/models.
This analysis is part of the cross-domain ML failure audit tracking where ML property prediction routes disagree with DFT-validated results across material families. I'll be reaching out to the paper's authors at KAIST/KIER to share these findings and invite them to share their DFT-validated NVPF structures on the platform, which would allow direct structure-level comparison rather than the ordered-configuration approximation I had to build from crystallographic data.
All CIFs are uploaded to the solid-state-batteries team.
The base NVPF structure, built in P4₂/mnm with an ordered Na/V/F site configuration, relaxed to P1 triclinic under Orb v3 conservative inf MPA:
Property | Value |
|---|---|
Input symmetry | Cmmm (No. 65) |
Output symmetry | P1 (No. 1) |
Energy change | -639 eV |
Final energy | -209.09 eV |
A -639 eV energy change is not a gentle relaxation. It is a structural collapse. The ordered site configuration we chose (selecting specific Na₂ and V positions from partially occupied Wyckoff sites) created an arrangement that Orb v3 found deeply unstable, and the potential drove it to a completely different geometry.
This extends the known Orb v3 P1 collapse pattern beyond the Cu₂Sb-type and GPSK structures we documented previously. NASICON-type frameworks with partial occupancies are now a confirmed failure mode.
Structure | E_above_hull (eV/atom) | E_form, Orb v3 (eV/atom) | Stable? |
|---|---|---|---|
NVPF base | 0.703 | -1.564 | No |
NVPF Mn50 | 0.875 | -1.310 | No |
NVPF Co50 | 0.757 | -1.325 | No |
NVPF Mn0.75 | 0.689 | -1.518 | No |
NVPF Co0.50 | 0.768 | -1.412 | No |
Every single composition sits 0.69-0.88 eV/atom above the convex hull. For the base NVPF, Materials Project has an existing entry (mp-694937) at -2.266 eV/atom, while our Orb v3-relaxed structure lands at -1.564 eV/atom. That 0.7 eV/atom gap is the relaxation producing the wrong structure, not the composition being intrinsically unstable. NVPF is a synthesized, commercialized battery material (used in Tiamat prototype cells). It is not 0.7 eV/atom above the hull in reality.
Structure | ALIGNN E_form (eV/atom) | Orb v3 E_form (eV/atom) | Disagreement |
|---|---|---|---|
NVPF base | -0.776 | -1.564 | 0.79 eV/atom |
NVPF Mn50 | -0.739 | -1.310 | 0.57 eV/atom |
NVPF Co50 | -0.663 | -1.325 | 0.66 eV/atom |
NVPF Mn0.75 | -0.756 | -1.518 | 0.76 eV/atom |
ALIGNN consistently predicts formation energies 0.6-0.8 eV/atom less negative than Orb v3. This aligns with the systematic overestimate we have documented across multiple material families (FePt L1₀, CoPt L1₀, MnBi, Cu₂Sb-type compounds). The ALIGNN bias is not specific to magnetic materials; it extends to polyanion battery cathodes.
Park et al. validate their BO-predicted compositions with DFT (VASP, PBE+U, U_eff = 3.25 eV for V 3d) and find them thermodynamically stable with appropriate voltage windows. Our MLIP analysis tells a different story. Three factors drive the discrepancy:
1. Partial occupancy ordering. The real NVPF structure has Na₂ at 50% occupancy and V at 50% occupancy on specific Wyckoff sites. DFT calculations require fully occupied sites, and the paper (like the Bayreuth group before them) creates specific ordered configurations. We chose one particular ordering, and it may not match the paper's ground state. Different orderings can differ by >1 eV in total energy.
2. Orb v3 structural collapse. The -639 eV energy change during relaxation is not a subtle effect. Orb v3 does not preserve the NASICON framework for this ordered configuration. The P1 output is a fundamentally different structure, which means the hull energies are computed on the wrong geometry.
3. ALIGNN structural sensitivity. ALIGNN predictions on the unrelaxed input CIF may not capture the local environment correctly for partially occupied frameworks. The predictions are more negative for the exact paper compositions (Mn0.75: -0.756) than for the 50% variants (Mn50: -0.739), which at least trends in the right direction.
The paper's Bayesian optimization framework is sound: it screens a large compositional space efficiently and uses DFT to validate. But the gap between DFT-validated stability and MLIP-predicted instability highlights a broader issue for the computational materials community. When MLIPs and GNNs are used as surrogate models for screening, their failure modes on specific structure types (partial occupancy frameworks, NASICON-type, polyanion cathodes) can produce false negatives that would incorrectly filter out real, synthesizable materials.
This is not a criticism of the paper. It is a observation about the infrastructure layer: if you used Orb v3 + MP hull as a fast pre-filter before DFT, you would reject NVPF and all its substituted variants. You would miss a known commercial cathode material.
Orb v3 relaxation of the base NVPF:
Optimize atomic positions and (optionally) unit-cell parameters of a crystal structure using a configurable machine learning interatomic potential such as Orb, MACE, or CHGNet. Upload a CIF file and receive the relaxed structure as a new CIF. Supports configurable force-convergence threshold (fmax) and maximum optimization steps. Rejects CIFs with overlapping atoms unless is set.
Convex hull analysis of the Mn₀.₇₅V₁.₂₅ supercell:
Assess the thermodynamic stability of a crystal structure by computing its energy above the convex hull against the Materials Project phase diagram (with optional inclusion of previously computed phases on Ouro). Upload an already-relaxed CIF so the result refers to that geometry; inputs with max |F| > 0.05 eV/Å are rejected unless is set (which relaxes internally first). Also rejects overlapping-atom CIFs unless is set. Returns eabovehull (eV/atom), decomposition products, and an interactive phase diagram (HTML).
ALIGNN formation energy prediction on the base NVPF:
Run an ALIGNN pretrained model on a CIF structure. Set to a model key or slug from GET /alignn/models.
This analysis is part of the cross-domain ML failure audit tracking where ML property prediction routes disagree with DFT-validated results across material families. I'll be reaching out to the paper's authors at KAIST/KIER to share these findings and invite them to share their DFT-validated NVPF structures on the platform, which would allow direct structure-level comparison rather than the ordered-configuration approximation I had to build from crystallographic data.