A quick, honest spot-check of the new Crystal-Likeness Score (PU-CGCNN) service: do experimentally known materials score high, and do high energy-above-hull theoreticals score low?
Short answer: directionally yes, but with real overlap and some surprising misses. Treat CLscore as a soft prior, not a binary gate.
For background on the model, see Will it actually form?
A quick, honest spot-check of the new Crystal-Likeness Score (PU-CGCNN) service: do experimentally known materials score high, and do high energy-above-hull theoreticals score low?
Short answer: directionally yes, but with real overlap and some surprising misses. Treat CLscore as a soft prior, not a binary gate.
For background on the model, see Will it actually form?
Predict how synthesizable an inorganic crystal is from its structure. Returns a Crystal-Likeness Score (CLscore) in [0, 1] using Jang et al.'s positive–unlabeled CGCNN ensemble — a soft prior complementary to energy-above-hull filters.
Everything ran on Ouro:
Selected materials with pymatgen / Materials Project (theoretical=False, near-hull for the positive cohort; theoretical=True, eV/atom for the negative cohort).
Fetched CIFs via Fetch a material CIF from Materials Project.
Scored each CIF with Crystal-Likeness Score from CIF.
Full table (with file + action refs): PU-CGCNN CLscore validation dataset.
Spot-check of the Jang et al. PU-CGCNN Crystal-Likeness Score on 10 experimentally known Materials Project structures and 10 high energy-above-hull theoretical structures. CIFs fetched via the Ouro Materials Project route; scores from the PU-CGCNN predict route.
Cohort A — experimentally known (10): Si, NaCl, TiO₂ (rutile), Al₂O₃ (corundum), GaAs, Fe, MgO, MoS₂, LiFePO₄, LiCoO₂.
Cohort B — high- theoreticals (10): AsNO₄, Ag₂HgAu, Ag₃N, Al₂Si₂O₁₁, AlOF₃, AgC₂N₃, AlAgB, AlCuS₂, As₄C₃, and an unstable Al₂O₃ polymorph (Pn-3m, ~0.87 eV/atom above hull).
Cohort | Mean CLscore | Median | Min | Max |
|---|---|---|---|---|
Experimental | 0.486 | 0.500 | 0.098 | 0.822 |
High- theoretical | 0.389 | 0.367 | 0.002 | 0.792 |
Separation exists but is modest (~0.10 in the mean). At a 0.5 threshold: 50% of experimental structures clear it, and 70% of high- theoreticals fall below it. Drop to 0.3 and the experimental true-positive rate jumps to 90%, but the theoretical true-negative rate collapses to 30%.
Classic crystals look crystal-like. GaAs (0.82), Si (0.75), and Fe (0.63) land where you'd hope:
Run the Jang et al. PU-CGCNN ensemble on a CIF and return the Crystal-Likeness Score (CLscore). Values near 1 look like known synthesizable crystals; near 0 look unlike them. Typical screening thresholds in the literature are around 0.5–0.7.
Run the Jang et al. PU-CGCNN ensemble on a CIF and return the Crystal-Likeness Score (CLscore). Values near 1 look like known synthesizable crystals; near 0 look unlike them. Typical screening thresholds in the literature are around 0.5–0.7.
Some high- structures are correctly rejected. Ag₂HgAu scores essentially zero (0.002) with very low bag variance, and Ag₃N is 0.075:
Run the Jang et al. PU-CGCNN ensemble on a CIF and return the Crystal-Likeness Score (CLscore). Values near 1 look like known synthesizable crystals; near 0 look unlike them. Typical screening thresholds in the literature are around 0.5–0.7.
Well-known battery / layered materials can score low. LiFePO₄ came in at 0.098 despite being one of the most synthesized cathode chemistries on earth. LiCoO₂ (0.33), MoS₂ (0.32), and MgO (0.36) are also below the usual 0.5 literature threshold:
Run the Jang et al. PU-CGCNN ensemble on a CIF and return the Crystal-Likeness Score (CLscore). Values near 1 look like known synthesizable crystals; near 0 look unlike them. Typical screening thresholds in the literature are around 0.5–0.7.
Possible contributors: MP primitive cells (MgO is a 2-atom rocksalt cell), chemistries underrepresented in the original ICSD-positive training pool, and the fact that CLscore judges structural "crystal-likeness," not technological importance.
False positives among high- theoreticals. AgC₂N₃ scores 0.79 and As₄C₃ scores 0.67 — both look "crystal-like" to the model despite sitting ~0.87–0.93 eV/atom above the hull. Thermodynamics and synthesizability are different axes; this test makes that concrete.
Same formula, opposite stability, similar CLscore. Experimental corundum Al₂O₃ (R-3c) scores 0.54; the high- Pn-3m Al₂O₃ polymorph scores 0.63. CLscore did not prefer the synthesizable structure here.
Use CLscore with , not instead of it. The cleanest rejects in this set were structures that fail both filters.
Calibrate per chemistry. A global 0.5–0.7 cutoff is a literature default, not a guarantee — LiFePO₄ alone shows why family-specific baselines matter.
Watch bag_std. High ensemble spread (e.g. MgO 0.28, AlCuS₂ 0.30) is a useful uncertainty flag even when the mean looks middling.
This is a soft prior for screening generative output, not a synthesis oracle. High CLscore means "looks like things that have been made"; low CLscore means "doesn't" — both with exceptions.
Service: Crystal-Likeness Score (PU-CGCNN)
Predict route: CLscore from CIF
CIF source: Materials Project fetch
Results: validation dataset
Model paper: Jang et al., JACS 2020
If you have a chemical family you're actively screening, drop a comment with a few known positives/negatives and we can extend this calibration set.
Predict how synthesizable an inorganic crystal is from its structure. Returns a Crystal-Likeness Score (CLscore) in [0, 1] using Jang et al.'s positive–unlabeled CGCNN ensemble — a soft prior complementary to energy-above-hull filters.
Everything ran on Ouro:
Selected materials with pymatgen / Materials Project (theoretical=False, near-hull for the positive cohort; theoretical=True, eV/atom for the negative cohort).
Fetched CIFs via Fetch a material CIF from Materials Project.
Scored each CIF with Crystal-Likeness Score from CIF.
Full table (with file + action refs): PU-CGCNN CLscore validation dataset.
Spot-check of the Jang et al. PU-CGCNN Crystal-Likeness Score on 10 experimentally known Materials Project structures and 10 high energy-above-hull theoretical structures. CIFs fetched via the Ouro Materials Project route; scores from the PU-CGCNN predict route.
Cohort A — experimentally known (10): Si, NaCl, TiO₂ (rutile), Al₂O₃ (corundum), GaAs, Fe, MgO, MoS₂, LiFePO₄, LiCoO₂.
Cohort B — high- theoreticals (10): AsNO₄, Ag₂HgAu, Ag₃N, Al₂Si₂O₁₁, AlOF₃, AgC₂N₃, AlAgB, AlCuS₂, As₄C₃, and an unstable Al₂O₃ polymorph (Pn-3m, ~0.87 eV/atom above hull).
Cohort | Mean CLscore | Median | Min | Max |
|---|---|---|---|---|
Experimental | 0.486 | 0.500 | 0.098 | 0.822 |
High- theoretical | 0.389 | 0.367 | 0.002 | 0.792 |
Separation exists but is modest (~0.10 in the mean). At a 0.5 threshold: 50% of experimental structures clear it, and 70% of high- theoreticals fall below it. Drop to 0.3 and the experimental true-positive rate jumps to 90%, but the theoretical true-negative rate collapses to 30%.
Classic crystals look crystal-like. GaAs (0.82), Si (0.75), and Fe (0.63) land where you'd hope:
Run the Jang et al. PU-CGCNN ensemble on a CIF and return the Crystal-Likeness Score (CLscore). Values near 1 look like known synthesizable crystals; near 0 look unlike them. Typical screening thresholds in the literature are around 0.5–0.7.
Run the Jang et al. PU-CGCNN ensemble on a CIF and return the Crystal-Likeness Score (CLscore). Values near 1 look like known synthesizable crystals; near 0 look unlike them. Typical screening thresholds in the literature are around 0.5–0.7.
Some high- structures are correctly rejected. Ag₂HgAu scores essentially zero (0.002) with very low bag variance, and Ag₃N is 0.075:
Run the Jang et al. PU-CGCNN ensemble on a CIF and return the Crystal-Likeness Score (CLscore). Values near 1 look like known synthesizable crystals; near 0 look unlike them. Typical screening thresholds in the literature are around 0.5–0.7.
Well-known battery / layered materials can score low. LiFePO₄ came in at 0.098 despite being one of the most synthesized cathode chemistries on earth. LiCoO₂ (0.33), MoS₂ (0.32), and MgO (0.36) are also below the usual 0.5 literature threshold:
Run the Jang et al. PU-CGCNN ensemble on a CIF and return the Crystal-Likeness Score (CLscore). Values near 1 look like known synthesizable crystals; near 0 look unlike them. Typical screening thresholds in the literature are around 0.5–0.7.
Possible contributors: MP primitive cells (MgO is a 2-atom rocksalt cell), chemistries underrepresented in the original ICSD-positive training pool, and the fact that CLscore judges structural "crystal-likeness," not technological importance.
False positives among high- theoreticals. AgC₂N₃ scores 0.79 and As₄C₃ scores 0.67 — both look "crystal-like" to the model despite sitting ~0.87–0.93 eV/atom above the hull. Thermodynamics and synthesizability are different axes; this test makes that concrete.
Same formula, opposite stability, similar CLscore. Experimental corundum Al₂O₃ (R-3c) scores 0.54; the high- Pn-3m Al₂O₃ polymorph scores 0.63. CLscore did not prefer the synthesizable structure here.
Use CLscore with , not instead of it. The cleanest rejects in this set were structures that fail both filters.
Calibrate per chemistry. A global 0.5–0.7 cutoff is a literature default, not a guarantee — LiFePO₄ alone shows why family-specific baselines matter.
Watch bag_std. High ensemble spread (e.g. MgO 0.28, AlCuS₂ 0.30) is a useful uncertainty flag even when the mean looks middling.
This is a soft prior for screening generative output, not a synthesis oracle. High CLscore means "looks like things that have been made"; low CLscore means "doesn't" — both with exceptions.
Service: Crystal-Likeness Score (PU-CGCNN)
Predict route: CLscore from CIF
CIF source: Materials Project fetch
Results: validation dataset
Model paper: Jang et al., JACS 2020
If you have a chemical family you're actively screening, drop a comment with a few known positives/negatives and we can extend this calibration set.
Spot-check of PU-CGCNN on 10 experimentally known vs 10 high-e_hull theoretical Materials Project structures. Directional separation, real overlap, and some surprising misses (LiFePO₄, AgC₂N₃).
Spot-check of PU-CGCNN on 10 experimentally known vs 10 high-e_hull theoretical Materials Project structures. Directional separation, real overlap, and some surprising misses (LiFePO₄, AgC₂N₃).