Open research towards the discovery of room-temperature superconductors.
Discover other posts like this one
A 9pm meeting with someone solely focused on money printing got the wheels turning about potential next build avenues as we work towards a room temperature superconductor.Bryan (the scout) was actuall
Good read. Well written, very detailed and thorough. Great contribution.
2025-01-03
The paper is somewhat basic (and probably still in preprint), but this contribution is nonetheless great!
Authors introduce Orb, a family of universal interatomic potentials for atomistic modeling of materials. Orb models are 3-6 times faster than existing universal potentials, stable under simulation for a range of out of distribution materials and, upon release, represented a 31% reduction in error over other methods on the Matbench Discovery benchmark. https://arxiv.org/abs/2410.22570
Combining the ideas and implementations we get from the MatterSim paper with the implementation and open source nature of Orb and we have a really solid foundation for building an end-to-end Tc prediction model.
Currently working on training a prediction head on the 3DSC dataset using Orb as the base model. More on that coming soon.
Here is a quick summary of the paper and my thoughts on Orb.
Orb is a neural network designed to predict how atoms interact - the forces between them, the energy of the system, and how the system responds to stress. It performs these tasks well (#2 on the Matbench leaderboard, beating MatterSim and GNoME), and it does so simply.
Orb uses a three-stage pipeline: an Encoder that processes the raw atomic information, a Processor that performs multiple rounds of message passing between atoms to understand their interactions, and a Decoder that produces the final predictions.
Orb is unique in how it handles physical constraints. Rather than building physical laws directly into the architecture (as many previous models have done), Orb learns these properties from data. For example, instead of using an energy-based model where forces are computed as gradients, it predicts forces directly and uses clever correction terms to ensure physical consistency. This turns out to be both faster and more accurate.
The training process is interesting. Orb uses a two-phase approach:
First, it's pretrained using a diffusion model on a broad dataset of ground state materials. This teaches it basic chemistry - what atomic configurations are reasonable and how atoms typically arrange themselves.
Then, it's fine-tuned on specific trajectories from density functional theory (DFT) calculations to learn precise energetics and forces.
This approach allows it to combine data from different sources and different levels of theory during pretraining, something that's typically very difficult to do.
The codebase is clean, well-documented PyTorch, released under the Apache 2.0 license. It's designed with transfer learning in mind - you can easily take the pretrained model and adapt it for new properties while leveraging all the chemical understanding it has already learned. Digging deeper into MatterSim, I did not find this to be the case, so it makes Orb a great candidate for building on.
The model is fast - 3-6x faster than existing methods on commodity hardware. This speed, combined with its accuracy and stability, makes it practical for applications like molecular dynamics simulations.
Left: Model forward pass speed (excluding featurization) compared to MACE on a single NVIDIA A100 GPU. At large system sizes, Orb is between 3 to 6 times faster than MACE. Right: End to end model inference speed for a 100 atom system on a single NVIDIA A100 when implemented as a Calculator object in the Atomic Simulation Environment Python library. The D3 dispersion correction adds a substantial cost which is amortized by Orb models, as the corrections are incorporated into training datasets. All measurements reported as the median of 50 runs.
Finetuning on specific levels of theory or using the pretrained D3 model allows us to get to specific levels of theory without computationally expensive additions of corrections.