Comparing MLIP and MLFF, aggregation methods

Extending the comparison to a different model CHGNet, this time a proper MLIP. Similar to the Orb model, this model predicts energy, force, and stress, but with the addition of the magnetic moment for each atom.

The results were interesting and another piece of evidence pointing to the need of a more robust electronic/magnetic representation. Unsurprisingly, the model doesn't perform as well in general as it's a bit older and just not as good. But there are areas it performs better, specifically around cell features and electric properties.

Using mean aggregation

Interestingly, we got our best performance predicting Tc with this model. It's not better by much (+0.01), but it's something. It's likely additional understanding of the physical drivers of the magnetic moment are beneficial in predicting critical temperature.

Using sum aggregation

CHGNet performs well on cell volume in all aggregation modes because there is a subset of latent features that come directly from a cell representation, so they're directly representing the cell as a whole and will be the same across aggregations.

It's cool to see total magnetization perform this well. It's not great still, but it's a significant improvement from the other models and aggregation modes.

Still waiting to find a representation that can predict band gap well and then I think we'll have a good amount of the coverage in latent representation we're looking for.