Evaluation of aggregation methods in an MLFF model for material property prediction

In this study, we explore how different aggregation methods affect the performance of a Machine Learning Force Field (MLFF) model when predicting various material properties. When using graph-based representations (e.g., molecules or crystalline materials), individual node (atom) features must be aggregated to form a global fingerprint of the material. Three common aggregation strategies were evaluated:

• Sum Aggregation: Adds the features of all nodes.
  Useful for extensive, size-dependent properties (e.g., total mass, cell volume).

• Mean Aggregation: Averages the features across all nodes, yielding a size-invariant representation.
  Generally effective for a wide range of properties, including intensive ones (e.g., average electronegativity).

• Max Aggregation: Selects the maximum value for each feature across nodes.
  Capable of highlighting dominant signals, potentially beneficial for properties where the presence of a strong local effect is critical (e.g., maximum magnetic moment).

Although the mean aggregation method is shown for most targets because it generally performs the best, the sum and max methods provided improvements in select cases.

Experimental Setup

• Dataset and Splits: The experiments were conducted on datasets derived from the orb_d3_v2 model's latent space.

• Properties Predicted: The following target properties were evaluated:

  • energy_per_atom_2

  • density_2

  • band_gap_2

  • tc (critical temperature for superconductors)

  • efermi_2

  • total_magnetization_2

  • cell_volume_2

• Metrics: The model performance was assessed using the coefficient of determination (R²), Mean Absolute Error (MAE), Mean Squared Error (MSE), and Root Mean Squared Error (RMSE) where applicable.

Results

Mean Aggregation Results

For most targets, mean aggregation provided good performance:

Sum Aggregation Results

While the mean method performed well overall, sum aggregation outperformed mean aggregation on certain properties:

Max Aggregation Results

For total magnetization, max aggregation further improved performance compared to mean aggregation:

Discussion

• General Trends:
  Mean aggregation generally yielded the best overall performance across most of the material properties. This is likely due to its ability to provide a balanced, size-invariant representation that works well for both intensive and many extensive properties.

• Select Cases – Sum vs. Mean:
  For properties such as cell_volume_2 and total_magnetization_2, which are inherently size-dependent or reflect cumulative effects across nodes, sum aggregation significantly improved the R² and reduced the MAE. This suggests that when the total contribution from all nodes is important, the sum method better captures the relevant information.

• Selective Improvement – Max Aggregation:
  The max aggregation strategy, especially for total_magnetization_2, indicates that emphasizing the most dominant node feature can be beneficial. This method may capture the presence of strong local signals that are diluted by averaging, thereby providing a better prediction for properties where extreme values play a critical role.

• Implications for Superconductor Data:
  The target tc (critical temperature) is central to superconductivity studies. While mean aggregation provided solid results (R² of 0.7598), further exploration with other aggregation methods may be warranted if specific superconducting phenomena are influenced by localized or extensive atomic contributions.

Conclusion

This investigation highlights that while mean aggregation is a robust choice for many material property predictions within an MLFF framework, alternative aggregation methods—specifically sum and max—can offer superior performance for certain properties.

• Sum aggregation is particularly effective for extensive properties such as cell volume and total magnetization, where the cumulative effect of all atoms is important.

• Max aggregation appears to capture dominant node-level signals that enhance prediction quality for properties like total magnetization.

These insights provide valuable guidance for tailoring the aggregation method to the property of interest in future MLFF model designs and applications, particularly in materials science and superconductivity research.