Large Language Diffusion Models Paper

This paper introduces LLaDA, a diffusion model trained from scratch under the pre-training and supervised finetuning (SFT) paradigm. LLaDA models distributions through a forward data masking process and a reverse process, parameterized by a vanilla Transformer to predict masked tokens. https://arxiv.org/abs/2502.09992

Large Language Diffusion Models Paper · Files on Ouro

23dupdated 23d ago

Large Language Diffusion Models Paper

Connections

How this file is connected to other assets

1 reference

23d

comment

Similar files

Discover other files like this one

MatterGen paper

This paper presents MatterGen, a model that generates stable, diverse inorganic materials across the periodic table and can further be fine-tuned to steer the generation towards a broad range of property constraints. To enable this, the authors introduce a new diffusion-based generative process that produces crystalline structures by gradually refining atom types, coordinates, and the periodic lattice. https://arxiv.org/abs/2312.03687

16d

PDF file

3D latent space exploration with Tc

Using the 256 dimensional latent space output from the Orb model, we visualize the 3DSC(MP) dataset using UMAP with direction from Tc labels. Hover a point to see Tc, formula, and Material Project identifier.

2mo

.html file

Interactive 3D latent space exploration with Tc

Using the 256 dimensional latent space output from the Orb model, we visualize the 3DSC(MP) dataset using t-SNE and UMAP. The UMAP projection has been given the target for learning a manifold that keeps similar Tc materials close together.

2mo

.html file

MatterSim

Authors present MatterSim, a deep learning model actively learned from large-scale first-principles computations, for efficient atomistic simulations at first-principles level and accurate prediction of broad material properties across the periodic table, spanning temperatures from 0 to 5000 K and pressures up to 1000 GPa. https://arxiv.org/abs/2405.04967

2mo

PDF file