LeCun Team Paper: 109M-Parameter Text JEPA Outperforms 72B-Parameter Qwen2.5 in Cross-Modal Alignment
The paper by LeCun and Randall Balestriero uses the JEPA framework to provide a mathematical proof of the Platonic Representation Hypothesis, and claims that on the cross-modal representation alignment task, a 109M-parameter text JEPA outperforms the 72B-parameter Qwen2.5. The research shows that Good Views training requires only 21.6 million samples to reach the alignment level that a next-token prediction model achieves with 900 million samples, reducing the sample count to 1/42, or to 1/14 when counted in tokens. The authors also point out that this comparison is limited to specific alignment metrics, and that the mathematical proof relies on idealized conditions such as a finite world and reversible observations.
#Papers/Research#Multimodal#English text is being prepared.