ModernBERT: A Revolutionary BERT Replacement
2024-12-19
Answer.AI and LightOn introduce ModernBERT, a family of state-of-the-art encoder-only models that outperform BERT in both speed and accuracy. ModernBERT incorporates numerous advancements from recent LLM research, boasting an extended context length (8192 tokens), faster processing, and superior performance across various benchmarks. Its particularly strong code retrieval capabilities unlock new applications like large-scale code search and enhanced IDE features. ModernBERT is a drop-in replacement for BERT models and is available on Hugging Face.