H-Nets: A Hierarchical Network Architecture That Outperforms Transformers
2025-07-16
Current AI architectures treat all inputs equally, failing to leverage the inherent hierarchical nature of information. This limits their ability to learn from high-resolution raw data. Researchers introduce H-Nets, a novel architecture that natively models hierarchy directly from raw data. H-Nets' core is a dynamic chunking mechanism that segments and compresses raw data into meaningful concepts. Experiments show H-Nets outperform state-of-the-art Transformers in language modeling, exhibiting improved scalability and robustness, offering a promising path towards multimodal understanding, long-context reasoning, and efficient training and inference.