Revolutionary Technique Cuts LLM Memory Costs by Up to 75%
2024-12-17
Sakana AI, a Tokyo-based startup, has developed a groundbreaking technique called "universal transformer memory" that significantly improves the memory efficiency of large language models (LLMs). Using neural attention memory modules (NAMMs), the technique acts like a smart editor, discarding redundant information while retaining crucial details. This results in up to a 75% reduction in memory costs and improved performance across various models and tasks, offering substantial benefits for enterprises utilizing LLMs.