AWS S3 Vectors: The Rise of Tiered Storage for Vector Databases?

AWS recently launched S3 Vectors, a vector database built on top of its S3 object storage. This has sparked debate about whether it will replace existing vector databases like Milvus, Pinecone, etc. The author, a engineering architect at Milvus, argues that S3 Vectors is not a replacement but a complement, particularly suitable for low-cost, low-query frequency cold data storage scenarios. He analyzes S3 Vectors' technical architecture, highlighting its advantages in cost and scalability, but also its limitations in high query latency, low precision, and limited functionality. The author further elaborates on the evolution of vector databases: from in-memory storage to disk storage, and now to object storage, ultimately leading to a tiered storage architecture (hot, warm, and cold data layers) to balance performance, cost, and scalability. Milvus is also moving in this direction, with the upcoming 3.0 release featuring a vector data lake for unified management of hot and cold data. The emergence of S3 Vectors proves the maturity and growth of the vector database market, rather than disruption.