Compiler Explorer: 92 Million Compilations a Year and Still Going Strong

2025-06-08

Compiler Explorer, the online compiler exploration website, handles a staggering 92 million compilations annually. This article dives deep into its architecture, from the Monaco editor frontend and CloudFront/load balancer to the secure sandboxing with nsjail. To manage this massive workload, it leverages AWS autoscaling and boasts nearly 4TB of over 3000 compiler versions supporting 81 languages. The author details the challenges and solutions in security, version management, cross-platform support (Windows, ARM, and GPU), and cost optimization, showcasing the evolution from a weekend project to a robust platform serving thousands of developers.

Read more
Development

Build Your Own Federated Microblog with Fedify

2025-04-24
Build Your Own Federated Microblog with Fedify

This tutorial guides you through building a small, ActivityPub-compliant federated microblog using the Fedify framework. No prior knowledge of TypeScript, JSX, or ActivityPub is required. You'll learn to create accounts, set up a database, interact with other servers (including Mastodon), and implement features like following, posting, and displaying a timeline. The tutorial provides detailed code examples and steps, along with concise explanations of TypeScript and JSX. By the end, you'll have a working single-user microblog that can communicate with the fediverse.

Read more
Development Federated Social

Firefly's Rocket Expertise Fuels Deep-Sea Probe

2025-03-02
Firefly's Rocket Expertise Fuels Deep-Sea Probe

Firefly, initially focused on rockets for launching satellites and spacecraft, has leveraged its experience building the Alpha rocket – despite some early setbacks – to design the propulsion system for its Blue Ghost deep-sea probe. CEO Jason Kim highlights the use of flight-proven attitude control thrusters from the Alpha rocket and in-house developed reaction control thrusters, reducing risk and capitalizing on existing expertise. Firefly's Texas facility, where rocket engine tests are conducted alongside a cow pasture, has expanded significantly. While thrusters were built on-site, the main engine for Blue Ghost was sourced from a supplier.

Read more

PostgreSQL Cracks Top 10 in ClickBench: pg_mooncake's Analytics Breakthrough

2025-03-08
PostgreSQL Cracks Top 10 in ClickBench: pg_mooncake's Analytics Breakthrough

pg_mooncake, a PostgreSQL extension, has propelled PostgreSQL into the ClickBench top 10, a benchmark typically dominated by specialized analytics databases. This wasn't achieved through a simple wrapper, but by leveraging PostgreSQL's extensibility to implement a columnar storage format, vectorized execution using DuckDB, and in-database metadata management. This demonstrates that with careful optimization, PostgreSQL can deliver analytics performance comparable to specialized databases, while maintaining its flexibility and ecosystem advantages.

Read more

Hidden in Plain Sight: The Anti-Capitalist Potential of Technical Standards

2025-05-02
Hidden in Plain Sight: The Anti-Capitalist Potential of Technical Standards

Technical standards, like electrical outlets and internet protocols, aren't accidental; they're created and maintained by Standards Developing Organizations (SDOs) like ISO, ANSI, and IEEE through open collaboration, consensus-building, and public knowledge sharing. This offers a rare example of an economic system that doesn't rely on capitalism. Unlike patents, standards are collaboratively developed and published by SDOs on 'reasonable and non-discriminatory' terms, ensuring wide availability. The article argues that actively participating in the standards development process, encouraging organizations to operate within principles of openness, consensus, and a lack of dominance, undermines capitalist power and promotes information sharing as a public good, representing a practical anti-capitalist action.

Read more

Moscow's Mandatory Tracking App for Foreign Nationals

2025-05-22
Moscow's Mandatory Tracking App for Foreign Nationals

A new Russian law mandates that all foreign nationals in the Moscow region install a tracking app. This app collects residence location, fingerprints, facial photographs, and real-time geolocation data. While presented as a crime-fighting measure targeting migrant crime, the law has sparked privacy concerns. Critics argue it violates Russia's constitutional right to privacy and may deter potential labor migrants. The mass-surveillance experiment runs until September 2029, with potential expansion nationwide if deemed successful.

Read more
Tech

Judge Sidesteps Google's AI Monopoly in Antitrust Case

2025-09-03
Judge Sidesteps Google's AI Monopoly in Antitrust Case

While Judge Amit Mehta's ruling partially blocks some of Google's anti-competitive practices, it fails to address the company's dominance in generative AI. The decision relies on speculative arguments about the future of AI, overlooking Google's existing monopolies and distribution advantages. Search is a key gateway to future AI interactions, and the judge's leniency allows Google to continue shaping the internet and economy, rather than enforcing laws designed for fair competition and fostering innovation.

Read more
Tech

Meta and Yandex Caught Bypassing Privacy Protections with Localhost Tracking

2025-06-04
Meta and Yandex Caught Bypassing Privacy Protections with Localhost Tracking

Security researchers revealed that Meta and Yandex used native Android apps to listen on localhost ports, linking web browsing data to user identities and bypassing typical privacy safeguards. Meta's Pixel script has stopped sending data to localhost and removed much of the tracking code, likely to avoid violating Google Play policies. Researchers discovered that Facebook, Instagram, and Yandex apps silently collected cookie data via fixed local ports, linking browsing activity to user identities and circumventing cookie clearing, incognito mode, and app permission systems. Meta employed this technique starting in September 2024, using HTTP, WebSocket, and WebRTC protocols. Meta has since ceased this practice, but Yandex's use continues. Chrome 137 includes some mitigations, and Firefox and DuckDuckGo are also taking action.

Read more

Run a Certificate Transparency Log on a Single Server!

2025-07-08
Run a Certificate Transparency Log on a Single Server!

This article explains how easy it is to run a Certificate Transparency (CT) log, a critical component of web security. The author highlights that with the new Static CT API and Sunlight implementation, running a CT log is now significantly cheaper and easier, requiring only a single server, minimal bandwidth, and storage. This presents an opportunity for organizations or individuals with spare resources to contribute to web security and enhance their technical skills.

Read more
Tech

LLMs: Lossy Encyclopedias

2025-09-02

Large language models (LLMs) are like lossy encyclopedias; they contain a vast amount of information, but this information is compressed, leading to data loss. The key is discerning which questions LLMs can answer effectively versus those where the lossiness significantly impacts accuracy. For example, asking an LLM to create a Zephyr project skeleton with specific configurations is a 'lossless' question requiring precise details, which LLMs struggle with. The solution is to provide a correct example, allowing the LLM to operate on existing facts rather than relying on potentially missing details within its knowledge base.

Read more

SpaceX Starship Flight 9: Partial Success, Both Stages Lost

2025-05-28
SpaceX Starship Flight 9: Partial Success, Both Stages Lost

SpaceX launched its Starship megarocket for the ninth time, marking the first significant reuse of Starship hardware. While the upper stage reached space – a major improvement – both stages were ultimately lost before completing their flight goals. Leaks caused a loss of main tank pressure during the coast and re-entry phases, according to SpaceX CEO Elon Musk. Despite the loss, valuable data was gathered, and SpaceX plans three more Starship test launches in the next three to four weeks.

Read more
Tech

Human Ingenuity vs. LLMs: Debugging Redis Vector Sets

2025-05-29

Redis developer antirez recounts a fascinating debugging experience where he pitted his wits against Gemini 2.5 PRO, a large language model. A complex bug in Redis's vector set (HNSW) implementation, stemming from data corruption leading to inconsistent node links, required a solution beyond a naive O(N²) approach. While Gemini suggested binary search, antirez ultimately devised a creative solution involving an XOR accumulator, further refined by incorporating MurmurHash128 and a random seed. This anecdote highlights the power of human creative thinking in tackling complex problems, showcasing how LLMs can assist but ultimately fall short of human ingenuity in generating truly novel solutions.

Read more
Development

America at a Crossroads: Empire's Decline or Rebirth?

2025-03-05

This article analyzes the current state of the American economy, highlighting its long-term reliance on dollar hegemony and financial speculation, leading to a hollowed-out real economy. The author argues that America faces two paths: one is to shift towards protectionism, becoming a regional power but facing stagnation; the other is to strive to maintain its imperial status, enhancing competitiveness by attracting global talent and backing the dollar with gold reserves. The author believes that the latter, while challenging, is the only hope for America's resurgence.

Read more

Waymo's 10 Million Rides: Tesla's Autopilot Strategy Under Pressure?

2025-06-04
Waymo's 10 Million Rides: Tesla's Autopilot Strategy Under Pressure?

In 2019, Elon Musk dismissed lidar and Waymo. Fast forward to 2024, and Waymo's driverless taxi service has surpassed 10 million rides, doubling its trips in just months. Conversely, Tesla's robotaxi service is launching with a mere 10 vehicles. The author argues Waymo's focus on densely populated urban areas, leveraging lidar and other technologies, has yielded significant progress. Tesla's approach may be too aggressive, overlooking the 80/20 rule of city driving—solving the last 20% of self-driving might require 80% of the effort. Waymo's success suggests a steady, controlled market approach might be more effective than striving for all-scenario coverage in the autonomous driving field.

Read more
Tech

Saturn's Rings: Older Than We Thought?

2024-12-19
Saturn's Rings: Older Than We Thought?

A new study challenges the long-held belief that Saturn's rings are relatively young. Previous research, based on data from the Cassini spacecraft, suggested an age of only a few hundred million years. However, a team led by planetary scientist Ryuki Hyodo used computer simulations to show that micrometeoroid impacts on the rings produce far less contamination than previously thought. This implies the rings could be billions of years old, as old as Saturn itself, predating even the dinosaurs. This finding significantly alters our understanding of early solar system evolution and points towards future missions for more detailed investigation.

Read more

C0: A High-Performance Read-Through Cache for Object Storage

2025-09-20
C0: A High-Performance Read-Through Cache for Object Storage

C0 is a high-performance HTTP API designed for caching immutable blobs in object storage. It leverages a hybrid memory and disk cache (powered by foyer) and works with any S3-compatible backend, but uses its own /fetch API requiring a precise Range header. Employing a fixed page size of 16 MiB, C0 maps requested byte ranges to page-aligned lookups, coalesces concurrent requests for the same page, and uses hedged requests to manage object storage tail latency. It can even attempt redundant buckets. C0 offers extensive configuration options, including timeout settings, retry mechanisms, and bucket prioritization, and provides performance monitoring via /stats and /metrics endpoints. Docker images are available.

Read more
Development

More Trees Than Stars in the Milky Way?

2025-01-02
More Trees Than Stars in the Milky Way?

A recent study suggests there might be more trees on Earth than stars in the Milky Way galaxy. While estimates for both are imprecise, a 2015 Nature paper estimated 3.04 trillion trees on Earth, while estimates for stars in the Milky Way range from 100 to 400 billion. Despite uncertainties, the tree count significantly exceeds even the highest star estimates. This surprising fact sparks reflection on Earth's ecosystems and the vastness of space.

Read more

Magnitude: AI-Native Testing Framework for Web Apps

2025-04-25
Magnitude: AI-Native Testing Framework for Web Apps

Magnitude is a revolutionary open-source end-to-end testing framework for web applications, powered by visual AI agents that 'see' your interface and adapt to any changes. Build test cases easily with natural language, leverage a powerful reasoning agent for planning and adjustment, and rely on a fast visual agent for reliable execution. Run tests locally or within CI/CD pipelines; the framework automatically handles problems encountered during testing. Magnitude supports various LLMs, including Gemini 2.5 pro, Anthropic, and OpenAI, and offers a free visual model, Moondream. It simplifies test creation and execution significantly.

Read more

All Roses Were Once Yellow: A Genomic Analysis Reveals the Truth

2025-05-01
All Roses Were Once Yellow: A Genomic Analysis Reveals the Truth

A groundbreaking genomic analysis reveals that all roses—red, white, pink, and peach—descended from a single-petaled yellow rose. Researchers from Beijing Forestry University sequenced the genomes of 205 rose samples, tracing the genus's evolutionary history and geographic distribution. Their findings point to an ancestor with seven leaflets and a single whorl of yellow petals. This discovery not only enhances our understanding of rose evolution but also opens new avenues for breeding more resilient and low-maintenance rose varieties, aiding in the conservation of endangered species.

Read more

eBPF Verifier's Security Dilemma: A Novel Isolated Execution Environment

2025-04-15

eBPF, a foundational technology in the Linux kernel, faces security vulnerabilities and complexity challenges in its verifier. Researchers propose a paradigm shift: defining BPF programs as kernel-mode applications requiring dedicated isolation. A novel execution environment is designed to isolate BPF programs, enhancing eBPF's security and scalability. This research delves into the Linux v6.16 eBPF verifier, revealing security properties, capability dilemmas, and correctness dilemmas within its full-path analysis. A hybrid security framework combining verification and isolation is proposed, pointing towards a more secure future for eBPF.

Read more
Development

Sonic Heritage: The Overlooked Sounds of Tourism

2025-04-21
Sonic Heritage: The Overlooked Sounds of Tourism

In our visually-driven tourism industry, sound is often overlooked. This project explores the crucial role sound plays in tourist experiences and heritage preservation. With overtourism becoming a growing crisis, sound offers a fresh perspective on tourist destinations and potential solutions. 'Sonic Heritage' aims to examine the soundscapes of the world's most culturally significant sites, advocating for the identification, celebration, and preservation of culturally or socially significant soundscapes before they disappear.

Read more
Design sound heritage

SpaceX Dragon Docks with ISS: A Mission Breakdown

2025-01-16
SpaceX Dragon Docks with ISS: A Mission Breakdown

SpaceX's Dragon capsule successfully docked with the International Space Station. The mission involved several phases: first, a Falcon 9 rocket launched Dragon into orbit; then, Dragon performed orbital adjustments to gradually approach the ISS; finally, Dragon docked with the ISS, completing pressurization and crew ingress. The entire process involved a series of precise thruster burns and orbital maneuvers, showcasing SpaceX's advanced aerospace technology.

Read more
Tech

The Paradox of Network Building: Starting Small to Go Big

2025-04-22
The Paradox of Network Building: Starting Small to Go Big

Andrew Chen's new book delves into the experiences and strategies of building networked products and platforms, revealing a core paradox: massive successful network effects require starting with a small, stable "atomic network." The book analyzes case studies of companies like Uber, Airbnb, and Reddit, summarizing key strategies to overcome the "cold start" problem, such as solving core user pain points, creating "magic moments," and cleverly using invite-only systems and subsidies. The author emphasizes that consistently focusing on user value and adapting strategies based on reality is key to achieving explosive growth through network effects.

Read more
Development cold start

Android's New Canary Channel: Continuous Early Access for Developers

2025-07-11
Android's New Canary Channel: Continuous Early Access for Developers

Google is replacing its Developer Preview program with a new Canary channel for Android, offering developers rolling updates throughout the year. This allows for earlier and more consistent access to experimental features and APIs. Unlike previous manual installations, Canary builds are delivered over-the-air and run concurrently with the beta program. While intended for testing and not daily use, Canary provides valuable early feedback, enabling developers to identify issues and test their apps continuously. Support is currently available for Pixel devices and the Android Studio Canary version.

Read more

May Day: A Century-Long Celebration of Labor

2025-05-01
May Day: A Century-Long Celebration of Labor

May Day, or International Workers' Day, commemorates the 1886 Haymarket affair in Chicago, a pivotal event in the struggle for the eight-hour workday. The tragic events led international socialists to establish May 1st as International Workers' Day in 1889, honoring those who sacrificed for workers' rights. Today, many countries observe it as a public holiday, celebrating the contributions of laborers and the working class, often marked by parades, demonstrations, and discussions advocating for workers' rights and improved conditions.

Read more

Reichstag Fire Decree: The Enabling Act's Precursor

2025-09-16
Reichstag Fire Decree: The Enabling Act's Precursor

The Reichstag fire of February 27, 1933, blamed on Dutch communist Marinus van der Lubbe, provided the pretext for the Nazi regime to enact the Decree for the Protection of the People and the State on February 28th. This decree, suspending fundamental constitutional rights, including freedom of speech and assembly, allowed for the arbitrary arrest and imprisonment of political opponents, the suppression of dissent, and the consolidation of Nazi power. It effectively dismantled democratic processes, paving the way for the Enabling Act and the establishment of a totalitarian police state.

Read more

Building a Web Search Engine from Scratch: 3 Billion Embeddings and 2 Months of Hustle

2025-08-13

The author recounts their two-month journey building a web search engine from scratch, leveraging 3 billion SBERT embeddings. Motivated by the shortcomings of existing search engines – excessive SEO spam and insufficient high-quality content – the project aimed to improve search relevance and understanding of complex queries. The post details the process, covering data crawling, text normalization, chunking, semantic context handling, embedding generation, storage (using RocksDB and HNSW), and retrieval. The resulting engine boasts 500ms query latency and handles complex natural language queries, surfacing high-quality results.

Read more

Near 100% GPU Utilization for Embedding Millions of Documents with Daft

2025-08-17
Near 100% GPU Utilization for Embedding Millions of Documents with Daft

The Daft team achieved near-100% GPU utilization while embedding millions of text documents using the Qwen3-Embedding-0.6B model. This blog post details a three-step data pipeline: text chunking, embedding generation, and distributed processing, providing code examples. They subsequently improved performance by 3x without relying on maximum GPU utilization.

Read more

The Hobbit in Five Celtic Languages: A Publishing Milestone

2025-06-27
The Hobbit in Five Celtic Languages: A Publishing Milestone

Welsh publisher Melin Bapur has compiled all current Celtic language editions of J.R.R. Tolkien's classic, *The Hobbit*, including the recently released Scottish Gaelic translation, *A' Hobat*. This marks a significant achievement in bringing the story to a wider Celtic audience, with only the Manx Gaelic version remaining untranslated. The Welsh translation, *Yr Hobyd*, released in 2024, uniquely uses the 18th-century Welsh Coelbren y Beirdd runes instead of Anglo-Saxon runes, adding a distinctly Welsh flavor. The publisher highlights the value of translating familiar books in encouraging wider readership and providing invaluable resources for language learners.

Read more

PURLs: Uniquely Identifying Software Packages

2025-06-05
PURLs: Uniquely Identifying Software Packages

Package URLs (PURLs) are an open standard for uniquely identifying software packages across various ecosystems. Created in 2017, PURLs address the need for standardized software package references. A PURL is a specially formatted URL encoding package type, name, version, and other qualifiers. Crucially, PURLs are essential for Software Bills of Materials (SBOMs), acting as unique identifiers for software components. This enables accurate vulnerability tracking and license verification. While offering simplicity and broad open-source support, PURLs currently lack comprehensive commercial product coverage, a domain where CPE remains more prevalent.

Read more
1 2 99 100 101 103 105 106 107 596 597