Category: Development

toyDB: A Distributed SQL Database in Rust for Educational Purposes

2025-05-11
toyDB: A Distributed SQL Database in Rust for Educational Purposes

toyDB is a distributed SQL database built from scratch in Rust as an educational project. It aims to illustrate the architecture and concepts behind distributed SQL databases in a simple and understandable way, supporting most common SQL features including joins, aggregates, and transactions. While performance and scalability aren't primary goals, a benchmark tool is included to evaluate performance under various workloads. toyDB uses Raft for consensus to manage a transactional key/value store, with a SQL query engine built on top.

Development Distributed Database

Solving the Eikonal Equation with Fast Sweeping in JAX

2025-05-11

This blog post explores solving the Eikonal equation, crucial in interface evolution and image processing, using the fast sweeping method implemented in JAX. It begins by explaining level set methods and the Eikonal equation itself. The core of the post details the fast sweeping algorithm, covering grid setup, iterative updates, and the Godunov upwind scheme. NumPy and JAX code implementations are provided, with benchmarks demonstrating JAX's significant speed advantage. The author also discusses attempts at parallelizing the algorithm and the challenges encountered.

Scraperr: Precise Web Data Extraction with XPath

2025-05-11
Scraperr: Precise Web Data Extraction with XPath

Scraperr is a self-hosted web scraping application that uses XPath selectors for precise data extraction. It offers a clean interface to manage scraping jobs, view results, and export data in various formats. Features include queue management, domain spidering, custom headers, media downloads, results visualization, and notification channels. Remember to respect robots.txt, terms of service, and implement rate limiting. Use only on websites that explicitly permit scraping.

Development

Sandboxing Libraries: A Thorny Path to Security

2025-05-11
Sandboxing Libraries: A Thorny Path to Security

This article delves into the significant challenges of building sandboxed libraries. The author explores threat modeling, memory tampering, cross-language interoperability, and code reuse. Even languages like Java struggle with complete sandboxing due to features like reflection, global methods, and native methods. Joe-E, a capability-based programming language, while severely restricting Java's functionality, offers valuable insights. However, this approach introduces the "God Object" problem, requiring a single object to control all permissions, contradicting good OOP principles. Chrome's Mojo system presents a process-based sandboxing solution, but faces challenges like unstable APIs, code reuse difficulties, and performance overhead. Ultimately, creating secure sandboxed libraries remains a complex and unsolved problem.

Development sandboxing

LSP Client in Clojure: 200 Lines of Code, Minimalist Language Server Interaction

2025-05-11

This blog post details how the author implemented a minimal LSP client in under 200 lines of Clojure code and used it to build a command-line code linter. It walks through the implementation of the base communication layer, JSON-RPC layer, and client API for the LSP protocol. The author then discusses the challenges of using LSP in practice, particularly the reliance of most language servers on notifications instead of requests for diagnostics, making a simple command-line tool more complex than expected. Finally, the author summarizes the pros and cons of LSP and speculates on the future of WASM-based language servers.

Development

Rust Devs Think We're Hopeless: Let's Prove Them Wrong (with C++ Memory Leaks!)

2025-05-11
Rust Devs Think We're Hopeless: Let's Prove Them Wrong (with C++ Memory Leaks!)

This video tutorial playfully challenges the perception of C++ developers as perpetually battling memory leaks. It demonstrates, through a simple C++ program, how to intentionally create and then effectively track down these leaks using debugging tools like `_CrtDumpMemoryLeaks`. The author presents memory leak hunting as a fun, almost reckless adventure, ultimately providing a practical guide for identifying and fixing memory issues in C++.

Development

C++ Memory Leaks: Show Rust Devs Who's Boss!

2025-05-11
C++ Memory Leaks: Show Rust Devs Who's Boss!

This video tutorial demonstrates how to intentionally create and hunt down memory leaks in C++. While Rust developers often poke fun at C++'s memory management, this tutorial shows how to use _CrtDumpMemoryLeaks and the DEBUG_NEW macro to detect and pinpoint memory leaks, proving that C++ developers can effectively handle memory issues without needing a borrow checker or lifetimes. Starting with a simple example, the tutorial progressively teaches how to use debugging tools to track down memory leaks, even in complex programs.

Development Memory Leaks

Native Windows Todo App in Pure C

2025-05-11
Native Windows Todo App in Pure C

A modern, native Windows todo application built with C and the Win32 API. It allows users to create, edit, delete, and mark todo items as complete, with persistent storage in AppData. Features include system tray integration and a native Windows look and feel. The application supports up to 100 todo items. The source code is open-source and includes build instructions.

Development Todo App

Framework-Free Web Development: A Guide to Vanilla JS

2025-05-11

This site provides a comprehensive guide to building websites and web applications using only an editor, a browser, and web standards—no build tools or frameworks required. It dives into using Web Components as fundamental building blocks, leveraging modern CSS capabilities, and creating web projects and single-page applications without build tools, frameworks, or server-side logic. Targeted at developers already familiar with HTML, CSS, and JavaScript, this tutorial champions a simpler, more maintainable approach to web development, addressing the complexity and maintenance overhead often associated with modern frameworks.

Development vanilla JS modern CSS

Visualizing Mathematical Proof: Insights from Lean Blueprints

2025-05-11

Lean Blueprints, a project planning tool inspired by mathematicians' blueprint method for formalizing proofs, offers unique insights into the proof process. It uses a graph where nodes represent definitions, lemmas, and theorems, with colors indicating their status (green: proven, blue: stated but unproven, black: unwritten). By tracking version control of Lean projects (e.g., Terrence Tao's formalization of the PFR conjecture), we can visualize the evolution of proofs, revealing how mathematicians work and providing data for building tools to assist them. This visualization offers previously hidden insights into how mathematical proofs are constructed.

Development Formal Proof

Developing iOS Apps on Non-Apple Devices: Legal Minefield

2025-05-11
Developing iOS Apps on Non-Apple Devices: Legal Minefield

The possibility of building iOS apps on non-Apple devices has long been debated. This post highlights that the primary challenge is legal and licensing, not technical. Apple's Developer Program License Agreement explicitly forbids installing, using, or running iOS SDKs or other Apple software on non-Apple devices. While the Oracle v. Google case altered the copyright landscape regarding reverse engineering for compatibility, the author argues that deploying apps built on non-Apple devices directly to the App Store carries significant legal risk, with Apple retaining the right to terminate violating developer accounts.

Faster Java Startup with AOT Cache Profile Improvements

2025-05-11

This improvement significantly reduces Java application warmup time by collecting method execution profiles during application training runs and storing them in the AOT cache. At startup in production, the JIT compiler can immediately use these profiles to generate native code, eliminating the wait for profile collection and resulting in faster startup and peak performance. This technique requires no code changes and is compatible with existing AOT cache creation commands. Experiments show a 19% reduction in warmup time for a simple example program.

Development AOT cache

GlassFlow: Real-time Streaming ETL for ClickHouse

2025-05-11
GlassFlow: Real-time Streaming ETL for ClickHouse

GlassFlow is a real-time stream processor designed for data engineers to simplify creating and managing data pipelines between Kafka and ClickHouse. It boasts a user-friendly interface for building and managing real-time data pipelines, featuring built-in deduplication and temporal joins. Handling late-arriving events and ensuring exactly-once processing, GlassFlow scales to handle high-throughput data, delivering accurate, low-latency results without sacrificing simplicity or performance. The intuitive web interface simplifies pipeline configuration and monitoring, while its robust architecture guarantees reliable data processing. It supports local development and Docker deployment, and includes a comprehensive demo setup for quick onboarding.

Development real-time processing

Rust Ecosystem Documentation Quality Review: Hits and Misses

2025-05-11
Rust Ecosystem Documentation Quality Review: Hits and Misses

This article provides an in-depth assessment of the documentation quality across numerous popular crates in the Rust ecosystem. It covers various domains, including random number generation, time handling, web frameworks, game engines, and error handling. The author evaluates each crate's documentation based on four quadrants (explanations, how-to guides, tutorials, reference) and highlights excellent examples (like `jiff`'s comprehensive documentation and design rationale) and areas for improvement (incomplete documentation or lack of practical guidance in some crates). This review offers valuable insights for Rust developers and points to directions for improving the Rust ecosystem's documentation.

Development

Roame Hiring: Lead Full-Stack Engineer to Revolutionize Travel Booking with AI

2025-05-11
Roame Hiring: Lead Full-Stack Engineer to Revolutionize Travel Booking with AI

Roame, a flight search engine leveraging credit card points and miles, is seeking a Lead Full-Stack Engineer. Backed by top-tier investors, Roame empowers millions to maximize their points' value. The ideal candidate possesses 4+ years of experience, mastery of Next.js, Firebase, and Go, and a strong understanding of AI pipelines. The San Francisco-based role requires in-person work. Roame offers competitive salary, equity, benefits, and a culture emphasizing collaboration and rapid iteration.

Development Travel Booking

Rails 8's Solid Queue: A Deep Dive into the New Background Job Processor (Part 1)

2025-05-11
Rails 8's Solid Queue: A Deep Dive into the New Background Job Processor (Part 1)

Rails 8 introduces Solid Queue, a novel background job processing library that eliminates the need for external services like Redis. This article delves into Solid Queue's architecture, explaining the interplay between Jobs and Workers and how database tables (solid_queue_jobs, solid_queue_ready_executions, solid_queue_claimed_executions, solid_queue_processes) manage job queuing, execution, and monitoring. Solid Queue achieves high performance and reliability through clever database design and the use of FOR UPDATE SKIP LOCKED statements, employing a supervisor process to prevent job loss. The article also highlights SQLite limitations and AppSignal's performance monitoring capabilities, promising a deeper dive in part two.

Development Background Jobs

arXivLabs: Experimental Projects with Community Collaborators

2025-05-11
arXivLabs: Experimental Projects with Community Collaborators

arXivLabs is a framework for collaborators to develop and share new arXiv features directly on the website. Individuals and organizations involved embrace arXiv's values of openness, community, excellence, and user data privacy. arXiv only partners with those sharing these commitments. Got an idea to enhance the arXiv community? Learn more about arXivLabs.

Development

Lazarus IDE 4.0 Released!

2025-05-11

The Lazarus team is thrilled to announce the release of Lazarus 4.0! Built with FPC 3.2.2, this release includes numerous improvements and bug fixes. Downloads are available on SourceForge (and mirrors) for Windows, Linux, and macOS. Minimum system requirements have been updated; check the official website for details. The source code is also available on Gitlab for community contributions.

Development

Running NetBSD on an Ancient i486SX: A Soft FPU Emulator is Born

2025-05-11
Running NetBSD on an Ancient i486SX: A Soft FPU Emulator is Born

A developer successfully emulated a missing floating-point unit (FPU) on an old i486SX laptop. After months of work, they successfully implemented a soft FPU emulator on NetBSD 10 and open-sourced the project. This project not only brings old 486SX machines back to life but also offers new possibilities for retrocomputing enthusiasts. It has now been successfully tested on the 86Box virtual machine.

Development

Writing an ASUS ACPI WMI Driver for OpenBSD Fan Control

2025-05-11

The author details their journey of writing an ASUS ACPI WMI driver for OpenBSD to control their laptop's fan speed. The process involved overcoming challenges with ACPI and WMI byte order, utilizing acpidump and iasl to analyze system ACPI code, and finally achieving fan speed control. The author compares the driver development approaches in Linux and FreeBSD, highlighting OpenBSD's convenient code organization.

Development Driver Development

Longtime Google Engineer Leaves After 19 Years, Reflecting on Go's Evolution

2025-05-11

A senior engineer who spent 19 years at Google, heavily involved in the development of the Go programming language, has left the company. The article reflects on their 14-year journey with Go, from contributing to the compiler frontend to shaping language features like generics and influencing the project's overall direction. The author believes Go still has significant room for growth and hopes to contribute to its future development.

Development

Adaptive Hashing in SBCL: Making Hash Tables Faster and More Robust

2025-05-11

A talk at the 2024 ELS focused on adaptive hashing, aiming to make general-purpose hash tables faster and more robust. Traditional hash table theory primarily concerns itself with asymptotic worst-case costs, neglecting the impact of constant factors on real-world performance. This research introduces an online adaptive approach, adjusting the hash function based on the actual key distribution to reduce collisions and improve cache utilization. Experiments demonstrate significant improvements in reducing expected comparisons and speeding up PUT operations, particularly with specific key distributions. SBCL's built-in hash tables now employ this technique, dynamically switching hash functions (including linear search, bit-shifting hash, and MurmurHash) based on collision counts and hash table size. For composite keys like strings and lists, a truncation strategy is used, dynamically adjusting the truncation length when too many collisions occur. This improvement enhances SBCL hash table speed in common cases and robustness in others.

Development adaptive hashing lisp

The Rise and Fall (and Rise?) of Visual Basic

2025-05-11
The Rise and Fall (and Rise?) of Visual Basic

In 1988, Alan Cooper showed Bill Gates Tripod, a groundbreaking shell construction kit for Windows. Its intuitive drag-and-drop interface allowed users to create personalized shells without deep OS knowledge. This evolved into Visual Basic, a revolutionary visual programming language that empowered countless developers, especially those transitioning from mainframes. VB's ease of use democratized programming, making it accessible to millions. However, Microsoft's shift to .NET and VB.NET, while adding features, sacrificed the simplicity that made VB so popular, leading many developers to abandon it. This story highlights the tension between innovation and business decisions, and the lasting impact of a seemingly simple tool.

Development

OpenSSL 3.0 Performance Disaster and the Future of the SSL Library Ecosystem

2025-05-11
OpenSSL 3.0 Performance Disaster and the Future of the SSL Library Ecosystem

The HAProxy team delves into the performance disaster caused by OpenSSL 3.0 and its impact on the SSL library ecosystem. The release of OpenSSL 3.0 resulted in significant performance degradation in multi-threaded environments, posing challenges for many projects reliant on OpenSSL. The article compares alternative solutions like BoringSSL, LibreSSL, WolfSSL, and AWS-LC, analyzing their trade-offs in functionality, performance, and maintenance. Performance testing reveals that OpenSSL 3.0 significantly underperforms compared to other libraries in multi-threaded scenarios, forcing organizations to provision more hardware to maintain throughput. The article also explores the QUIC protocol and its relationship with SSL libraries, along with the OpenSSL team's handling of the QUIC API. Ultimately, the article recommends HAProxy users choose suitable SSL libraries based on their needs, such as AWS-LC or WolfSSL, and calls for the community to collaboratively address the performance issues in OpenSSL 3.0.

Development SSL Libraries

The Misunderstood Origins of OOP: Setting the Record Straight

2025-05-10

This article debunks the common misconception that Alan Kay invented both object-oriented programming (OOP) and the concept of objects. In reality, the Simula language was a significant precursor to OOP, a fact acknowledged by the Smalltalk team. While Kay coined the term "objects," he later regretted it, emphasizing message passing as the core idea of OOP. The article further explores Smalltalk's unique contributions to OOP, particularly its design principle of treating everything as an object and its innovative message-handling mechanism. The author argues that OOP's evolution wasn't a singular event but a confluence of ideas and needs, with different developers understanding and applying OOP differently in various contexts.

Development

LoopMix128: Blazing Fast and Robust 2^128 Period PRNG

2025-05-10
LoopMix128: Blazing Fast and Robust 2^128 Period PRNG

LoopMix128 is an extremely fast pseudo-random number generator (PRNG) with a guaranteed period of 2^128, proven injectivity, and clean passes in both BigCrush and PractRand (32TB). Designed for non-cryptographic applications where speed and statistical quality are paramount, it significantly outperforms standard library generators and rivals or surpasses modern high-speed PRNGs like wyrand and xoroshiro128++. Its performance is backed by rigorous testing, passing BigCrush and PractRand with zero anomalies, and boasting a proven 192-bit injective state enabling parallel streams.

Drawing the Sierpinski Triangle with Bitwise Operations: A Stunning Bit Twiddling Hack

2025-05-10
Drawing the Sierpinski Triangle with Bitwise Operations: A Stunning Bit Twiddling Hack

This article unveils a stunning bit manipulation trick: generating the famous Sierpinski triangle fractal using only a simple bitwise AND operation (&). The author meticulously breaks down the bitwise operation, revealing the underlying mathematical principles. It shows how the inherent fractal nature of binary counting and iterative block removal, achieved through bitwise manipulation, generates the classic Sierpinski triangle. This technique cleverly leverages the binary operation capabilities of computers, simplifying the seemingly complex process of generating graphics into concise code, resulting in an astonishingly elegant solution.

Development

arXivLabs: Experimenting with Community Collaboration on New Features

2025-05-10
arXivLabs: Experimenting with Community Collaboration on New Features

arXivLabs is a framework for collaborators to develop and share new arXiv features directly on the website. Individuals and organizations involved share arXiv's values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners who adhere to them. Have an idea to enhance the arXiv community? Learn more about arXivLabs.

Development

Unique Games Conjecture: A Surprisingly Divisive Problem in Computational Complexity

2025-05-10

Proposed by Subhash Khot in 2002, the Unique Games Conjecture (UGC) posits that approximating the value of a specific type of game, known as a unique game, is NP-hard. This conjecture has significant implications for the theory of approximation algorithms; if true and P≠NP, many crucial problems wouldn't allow for good polynomial-time approximations, not just exact solutions. The academic community is split on its validity, with equivalent formulations including label cover and Max2Lin(k) problems. While stronger versions have been disproven, the UGC's exploration has spurred substantial mathematical research, and some progress towards proving it has been made, including proving a related conjecture, the 2-2 games conjecture.

Flutter Local-First Architecture: A Guide to Building Offline-First Apps

2025-05-10
Flutter Local-First Architecture: A Guide to Building Offline-First Apps

This article explores Flutter's local-first application architecture, prioritizing local data storage and synchronization for superior user experiences. Unlike traditional online-first approaches, local-first architecture designates the local database as the primary data source, ensuring app functionality even offline. The article details the advantages of local-first architecture, the challenges of building a sync engine (including change tracking, conflict resolution, edge cases and error handling, and performance optimization), and demonstrates building a Todo app with Riverpod, Drift, and PowerSync connected to a Supabase backend. These tools simplify building robust offline-capable apps, enhancing user experience.

Development Offline App
1 2 91 92 93 95 97 98 99 214 215