arXivLabs is a framework enabling collaborators to develop and share new arXiv features directly on the website. Individuals and organizations involved share arXiv's commitment to openness, community, excellence, and user data privacy. Got an idea to enhance the arXiv community? Learn more about arXivLabs.
Computer scientists have cracked the 'bookshelf problem' (list labeling problem), a decades-old challenge in efficiently inserting new data into sorted data structures. Researchers developed a new algorithm that approaches the theoretical lower bound, achieving a significant breakthrough in insertion cost. This advance has the potential to challenge the dominance of binary search trees in data management, revolutionizing how we handle massive datasets.
In 2022, Philip Thomas launched Postcard, a personal website and newsletter tool, as a replacement for social media to stay connected with friends. Postcard gained thousands of users, and despite modest revenue, the author continues to maintain it. Now, the author is open-sourcing the code, allowing developers to contribute and customize. Postcard is a simple Ruby on Rails application, supporting both single-user and multi-user modes. Deployment is straightforward with a Dockerfile and render.yaml for easy deployment on Render.
The Linux errno(3) man page reveals a peculiar limitation: errno can be modified even on successful function calls, and it's never set to zero by any system call or library function. This stems from traditional Unix design where system calls typically return errno only on failure, leaving it unchanged on success. C library functions might make multiple system calls, some of which could fail without affecting the library function's overall success, leaving errno with the failure value. ANSI C and POSIX inherited this behavior, requiring errno to be meaningful only when a function fails and its documentation specifies setting errno.
The conventional wisdom is that caching speeds up software. The author argues this is only part of the story. After working with data movement between object storage, disk, and memory, the author posits that caching's more crucial role is simplifying software. The article explores the limitations of pre-baked caching algorithms (LRU, LFU, etc.) and suggests caching acts more as an abstraction layer, hiding the underlying storage details, freeing programmers from worrying about data tier location. Database and OS caching mechanisms exemplify this abstraction. While caching can have issues, like OS page cache and fsync misuse, this doesn't necessitate abandoning caching but rather understanding and using it better.
Pennybase is a lightweight Backend-as-a-Service (BaaS) solution implemented in under 1000 lines of Go code, offering a simplified alternative to Firebase, Supabase, and Pocketbase. It relies solely on the Go standard library, requiring no external dependencies. Core features include file storage (versioned CSV), a REST API (JSON responses), session cookie and Basic Auth authentication, RBAC & ownership-based permissions, real-time updates via SSE, schema validation, and Go template rendering. Data is stored in human-readable CSVs, with updates creating new record versions. A clever in-memory index allows for fast lookups and updates. Permission control uses a simple RBAC model, and custom functionality is possible via hook functions.
Applying code suggestions in bulk during GitHub code review has several limitations. These include: invalid suggestions (no code changes), closed pull requests, viewing a subset of changes, only one suggestion per line in a batch, inability to apply suggestions to deleted lines, suggestions already applied or marked resolved, suggestions from pending reviews, suggestions on multi-line comments, inability to apply while the pull request is queued to merge, and an unspecified 'cannot apply suggestion right now' error.
Wasm-agents aims to revolutionize open-source agent accessibility by packaging them as HTML files runnable directly in your browser, eliminating the need for external tools or frameworks. Leveraging WebAssembly and Pyodide, this project enables near-native speed execution of Python-based AI agents within a browser sandbox. Currently supporting OpenAI APIs and self-hosted models (like those served via Ollama), Wasm-agents provides example HTML files demonstrating simple conversational agents, multi-agent systems, and advanced agents with built-in tools. While limitations exist, such as reliance on the openai-agents framework and CORS considerations, this project offers a novel approach to AI agent development and sharing, ripe for exploration.
Over six months, the author painstakingly ported the ~67,000 lines of C code behind tmux, the terminal multiplexer, to Rust, resulting in a ~81,000 line codebase. Initial attempts using the C2Rust transpiler proved unwieldy, leading to a manual rewrite. The author details the build process, encountered bugs, and strategies for converting C idioms to Rust, including handling raw pointers, goto statements, and intrusive macros. The project culminates in a lalrpop-based reimplementation of the yacc parser, achieving a complete C-to-Rust migration and culminating in a 0.0.1 release. The journey showcases the challenges and rewards of large-scale code porting.
This article introduces the principle of Locality of Behavior (LoB), which emphasizes that the behavior of a code unit should be readily apparent within that unit itself. The author uses examples of AJAX requests in htmx and jQuery to illustrate how LoB improves code maintainability. While LoB may conflict with principles like DRY (Don't Repeat Yourself) and SoC (Separation of Concerns), the author argues that judiciously prioritizing LoB enhances code readability and maintainability, ultimately leading to higher software quality and sustainability.
arXivLabs is a framework for collaborators to build and share new arXiv features directly on the website. Individuals and organizations involved share arXiv's values of openness, community, excellence, and user data privacy. arXiv only partners with those who uphold these values. Got an idea to improve the arXiv community? Learn more about arXivLabs.
uv's native build backend, uv_build, significantly improves the speed and user experience of building Python projects. It features sensible defaults, aiming for zero configuration for most users, while offering flexible configuration to accommodate diverse project structures. uv_build currently supports pure Python code; alternative backends are needed for libraries with extension modules. Use this backend by adding `uv_build` to your `pyproject.toml` or by creating a new project with `uv init --build-backend uv`. uv_build also optimizes package name normalization, module discovery, and file inclusion/exclusion strategies, leading to more predictable and repeatable builds.
This article challenges the practicality of Multi-Component Pipelines (MCPs) for many tasks, arguing that their heavy reliance on inference makes them inefficient and difficult to scale. The author uses a personal example – converting reStructuredText to Markdown – to demonstrate a superior approach: using LLMs to generate code that performs the task, followed by LLM-based validation. This method reduces inference dependency, enhances reliability, and scales well, especially for repetitive tasks. While acknowledging MCP's strengths in niche scenarios, the author concludes that its inherent limitations hinder large-scale automation. The future, they suggest, lies in developing more effective code generation techniques coupled with LLM validation and explanation to improve usability and applicability.
This story recounts the tale of Max, a programmer, and his Imagebin, a 15-year-old PHP image upload script. Imagebin's code is remarkably simple, a mere 233 lines, mostly changelog. The author attempted a Go rewrite, only to find the codebase ballooned and readability suffered. This led to a reflection on the complexity of software design, highlighting the ease of maintenance provided by concise code. Max's Imagebin's longevity is attributed to this simplicity. Ultimately, the author decided to keep Max's code and stick with PHP.
This post details the author's experience building a Linux kernel (RISC-V architecture) natively on macOS. The journey was fraught with challenges: an outdated make version, macOS's ld being incompatible with Linux expectations, missing elf.h and byteswap.h headers, and version conflicts with uuid_t and sed. The author overcame these by using Homebrew to install updated toolchains (make, llvm, lld, libelf, gnu-sed) and creating patches to address header file discrepancies and type definition conflicts. The kernel was successfully built, with macOS showing faster build times than Docker for clean builds, but slower incremental builds. The author concludes that fixing build issues is an ongoing, iterative process similar to continuous integration.
This post summarizes frequently asked questions encountered while teaching 700+ engineers and product managers an AI evaluation course. Topics covered include whether RAG is dead, model selection, annotation tools, evaluation methodologies, synthetic data generation, and gaps in existing evaluation tooling. The authors stress the importance of error analysis, advocating for binary evaluations over Likert scales, and sharing best practices for building custom annotation tools, choosing appropriate chunk sizes, and evaluating RAG systems. The post also discusses the differences between guardrails and evaluators, minimum viable evaluation setup, evaluating agentic workflows, and the different uses of evaluations in CI/CD versus production monitoring.
This article delves into the nuances of JavaScript's three function declaration styles: function declarations, function expressions, and arrow functions. Function declarations are hoisted; function expressions can be named, but their names aren't bound to the scope; arrow functions are concise but lack `this` binding, constructor capabilities, and generator functionality. The choice depends on whether you're using the `this` keyword, need generator functions, and prioritize code readability. A flowchart is provided to guide developers in choosing the appropriate function declaration style for different situations.
A young programmer's anxiety sparks a reflection on the future of coding careers in the age of AI. The author, drawing on 28 years in tech, addresses the anxieties surrounding AI-driven job displacement. The argument posits that AI will not replace programmers, but rather augment their abilities, freeing them to focus on creative problem-solving. The future programmer will need to master context, effectively guide AI tools, and remain a lifelong learner to avoid stagnation. The core value of a programmer – problem-solving, critical thinking, and uniquely human ingenuity – remains irreplaceable by AI.
The advent of Large Language Models (LLMs) has dramatically increased code generation speed. However, this hasn't solved the real bottlenecks in software engineering: code reviews, knowledge transfer, testing, debugging, and team collaboration. The article argues that LLMs lower the cost of writing code, but the cost of understanding, testing, and trusting that code is higher than ever. LLM-generated code can be hard to understand, violate established conventions, or introduce unintended side effects, increasing review and maintenance difficulty. Ultimately, software engineering still relies on team trust, shared context, and meticulous code review—these are the keys to efficiency.
The author envisions a novel software development paradigm: using Large Language Models (LLMs) as compilers. Instead of directly writing code, engineers provide context (requirements, test cases, etc.) to the LLM, which iteratively generates and integrates code. Engineers focus on context and testing, while the LLM handles code generation and integration. This lowers the barrier to entry, increases development speed, and potentially revolutionizes software engineering. While LLM outputs aren't perfectly predictable, testing and iteration ensure reliability. The core idea is treating code as an intermediate layer; the goal is functionality, not perfect code.
This article explores the evolution of Go caching libraries. Early Go cache libraries suffered from concurrency and performance bottlenecks. Ristretto emerged as a leader, offering high throughput and good hit rates, but its design flaws eventually led to decreased hit rates and suboptimal performance under specific loads. Theine and Otter v1 followed, attempting to address Ristretto's shortcomings, each with its own strengths and weaknesses. Finally, the author developed Otter v2, which incorporates lessons learned from its predecessors and borrows from Java's Caffeine library to achieve high throughput, high hit rates, and rich features, making it a top contender among Go caching libraries.
Engineers encountered a difficult-to-reproduce bug causing file copy failures (-EACCES) in Gord, a critical trading data system. Disabling Kerberos resolved the issue, pointing to authentication problems. Investigation revealed the kernel obtains Kerberos credentials via the rpc_gssd daemon, but logs showed no anomalies. Extensive testing, including creating an in-memory fake filesystem and using bpftrace for kernel tracing, finally pinpointed the issue: high NFS server load caused request retransmissions. The kernel mishandled requests/responses with identical XIDs but different GSS sequence numbers, leading to checksum mismatches and errors. The engineer fixed the kernel to prevent immediate retransmission due to sequence number mismatches.
This beginner-friendly post explores ten captivating features of the D programming language. From smaller quality-of-life improvements to major features like automatic constructors, design by contract, compile-time function execution (CTFE), and built-in unit testing, the article provides clear explanations. D's powerful metaprogramming capabilities are also highlighted, rivaling few statically compiled languages in flexibility and modeling power. The post also covers unique D syntax features such as the dollar operator, parenthesis omission, and uniform function call syntax (UFCS), significantly improving code readability and efficiency. Additionally, D supports scoped and selective imports and a built-in documentation generator, further enhancing code maintainability and readability. In short, D offers a compelling blend of features for efficient and convenient programming.
Hugo, an expert advising teams at Netflix, Meta, and the US Air Force on building LLM-powered systems, argues that many teams prematurely adopt AI agents, resulting in complex, hard-to-debug systems. He contends that simpler workflows like chaining, parallel processing, routing, and orchestrator-worker patterns are often more effective than complex agents. Agents are only the right tool when dealing with dynamic workflows requiring memory, delegation, and planning. The author shares five LLM workflow patterns and emphasizes the importance of building observable and controllable systems. Avoid agents for stable enterprise systems; they are better suited for human-in-the-loop scenarios.
Gmailtail is a command-line tool for real-time monitoring of Gmail messages, outputting them as JSON. It offers flexible filtering (sender, subject, labels, attachments, etc.), multiple output formats (JSON, JSON Lines, compact), OAuth2 and service account authentication, and checkpointing. Users can customize monitoring rules via simple command-line arguments or a YAML configuration file, making it ideal for automation, monitoring, and integration with other tools.
This post unveils a clever trick using the XOR bitwise operator to solve several common interview problems. XOR possesses unique properties: identical numbers XOR to 0, different numbers XOR to a non-zero value, and the order of operations doesn't affect the outcome. Leveraging these properties, you can elegantly find missing or duplicate numbers in an array, even two missing/duplicate numbers, without resorting to complex algorithms or data structures. The post progressively explains applications of the XOR trick with code examples, offering an efficient and concise problem-solving approach.
Launched in 2004 under the MIT License, Ruby on Rails revolutionized web development with its freedom, flexibility, and strong community. Its 'convention over configuration' philosophy, ActiveRecord ORM, and powerful scaffolding tools drastically improved developer productivity, profoundly influencing subsequent frameworks like Laravel, Django, and Phoenix. Rails proved open-source could compete with, and surpass, commercial alternatives, setting a model for others and continuing to shape web development.
Inspired by the challenges of Intellivision game development, the author created IntyBASIC, a BASIC compiler for the Intellivision console. Launched in 2014, IntyBASIC translates BASIC source code into CP1610 assembly, supporting Intellivision's video and sound capabilities. Its success sparked a surge of new Intellivision games and homebrews, even leading to a programming contest. The author also penned two books sharing game development insights, covering basic to advanced techniques with complete game source code included.
A browser-based visual tool for designing, testing, and exporting chatbot conversation flows as JSON. Add nodes, edit content, test the flow, and import/export JSON. The exported JSON is compatible with any chatbot application. Installation options include: installing as a dev dependency, adding to project scripts, or global installation.
An osu! game developer reported a bizarre issue in Windows 11 Canary build (27881.1000) where `Math.Pow(-1, 2)` in .NET 8 unexpectedly returns -1 instead of 1. The problem also occurs in C++'s `std::pow()`, but works correctly in Python. The developer has filed a GitHub issue and suggests joining the osu! Discord server for further details.