Category: Development

Stubborn Feed Readers Bring Down Website

2024-12-22

A blogger experienced website unavailability, tracing it not to carriers or hosting, but to misbehaving feed reader software. These readers ignore best practices, sending unconditional requests and ignoring 429 errors (too many requests), ultimately causing the server to defensively shut down. The blogger resorted to a blog post urging users to check their feed readers, offering a tool called "Feed Reader Score" to analyze reader behavior and resolve the issue.

Tao's New Paper: Delving into Eigenvalue Distribution of GUE and its Minors

2024-12-22

In his latest arXiv preprint, renowned mathematician Terence Tao delves into the distribution of eigenvalues of the Gaussian Unitary Ensemble (GUE) and its minors at fixed indices. Employing determinantal processes and sophisticated analytical techniques, the paper establishes several estimates regarding eigenvalue gaps, addressing previously unanswered questions and paving the way for future work on the limiting behavior of 'hives' with GUE boundary conditions. This research significantly contributes to the understanding of random matrix models and related fields.

Ethereum On-Chain Upgrade Signaling EIP: Community Consensus Drives Upgrades

2024-12-22

This proposal introduces an on-chain mechanism allowing Ethereum clients to signal their readiness for protocol upgrades by embedding a "vote" indicator in the extraData field of newly mined blocks. Future fork activation blocks only occur if enough blocks within a specified window signal "for" the upgrade. This shifts upgrade decisions from the Ethereum Foundation to network participants, enhancing decentralization and mitigating centralization risks. The proposal also addresses trademark concerns and potential security vulnerabilities.

Minimizing Wifi Battery Drain in IoT Projects

2024-12-22

In the early development of PetDrifts, an IoT device using an ESP32 C3, battery life was a major challenge, lasting only a day. The culprit was identified as high power consumption during HTTP POST requests every 15 minutes. To address this, the team explored two solutions: switching from TCP to UDP to reduce network overhead, and adopting MQTT for persistent connections and lower power usage. While improvements were observed, the battery life wasn't sufficient. Ultimately, they opted for a Bluetooth MCU to achieve significantly longer battery life.

Development IoT Power Optimization

SQLite-Backed Key-Value Store with JS-Like Object Manipulation

2024-12-22

A GitHub project introduces a key-value store built on SQLite, enabling JavaScript-like object manipulation with automatic JSON serialization. The `createDatabaseClient` function creates a parallel client with separate reader (`rdr`) and writer (`wtr`) components. The writer utilizes proxies for partial JSON updates, while the reader returns plain JavaScript objects. Comprehensive tests cover basic CRUD operations, nested updates, deletions, and array manipulations.

Nix Home Manager: A Guide to Dotfiles Management

2024-12-22

This article delves into the advantages and techniques of using Nix Home Manager for dotfiles management. The author begins by acknowledging the steep learning curve of Nix and Home Manager, recommending a gradual approach to mastering its features. The article explains various Home Manager use cases, including software installation, declarative program and service configuration, and dotfiles management. A key focus is on the `mkOutOfStoreSymlink` function, which creates symlinks to dotfiles, allowing modifications without rebuilding the entire system. A custom module example is provided for easy switching between mutable and immutable configurations. Finally, the author compares Home Manager to other dotfiles management tools, highlighting its reproducibility benefits.

Development

GCC Build Failure: A Debugging Mystery Caused by sbuild Refactoring

2024-12-22

Official Debian GCC builds started failing mysteriously after an sbuild refactoring. A team embarked on a six-stage investigation, ultimately uncovering a conflict between the new sbuild init system and a D language unit test. The test accidentally terminated its own process group, halting the build. The root cause was the use of -2 as a special PID value in the D language unit test, leading to SIGTERM signals being sent to the wrong process group. Switching back to the old init system or modifying the test code resolved the issue.

Development debugging

The Enduring Legacy of NeXT in OS X and iOS

2024-12-22

This article explores the profound impact of NeXT and its NeXTSTEP operating system on Apple's OS X and iOS. NeXTSTEP's UNIX foundation brought crucial features like protected memory, preemptive multitasking, and daemons, enhancing stability and efficiency. It also introduced the Objective-C programming language and Cocoa framework, simplifying software development and giving rise to powerful tools like Interface Builder. Furthermore, NeXTSTEP's Display PostScript technology laid the groundwork for OS X's Quartz graphics system. These technologies remain core to Apple devices today, highlighting NeXT's significant contribution to modern computing.

Development

Slow Deployments Breed Meetings: A Reverse Causality Argument

2024-12-22

Programmers often complain about too many meetings hindering productivity. Kent Beck challenges this notion, suggesting that meetings are a consequence, not the cause, of slow deployments. Facebook's experience shows that increasing deployment frequency is key. When deployment speed lags behind code changes, organizations add meetings and reviews to mitigate risk, ultimately reducing efficiency. Instead of reducing meetings, focus on improving deployment capacity by shortening cycles or enhancing code quality. This essay offers a fresh perspective, exploring the counter-intuitive relationship between slow deployments and increased organizational overhead.

Improving F# Error Handling: Introducing FaultReport

2024-12-22

This article critiques the shortcomings of F#'s Result type in error handling, highlighting inconsistencies in error types and the problems stemming from using strings as error types. The author proposes FaultReport as an alternative, using an IFault interface to standardize error types and a Report<'Pass', 'Fail> type to represent operation outcomes, where 'Fail must implement IFault. This ensures consistent and type-safe error handling, avoiding the inconveniences of string-based errors. FaultReport further provides Report.generalize for upcasting and a FailAs active pattern for downcasting, facilitating handling of diverse error types. While replacing FSharp.Core's Result is a significant undertaking, the author argues that FaultReport's design offers a valuable improvement to F#'s error handling.

Development

GitHub Assistant: Explore GitHub Repositories with Natural Language

2024-12-22

GitHub Assistant is a proof-of-concept project that lets users explore GitHub repositories using natural language questions. Built with Relta and assistant-ui, it allows users to ask questions in plain English and receive relevant repository information. The Relta sub-module is currently closed source but available upon request. Requires Python 3.9+, npm, Git, and configuration of an OpenAI API key and database connection URI.

Development Code Search

Rosetta 2 Creator Joins Lean FRO to Enhance Code Generator

2024-12-22

Leonardo de Moura, Senior Principal Applied Scientist at AWS and Chief Architect at Lean FRO (a non-profit), announced that Cameron Zwarich, the brilliant creator of Rosetta 2 and an exceptional software developer with over 15 years of experience at Apple specializing in low-level systems software, has joined the Lean FRO team. Zwarich will focus on improving Lean's code generator, promising a significant impact on the Lean ecosystem.

Development Code Generator

Revolutionizing Workflow: The Power of a Public CHANGELOG

2024-12-22

AWS engineer Daniel Doubrovkine shares his experience with maintaining a public CHANGELOG of his work. By openly documenting his weekly tasks, he fosters transparency and collaboration. This practice has yielded significant benefits: more productive 1:1s, smoother onboarding for new engineers, easy access to past work, enhanced self-reflection, and increased trust among colleagues. He encourages others to adopt this approach and shares his simple logging method along with a Ruby script for generating a yearly table of contents.

Development work log

A Curious Case of Slow USD Import in Blender

2024-12-22

A developer encountered unexpectedly slow import times when importing USD scenes into Blender. Profiling revealed the bottleneck to be Blender's internal ID sorting function, `id_sort_by_name`. This function, expected to be O(N), degraded to O(N^2) due to the naming scheme in the USD files. By modifying the naming convention and optimizing the sorting algorithm, the developer reduced import times from 4 minutes 40 seconds to 8 seconds for smaller files. However, the underlying issue stems from Blender's requirement for sorted IDs, leading to suggestions for replacing the linked list with a Trie or hash table. This optimization highlights a common challenge in performance tuning: identifying and addressing unexpected complexity.

Development

cqd: A Colorful Python Utility for Inspecting Object Attributes

2024-12-22

cqd is a lightweight Python utility that provides a colorful visualization of object attributes, simplifying object inspection during development and debugging. It color-codes attributes: dunder methods (blue), protected attributes (yellow), and public attributes/methods (green). For example, it's useful for easily viewing attributes of a Hugging Face tokenizer. Installation is easy via `pip install cqd`. Usage involves importing the `cqd` function and calling `cqd(your_object).

Java JEP 483: Ahead-of-Time Class Loading & Linking Boosts Startup Time

2024-12-22

JEP 483 significantly improves Java application startup time by loading and linking application classes ahead of time when the HotSpot JVM starts. It achieves this by monitoring a single application run, storing the loaded and linked forms of all classes in a cache for reuse in subsequent runs. This feature requires no code changes and offers substantial speed improvements for large server applications, such as Spring PetClinic showing a 42% reduction in startup time. While currently a two-step process, future versions will streamline cache creation to a single step and offer more flexible training run configuration.

Development

Meta's Massive Java-to-Kotlin Translation: Conquering Millions of Lines of Code

2024-12-22

Meta has undertaken a multi-year effort to translate its massive Android codebase from Java to Kotlin. This post details how Meta built the Kotlinator, an automation tool, to overcome challenges like slow build speeds and insufficient linters, successfully converting over half of its code. The Kotlinator comprises several phases: preprocessing, headless J2K conversion, postprocessing, and error fixing. Meta also collaborated with JetBrains to improve J2K and open-sourced parts of the process to foster community collaboration. The article highlights null safety handling and various code issues encountered and resolved during the conversion.

Development code migration

A Wall Conversation Changed My Programming Career

2024-12-21

In 1983, a programmer working at a large defense contractor planned to pursue a Ph.D. in Chemistry. A chance conversation over a wall with the manager of the neighboring "Microcomputer Group" (a tinkerer) led to an invitation to a meeting about Apple II. There, he was tasked with building a VT-100 terminal emulator in 6502 assembly language within a week to enable the company president to read email at home. This experience not only redirected his career path, leading him to join the Microcomputer Group and become the company's sole PC programmer, but also ultimately led him to start his own company. Years later, he reflected on how chance encounters and interpersonal connections significantly shaped his life.

Development career opportunity

Software Design Philosophy: Taming Complexity

2024-12-21

This post summarizes three key ideas from the book "A Philosophy of Software Design": zero tolerance for complexity, the misconception that smaller components always equate to better modularity, and the complexities inherent in exception handling. The author argues that complexity isn't caused by single errors but accumulates over time. Examples of an order processing system and user registration illustrate how to avoid duplicated code and find the right balance between component size and modularity. Furthermore, the post details three techniques to reduce exception handling complexity: eliminating errors, masking exceptions, and exception aggregation, with file processing serving as an example. The book ultimately emphasizes the importance of consistently simplifying complexity in software design.

Enum of Arrays: A Novel Data Structure for Efficient Data Processing

2024-12-21

This article introduces a data structure called "Enum of Arrays" (EoA), similar to the popular "Struct of Arrays" (SoA), but with enums at its core. EoA packs multiple enum values into an array, using a single tag to identify the array's type. This reduces memory usage and branch prediction overhead, leading to more efficient data processing, particularly beneficial for SIMD optimization. The article uses the database system TigerBeetle as an example, illustrating how EoA enables efficient batch processing by effectively separating the control plane and data plane, resulting in significantly improved performance.

Yakari: Interactive Command Builder Simplifies Complex CLIs

2024-12-21

Yakari is an interactive command-building tool designed to simplify complex command-line interfaces. It guides users through command construction step-by-step, eliminating the need to memorize complex syntax. Supporting various argument types, Yakari offers contextual help and command history, significantly improving CLI usability. Users can build and execute commands with simple shortcuts, making even intricate commands accessible.

Development interactive interface

Go Iterators: Efficiently Handling Paginated APIs

2024-12-21

This article demonstrates how to efficiently handle paginated APIs using the iterator feature introduced in Go 1.23. Using the GitHub API as an example, the author shows how to write a custom iterator to abstract pagination logic, making the code more readable and reusable. The article focuses on the implementation and testing of the iterator, including mocking API calls and using pull iterators to ensure the iterator returns the expected results. Iterators allow developers to separate pagination logic from business logic, improving code maintainability and readability.

The CD Pipeline Manifesto: Building Better Software Delivery

2024-12-21

Modern software teams desperately need better tools for managing their Continuous Delivery pipelines. Today's CD pipeline ecosystem is fragmented, rigid, and inefficient. This manifesto advocates for code-first, developer-friendly pipelines designed to handle the complexities of modern engineering workflows. It emphasizes a single source of truth, reusable and typesafe components, dynamic and flexible pipelines, transparent and visual debugging, and mechanisms for handling change and fast feedback loops, ultimately aiming to improve efficiency and accelerate delivery.

Rivet: Run and Scale Realtime Applications with Actors

2024-12-21

Rivet is a platform for building and scaling real-time applications using the Actor model. It features built-in RPC, state, and events, simplifying modern application development. Rivet boasts automatic scaling, edge network deployment, and includes built-in monitoring and data localization capabilities. Powered by Rust, FoundationDB, V8 isolates, and the Deno runtime, it ensures performance and efficiency. Rivet is suitable for collaborative applications, local-first apps, AI agents, game servers, and more.

SingleFile: Save Entire Webpages as Single HTML Files

2024-12-21

SingleFile is a powerful web extension and CLI tool that saves complete web pages as a single HTML file. Compatible with Chrome, Firefox, Edge, and more, it offers convenient page saving, multi-tab processing, annotation capabilities, and even allows uploading saved pages to Google Drive or GitHub. Customize shortcuts and settings to tailor it to your needs.

Development webpage saving

Efficient German Language Learning: Is Anki the Answer?

2024-12-21

An engineer living in Germany for eight years confesses to still not knowing the language. To remedy this, they're trying Anki, leveraging spaced repetition to learn 10 new German words daily – aiming for C1 level proficiency within a year. They chose a frequency-ordered Anki deck, adding audio pronunciations themselves. The author invites readers to share their Anki experiences and German learning tips.

Saying Goodbye to C String Vulnerabilities: A Safer String Handling Approach

2024-12-21

Tired of C string vulnerabilities and insecurity? This article introduces a clever alternative: a custom string struct `struct str`, which contains a data pointer and length, avoiding the risks associated with null termination. The author's six-month experience in a bare-metal environment demonstrates that this approach effectively prevents errors such as buffer overflows. While using the macro `STR` is slightly verbose, the increase in safety and readability far outweighs this. Compiler optimizations also make the performance loss negligible, offering a new approach for developers prioritizing code security.

How an AI Code Review Bot Learned to Shut Up

2024-12-21

Greptile's AI code review bot initially faced criticism for generating excessive comments. To address this, they experimented with prompt engineering and having the LLM evaluate its own comments, but these methods proved ineffective. Their breakthrough came from vectorizing past comments, clustering them in a vector database, and filtering out new comments similar to those previously downvoted. This approach boosted the developer address rate from 19% to over 55%, significantly reducing LLM noise.

Development Code Review

Aegisub Subtitle Editor 3.4.0 Released!

2024-12-21

Aegisub 3.4.0 has been released! This free, cross-platform, open-source tool makes creating and editing subtitles quick and easy. It features powerful styling tools and a built-in real-time video preview. Perfect for both seasoned subtitlers and newcomers.

Development subtitle editor

Implementing Raft: A Deep Dive into Distributed Consensus

2024-12-21

This is the first post in a series detailing the Raft distributed consensus algorithm and its Go implementation. Raft solves the problem of replicating a deterministic state machine across multiple servers, ensuring service availability even with server failures. The post introduces core Raft components: the state machine, log, consensus module, leader/follower roles, and client interaction. It discusses Raft's fault tolerance, the CAP theorem, and the choice of Go as the implementation language. Subsequent posts will delve into the algorithm's implementation.

Development Distributed Consensus
← Previous 1 3 4 5 6 7 8 9