Category: Development

GitHub Assistant: Explore GitHub Repositories with Natural Language

2024-12-22

GitHub Assistant is a proof-of-concept project that lets users explore GitHub repositories using natural language questions. Built with Relta and assistant-ui, it allows users to ask questions in plain English and receive relevant repository information. The Relta sub-module is currently closed source but available upon request. Requires Python 3.9+, npm, Git, and configuration of an OpenAI API key and database connection URI.

Development Code Search

Rosetta 2 Creator Joins Lean FRO to Enhance Code Generator

2024-12-22

Leonardo de Moura, Senior Principal Applied Scientist at AWS and Chief Architect at Lean FRO (a non-profit), announced that Cameron Zwarich, the brilliant creator of Rosetta 2 and an exceptional software developer with over 15 years of experience at Apple specializing in low-level systems software, has joined the Lean FRO team. Zwarich will focus on improving Lean's code generator, promising a significant impact on the Lean ecosystem.

Development Code Generator

Revolutionizing Workflow: The Power of a Public CHANGELOG

2024-12-22

AWS engineer Daniel Doubrovkine shares his experience with maintaining a public CHANGELOG of his work. By openly documenting his weekly tasks, he fosters transparency and collaboration. This practice has yielded significant benefits: more productive 1:1s, smoother onboarding for new engineers, easy access to past work, enhanced self-reflection, and increased trust among colleagues. He encourages others to adopt this approach and shares his simple logging method along with a Ruby script for generating a yearly table of contents.

Development work log

A Curious Case of Slow USD Import in Blender

2024-12-22

A developer encountered unexpectedly slow import times when importing USD scenes into Blender. Profiling revealed the bottleneck to be Blender's internal ID sorting function, `id_sort_by_name`. This function, expected to be O(N), degraded to O(N^2) due to the naming scheme in the USD files. By modifying the naming convention and optimizing the sorting algorithm, the developer reduced import times from 4 minutes 40 seconds to 8 seconds for smaller files. However, the underlying issue stems from Blender's requirement for sorted IDs, leading to suggestions for replacing the linked list with a Trie or hash table. This optimization highlights a common challenge in performance tuning: identifying and addressing unexpected complexity.

Development

cqd: A Colorful Python Utility for Inspecting Object Attributes

2024-12-22

cqd is a lightweight Python utility that provides a colorful visualization of object attributes, simplifying object inspection during development and debugging. It color-codes attributes: dunder methods (blue), protected attributes (yellow), and public attributes/methods (green). For example, it's useful for easily viewing attributes of a Hugging Face tokenizer. Installation is easy via `pip install cqd`. Usage involves importing the `cqd` function and calling `cqd(your_object).

Java JEP 483: Ahead-of-Time Class Loading & Linking Boosts Startup Time

2024-12-22

JEP 483 significantly improves Java application startup time by loading and linking application classes ahead of time when the HotSpot JVM starts. It achieves this by monitoring a single application run, storing the loaded and linked forms of all classes in a cache for reuse in subsequent runs. This feature requires no code changes and offers substantial speed improvements for large server applications, such as Spring PetClinic showing a 42% reduction in startup time. While currently a two-step process, future versions will streamline cache creation to a single step and offer more flexible training run configuration.

Development

Meta's Massive Java-to-Kotlin Translation: Conquering Millions of Lines of Code

2024-12-22

Meta has undertaken a multi-year effort to translate its massive Android codebase from Java to Kotlin. This post details how Meta built the Kotlinator, an automation tool, to overcome challenges like slow build speeds and insufficient linters, successfully converting over half of its code. The Kotlinator comprises several phases: preprocessing, headless J2K conversion, postprocessing, and error fixing. Meta also collaborated with JetBrains to improve J2K and open-sourced parts of the process to foster community collaboration. The article highlights null safety handling and various code issues encountered and resolved during the conversion.

Development code migration

A Wall Conversation Changed My Programming Career

2024-12-21

In 1983, a programmer working at a large defense contractor planned to pursue a Ph.D. in Chemistry. A chance conversation over a wall with the manager of the neighboring "Microcomputer Group" (a tinkerer) led to an invitation to a meeting about Apple II. There, he was tasked with building a VT-100 terminal emulator in 6502 assembly language within a week to enable the company president to read email at home. This experience not only redirected his career path, leading him to join the Microcomputer Group and become the company's sole PC programmer, but also ultimately led him to start his own company. Years later, he reflected on how chance encounters and interpersonal connections significantly shaped his life.

Development career opportunity

Software Design Philosophy: Taming Complexity

2024-12-21

This post summarizes three key ideas from the book "A Philosophy of Software Design": zero tolerance for complexity, the misconception that smaller components always equate to better modularity, and the complexities inherent in exception handling. The author argues that complexity isn't caused by single errors but accumulates over time. Examples of an order processing system and user registration illustrate how to avoid duplicated code and find the right balance between component size and modularity. Furthermore, the post details three techniques to reduce exception handling complexity: eliminating errors, masking exceptions, and exception aggregation, with file processing serving as an example. The book ultimately emphasizes the importance of consistently simplifying complexity in software design.

Enum of Arrays: A Novel Data Structure for Efficient Data Processing

2024-12-21

This article introduces a data structure called "Enum of Arrays" (EoA), similar to the popular "Struct of Arrays" (SoA), but with enums at its core. EoA packs multiple enum values into an array, using a single tag to identify the array's type. This reduces memory usage and branch prediction overhead, leading to more efficient data processing, particularly beneficial for SIMD optimization. The article uses the database system TigerBeetle as an example, illustrating how EoA enables efficient batch processing by effectively separating the control plane and data plane, resulting in significantly improved performance.

Yakari: Interactive Command Builder Simplifies Complex CLIs

2024-12-21

Yakari is an interactive command-building tool designed to simplify complex command-line interfaces. It guides users through command construction step-by-step, eliminating the need to memorize complex syntax. Supporting various argument types, Yakari offers contextual help and command history, significantly improving CLI usability. Users can build and execute commands with simple shortcuts, making even intricate commands accessible.

Development interactive interface

Go Iterators: Efficiently Handling Paginated APIs

2024-12-21

This article demonstrates how to efficiently handle paginated APIs using the iterator feature introduced in Go 1.23. Using the GitHub API as an example, the author shows how to write a custom iterator to abstract pagination logic, making the code more readable and reusable. The article focuses on the implementation and testing of the iterator, including mocking API calls and using pull iterators to ensure the iterator returns the expected results. Iterators allow developers to separate pagination logic from business logic, improving code maintainability and readability.

The CD Pipeline Manifesto: Building Better Software Delivery

2024-12-21

Modern software teams desperately need better tools for managing their Continuous Delivery pipelines. Today's CD pipeline ecosystem is fragmented, rigid, and inefficient. This manifesto advocates for code-first, developer-friendly pipelines designed to handle the complexities of modern engineering workflows. It emphasizes a single source of truth, reusable and typesafe components, dynamic and flexible pipelines, transparent and visual debugging, and mechanisms for handling change and fast feedback loops, ultimately aiming to improve efficiency and accelerate delivery.

Rivet: Run and Scale Realtime Applications with Actors

2024-12-21

Rivet is a platform for building and scaling real-time applications using the Actor model. It features built-in RPC, state, and events, simplifying modern application development. Rivet boasts automatic scaling, edge network deployment, and includes built-in monitoring and data localization capabilities. Powered by Rust, FoundationDB, V8 isolates, and the Deno runtime, it ensures performance and efficiency. Rivet is suitable for collaborative applications, local-first apps, AI agents, game servers, and more.

SingleFile: Save Entire Webpages as Single HTML Files

2024-12-21

SingleFile is a powerful web extension and CLI tool that saves complete web pages as a single HTML file. Compatible with Chrome, Firefox, Edge, and more, it offers convenient page saving, multi-tab processing, annotation capabilities, and even allows uploading saved pages to Google Drive or GitHub. Customize shortcuts and settings to tailor it to your needs.

Development webpage saving

Efficient German Language Learning: Is Anki the Answer?

2024-12-21

An engineer living in Germany for eight years confesses to still not knowing the language. To remedy this, they're trying Anki, leveraging spaced repetition to learn 10 new German words daily – aiming for C1 level proficiency within a year. They chose a frequency-ordered Anki deck, adding audio pronunciations themselves. The author invites readers to share their Anki experiences and German learning tips.

Saying Goodbye to C String Vulnerabilities: A Safer String Handling Approach

2024-12-21

Tired of C string vulnerabilities and insecurity? This article introduces a clever alternative: a custom string struct `struct str`, which contains a data pointer and length, avoiding the risks associated with null termination. The author's six-month experience in a bare-metal environment demonstrates that this approach effectively prevents errors such as buffer overflows. While using the macro `STR` is slightly verbose, the increase in safety and readability far outweighs this. Compiler optimizations also make the performance loss negligible, offering a new approach for developers prioritizing code security.

How an AI Code Review Bot Learned to Shut Up

2024-12-21

Greptile's AI code review bot initially faced criticism for generating excessive comments. To address this, they experimented with prompt engineering and having the LLM evaluate its own comments, but these methods proved ineffective. Their breakthrough came from vectorizing past comments, clustering them in a vector database, and filtering out new comments similar to those previously downvoted. This approach boosted the developer address rate from 19% to over 55%, significantly reducing LLM noise.

Development Code Review

Aegisub Subtitle Editor 3.4.0 Released!

2024-12-21

Aegisub 3.4.0 has been released! This free, cross-platform, open-source tool makes creating and editing subtitles quick and easy. It features powerful styling tools and a built-in real-time video preview. Perfect for both seasoned subtitlers and newcomers.

Development subtitle editor

Implementing Raft: A Deep Dive into Distributed Consensus

2024-12-21

This is the first post in a series detailing the Raft distributed consensus algorithm and its Go implementation. Raft solves the problem of replicating a deterministic state machine across multiple servers, ensuring service availability even with server failures. The post introduces core Raft components: the state machine, log, consensus module, leader/follower roles, and client interaction. It discusses Raft's fault tolerance, the CAP theorem, and the choice of Go as the implementation language. Subsequent posts will delve into the algorithm's implementation.

Development Distributed Consensus

Gazzetta: A New Mastodon News Reader

2024-12-21

Gazzetta is a revolutionary news reader designed specifically for Mastodon. Unlike other Mastodon clients that prioritize the social network experience, Gazzetta functions more like an RSS reader for the platform. It provides a separate interface, allowing users to focus solely on reading news and links. Features include following servers and accounts to see trending links, full-text search, integration with Safari's view controller, bookmark management, link exporting, and extensive customization options such as font styles, hiding thumbnails, and filtering links by domain, keyword, or language.

Development News Reader

S2: Revolutionizing Stream Data Storage in the Cloud

2024-12-21

Bandar Systems introduces S2, a novel stream data storage service designed to revolutionize data processing in the cloud era. Unlike traditional object-based storage, S2 centers around streams, offering efficient, scalable, and cost-effective real-time data ingestion and processing. It supports high-throughput, low-latency read and write operations and provides multiple storage classes to meet varying performance and cost requirements. S2 aims to replace systems like Kafka and Kinesis, providing users with a more powerful and flexible stream data management solution.

(s2.dev)

Parasite SEO Operator Evaded Google Penalties with Finesse

2024-12-21

This article exposes how parasite SEO operator Finixio/Clickout Media swiftly and effectively evaded Google penalties. Following a Google algorithm update, several Finixio/Clickout Media websites faced severe penalties for violating Google's site reputation abuse policy, resulting in plummeting traffic and rankings. However, through clever use of redirects and cloaking techniques, they restored their operations within days and continued profiting from their parasite website network. They even expanded their operations after being penalized, leveraging new websites and existing high-authority sites (like CoinTelegraph) to continue promoting gambling and cryptocurrency. The article details their strategies, including using geolocation to hide content and placing content on various platforms. It points out that the root cause of this phenomenon is Google's weakening of topical authority in its algorithms, making domain authority the primary ranking factor.

AP5 Reference Manual: A Logic-Based Extension to Common Lisp

2024-12-21

AP5 is an extension to Common Lisp that allows users to "program" at a more "specitional" level, focusing on what the machine should do rather than how. It combines aspects of Lisp and the Gist specification language, incorporating compilable parts of Gist and offering annotation mechanisms for performance tuning. AP5 uses a relational model to represent data and supports a first-order logic language for data access and manipulation. Programmers define relations, rules, and constraints, optimizing performance through annotations. The manual details AP5's syntax, database operations, rules, types, equivalence, and implementation specifics, providing numerous examples and explanations.

FindMy.py: An All-in-One Python Library for Querying Apple's FindMy Network

2024-12-21

FindMy.py is a Python library providing everything needed to query Apple's FindMy network. It unifies the fragmented Find My ecosystem, offering a cross-platform solution supporting various devices (AirTags, iDevices, etc.) and authentication methods (including SMS and Trusted Device 2FA). It features both async and sync APIs. Currently in Alpha, the API may change, but core functionality is stable.

Development

MarkItDown: Free Online Markdown Converter

2024-12-21

MarkItDown is a free online tool that converts various file formats (like Word, PDF, HTML, etc.) into standard Markdown. Powered by Microsoft's open-source Markitdown project, it offers fast and reliable conversions, perfect for bloggers, note-takers, and technical writers. No downloads or installations are required; simply upload your file and get clean, organized Markdown output. It's a secure and efficient way to manage your content.

Development online tool

Technical Debt vs. Technical Assets: A Wise Investment Strategy

2024-12-21

This article explores the difference between technical debt and technical assets. Technical debt, similar to financial debt, represents code issues that must be addressed, such as bugs and poor code readability, hindering development efficiency. Technical assets, on the other hand, are proactive investments in known problems, like building high-quality SDKs, reducing future maintenance costs and increasing development freedom. The article advises prioritizing the repayment of technical debt before investing in technical assets, leveraging proven processes and technologies to avoid accumulating technical debt and ultimately achieving higher development efficiency and product quality.

Rules to Avoid Common Extended Inline Assembly Mistakes

2024-12-21

This article isn't an inline assembly tutorial, but rather a summary of six rules to avoid common mistakes. The author emphasizes that inline assembly is treacherous and should be avoided whenever possible, as modern compilers offer intrinsics and built-ins for most use cases. If unavoidable, the rules are: use the `volatile` qualifier; use a `memory` clobber if accessing memory; never modify input constraints; never call functions from inline assembly; don't define absolute assembly labels; and use the assembler's local label feature to avoid label conflicts. The article concludes by encouraging readers to practice applying these rules by reviewing online tutorials and even LLM-generated code.

NoDB: Processing Payments Without a Database

2024-12-21

Alvaro Duran's "The Payments Engineer Playbook" introduces a revolutionary approach to payment system design: processing payments without a database. He argues that the prevalence of asynchronous programming stems from the assumption of database necessity. Using event sourcing, each step in the payment process is recorded as an event, not as a persistent state. These events are temporarily stored in memory, and the system reconstructs the payment status from the event stream, eliminating the need for persistent storage. This high-performance, high-reliability approach, inspired by high-frequency trading, allows for quick recovery from outages through hot backups. The article details this concept using a payment flow example and looks toward future applications in payment systems.

@celine/bibhtml v3.0.3: A Web Components-Based Referencing System

2024-12-21

@celine/bibhtml, a Web Components-based referencing system for HTML documents, has released version 3.0.3. It aims to provide a user experience similar to LaTeX/BibTeX referencing, using Citation.js under the hood and gracefully degrading when citations and references are malformed or JavaScript is disabled. Supporting BibTeX, unstructured text, DOI, and Wikidata formats, it offers three custom elements: ``, ``, and ``, simplifying reference management in HTML.

Development Reference Management
← Previous 1 3 4 5 6 7 8