Blog

Notes on engineering, platforms, and shipping product. Latest first.

• Inbound AI Agent Traffic: System Architecture Lessons Nov 12, 2026
  United Airlines banned third-party AI agents. Here is how platform leaders must architect M2M endpoints, rate limits, and zero-trust auth for automated bots.
• OAuth Delegation for AI Agents: Identity Broker Layer Nov 11, 2026
  Decouple OAuth 2.1 ceremonies from custom MCP tools using an identity broker layer. Secure AI agent delegation without risking raw credentials.
• Distributed Edge Detection and Platform Engineering Realities Nov 10, 2026
  Exploring bio-inspired computer vision algorithms like HaCI alongside heterogeneous GPU management patterns and OAuth delegation for modern systems.
• Managing Agent GPU Kernels: vLLM's Kimi K3 Fast-Track RFC Nov 04, 2026
  vLLM's Kimi K3 RFC balances CuTe DSL kernel fusion on Blackwell GPUs with strict isolation and automated gates to manage agentic code generation.
• Agentic E2E Testing in CI: Auto-Healing & MCP Verification Nov 03, 2026
  How autonomous QA agents use Model Context Protocol and dynamic auto-healing to eliminate continuous integration bottlenecks.
• vLLM Constrained Decoding: Fix Tool Call Schema Errors Oct 27, 2026
  Enforcing server-side constrained decoding in vLLM 0.31 fixes tool-call schema failures, bumping open model accuracy from 92% to 100% in agent workflows.
• Desktop Agent Runtimes: LSPs and Native Rust Engines Oct 23, 2026
  Embedding LSPs, Rust AST diffing, and canvas drawing primitives inside desktop agent runtimes changes how engineers and AI agents collaborate locally.
• Linking Architecture Diagrams to Code via Agent Canvas SDKs Oct 22, 2026
  Architectural drift accelerates when AI agents write code. Open source tools like Whiteboard tie sequence diagrams directly to AST symbols via LSP bindings.
• Beyond Line Diffs: AST-Aware Semantic Code Mappings Oct 20, 2026
  Line-based git diffs obscure architectural intent during cross-service refactorings. AST-aware semantic diffing and interactive visual canvases restore clarity.
• Deterministic Replay: Time-Travel APIs for AI Agents Oct 19, 2026
  AI agents can now use deterministic time-travel APIs to step through nanosecond execution histories, isolate distributed race conditions, and verify fixes.
• Multi-Node LLM Inference Orchestration Topologies Oct 12, 2026
  An engineering leader's guide to multi-node LLM inference topologies, KV-aware routing, disaggregated prefill/decode, and heterogeneous GPU orchestrators.
• Cache Expiration Without CPU Spikes: Adaptive TTL Reapers Oct 08, 2026
  Learn how to balance memory leaks and CPU thrashing in high-throughput caches using Redis-style adaptive active TTL reapers.
• Async Agent Runtimes: Stateful Queues, Decoupled Execution Oct 07, 2026
  Explore the architectural shift to asynchronous AI agent runtimes with stateful checkpointing, background daemons, and explicit security boundaries.
• Bridging Systems Theory and Production Platform Engineering Oct 02, 2026
  How classical distributed primitives like FLP, Raft, and CRDTs translate to modern multi-region platform reliability and operational trade-offs.
• Evaluating Speculative Decoding in vLLM on AMD MI300X GPUs Oct 01, 2026
  An engineering evaluation of speculative decoding methods in vLLM on AMD Instinct MI300X hardware, analyzing throughput, acceptance rates, and tradeoffs.
• Hide or Reduce: Why Modularity Breaks Distributed Systems Sep 30, 2026
  Discover why traditional modular encapsulation fails in high-concurrency systems and how platform leaders can use modeling abstractions to prove system invar...
• Scheduling Agents Like Processes: Patterns for AI Fleets Sep 28, 2026
  Why scaling autonomous AI agent fleets requires OS kernel patterns, decoupled state, and process scheduling rather than bigger context windows.
• Evaluating AST-Aware Semantic Diffing for Code Review Sep 25, 2026
  An evaluation of Rust-based AST semantic diff engines, visual diagrams, and LSP integration for reducing PR noise during large architectural code reviews.
• Designing Tool-Scoped Subagents to Prevent Context Bloat Sep 23, 2026
  How isolating capabilities into tool-scoped subagents prevents context bloat, reduces inference latency, and stabilizes long-running AI agent runtimes.
• Multi-Node LLM Orchestrators: Exo, GPUStack, LocalAI Sep 22, 2026
  A pragmatic teardown of self-hosted multi-node LLM orchestrators comparing zero-config P2P, supervisor-worker topologies, and cache-aware routing.
• Interactive Architecture Models: Visualizing Tradeoffs Sep 21, 2026
  Learn how interactive architecture models reveal distributed system tradeoffs, fault boundaries, and stateful flow bottlenecks before shipping code.
• Least-Privilege Filesystem Sandboxing for Agent Runtimes Sep 18, 2026
  How Pizza Bot isolates background AI agent execution using explicit directory whitelisting, process separation, and tool-scoped subagents.
• Supervising Agent Daemons: Multi-Client SSE in Pizza Bot Sep 17, 2026
  How Pizza Bot uses supervised Node.js daemons, HTTP/SSE, and LangGraph checkpoints to build resilient, multi-client local AI workflows.
• Async Agent Workflows with Checkpoints in Pizza Bot Sep 16, 2026
  Learn how Pizza Bot uses stateful LangGraph runtimes, SSE, and explicit action queues to build durable, asynchronous background AI agents.
• Distributed Systems Classics Platform Engineers Must Master Sep 15, 2026
  Why platform engineering failures stem from ignoring classic distributed systems constraints like FLP impossibility, Lamport clocks, and CRDT limitations.
• KV Cache Offloading in vLLM: Breaking the GPU Memory Wall Sep 14, 2026
  How vLLM v0.28.0 breaks the GPU VRAM wall using native NVMe KV cache offloading, canonical CPU layouts, and Model Runner V2 disaggregation.
• Why Hiding Complexity Fails in Distributed Systems Sep 11, 2026
  Platform engineering leaders must distinguish between vertical modularity and horizontal modeling abstractions to build reliable distributed systems.
• Client-Side Determinism and Logit Control with WebLLM Sep 10, 2026
  How WebLLM brings OpenAI-compatible streaming, logit bias, and seed determinism directly to browsers via WebGPU and WebAssembly.
• Packaging Custom LLMs for the Edge with MLC and WebLLM Sep 09, 2026
  Learn how platform teams build edge CI/CD pipelines to package fine-tuned open-weight LLMs into MLC format for browser-native WebGPU execution.
• In-Browser LLM Execution: Zero-Refactoring with WebLLM Sep 08, 2026
  How WebLLM brings OpenAI API wire compatibility to WebGPU, enabling client-side LLM inference without refactoring frontend code.
• Zero-Backend Local Inference with WebLLM on Browser GPUs Sep 07, 2026
  Run open source LLMs directly inside client browsers using WebGPU and WebLLM without modifying application logic or paying backend token costs.
• Guaranteed Schema Output: WASM JSON Generation in WebLLM Sep 04, 2026
  How WebLLM enforces strict JSON schemas in the browser using WebAssembly and WebGPU to eliminate client-side LLM parse errors without server validation.
• Client-Side LLM Execution: In-Browser Workflows with WebLLM Sep 03, 2026
  Execute open-source LLMs inside web browsers using WebLLM and WebGPU to eliminate cloud compute costs, preserve user privacy, and streamline frontend AI arch...
• Deterministic Secret State: keep.lock Stops CI/CD Drift Sep 02, 2026
  Aligning secret state with Git commits using committed keep.lock manifests prevents build failures and runtime secret drift across engineering teams.
• Debugging Silent 200 OK Failures with Memory Snapshots Sep 01, 2026
  Silent 200 OK failures escape standard telemetry. Learn how asynchronous live memory snapshots isolate hidden state corruption without thread pauses.
• Branch-Aware Secrets: Aligning Env Scopes to Git Branches Aug 31, 2026
  Eliminate configuration drift across feature branches by pairing secrets directly to Git using versioned keep.lock manifests and CLI tools.
• Standardizing AI Agent Knowledge Across Developer Toolchains Aug 28, 2026
  Platform engineering strategies for eliminating context drift across Cursor, Claude Code, and Codex using shared Model Context Protocol endpoints.
• Unifying Multi-Tool AI Workflows with MCP Knowledge Routers Aug 27, 2026
  Eliminate context drift across Cursor, Claude Code, and ChatGPT by deploying central MCP knowledge routers for your engineering team.
• Two-Party Envelope Decryption and Zero-SDK Secret Injection Aug 25, 2026
  How zero-trust envelope decryption and process boundary injection solve developer secret sync without runtime SDKs or key rotation cascades.
• Managing Agent Context Drift with Shared Knowledge Layers Aug 24, 2026
  How platform engineering teams prevent context fragmentation across Claude, Cursor, and terminal agents using Model Context Protocol knowledge layers.
• Measuring Engineering Productivity: The DX Core 4 Guide Aug 21, 2026
  How the DX Core 4 framework combines DORA, SPACE, and DevEx to balance velocity, software quality, and developer experience.
• Zero-Trust Secrets Management: Stop Key Rotation Cascades Aug 20, 2026
  How local-first encryption and two-party cryptographic revocation eliminate key rotation cascades during developer offboarding.
• How Read-Only AI Debugging Agents Change Incident Response Aug 19, 2026
  Platform engineering teams are moving from redeploy-and-log loops to ephemeral, read-only AI probing in live production environments.
• Debugging Production Without Redeploys: AI Virtual Probes Aug 19, 2026
  How AI agents and read-only virtual probes eliminate speculative redeployments and capture live runtime state without pausing threads.