#LLM

13 articles tagged with LLM

Testing AI Features: Non-Deterministic Output, Deterministic Tests

How to write reliable CI tests for non-deterministic LLM features using semantic similarity, assertion rubrics, and mock evaluations.

8/6/202622 min read

FastAPI Rate Limiting and Cost Controls for LLM-Backed Endpoints

Traffic-shaped rate limiting isn't enough when every request has a dollar cost attached. A layered approach: request limits, token-aware budgets, and hard circuit breakers.

7/23/202611 min read

FastAPI Streaming Responses for LLM Token-by-Token Output

A step-by-step build of a real token-streaming endpoint in FastAPI: StreamingResponse, server-sent events, backpressure, client disconnects, and the gotchas that don't show up in a demo.

7/5/202610 min read

FastAPI + Pydantic v2: Building Type-Safe LLM API Endpoints

A hands-on walkthrough of the Pydantic v2 patterns that actually hold up when your endpoint's request or response body is generated, in part, by a language model.

6/29/202611 min read

Why FastAPI Became the Default Backend for AI Products in 2026

Node.js won the last decade of API backends. For AI products, the calculus flipped. Here's the actual reasoning — not hype — behind why most AI teams reach for FastAPI first.

6/26/20269 min read

streamUI: Dynamic React Components via Vercel AI SDK 3.x

Text generation is only the first step. Learn how to stream interactive, fully styled React components directly from LLM completions using Vercel AI SDK streamUI.

6/25/20269 min read

Structured Output & Agentic Routing with Vercel AI SDK 3.x

LLM responses are unpredictable. Discover how to use Vercel AI SDK Core APIs to guarantee structured JSON schema outputs and build dynamic agent routing loops.

6/17/20269 min read

Accelerating LLMs in the Browser: WebGPU vs WebNN — A 2026 Deep Dive

WebGPU and WebNN have fundamentally changed what's possible for LLM inference directly in the browser. This deep dive benchmarks both APIs, dissects WGSL shader code for matrix multiplication, and shows you exactly when to pick each technology.

6/7/202618 min read

Browser-Native AI Models with WebGPU: Running LLMs Locally Without a Server

Stop paying for inference APIs. This deep dive covers the full stack of running quantized LLMs directly in the browser using WebGPU compute pipelines, Transformers.js v3, and ONNX Runtime Web — with real benchmark numbers and production architecture patterns.