#LLM Benchmarking

2 articles tagged with LLM Benchmarking

Evaluating a New Model Release Before Migrating Your Production Prompts

The AI regression testing guide. How to build automated prompt eval suites, semantic similarity scoring, and LLM-as-a-Judge benchmarking in TypeScript.

8/1/202625 min read

Benchmarking Reasoning Models for Code Generation Tasks

Public coding benchmarks are a poor proxy for how a model will behave in your codebase. Here's the harness I built to benchmark reasoning models against tasks that look like actual work.

7/12/202611 min read