Tag

#llm-evaluation

Posts tagged with llm-evaluation

4 posts

The pass@k crossover at the heart of choosing between a base checkpoint and its RL-tuned sibling.

Engineering • Sep 22, 2026 •11 min read

Pass@k Crossover Tests Tell You When to Ship the Base Model

Pass@k crossovers show RLVR sharpens rather than adds capability. Learn the decision rule and statistical test that find your model's crossover point.

#passk #rlvr +4

A prompt dependency graph maps prompts and their consuming components as coupled nodes so teams can see exactly what a prompt change touches before retesting.

Guides • Sep 03, 2026 •11 min read

Prompt Dependency Graphs That Shrink Your Retest Set

Build a prompt dependency graph to compute the blast radius of any prompt change and rerun only the evals your multi-prompt LLM system needs.

#llm-evaluation #prompt-engineering +3

Text-to-SQL evaluation tests whether AI-generated SQL returns defensible answers on real warehouse data instead of just matching public benchmark scores.

Engineering • Aug 27, 2026 •12 min read

Text-to-SQL Evaluation That Catches Silent Wrong Answers

Public benchmarks say 89%, your warehouse says otherwise. Build a text-to-SQL evaluation with schema-specific oracles that catches silent wrong answers.

#text-to-sql #nl2sql +4

A decision framework for choosing between fine-tuning, RAG, and prompt engineering when working with small open-weight language models.

Guides • Jul 12, 2026 •13 min read

Fine-Tuning vs RAG vs Prompt Engineering Decision Framework

Fine-tuning vs RAG vs prompt engineering: run this eval-gated framework before training a custom model and inheriting its hidden maintenance tax.

#fine-tuning #rag +4