DAXBench evaluates language models on their ability to write correct, production-grade DAX code. Answers are verified by execution, and scoring goes beyond correctness to measure code quality, performance patterns, and clarity.
Benchmark v2 runs a new task set against prepared Power BI models and is scored separately from v1, so v1 and v2 scores are never compared or mixed.
The v2 tasks and expected results are provided by SQLBI. DAXBench is developed by Maxim Anatsko in collaboration with SQLBI.
A task is correct only if every probe query and every required structural check passes.
The probes-passed count shows how close an answer came. It is not partial credit and adds no points.
Correct tasks earn points weighted by complexity, plus a style bonus from the quality dimensions. Incorrect tasks earn nothing.
Whether the model followed the requested output format is reported on its own and never changes the score.
If the benchmark or the infrastructure (fixture, parser or provider) fails, the task is marked incomplete and is not counted against the model.
Does the code produce the expected output? We verify results through execution against real Power BI data when available.
Does the code follow DAX conventions recommended by the community and Microsoft documentation?
Does the code use efficient patterns that matter at scale?
Is the code readable and maintainable with clear structure and naming?
Correctness Gating: Style scores are only awarded when correctness exceeds a minimum threshold. This prevents models from ranking highly with well-formatted but incorrect code.
Tasks use the Contoso sample database with realistic business scenarios.
Different correct approaches receive full credit.
Code runs against live Power BI data to verify results.
Tasks span three complexity tiers. Advanced tasks contribute more to the final score.
See how different models perform across task categories, complexity levels, and scoring dimensions.