GrandpaCAD: AI-Powered 3D Modeling for Everyone

This dashboard benchmarks each model and CAD-engine combo on a fixed set of prompts, scoring prompt adherence, aesthetics, cost, speed, and code errors. The short version: medium reasoning is the value sweet spot, Opus 4.7 is our reliability benchmark, and cheap models usually cost more once you count error-repair loops and latency.

Sort by:

8/14/2026

$2.42

38m 49s

23 × 1 = 23 runs

Gemini 3.7 Flash — cheap suite (no judge)

model gemini-3.7-flash, thinking medium

gemini-3.7-flashThinking mediumopenscad0.2

GPT-6.1 Sol (OpenRouter, reasoning medium) — openscad

model gpt-6.1-sol, reasoning medium hex-bracket and pegboard-hook replaced with attempt 1 of the re-runs on the fully specified prompts (M8 nut / IKEA SKÅDIS), runs 6767d14 and 2e0f412.

gpt-6.1-solThinking mediumopenscad0.2active

GPT-6.1 Sol (OpenRouter, reasoning medium) — build123d

model gpt-6.1-sol, reasoning medium

gpt-6.1-solThinking mediumbuild123d0.2

Gemini 3.8 Flash — baseline vs 3.7

model gemini-3.8-flash, thinking medium

gemini-3.8-flashThinking mediumopenscad0.2

Claude Opus 5.5 (Anthropic, effort medium) — build123d

model claude-opus-5.5, thinking adaptive, effort medium

claude-opus-5.5Thinking mediumbuild123d0.2

R²: 0.00%

What we've learned

2025-10Medium reasoning beats High: about 2x the cost and time for little quality gain.
2025-11Nearly-free open models mostly failed or errored out. Sticker price is not real cost.
2025-12GPT-5.2 High was a cost trap: $0.65 per generation and 22 code errors.
2026-02Switched the target engine to OpenSCAD and began deliberate error-mining runs.
2026-04Opus 4.7 set the bar: fast, about $0.10 per generation, few code errors.
2026-06The cheapest models (like minimax-m3) hit 90+ code errors and multi-hour runs.
2026-07gemini-3.1-pro is the current OpenSCAD baseline.
2026-07Opus 5 at low reasoning beat itself at high on every axis: 28% cheaper, 2.4x faster, 5 code errors down to 0, same adherence.
2026-07Same Opus 5, same effort, 35% cheaper on the first-party Anthropic API than through OpenRouter. The gateway is not cost-neutral.

Other Stats

Average R-squared

9.50%

What is the average R-squared of all the runs? This tells us linear regression alignment human vs AI. 100% means AI perfectly predicts a human vote. 0% means AI doesn't predict it at all.

Combined R-squared

17.14%

When we set the model according to all pairs from all runs what is the R-squared?

Eval Runs

109

Total number of evaluation runs.

Total Cost of All Evals

$721.69

The sum of costs for all evaluation runs.

Average Cost per Test Run

$6.62

The average cost of a single evaluation run.

Total Duration of All Evals

7169m 49s

The sum of durations for all evaluation runs.

Average Duration per Test Run

65m 47s

The average duration of a single evaluation run.

All Generations

3264

The sum of all generations for all evaluation runs.