GPT-6.1 Sol vs Opus 5.5 - JEV: 9 Real Dev Tests

5 Просмотры

Спасибо! Поделитесь с друзьями!

Вам не понравилось видео. Спасибо за то что поделились своим мнением!

Издатель Oct 1, 2026

i gave GPT-6.1 Sol and Opus 5.5 nine developer jobs using Jev.
same prompts, separate tools, hidden tests. both coding models ran at High effort.

we check speed, correctness, Jev accuracy, Jev cost, integration, maintainability, security, scale and SOLID design.
each model has 81 scores, with 810 points available.

GPT-6.1 Sol: 623.738/810, 7.700/10
Opus 5.5: 543.816/810, 6.714/10

0:00 Cold open
1:15 File selection
2:21 Test selection
3:31 Duplicate bug reports
4:40 Commit types
5:50 Test failure triage
7:08 Next tool
8:33 Model routing
10:12 Browser navigation
11:42 Web research
13:40 Final scores

method notes:
- 147 development items and 430 hidden items across nine jobs.
- some task text retains public source links. classification runs were forbidden to look up answers; this is not airtight blinding.
- duplicate labels are recorded human issue links, not all maintainer decisions.
- next-tool results measure imitation of recorded successful work, not one uniquely best action.
- model routing replays historical outcomes and costs. cheap failure does not prove expensive success.
- browser access faults were repaired for both tools. cost includes every repair; the final repeat supplies timing only.
- recorded batch times use calendar clocks on a shared host. speed scores use coarse limits, not isolated monotonic benchmarks.
- the search tools are community projects. separate demonstrations do not prove a general search-speed or accuracy advantage.
- terminal replays and recorded development work are labeled.
- the separate cost comment uses returned Jev tokens and list prices. subscription bills and invoices are unavailable.

Jev documentation: https://docs.typesafe.ai/introduction
TypeSafe agent skill: https://github.com/typesafe-ai/skills
Community webctl: https://github.com/dorkitude/webctl
Community Jev Search: https://github.com/superagents-lab/jev-search
Model comparison: https://openrouter.ai/compare/openai/gpt-6.1-sol/anthropic/claude-opus-5.5
High-effort comparison: https://artificialanalysis.ai/models/comparisons/gpt-6-1-sol-high-vs-claude-opus-5-5-high