Skip to content

Merging staging into release#959

Closed
Nicolapps wants to merge 7 commits intoreleasefrom
staging
Closed

Merging staging into release#959
Nicolapps wants to merge 7 commits intoreleasefrom
staging

Conversation

@Nicolapps
Copy link
Member


By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Nicolapps and others added 7 commits October 23, 2025 22:49
If the response indicates that logging in with SSO is required, show a
more specific error message

----

By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice.
<!-- Describe your PR here. -->



<!--
  The following applies to third-party contributors.
  Convex employees and contractors can delete or ignore.
-->

----

By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice.
<!-- Describe your PR here. -->



<!--
  The following applies to third-party contributors.
  Convex employees and contractors can delete or ignore.
-->

----

By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice.
<!-- Describe your PR here. -->



<!--
  The following applies to third-party contributors.
  Convex employees and contractors can delete or ignore.
-->

----

By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice.
<!-- Describe your PR here. -->



<!--
  The following applies to third-party contributors.
  Convex employees and contractors can delete or ignore.
-->

----

By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice.
@github-actions
Copy link

github-actions bot commented Jan 8, 2026

Braintrust eval report

chef (chef-claude-4-sonnet-e62bd831)

Score Average Improvements Regressions
1/Deploys 50% - -
IsSuccess 100% - -
Llm_calls 6.25 (+2.75) 4 🟢 -
Tool_calls 0 (+0) - -
Errors 1 (-2) 4 🟢 -
Llm_errors 0 (-1) 4 🟢 -
Tool_errors 0 (+0) - -
Prompt_tokens 177066tok (+124672.25tok) - 4 🔴
Prompt_cached_tokens 0tok (+0tok) - -
Prompt_cache_creation_tokens 0tok (+0tok) - -
Completion_tokens 9952.25tok (+9656tok) - 4 🔴
Completion_reasoning_tokens 0tok (+0tok) - -
Total_tokens 187018.25tok (+134328.25tok) - 4 🔴
Estimated_cost 0.68$ (+0.52$) - 4 🔴
Duration 156.58s (+97.51s) - 4 🔴
Llm_duration 118.04s (+101.32s) - 4 🔴

‼️ chef (chef-claude-4.5-sonnet-0cc4cf3a)

Score Average Improvements Regressions
Llm_calls 3.5 (-3.5) - 4 🔴
Tool_calls 0 (+0) - -
Errors 3 (+2) - 4 🔴
Llm_errors 1 (+1) - 4 🔴
Tool_errors 0 (+0) - -
Prompt_tokens 52393.75tok (-155247.5tok) 4 🟢 -
Prompt_cached_tokens 0tok (+0tok) - -
Prompt_cache_creation_tokens 0tok (+0tok) - -
Completion_tokens 296.25tok (-10070.25tok) 4 🟢 -
Completion_reasoning_tokens 0tok (+0tok) - -
Total_tokens 52690tok (-165317.75tok) 4 🟢 -
Estimated_cost 0.16$ (-0.62$) 4 🟢 -
Duration 59.07s (-99.44s) 4 🟢 -
Llm_duration 16.71s (-106.17s) 4 🟢 -
Expand to see errors
AI_APICallError: prompt is too long: 216902 tokens > 200000 maximum
    at eval (eval at <anonymous> (/home/runner/work/chef/chef/node_modules/.pnpm/[email protected]_@[email protected][email protected][email protected]__wgerilt4ezktegc3ulzicjnwli/node_modules/braintrust/dist/cli.js:6941:5), <anonymous>:14228:14)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async postToApi (eval at <anonymous> (/home/runner/work/chef/chef/node_modules/.pnpm/[email protected]_@[email protected][email protected][email protected]__wgerilt4ezktegc3ulzicjnwli/node_modules/braintrust/dist/cli.js:6941:5), <anonymous>:14133:28)
    at async AnthropicMessagesLanguageModel.doGenerate (eval at <anonymous> (/home/runner/work/chef/chef/node_modules/.pnpm/[email protected]_@[email protected][email protected][email protected]__wgerilt4ezktegc3ulzicjnwli/node_modules/braintrust/dist/cli.js:6941:5), <anonymous>:18942:9)
    at async fn (eval at <anonymous> (/home/runner/work/chef/chef/node_modules/.pnpm/[email protected]_@[email protected][email protected][email protected]__wgerilt4ezktegc3ulzicjnwli/node_modules/braintrust/dist/cli.js:6941:5), <anonymous>:29803:30)
    at async eval (eval at <anonymous> (/home/runner/work/chef/chef/node_modules/.pnpm/[email protected]_@[email protected][email protected][email protected]__wgerilt4ezktegc3ulzicjnwli/node_modules/braintrust/dist/cli.js:6941:5), <anonymous>:28123:22)
    at async _retryWithExponentialBackoff (eval at <anonymous> (/home/runner/work/chef/chef/node_modules/.pnpm/[email protected]_@[email protected][email protected][email protected]__wgerilt4ezktegc3ulzicjnwli/node_modules/braintrust/dist/cli.js:6941:5), <anonymous>:27940:12)
    at async fn (eval at <anonymous> (/home/runner/work/chef/chef/node_modules/.pnpm/[email protected]_@[email protected][email protected][email protected]__wgerilt4ezktegc3ulzicjnwli/node_modules/braintrust/dist/cli.js:6941:5), <anonymous>:29763:32)
    at async eval (eval at <anonymous> (/home/runner/work/chef/chef/node_modules/.pnpm/[email protected]_@[email protected][email protected][email protected]__wgerilt4ezktegc3ulzicjnwli/node_modules/braintrust/dist/cli.js:6941:5), <anonymous>:28123:22)
    at async type (eval at <anonymous> (/home/runner/work/chef/chef/node_modules/.pnpm/[email protected]_@[email protected][email protected][email protected]__wgerilt4ezktegc3ulzicjnwli/node_modules/braintrust/dist/cli.js:6941:5), <anonymous>:35083:24)
AI_APICallError: prompt is too long: 201193 tokens > 200000 maximum
    at eval (eval at <anonymous> (/home/runner/work/chef/chef/node_modules/.pnpm/[email protected]_@[email protected][email protected][email protected]__wgerilt4ezktegc3ulzicjnwli/node_modules/braintrust/dist/cli.js:6941:5), <anonymous>:14228:14)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async postToApi (eval at <anonymous> (/home/runner/work/chef/chef/node_modules/.pnpm/[email protected]_@[email protected][email protected][email protected]__wgerilt4ezktegc3ulzicjnwli/node_modules/braintrust/dist/cli.js:6941:5), <anonymous>:14133:28)
    at async AnthropicMessagesLanguageModel.doGenerate (eval at <anonymous> (/home/runner/work/chef/chef/node_modules/.pnpm/[email protected]_@[email protected][email protected][email protected]__wgerilt4ezktegc3ulzicjnwli/node_modules/braintrust/dist/cli.js:6941:5), <anonymous>:18942:9)
    at async fn (eval at <anonymous> (/home/runner/work/chef/chef/node_modules/.pnpm/[email protected]_@[email protected][email protected][email protected]__wgerilt4ezktegc3ulzicjnwli/node_modules/braintrust/dist/cli.js:6941:5), <anonymous>:29803:30)
    at async eval (eval at <anonymous> (/home/runner/work/chef/chef/node_modules/.pnpm/[email protected]_@[email protected][email protected][email protected]__wgerilt4ezktegc3ulzicjnwli/node_modules/braintrust/dist/cli.js:6941:5), <anonymous>:28123:22)
    at async _retryWithExponentialBackoff (eval at <anonymous> (/home/runner/work/chef/chef/node_modules/.pnpm/[email protected]_@[email protected][email protected][email protected]__wgerilt4ezktegc3ulzicjnwli/node_modules/braintrust/dist/cli.js:6941:5), <anonymous>:27940:12)
    at async fn (eval at <anonymous> (/home/runner/work/chef/chef/node_modules/.pnpm/[email protected]_@[email protected][email protected][email protected]__wgerilt4ezktegc3ulzicjnwli/node_modules/braintrust/dist/cli.js:6941:5), <anonymous>:29763:32)
    at async eval (eval at <anonymous> (/home/runner/work/chef/chef/node_modules/.pnpm/[email protected]_@[email protected][email protected][email protected]__wgerilt4ezktegc3ulzicjnwli/node_modules/braintrust/dist/cli.js:6941:5), <anonymous>:28123:22)
    at async type (eval at <anonymous> (/home/runner/work/chef/chef/node_modules/.pnpm/[email protected]_@[email protected][email protected][email protected]__wgerilt4ezktegc3ulzicjnwli/node_modules/braintrust/dist/cli.js:6941:5), <anonymous>:35083:24)
AI_APICallError: prompt is too long: 201156 tokens > 200000 maximum
    at eval (eval at <anonymous> (/home/runner/work/chef/chef/node_modules/.pnpm/[email protected]_@[email protected][email protected][email protected]__wgerilt4ezktegc3ulzicjnwli/node_modules/braintrust/dist/cli.js:6941:5), <anonymous>:14228:14)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async postToApi (eval at <anonymous> (/home/runner/work/chef/chef/node_modules/.pnpm/[email protected]_@[email protected][email protected][email protected]__wgerilt4ezktegc3ulzicjnwli/node_modules/braintrust/dist/cli.js:6941:5), <anonymous>:14133:28)
    at async AnthropicMessagesLanguageModel.doGenerate (eval at <anonymous> (/home/runner/work/chef/chef/node_modules/.pnpm/[email protected]_@[email protected][email protected][email protected]__wgerilt4ezktegc3ulzicjnwli/node_modules/braintrust/dist/cli.js:6941:5), <anonymous>:18942:9)
    at async fn (eval at <anonymous> (/home/runner/work/chef/chef/node_modules/.pnpm/[email protected]_@[email protected][email protected][email protected]__wgerilt4ezktegc3ulzicjnwli/node_modules/braintrust/dist/cli.js:6941:5), <anonymous>:29803:30)
    at async eval (eval at <anonymous> (/home/runner/work/chef/chef/node_modules/.pnpm/[email protected]_@[email protected][email protected][email protected]__wgerilt4ezktegc3ulzicjnwli/node_modules/braintrust/dist/cli.js:6941:5), <anonymous>:28123:22)
    at async _retryWithExponentialBackoff (eval at <anonymous> (/home/runner/work/chef/chef/node_modules/.pnpm/[email protected]_@[email protected][email protected][email protected]__wgerilt4ezktegc3ulzicjnwli/node_modules/braintrust/dist/cli.js:6941:5), <anonymous>:27940:12)
    at async fn (eval at <anonymous> (/home/runner/work/chef/chef/node_modules/.pnpm/[email protected]_@[email protected][email protected][email protected]__wgerilt4ezktegc3ulzicjnwli/node_modules/braintrust/dist/cli.js:6941:5), <anonymous>:29763:32)
    at async eval (eval at <anonymous> (/home/runner/work/chef/chef/node_modules/.pnpm/[email protected]_@[email protected][email protected][email protected]__wgerilt4ezktegc3ulzicjnwli/node_modules/braintrust/dist/cli.js:6941:5), <anonymous>:28123:22)
    at async type (eval at <anonymous> (/home/runner/work/chef/chef/node_modules/.pnpm/[email protected]_@[email protected][email protected][email protected]__wgerilt4ezktegc3ulzicjnwli/node_modules/braintrust/dist/cli.js:6941:5), <anonymous>:35083:24)
AI_APICallError: prompt is too long: 216282 tokens > 200000 maximum
    at eval (eval at <anonymous> (/home/runner/work/chef/chef/node_modules/.pnpm/[email protected]_@[email protected][email protected][email protected]__wgerilt4ezktegc3ulzicjnwli/node_modules/braintrust/dist/cli.js:6941:5), <anonymous>:14228:14)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async postToApi (eval at <anonymous> (/home/runner/work/chef/chef/node_modules/.pnpm/[email protected]_@[email protected][email protected][email protected]__wgerilt4ezktegc3ulzicjnwli/node_modules/braintrust/dist/cli.js:6941:5), <anonymous>:14133:28)
    at async AnthropicMessagesLanguageModel.doGenerate (eval at <anonymous> (/home/runner/work/chef/chef/node_modules/.pnpm/[email protected]_@[email protected][email protected][email protected]__wgerilt4ezktegc3ulzicjnwli/node_modules/braintrust/dist/cli.js:6941:5), <anonymous>:18942:9)
    at async fn (eval at <anonymous> (/home/runner/work/chef/chef/node_modules/.pnpm/[email protected]_@[email protected][email protected][email protected]__wgerilt4ezktegc3ulzicjnwli/node_modules/braintrust/dist/cli.js:6941:5), <anonymous>:29803:30)
    at async eval (eval at <anonymous> (/home/runner/work/chef/chef/node_modules/.pnpm/[email protected]_@[email protected][email protected][email protected]__wgerilt4ezktegc3ulzicjnwli/node_modules/braintrust/dist/cli.js:6941:5), <anonymous>:28123:22)
    at async _retryWithExponentialBackoff (eval at <anonymous> (/home/runner/work/chef/chef/node_modules/.pnpm/[email protected]_@[email protected][email protected][email protected]__wgerilt4ezktegc3ulzicjnwli/node_modules/braintrust/dist/cli.js:6941:5), <anonymous>:27940:12)
    at async fn (eval at <anonymous> (/home/runner/work/chef/chef/node_modules/.pnpm/[email protected]_@[email protected][email protected][email protected]__wgerilt4ezktegc3ulzicjnwli/node_modules/braintrust/dist/cli.js:6941:5), <anonymous>:29763:32)
    at async eval (eval at <anonymous> (/home/runner/work/chef/chef/node_modules/.pnpm/[email protected]_@[email protected][email protected][email protected]__wgerilt4ezktegc3ulzicjnwli/node_modules/braintrust/dist/cli.js:6941:5), <anonymous>:28123:22)
    at async type (eval at <anonymous> (/home/runner/work/chef/chef/node_modules/.pnpm/[email protected]_@[email protected][email protected][email protected]__wgerilt4ezktegc3ulzicjnwli/node_modules/braintrust/dist/cli.js:6941:5), <anonymous>:35083:24)

@vercel
Copy link

vercel bot commented Jan 8, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Review Updated (UTC)
chef (staging) Ready Ready Preview Jan 8, 2026 3:54pm

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants