fix(auto-monitor-setups): update bulk upsert to atomic semantics#173
Open
galzilber wants to merge 1 commit into
Open
fix(auto-monitor-setups): update bulk upsert to atomic semantics#173galzilber wants to merge 1 commit into
galzilber wants to merge 1 commit into
Conversation
The bulk upsert endpoint changed from per-item partial success (207 Multi-Status) to atomic all-or-nothing. Update the docs to match: 200 OK with a flat list of full setups on success, 400 for request- and per-item validation errors, 500 for DB failures. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
galzilber
changed the title
Contributor
📝 WalkthroughWalkthroughDocumentation for POST /v2/auto-monitor-setups/bulk was updated to describe atomic all-or-nothing batch semantics instead of partial success. The 207 Multi-Status per-item model was removed; 200 OK, 400 Bad Request, and 500 Internal Server Error response examples were revised accordingly. ChangesDocumentation update for atomic bulk semantics
Estimated code review effort: 1 (Trivial) | ~5 minutes Possibly related PRs
Suggested reviewers: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@api-reference/auto-monitor-setups/bulk-upsert-auto-monitor-setups.mdx`:
- Around line 64-107: The 200 OK example and description in
bulk-upsert-auto-monitor-setups.mdx are inconsistent with the stated flat-array
response shape. Update the documented success payload to match the actual
contract used by the bulk upsert endpoint, and make sure the narrative near the
example aligns with that shape by referencing the bulk-upsert auto monitor
setups response and its `200 OK` section.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 1c87cc6c-4d96-45df-b531-5a66da712719
📒 Files selected for processing (1)
api-reference/auto-monitor-setups/bulk-upsert-auto-monitor-setups.mdx
Comment on lines
+64
to
107
| ### 200 OK | ||
|
|
||
| Returned when the batch is accepted. The `setups` array contains one entry per input item, in the **same order** as the request. | ||
|
|
||
| Each entry has: | ||
|
|
||
| | Field | Type | Description | | ||
| |-------|------|-------------| | ||
| | `external_id` | string | The `external_id` of the input item. | | ||
| | `status` | string | `ok` if the item was upserted, `error` if it failed. | | ||
| | `result` | object | Present when `status` is `ok`. The full upserted setup, matching the [Create](/api-reference/auto-monitor-setups/create-an-auto-monitor-setup) response shape. | | ||
| | `error` | string | Present when `status` is `error`. Human-readable message prefixed with the item index, e.g. `setups[1]: unknown evaluator slug "..."`. Internal errors are reported as `setups[N]: internal error` — the underlying cause is logged server-side but not returned to the client. | | ||
| Returned when the entire batch is upserted successfully. The `setups` array contains one entry per input item, in the **same order** as the request. Each entry is a full setup object, matching the [Create](/api-reference/auto-monitor-setups/create-an-auto-monitor-setup) response shape. | ||
|
|
||
| ```json | ||
| { | ||
| "setups": [ | ||
| { | ||
| "id": "cmm...", | ||
| "external_id": "openai-gpt4o-monitor", | ||
| "status": "ok", | ||
| "result": { | ||
| "id": "cmm...", | ||
| "external_id": "openai-gpt4o-monitor", | ||
| "org_id": "c108269c-...", | ||
| "project_id": "cm9v2g95l...", | ||
| "env_project_id": "cm9v2ga9i...", | ||
| "init_rules": [ | ||
| { "key": "gen_ai.system", "value": "openai", "source": "span_attributes", "operator": "equals" }, | ||
| { "key": "gen_ai.request.model", "value": "gpt-4o", "source": "span_attributes", "operator": "equals" } | ||
| ], | ||
| "evaluators": [ | ||
| { "evaluator_type": "answer-relevancy", "status": "pending" }, | ||
| { "evaluator_type": "toxicity-detector", "status": "pending" } | ||
| ], | ||
| "status": "pending", | ||
| "created_at": "2026-05-20T10:30:00Z", | ||
| "updated_at": "2026-05-20T10:30:00Z" | ||
| } | ||
| "org_id": "c108269c-...", | ||
| "project_id": "cm9v2g95l...", | ||
| "env_project_id": "cm9v2ga9i...", | ||
| "init_rules": [ | ||
| { "key": "gen_ai.system", "value": "openai", "source": "span_attributes", "operator": "equals" }, | ||
| { "key": "gen_ai.request.model", "value": "gpt-4o", "source": "span_attributes", "operator": "equals" } | ||
| ], | ||
| "evaluators": [ | ||
| { "evaluator_type": "answer-relevancy", "status": "pending" }, | ||
| { "evaluator_type": "toxicity-detector", "status": "pending" } | ||
| ], | ||
| "status": "pending", | ||
| "created_at": "2026-05-20T10:30:00Z", | ||
| "updated_at": "2026-05-20T10:30:00Z" | ||
| }, | ||
| { | ||
| "id": "cmm...", | ||
| "external_id": "anthropic-monitor", | ||
| "status": "error", | ||
| "error": "setups[1]: unknown evaluator slug \"answer-relevancy-typo\"" | ||
| "org_id": "c108269c-...", | ||
| "project_id": "cm9v2g95l...", | ||
| "env_project_id": "cm9v2ga9i...", | ||
| "init_rules": [ | ||
| { "key": "gen_ai.system", "value": "anthropic", "source": "span_attributes", "operator": "equals" } | ||
| ], | ||
| "evaluators": [ | ||
| { "evaluator_type": "answer-relevancy", "status": "pending" } | ||
| ], | ||
| "status": "pending", | ||
| "created_at": "2026-05-20T10:30:00Z", | ||
| "updated_at": "2026-05-20T10:30:00Z" | ||
| } | ||
| ] | ||
| } | ||
| ``` |
Contributor
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win
Response shape contradicts stated behavior — flat array vs. object wrapper.
The PR objectives state the success response is "a flat array of full setup objects," but the documented 200 OK example wraps the array in a setups key ({"setups": [...]}). One of these is wrong; if the actual API returns a bare array, this example and the accompanying description ("The setups array contains one entry per input item...") need correcting to match the real contract.
📝 Proposed fix if the API returns a bare array
### 200 OK
-Returned when the entire batch is upserted successfully. The `setups` array contains one entry per input item, in the **same order** as the request. Each entry is a full setup object, matching the [Create](/api-reference/auto-monitor-setups/create-an-auto-monitor-setup) response shape.
+Returned when the entire batch is upserted successfully. The response is a flat array with one entry per input item, in the **same order** as the request. Each entry is a full setup object, matching the [Create](/api-reference/auto-monitor-setups/create-an-auto-monitor-setup) response shape.
```json
-{
- "setups": [
- {
+[
+ {
...
- }
- ]
-}
+ }
+]📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| ### 200 OK | |
| Returned when the batch is accepted. The `setups` array contains one entry per input item, in the **same order** as the request. | |
| Each entry has: | |
| | Field | Type | Description | | |
| |-------|------|-------------| | |
| | `external_id` | string | The `external_id` of the input item. | | |
| | `status` | string | `ok` if the item was upserted, `error` if it failed. | | |
| | `result` | object | Present when `status` is `ok`. The full upserted setup, matching the [Create](/api-reference/auto-monitor-setups/create-an-auto-monitor-setup) response shape. | | |
| | `error` | string | Present when `status` is `error`. Human-readable message prefixed with the item index, e.g. `setups[1]: unknown evaluator slug "..."`. Internal errors are reported as `setups[N]: internal error` — the underlying cause is logged server-side but not returned to the client. | | |
| Returned when the entire batch is upserted successfully. The `setups` array contains one entry per input item, in the **same order** as the request. Each entry is a full setup object, matching the [Create](/api-reference/auto-monitor-setups/create-an-auto-monitor-setup) response shape. | |
| ```json | |
| { | |
| "setups": [ | |
| { | |
| "id": "cmm...", | |
| "external_id": "openai-gpt4o-monitor", | |
| "status": "ok", | |
| "result": { | |
| "id": "cmm...", | |
| "external_id": "openai-gpt4o-monitor", | |
| "org_id": "c108269c-...", | |
| "project_id": "cm9v2g95l...", | |
| "env_project_id": "cm9v2ga9i...", | |
| "init_rules": [ | |
| { "key": "gen_ai.system", "value": "openai", "source": "span_attributes", "operator": "equals" }, | |
| { "key": "gen_ai.request.model", "value": "gpt-4o", "source": "span_attributes", "operator": "equals" } | |
| ], | |
| "evaluators": [ | |
| { "evaluator_type": "answer-relevancy", "status": "pending" }, | |
| { "evaluator_type": "toxicity-detector", "status": "pending" } | |
| ], | |
| "status": "pending", | |
| "created_at": "2026-05-20T10:30:00Z", | |
| "updated_at": "2026-05-20T10:30:00Z" | |
| } | |
| "org_id": "c108269c-...", | |
| "project_id": "cm9v2g95l...", | |
| "env_project_id": "cm9v2ga9i...", | |
| "init_rules": [ | |
| { "key": "gen_ai.system", "value": "openai", "source": "span_attributes", "operator": "equals" }, | |
| { "key": "gen_ai.request.model", "value": "gpt-4o", "source": "span_attributes", "operator": "equals" } | |
| ], | |
| "evaluators": [ | |
| { "evaluator_type": "answer-relevancy", "status": "pending" }, | |
| { "evaluator_type": "toxicity-detector", "status": "pending" } | |
| ], | |
| "status": "pending", | |
| "created_at": "2026-05-20T10:30:00Z", | |
| "updated_at": "2026-05-20T10:30:00Z" | |
| }, | |
| { | |
| "id": "cmm...", | |
| "external_id": "anthropic-monitor", | |
| "status": "error", | |
| "error": "setups[1]: unknown evaluator slug \"answer-relevancy-typo\"" | |
| "org_id": "c108269c-...", | |
| "project_id": "cm9v2g95l...", | |
| "env_project_id": "cm9v2ga9i...", | |
| "init_rules": [ | |
| { "key": "gen_ai.system", "value": "anthropic", "source": "span_attributes", "operator": "equals" } | |
| ], | |
| "evaluators": [ | |
| { "evaluator_type": "answer-relevancy", "status": "pending" } | |
| ], | |
| "status": "pending", | |
| "created_at": "2026-05-20T10:30:00Z", | |
| "updated_at": "2026-05-20T10:30:00Z" | |
| } | |
| ] | |
| } | |
| ``` | |
| ### 200 OK | |
| Returned when the entire batch is upserted successfully. The response is a flat array with one entry per input item, in the **same order** as the request. Each entry is a full setup object, matching the [Create](/api-reference/auto-monitor-setups/create-an-auto-monitor-setup) response shape. | |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@api-reference/auto-monitor-setups/bulk-upsert-auto-monitor-setups.mdx` around
lines 64 - 107, The 200 OK example and description in
bulk-upsert-auto-monitor-setups.mdx are inconsistent with the stated flat-array
response shape. Update the documented success payload to match the actual
contract used by the bulk upsert endpoint, and make sure the narrative near the
example aligns with that shape by referencing the bulk-upsert auto monitor
setups response and its `200 OK` section.
doronkopit5
approved these changes
Jul 9, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Updates the bulk upsert endpoint docs (
POST /v2/auto-monitor-setups/bulk) to match the current api-service implementation, which changed from per-item partial success to atomic all-or-nothing. Success is now200 OKwith a flat array of full setup objects (previously207 Multi-Statuswith per-item{status, result, error}wrappers).400now also covers per-item validation errors with the exact error strings from the handler, and the500body was corrected to{"error": "Internal server error"}.🤖 Generated with Claude Code
Summary by CodeRabbit
200 OKwith full setup details in request order.