ChatGPT↔Codex handoffs can cause false PASS results by losing requirements and completion criteria — request for unified context and locked acceptance criteria

[buraburachinpokou]

ChatGPT↔Codex handoffs can break continuity of intent: requirements and “done” criteria agreed in ChatGPT can be reinterpreted downstream, causing completion judgment to drift.

This can produce false PASS results when Codex validates a proxy metric (for example, “time progressed”) while the actual human-observable usability requirements are still failing.

A real example was a “Detached simulation speed” feature that advanced simulation time quickly but effectively time-warped intermediate states, making mid-run observation, Stop, and manual actions unusable — yet it was reported as PASS based on the proxy.

Environment / component

• Component: Codex chat used from ChatGPT
• Not using Codex CLI / SDK / IDE extension
• Exact Codex version: unknown; not preserved in my logs
• OS: Windows
• Date of incident: August 23, 2026
• As of August 24, 2026 07:07 JST, this remained HUMAN FAIL / DESIGN MISMATCH in my project log
• Browser: unknown; I cannot identify it reliably from the retained logs, so I do not want to guess
• Target application: local Windows HTML/CSS/JavaScript application with a PowerShell launcher

Target feature

A self-developed game/project called “SANKAI CAPITAL,” specifically the Investment module.

The affected feature was simulation speed in a Detached Window:

• 1x
• 5x
• 20x
• 100x

Agreed requirements / acceptance criteria

The intended behavior was:

• Increasing speed should replay the same small simulation time steps faster rather than increasing the size of each simulation jump.
• Intermediate chart states should remain visually observable during playback.
• Stop should remain practically responsive during accelerated playback.
• The user should be able to manually close a position at an intermediate point during accelerated playback.

In other words, accelerated simulation should remain observable and interruptible.

What Codex treated as PASS

The automated test primarily measured how much simulation time advanced in approximately one real second.

The observed results were approximately:

• 1x: several simulation minutes
• 5x: more than 20 simulation minutes
• 20x: approximately 80 simulation minutes
• 100x: more than 400 simulation minutes

Because simulation time advanced according to the speed setting, the feature was treated as PASS.

The exact internal test command was not preserved in my retained logs, so I will not reconstruct or guess it.

Human test result: FAIL

The actual user-visible behavior did not satisfy the intended feature.

At higher speeds such as 100x, the system behaved closer to a time warp: one update could jump tens or hundreds of simulation minutes.

As a result:

• intermediate chart states were not practically observable,
• it was difficult or impossible to Stop at a desired intermediate point,
• manual intervention / position closing during the run was not practically usable.

So the proxy metric passed, while the actual user-facing requirement failed.

Relevant implementation detail

From the retained implementation information:

• The Detached Window does not have its own independent simulation timer.
• The simulation is driven by the main application's single simulation engine.

I cannot reliably determine from the retained logs whether all rendering and input processing were on exactly the same JavaScript thread/event loop, so I do not want to claim that.

Core problem

The central problem is not only this particular simulation-speed bug.

The larger problem is that requirements and completion criteria can become weakened or reinterpreted when work is handed from ChatGPT to Codex.

In this case:

“Final simulation datetime advanced sufficiently”

became a successful proxy for:

“The accelerated simulation remains observable, responsive, and manually interruptible.”

Those are not equivalent.

The proxy passed, but the actual human requirement failed.

This creates a dangerous failure mode:

ChatGPT conversation
→ requirements and intended behavior are agreed
→ task is handed to Codex
→ Codex creates or chooses its own measurable proxy
→ proxy passes
→ Codex reports PASS
→ human uses the result
→ actual feature is still FAIL

The problem becomes especially costly in larger projects because subsequent work may continue on top of a false PASS.

Primary feature request: Integrated ChatGPT + Codex mode

I would like an integrated mode where ChatGPT and Codex do not behave as separate stages with a lossy handoff.

The same conversation context, requirement state, and completion judgment should persist end-to-end across:

specification
→ implementation
→ execution
→ testing
→ PASS / FAIL decision

The agent performing implementation and testing should evaluate completion against the same criteria that were agreed with the user, rather than against a re-summarized or reinterpreted version of the task.

Ideally, ChatGPT could retain its conversation-level understanding and judgment while directly using Codex-class capabilities for:

• editing code
• executing commands
• running tests
• inspecting results
• iterating on implementation

This would reduce the need for a separate AI-to-AI handoff entirely.

Interim / complementary solution: Criteria Lock

If a fully integrated mode cannot be provided immediately, an intermediate solution would still be very useful.

When ChatGPT delegates work to Codex, allow a structured and immutable “Criteria Lock” contract to be attached to the task.

This contract should remain authoritative for the entire task.

It should include at minimum:

• acceptance criteria
• stop / failure conditions
• human-observable usability requirements
• explicit forbidden substitutions or proxies
• required evidence for each PASS condition

For example:

“Simulation time advancing quickly is NOT sufficient for PASS.

PASS additionally requires:

1. intermediate states remain observable,
2. Stop remains responsive,
3. manual close can succeed during accelerated playback.”

Codex should not silently replace these criteria with an easier proxy.

A GitHub-Issue-like structured format could work well for this purpose because the success conditions remain explicit and auditable.

Evidence-based PASS gating

Codex should map evidence to each locked acceptance criterion before reporting PASS.

For example:

Acceptance criterion	Evidence	Result
Simulation advances faster	measured simulation timestamps	PASS
Intermediate states observable	no valid evidence	INCONCLUSIVE / FAIL
Stop responsive during accelerated playback	no valid evidence	INCONCLUSIVE / FAIL
Manual close works mid-run	no valid evidence	INCONCLUSIVE / FAIL

If evidence for any required criterion is missing or ambiguous, Codex should return INCONCLUSIVE or FAIL.

It must not declare PASS solely because a proxy metric passed.

In particular:

If evidence is missing or ambiguous, Codex must not declare PASS based solely on proxy metrics.

Proposed solutions

1. Integrated mode — primary request

   Provide a unified ChatGPT + Codex mode that preserves the same conversation context and judgment state end-to-end:

   spec → implementation → execution → testing → PASS/FAIL.

   Acceptance decisions should be made against the original agreed criteria rather than a re-summarized handoff.

2. Criteria Lock — interim/complementary solution

   When delegating to Codex, attach a structured, immutable checklist that Codex must treat as authoritative:

   • acceptance criteria
   • stop/failure conditions
   • human-observable usability requirements
   • forbidden proxy substitutions
   • evidence required for PASS

   Proxy tests must not produce PASS unless all required human-observable criteria are evidenced.

3. Evidence-based PASS gating

   Require Codex to provide concrete evidence mapped to every locked criterion.

   If evidence is missing or ambiguous, Codex must return INCONCLUSIVE/FAIL and must not declare PASS based solely on proxy metrics.

Why this matters

Automated tests are valuable, but a technically measurable proxy is not necessarily the user's actual requirement.

The more autonomous Codex becomes, the more important this distinction becomes.

A system capable of making large code changes needs an equally strong mechanism for preserving the user's original definition of “done.”

Otherwise, greater implementation capability can also increase the cost of a false PASS.

The goal of this request is not to remove automated testing.

It is to make sure automated testing remains subordinate to the acceptance criteria that were actually agreed with the user.

[muhammadyusufpov]

Step 1 (Update your local branch): Switch to your default branch (main or master) and pull the latest changes:

git checkout main
git pull

Step 2 (Merge into your working branch): Switch back to your feature/pull request branch and merge the main branch to resolve any state discrepancies:

git checkout <your-branch-name>
git merge main

Step 3 (Resolve conflicts and push): If any merge conflicts appear, fix them in your code editors, stage the files (git add .), commit the changes, and push them to GitHub:

git push origin <your-branch-name>

(Note: If the "Pull request object state is invalid" error persists after pushing, closing this pull request and opening a fresh one usually clears the invalid database state on GitHub).