Skip to content

refactor(retry): server retry use tenacity#2254

Open
salignatmoandal wants to merge 2 commits intobasetenlabs:mainfrom
salignatmoandal:refactor/server-retry-use-tenacity
Open

refactor(retry): server retry use tenacity#2254
salignatmoandal wants to merge 2 commits intobasetenlabs:mainfrom
salignatmoandal:refactor/server-retry-use-tenacity

Conversation

@salignatmoandal
Copy link

What

  • Replaces the custom retry implementation in truss/templates/server/common/retry.py with tenacity.
  • Adds tenacity to truss/templates/server/requirements.txt so the server runtime has the dependency when building/running the image.

This addresses the existing TODO in the file and aligns the server template with the rest of the codebase, which already uses tenacity (e.g. truss_handle, remote/baseten/service, control server).

How

  • retry.py: Implemented the same retry behaviour using tenacity.Retrying with:
    • stop_after_attempt(count + 1) for the same total number of attempts (1 initial + count retries).
    • wait_fixed(gap_seconds) for the delay between attempts.
    • before_sleep callback to log the same messages as before ("Retrying..." on first failure, "Retry count: N" on subsequent ones).
    • reraise=True so the original exception is propagated after the last failure.
  • requirements.txt: Added tenacity>=8.0.1 (aligned with the version in the project root pyproject.toml).

The public API of retry() is unchanged, so no updates were needed in model_wrapper.py.

Testing

  • All existing tests pass without modification:
    • uv run pytest truss/tests/templates/server/common/test_retry.py -v — 5/5 passed.
  • Behaviour is preserved: same number of attempts, same log messages, same exception propagation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant