NAT Automatic Retries don't work with caching disabled

[katherineh123]

If caching is turned off, NAT automatic retries do not trigger.

Steps to reproduce:
First, trigger a 401 Unauthorized error in the checklist step by setting NVIDIA_API_KEY to a wrong key. To do this, run export NVIDIA_API_KEY="wrong-key" in your terminal. Then, in config.yml, configure the NAT automatic retry feature to retry on this error: add the string "Unauthorized" to the config field llms.checklist_llm.retry_on_errors

Now, if you run the workflow, you should see an error and a retry attempt:

2025-10-29 01:46:50 - INFO     - nat.utils.exception_handlers.automatic_retries:103 - Retrying on exception [401] Unauthorized
Authentication failed
Please check or regenerate your API key. with matched message [401] unauthorized
authentication failed
please check or regenerate your api key.

Next, turn off caching in config.yml by setting llm.checklist_llm.base_url to https://integrate.api.nvidia.com/v1.
If you try to run the workflow again, you will no longer see the retry attempt on the error.

[efajardo-nv]

@katherineh123 Could you try adding 401/Unauthorized retries to nim_embedder. I'm wondering if it's because VDB needs to be created and workflow is failing there before reaching checklist.