docs: document Tesla T4 (Turing/sm_75) GPU setup pitfalls

[moduvoice]

Motivation

Verifying TabPFN inference end-to-end on a real Tesla T4 (16GB, Turing/sm_75)
surfaced two doc gaps that cost time on this common hardware/driver
combination, plus a couple of things worth stating explicitly since they were
non-obvious until measured:

1. Silent CPU fallback. README-verbatim pip install tabpfn resolved
   torch==2.13.0+cu130 on this box. The driver (550.163.01) only supports
   CUDA up to 12.4, so torch.cuda.is_available() was False —
   RuntimeError: The NVIDIA driver on your system is too old (found version 12040) during a manual check. device="auto" doesn't raise on this; it
   just returns a CPU device with no warning, so an agent/user can end up
   silently running on CPU while believing GPU is in use.
2. License gate not visible where Quick Start is read. The default
   TabPFN-3 checkpoint requires a one-time interactive browser login or a
   TABPFN_TOKEN (documented later, in the License / FAQ sections). Running
   the Quick Start example headless raises tabpfn.errors.TabPFNLicenseError
   with no forward-reference from Basic Usage to the ungated TabPFN-2
   (ModelVersion.V2, Apache-2.0) alternative.

This is a documentation-only PR — no code or default behavior changes — filed
directly per the doc_improvement issue template's own guidance ("Alternatively
you can just open a pull request with the suggested change").

Changes

• README — Installation: add a note that pip install tabpfn can resolve
  a PyTorch build newer than the local CUDA driver supports, with the
  matching-wheel command for CUDA-12.4-only drivers and a one-line
  torch.cuda.is_available() verification snippet.
• README — Basic Usage: add a note that device="auto" silently falls
  back to CPU with no warning, and surface the TabPFN-3 license
  requirement + ungated V2 alternative right next to the Quick Start code
  (previously only in the License section further down).
• New AGENTS.md: documents, from direct T4 measurement:
  • autocast dtype under inference_precision="auto" is torch.float16
    (verified via SDPA input dtype) — correct for Turing, since sm_75 has no
    bf16 tensor cores; the # bfloat16 comment next to
    AUTOCAST_DTYPE_BYTE_SIZE in constants.py refers to byte size, not the
    autocast dtype, and shouldn't be read as "bf16 is the default."
  • SDPA backend selection on sm_75: FlashAttention/cuDNN kernels are
    unavailable (No available kernel), so the repo's existing backend list
    ([FLASH, EFFICIENT, CUDNN, MATH]) already falls through to EFFICIENT
    (memory-efficient attention) — no config needed, just documenting that
    this is expected and already handled.
  • measured peak VRAM / timing on T4 across dataset sizes up to TabPFN-2's
    10,000-sample validation limit (rejected via TabPFNValidationError,
    not OOM).

No dependency pins, defaults, or code paths are changed.

Testing

Environment: Tesla T4 16GB · driver 550.163.01 (CUDA 12.4 runtime) ·
Python 3.10.12 · torch==2.6.0+cu124 · tabpfn==8.0.8 · numpy==2.2.6 ·
scikit-learn==1.7.2.

Reproduction of the documented issues:

# ENV — verbatim `pip install tabpfn` install:
torch 2.13.0+cu130 cuda_build 13.0
is_available False
RuntimeError('The NVIDIA driver on your system is too old (found version 12040). ...')

# After installing the matching cu124 build:
torch 2.6.0+cu124, is_available=True, get_device_name(0)='Tesla T4'

# DOC — default TabPFN-3 headless:
tabpfn.errors.TabPFNLicenseError: TabPFN requires a one-time license acceptance
to download model weights for local inference, but no interactive terminal is
available. ... export TABPFN_TOKEN="<your-api-key>" ...

Measured via the repo's own fit/predict_proba API
(TabPFNClassifier.create_default_for_version(ModelVersion.V2), no code
changes), torch.autocast/SDPA dtype introspection, and
torch.cuda.max_memory_allocated():

Dataset	autocast dtype	SDPA backend	Peak VRAM	fit / predict
breast-cancer (569×30)	float16	EFFICIENT	0.15 GB	0.42s / 0.41s (ROC-AUC 0.997)
synthetic 2,000×50	float16	EFFICIENT	0.59 GB	0.70s / 3.57s
synthetic 10,000×100	float16	EFFICIENT	5.33 GB	3.49s / 84.3s
synthetic 24,000×200	—	—	—	rejected: TabPFNValidationError (>10,000 samples)

SDPA backend probe on sm_75, each backend forced explicitly:
FLASH: RuntimeError: No available kernel · EFFICIENT: OK ·
CUDNN: RuntimeError: No available kernel · MATH: OK.

Raw metrics JSON

{"breast_cancer_v2": {"device":"auto","use_autocast_":true,"forced_inference_dtype_":"None",
  "n_estimators":8,"inference_precision":"auto","autocast_enabled":true,
  "autocast_dtype":"torch.float16","q_dtype":"torch.float16","q_device":"cuda:0",
  "roc_auc":0.997,"acc":0.9734,"fit_s":0.423,"predict_warm_s_mean":0.4119,
  "peak_vram_gb":0.146},
 "sdpa_backend_probe_sm75": {"FLASH":"FAIL:RuntimeError:No available kernel. Aborting execution.",
  "EFFICIENT":"OK","CUDNN":"FAIL:RuntimeError:No available kernel. Aborting execution.","MATH":"OK"},
 "scale": [{"n":2000,"f":50,"n_est":8,"fit_s":0.7,"pred_s":3.57,"peak_vram_gb":0.59,"ok":true},
  {"n":10000,"f":100,"n_est":8,"fit_s":3.49,"pred_s":84.34,"peak_vram_gb":5.33,"ok":true},
  {"n":30000,"f":200,"ok":false,"err":"TabPFNValidationError: Number of samples 24,000 ... greater than the maximum number of samples 10,000 officially supported"}]}

• The changes have been tested locally.
• Documentation has been updated (this PR is documentation only).
• Changelog: docs-only change, no changelog entry added — happy to add one if maintainers prefer.
• The code follows the project's style guidelines (Markdown only, no code changes).
• I have considered the impact of these changes on the public API: none — no public API changes.

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[gemini-code-assist]

Code Review

This pull request adds a new documentation file AGENTS.md and updates README.md to guide users on GPU detection, CUDA driver mismatches, and hardware-specific configurations for Turing-class GPUs. The review comments correctly point out that the suggested verification command will crash with an AssertionError if CUDA is unavailable, and provide a safer alternative using Python's short-circuiting and operator.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

[gemini-code-assist]

If torch.cuda.is_available() is False, calling torch.cuda.get_device_name(0) will raise an unhandled RuntimeError or AssertionError (e.g., "AssertionError: Torch not compiled with CUDA enabled"). This causes the verification command to crash with a traceback instead of cleanly returning False. Using Python's short-circuiting and operator prevents this crash by only evaluating get_device_name(0) if CUDA is actually available.

Suggested change

	> python -c "import torch; print(torch.cuda.is_available(), torch.cuda.get_device_name(0))"
	> python -c "import torch; print(torch.cuda.is_available() and torch.cuda.get_device_name(0))"

[gemini-code-assist]

If torch.cuda.is_available() is False, calling torch.cuda.get_device_name(0) will raise an unhandled RuntimeError or AssertionError. This causes the verification command to crash with a traceback instead of cleanly returning False. Using Python's short-circuiting and operator prevents this crash by only evaluating get_device_name(0) if CUDA is actually available.

Suggested change

	python -c "import torch; print(torch.cuda.is_available(), torch.cuda.get_device_name(0))"
	python -c "import torch; print(torch.cuda.is_available() and torch.cuda.get_device_name(0))"

[alanprior]

Thank you for your contribution @moduvoice

While I believe that some of the concerns raised here are valid and should be well-documented, I'm unsure whether we want to add a new file the way it was implemented. Also, as we want that the tone and style will be consistent across applications.

Could you raise the main concerns you think are worthy of highlighting?