RTX 3050TI Not detecting m.get_device()

[the15thwiseone]

In .venv/lib/python3.10/site-packages/torch/_inductor there is this utils file.
The utils file basically keeps track of what GPU/CPU is currently running the main script for the sake of Torchrun.

I have a laptop with 3050TI, Cuda toolkit installed both in my windows, and my wsl. I am updating my driver.

In order to run the script runs/speedrun.sh after I made the changes to it so that it runs with 1 gpu in mind and is at 8 Size I must be in the wsl envrionment which is currently set to ubuntu 22.04.

While running it pitches the fault that Not enough SM to run MAX_AUTOTUNE_GMM. and then just stops there.

This is immediately after it basically tells me how many flops it would need to perform the tune, and the size of each batch.

I had antigravity set all torch.compile(mode=default) and it still had the same issue, since it's a prompt and it just haults the process entirely.

It even loads cuda when initiating the script, its several of the 81 dependencies it installs when it recreates the virtual environment.

I'm confused what to do here, since I did remove all torchrun and set the gpu=1 and batch_size=8. It just won't do anything.

Is it that I'm running it in wsl? I have tried in python and in base windows, and they both just don't work since the commands are just not supported.

Much appreciated for your response.

Also, most the windows repos in these forks have the same issues, they don't get past this issue.

[the15thwiseone]

This is the specified warning:
W0412 02:43:36.979000 716 .venv/lib/python3.10/site-packages/torch/_inductor/utils.py:1613] [0/0] Not enough SMs to use max_autotune_gemm mode
This is a file inside of the torch, and it uses triton.

SM logic is not relevant to ROCm gpus

# Arbitrarily skipping the older models

if torch.version.hip:
assert prop.major is not None
if prop.major < 9 or prop.major == 10:
log.warning("GPU arch does not support max_autotune_gemm mode usage")
return False
return True

min_sms = 16 if device.type == "xpu" else 68  # 3080
avail_sms = prop.multi_processor_count
if avail_sms < min_sms:
    log.warning(
        "Not enough SMs to use max_autotune_gemm mode",
        extra={"min_sms": min_sms, "avail_sms": avail_sms},
    )
    return False

This warning happens and the script just stops.
I do have a 3050TI, and it wont proceed past that point. The only thing I did was follow the instructions when it comes to speedrun.sh.

GPU =1, batch_size=8, replace torchrun with python relavent script.

I guess people were having issues with main_train, but I'm having issues with this script assigning my clearly CUDA cpu as a XPU and just not running.

[gouthamk16]

Does the process fail? Define "stops there". Because I don't think this is an issue, its a hardware constraint. The SM check is just a warning, does not stop execution.

If its just silence after the warning and the process itself does not fail, then its most probably just zero otuput, i.e., at step = 0, --eval-every defaults to 250, 0 % 250 == 0, so validation runs before the first training step. eval_steps = 80 * 524288 // (8 * 2048 * 1) = 2560 forward passes, and evaluate_bpb (loss_eval.py:31-53) prints nothing inside the loop. On one 3050 Ti that could lead to hours of zero output.

Or it might be getting OOM killed. At B=8, T=2048, V=32768: 8 * 2048 * 3768 * 4 = 2.15 GB, allocated twice. That alone exceeds a 4 GB card before model, grads, or optimizer state. And if you kept --depth=24 from speedrun.sh:67, that's a ~560M-param model with fp32 master weights plus Muon/Adam state, not remotely feasible on 4 gigs.

Try looking at runs/runcpu.sh :)