apple m1 아키텍춰에서 gpt2 사용시 오류 발생 이슈

[nuri428]

os : mac os 12.5.1
python version : 3.9.7
python packages :
torch 1.12.1
tensorflow-macos 2.8.0
tensorflow-metadata 1.7.0
tensorflow-metal 0.4.0

위와 같은 환경에서 아래와 같은 코드를 실행 시켰을때 오류 발생.

---

code

import torch
from transformers import GPT2LMHeadModel
from transformers import PreTrainedTokenizerFast

device = torch.device("mps")

device = torch.device("cpu")

fast = "skt/kogpt2-base-v2"
tokenizer = PreTrainedTokenizerFast.from_pretrained(fast)

model = GPT2LMHeadModel.from_pretrained("skt/kogpt2-base-v2").to(device=device, non_blocking=True)

text = "테스트입니당"
inputs = tokenizer(text, return_tensors="pt").to(device=device)

gen_ids = model.generate(

inputs["input_ids"],
max_length=128,
repetition_penalty=2.0,
pad_token_id=tokenizer.pad_token_id,
eos_token_id=tokenizer.eos_token_id,
bos_token_id=tokenizer.bos_token_id,
use_cache=True,

)
generated = tokenizer.decode(gen_ids[0])
print(generated)

---

에러 메세지

gen_ids = model.generate(

File "/Users/nuri/miniforge3/envs/env_nlp/lib/python3.9/site-packages/torch/autograd/grad_mode.py", line 27, in decorate_context
return func(*args, **kwargs)
File "/Users/nuri/miniforge3/envs/env_nlp/lib/python3.9/site-packages/transformers/generation_utils.py", line 1294, in generate
return self.greedy_search(
File "/Users/nuri/miniforge3/envs/env_nlp/lib/python3.9/site-packages/transformers/generation_utils.py", line 1689, in greedy_search
outputs = self(
File "/Users/nuri/miniforge3/envs/env_nlp/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
return forward_call(*input, **kwargs)
File "/Users/nuri/miniforge3/envs/env_nlp/lib/python3.9/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1058, in forward
transformer_outputs = self.transformer(
File "/Users/nuri/miniforge3/envs/env_nlp/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
return forward_call(*input, **kwargs)
File "/Users/nuri/miniforge3/envs/env_nlp/lib/python3.9/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 901, in forward
outputs = block(
File "/Users/nuri/miniforge3/envs/env_nlp/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
return forward_call(*input, **kwargs)
File "/Users/nuri/miniforge3/envs/env_nlp/lib/python3.9/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 401, in forward
attn_outputs = self.attn(
File "/Users/nuri/miniforge3/envs/env_nlp/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
return forward_call(*input, **kwargs)
File "/Users/nuri/miniforge3/envs/env_nlp/lib/python3.9/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 323, in forward
query, key, value = self.c_attn(hidden_states).split(self.split_size, dim=2)
File "/Users/nuri/miniforge3/envs/env_nlp/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
return forward_call(*input, **kwargs)
File "/Users/nuri/miniforge3/envs/env_nlp/lib/python3.9/site-packages/transformers/pytorch_utils.py", line 109, in forward
x = torch.addmm(self.bias, x.view(-1, x.size(-1)), self.weight)
RuntimeError: tensors must be 2-D

위의 코드에서 device를 cpu로 설정시는 정상 동작이 됩니다.

위 문제 관련하여 문의 드립니다.

[bage79]

@nuri428 답변이 늦어져서 죄송합니다.
제가 m1 맥북이 있지 않아서, 테스트 환경을 알아보는데 시간이 걸렸습니다.
테스트해보니, 내부적으로 아래와 같은 오류가 발생하였습니다.

File ~/.pyenv/versions/3.9.10/envs/kogpt2/lib/python3.9/site-packages/transformers/models/gpt2/modeling_gpt2.py:1011, in GPT2LMHeadModel.prepare_inputs_for_generation(self, input_ids, past, **kwargs)
   1007 position_ids = kwargs.get("position_ids", None)
   1009 if attention_mask is not None and position_ids is None:
   1010     # create position_ids on the fly for batch generation
-> 1011     position_ids = attention_mask.long().cumsum(-1) - 1
   1012     position_ids.masked_fill_(attention_mask == 0, 1)
   1013     if past:

NotImplementedError: The operator 'aten::cumsum.out' is not current implemented for the MPS device. If you want this op to be added in priority during the prototype phase of this feature, please comment on https://github.com/pytorch/pytorch/issues/77764. As a temporary fix, you can set the environment variable `PYTORCH_ENABLE_MPS_FALLBACK=1` to use the CPU as a fallback for this op. WARNING: this will be slower than running natively on MPS.

aten:cumsum이 MPS(m1 device)에서 지원되지 않아서 실행할 수 없는 것으로 보입니다.
cpu 또는 gpu(cuda) 를 사용하셔야할 것 같아요.

[nuri428]

@nuri428 답변이 늦어져서 죄송합니다. 제가 m1 맥북이 있지 않아서, 테스트 환경을 알아보는데 시간이 걸렸습니다. 테스트해보니, 내부적으로 아래와 같은 오류가 발생하였습니다.

File ~/.pyenv/versions/3.9.10/envs/kogpt2/lib/python3.9/site-packages/transformers/models/gpt2/modeling_gpt2.py:1011, in GPT2LMHeadModel.prepare_inputs_for_generation(self, input_ids, past, **kwargs)
   1007 position_ids = kwargs.get("position_ids", None)
   1009 if attention_mask is not None and position_ids is None:
   1010     # create position_ids on the fly for batch generation
-> 1011     position_ids = attention_mask.long().cumsum(-1) - 1
   1012     position_ids.masked_fill_(attention_mask == 0, 1)
   1013     if past:

NotImplementedError: The operator 'aten::cumsum.out' is not current implemented for the MPS device. If you want this op to be added in priority during the prototype phase of this feature, please comment on https://github.com/pytorch/pytorch/issues/77764. As a temporary fix, you can set the environment variable `PYTORCH_ENABLE_MPS_FALLBACK=1` to use the CPU as a fallback for this op. WARNING: this will be slower than running natively on MPS.

aten:cumsum이 MPS(m1 device)에서 지원되지 않아서 실행할 수 없는 것으로 보입니다. cpu 또는 gpu(cuda) 를 사용하셔야할 것 같아요.

답변 감사합니다.

우선은 cpu 모드로 사용중입니다.

혜웅님을 여기서 뵙네요 ^^

[bage79]

@nuri428 재호님 잘 지내시죠?
이 곳이 공개적인 공간이라서 아는 척을 못했네요. ^^