Skip to content

Commit 43bdec4

Browse files
committed
clean up example
Signed-off-by: Kyle Sayers <kylesayrs@gmail.com>
1 parent 87c79d8 commit 43bdec4

File tree

1 file changed

+3
-1
lines changed

1 file changed

+3
-1
lines changed

examples/disk_offloading/kimi_k2_example.py

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -11,8 +11,9 @@
1111
model_id,
1212
dtype="auto",
1313
device_map="auto_offload", # fit as much as possible on cpu, rest goes on disk
14+
trust_remote_code=True,
1415
)
15-
tokenizer = AutoTokenizer.from_pretrained(model_id)
16+
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
1617

1718
# Confirm that model is dispatched correctly
1819
devices = {offloaded for _onloaded, offloaded in get_device_map(model).values()}
@@ -34,6 +35,7 @@
3435
# Apply algorithms.
3536
oneshot(
3637
model=model,
38+
processor=tokenizer,
3739
dataset=DATASET_ID,
3840
splits={"calibration": f"{DATASET_SPLIT}[:{NUM_CALIBRATION_SAMPLES}]"},
3941
recipe=recipe,

0 commit comments

Comments
 (0)