Getting an error Error(s) in loading state_dict for LCFS_ATEPC

[kirankm]

I was running this code

from pyabsa import AspectTermExtraction as ATEPC
config = (
ATEPC.ATEPCConfigManager.get_atepc_config_english()
)

config.model = ATEPC.ATEPCModelList.LCFS_ATEPC

I tried other models as well

trainer = ATEPC.ATEPCTrainer(
config=config,
dataset=dataset,
from_checkpoint="english", # if you want to resume training from our pretrained checkpoints, you can pass the checkpoint name here
auto_device=DeviceTypeOption.AUTO, # use cuda if available
checkpoint_save_mode=ModelSaveOption.SAVE_MODEL_STATE_DICT, # save state dict only instead of the whole model
load_aug=False, # there are some augmentation dataset for integrated datasets, you use them by setting load_aug=True to improve performance
)

I am getting the following error.

---

RuntimeError Traceback (most recent call last)
/var/tmp/ipykernel_9733/2525470414.py in
19 auto_device=DeviceTypeOption.AUTO, # use cuda if available
20 checkpoint_save_mode=ModelSaveOption.SAVE_MODEL_STATE_DICT, # save state dict only instead of the whole model
---> 21 load_aug=False, # there are some augmentation dataset for integrated datasets, you use them by setting load_aug=True to improve performance
22 )

/opt/conda/lib/python3.7/site-packages/pyabsa/tasks/AspectTermExtraction/trainer/atepc_trainer.py in init(self, config, dataset, from_checkpoint, checkpoint_save_mode, auto_device, path_to_save, load_aug)
62 )
63
---> 64 self._run()

/opt/conda/lib/python3.7/site-packages/pyabsa/framework/trainer_class/trainer_template.py in _run(self)
216 self.config.seed = s
217 if self.config.checkpoint_save_mode:
--> 218 model_path.append(self.training_instructor(self.config).run())
219 else:
220 # always return the last trained model if you don't save trained model

/opt/conda/lib/python3.7/site-packages/pyabsa/tasks/AspectTermExtraction/instructor/atepc_instructor.py in run(self)
793
794 def run(self):
--> 795 return self._train(criterion=None)

/opt/conda/lib/python3.7/site-packages/pyabsa/framework/instructor_class/instructor_template.py in _train(self, criterion)
291 pass
292
--> 293 self._resume_from_checkpoint()
294
295 if self.config.warmup_step >= 0:

/opt/conda/lib/python3.7/site-packages/pyabsa/framework/instructor_class/instructor_template.py in _resume_from_checkpoint(self)
354 self.model.load_state_dict(
355 torch.load(
--> 356 state_dict_path[0], map_location=self.config.device
357 )
358 )

/opt/conda/lib/python3.7/site-packages/torch/nn/modules/module.py in load_state_dict(self, state_dict, strict)
1603 if len(error_msgs) > 0:
1604 raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
-> 1605 self.class.name, "\n\t".join(error_msgs)))
1606 return _IncompatibleKeys(missing_keys, unexpected_keys)
1607

RuntimeError: Error(s) in loading state_dict for LCFS_ATEPC:
Missing key(s) in state_dict: "bert4global.embeddings.position_ids", "bert4global.embeddings.word_embeddings.weight", "bert4global.embeddings.LayerNorm.weight", "bert4global.embeddings.LayerNorm.bias", "bert4global.encoder.layer.0.attention.self.query_proj.weight", "bert4global.encoder.layer.0.attention.self.query_proj.bias", "bert4global.encoder.layer.0.attention.self.key_proj.weight", "bert4global.encoder.layer.0.attention.self.key_proj.bias", "bert4global.encoder.layer.0.attention.self.value_proj.weight", "bert4global.encoder.layer.0.attention.self.value_proj.bias", "bert4global.encoder.layer.0.attention.output.dense.weight", "bert4global.encoder.layer.0.attention.output.dense.bias", "bert4global.encoder.layer.0.attention.output.LayerNorm.weight", "bert4global.encoder.layer.0.attention.output.LayerNorm.bias", "bert4global.encoder.layer.0.intermediate.dense.weight", "bert4global.encoder.layer.0.intermediate.dense.bias", "bert4global.encoder.layer.0.output.dense.weight", "bert4global.encoder.layer.0.output.dense.bias", "bert4global.encoder.layer.0.output.LayerNorm.weight", "bert4global.encoder.layer.0.output.LayerNorm.bias", "bert4global.encoder.layer.1.attention.self.query_proj.weight", "bert4global.encoder.layer.1.attention.self.query_proj.bias", "bert4global.encoder.layer.1.attention.self.key_proj.weight", "bert4global.encoder.layer.1.attention.self.key_proj.bias", "bert4global.encoder.layer.1.attention.self.value_proj.weight", "bert4global.encoder.layer.1.attention.self.value_proj.bias", "bert4global.encoder.layer.1.attention.output.dense.weight", "bert4global.encoder.layer.1.attention.output.dense.bias", "bert4global.encoder.layer.1.attention.output.LayerNorm.weight", "bert4global.encoder.layer.1.attention.output.LayerNorm.bias", "bert4global.encoder.layer.1.intermediate.dense.weight", "bert4global.encoder.layer.1.intermediate.dense.bias", "bert4global.encoder.layer.1.output.dense.weight", "bert4global.encoder.layer.1.output.dense.bias", "bert4global.encoder.layer.1.output.LayerNorm.weight", "bert4global.encoder.layer.1.output.LayerNorm.bias", "bert4global.encoder.layer.2.attention.self.query_proj.weight", "bert4global.encoder.layer.2.attention.self.query_proj.bias", "bert4global.encoder.layer.2.attention.self.key_proj.weight", "bert4global.encoder.layer.2.attention.self.key_proj.bias", "bert4global.encoder.layer.2.attention.self.value_proj.weight", "bert4global.encoder.layer.2.attention.self.value_proj.bias", "bert4global.encoder.layer.2.attention.output.dense.weight", "bert4global.encoder.layer.2.attention.output.dense.bias", "bert4global.encoder.layer.2.attention.output.LayerNorm.weight", "bert4global.encoder.layer.2.attention.output.LayerNorm.bias", "bert4global.encoder.layer.2.intermediate.dense.weight", "bert4global.encoder.layer.2.intermediate.dense.bias", "bert4global.encoder.layer.2.output.dense.weight", "bert4global.encoder.layer.2.output.dense.bias", "bert4global.encoder.layer.2.output.LayerNorm.weight", "bert4global.encoder.layer.2.output.LayerNorm.bias", "bert4global.encoder.layer.3.attention.self.query_proj.weight", "bert4global.encoder.layer.3.attention.self.query_proj.bias", "bert4global.encoder.layer.3.attention.self.key_proj.weight", "bert4global.encoder.layer.3.attention.self.key_proj.bias", "bert4global.encoder.layer.3.attention.self.value_proj.weight", "bert4global.encoder.layer.3.attention.self.value_proj.bias", "bert4global.encoder.layer.3.attention.output.dense.weight", "bert4global.encoder.layer.3.attention.output.dense.bias", "bert4global.encoder.layer.3.attention.output.LayerNorm.weight", "bert4global.encoder.layer.3.attention.output.LayerNorm.bias", "bert4global.encoder.layer.3.intermediate.dense.weight", "bert4global.encoder.layer.3.intermediate.dense.bias", "bert4global.encoder.layer.3.output.dense.weight", "bert4global.encoder.layer.3.output.dense.bias", "bert4global.encoder.layer.3.output.LayerNorm.weight", "bert4global.encoder.layer.3.output.LayerNorm.bias", "bert4global.encoder.layer.4.attention.self.query_proj.weight", "bert4global.encoder.layer.4.attention.self.query_proj.bias", "bert4global.encoder.layer.4.attention.self.key_proj.weight", "bert4global.encoder.layer.4.attention.self.key_proj.bias", "bert4global.encoder.layer.4.attention.self.value_proj.weight", "bert4global.encoder.layer.4.attention.self.value_proj.bias", "bert4global.encoder.layer.4.attention.output.dense.weight", "bert4global.encoder.layer.4.attention.output.dense.bias", "bert4global.encoder.layer.4.attention.output.LayerNorm.weight", "bert4global.encoder.layer.4.attention.output.LayerNorm.bias", "bert4global.encoder.layer.4.intermediate.dense.weight", "bert4global.encoder.layer.4.intermediate.dense.bias", "bert4global.encoder.layer.4.output.dense.weight", "bert4global.encoder.layer.4.output.dense.bias", "bert4global.encoder.layer.4.output.LayerNorm.weight", "bert4global.encoder.layer.4.output.LayerNorm.bias", "bert4global.encoder.layer.5.attention.self.query_proj.weight", "bert4global.encoder.layer.5.attention.self.query_proj.bias", "bert4global.encoder.layer.5.attention.self.key_proj.weight", "bert4global.encoder.layer.5.attention.self.key_proj.bias", "bert4global.encoder.layer.5.attention.self.value_proj.weight", "bert4global.encoder.layer.5.attention.self.value_proj.bias", "bert4global.encoder.layer.5.attention.output.dense.weight", "bert4global.encoder.layer.5.attention.output.dense.bias", "bert4global.encoder.layer.5.attention.output.LayerNorm.weight", "bert4global.encoder.layer.5.attention.output.LayerNorm.bias", "bert4global.encoder.layer.5.intermediate.dense.weight", "bert4global.encoder.layer.5.intermediate.dense.bias", "bert4global.encoder.layer.5.output.dense.weight", "bert4global.encoder.layer.5.output.dense.bias", "bert4global.encoder.layer.5.output.LayerNorm.weight", "bert4global.encoder.layer.5.output.LayerNorm.bias", "bert4global.encoder.layer.6.attention.self.query_proj.weight", "bert4global.encoder.layer.6.attention.self.query_proj.bias", "bert4global.encoder.layer.6.attention.self.key_proj.weight", "bert4global.encoder.layer.6.attention.self.key_proj.bias", "bert4global.encoder.layer.6.attention.self.value_proj.weight", "bert4global.encoder.layer.6.attention.self.value_proj.bias", "bert4global.encoder.layer.6.attention.output.dense.weight", "bert4global.encoder.layer.6.attention.output.dense.bias", "bert4global.encoder.layer.6.attention.output.LayerNorm.weight", "bert4global.encoder.layer.6.attention.output.LayerNorm.bias", "bert4global.encoder.layer.6.intermediate.dense.weight", "bert4global.encoder.layer.6.intermediate.dense.bias", "bert4global.encoder.layer.6.output.dense.weight", "bert4global.encoder.layer.6.output.dense.bias", "bert4global.encoder.layer.6.output.LayerNorm.weight", "bert4global.encoder.layer.6.output.LayerNorm.bias", "bert4global.encoder.layer.7.attention.self.query_proj.weight", "bert4global.encoder.layer.7.attention.self.query_proj.bias", "bert4global.encoder.layer.7.attention.self.key_proj.weight", "bert4global.encoder.layer.7.attention.self.key_proj.bias", "bert4global.encoder.layer.7.attention.self.value_proj.weight", "bert4global.encoder.layer.7.attention.self.value_proj.bias", "bert4global.encoder.layer.7.attention.output.dense.weight", "bert4global.encoder.layer.7.attention.output.dense.bias", "bert4global.encoder.layer.7.attention.output.LayerNorm.weight", "bert4global.encoder.layer.7.attention.output.LayerNorm.bias", "bert4global.encoder.layer.7.intermediate.dense.weight", "bert4global.encoder.layer.7.intermediate.dense.bias", "bert4global.encoder.layer.7.output.dense.weight", "bert4global.encoder.layer.7.output.dense.bias", "bert4global.encoder.layer.7.output.LayerNorm.weight", "bert4global.encoder.layer.7.output.LayerNorm.bias", "bert4global.encoder.layer.8.attention.self.query_proj.weight", "bert4global.encoder.layer.8.attention.self.query_proj.bias", "bert4global.encoder.layer.8.attention.self.key_proj.weight", "bert4global.encoder.layer.8.attention.self.key_proj.bias", "bert4global.encoder.layer.8.attention.self.value_proj.weight", "bert4global.encoder.layer.8.attention.self.value_proj.bias", "bert4global.encoder.layer.8.attention.output.dense.weight", "bert4global.encoder.layer.8.attention.output.dense.bias", "bert4global.encoder.layer.8.attention.output.LayerNorm.weight", "bert4global.encoder.layer.8.attention.output.LayerNorm.bias", "bert4global.encoder.layer.8.intermediate.dense.weight", "bert4global.encoder.layer.8.intermediate.dense.bias", "bert4global.encoder.layer.8.output.dense.weight", "bert4global.encoder.layer.8.output.dense.bias", "bert4global.encoder.layer.8.output.LayerNorm.weight", "bert4global.encoder.layer.8.output.LayerNorm.bias", "bert4global.encoder.layer.9.attention.self.query_proj.weight", "bert4global.encoder.layer.9.attention.self.query_proj.bias", "bert4global.encoder.layer.9.attention.self.key_proj.weight", "bert4global.encoder.layer.9.attention.self.key_proj.bias", "bert4global.encoder.layer.9.attention.self.value_proj.weight", "bert4global.encoder.layer.9.attention.self.value_proj.bias", "bert4global.encoder.layer.9.attention.output.dense.weight", "bert4global.encoder.layer.9.attention.output.dense.bias", "bert4global.encoder.layer.9.attention.output.LayerNorm.weight", "bert4global.encoder.layer.9.attention.output.LayerNorm.bias", "bert4global.encoder.layer.9.intermediate.dense.weight", "bert4global.encoder.layer.9.intermediate.dense.bias", "bert4global.encoder.layer.9.output.dense.weight", "bert4global.encoder.layer.9.output.dense.bias", "bert4global.encoder.layer.9.output.LayerNorm.weight", "bert4global.encoder.layer.9.output.LayerNorm.bias", "bert4global.encoder.layer.10.attention.self.query_proj.weight", "bert4global.encoder.layer.10.attention.self.query_proj.bias", "bert4global.encoder.layer.10.attention.self.key_proj.weight", "bert4global.encoder.layer.10.attention.self.key_proj.bias", "bert4global.encoder.layer.10.attention.self.value_proj.weight", "bert4global.encoder.layer.10.attention.self.value_proj.bias", "bert4global.encoder.layer.10.attention.output.dense.weight", "bert4global.encoder.layer.10.attention.output.dense.bias", "bert4global.encoder.layer.10.attention.output.LayerNorm.weight", "bert4global.encoder.layer.10.attention.output.LayerNorm.bias", "bert4global.encoder.layer.10.intermediate.dense.weight", "bert4global.encoder.layer.10.intermediate.dense.bias", "bert4global.encoder.layer.10.output.dense.weight", "bert4global.encoder.layer.10.output.dense.bias", "bert4global.encoder.layer.10.output.LayerNorm.weight", "bert4global.encoder.layer.10.output.LayerNorm.bias", "bert4global.encoder.layer.11.attention.self.query_proj.weight", "bert4global.encoder.layer.11.attention.self.query_proj.bias", "bert4global.encoder.layer.11.attention.self.key_proj.weight", "bert4global.encoder.layer.11.attention.self.key_proj.bias", "bert4global.encoder.layer.11.attention.self.value_proj.weight", "bert4global.encoder.layer.11.attention.self.value_proj.bias", "bert4global.encoder.layer.11.attention.output.dense.weight", "bert4global.encoder.layer.11.attention.output.dense.bias", "bert4global.encoder.layer.11.attention.output.LayerNorm.weight", "bert4global.encoder.layer.11.attention.output.LayerNorm.bias", "bert4global.encoder.layer.11.intermediate.dense.weight", "bert4global.encoder.layer.11.intermediate.dense.bias", "bert4global.encoder.layer.11.output.dense.weight", "bert4global.encoder.layer.11.output.dense.bias", "bert4global.encoder.layer.11.output.LayerNorm.weight", "bert4global.encoder.layer.11.output.LayerNorm.bias", "bert4global.encoder.rel_embeddings.weight", "bert4global.encoder.LayerNorm.weight", "bert4global.encoder.LayerNorm.bias", "bert4local.embeddings.position_ids", "bert4local.embeddings.word_embeddings.weight", "bert4local.embeddings.LayerNorm.weight", "bert4local.embeddings.LayerNorm.bias", "bert4local.encoder.layer.0.attention.self.query_proj.weight", "bert4local.encoder.layer.0.attention.self.query_proj.bias", "bert4local.encoder.layer.0.attention.self.key_proj.weight", "bert4local.encoder.layer.0.attention.self.key_proj.bias", "bert4local.encoder.layer.0.attention.self.value_proj.weight", "bert4local.encoder.layer.0.attention.self.value_proj.bias", "bert4local.encoder.layer.0.attention.output.dense.weight", "bert4local.encoder.layer.0.attention.output.dense.bias", "bert4local.encoder.layer.0.attention.output.LayerNorm.weight", "bert4local.encoder.layer.0.attention.output.LayerNorm.bias", "bert4local.encoder.layer.0.intermediate.dense.weight", "bert4local.encoder.layer.0.intermediate.dense.bias", "bert4local.encoder.layer.0.output.dense.weight", "bert4local.encoder.layer.0.output.dense.bias", "bert4local.encoder.layer.0.output.LayerNorm.weight", "bert4local.encoder.layer.0.output.LayerNorm.bias", "bert4local.encoder.layer.1.attention.self.query_proj.weight", "bert4local.encoder.layer.1.attention.self.query_proj.bias", "bert4local.encoder.layer.1.attention.self.key_proj.weight", "bert4local.encoder.layer.1.attention.self.key_proj.bias", "bert4local.encoder.layer.1.attention.self.value_proj.weight", "bert4local.encoder.layer.1.attention.self.value_proj.bias", "bert4local.encoder.layer.1.attention.output.dense.weight", "bert4local.encoder.layer.1.attention.output.dense.bias", "bert4local.encoder.layer.1.attention.output.LayerNorm.weight", "bert4local.encoder.layer.1.attention.output.LayerNorm.bias", "bert4local.encoder.layer.1.intermediate.dense.weight", "bert4local.encoder.layer.1.intermediate.dense.bias", "bert4local.encoder.layer.1.output.dense.weight", "bert4local.encoder.layer.1.output.dense.bias", "bert4local.encoder.layer.1.output.LayerNorm.weight", "bert4local.encoder.layer.1.output.LayerNorm.bias", "bert4local.encoder.layer.2.attention.self.query_proj.weight", "bert4local.encoder.layer.2.attention.self.query_proj.bias", "bert4local.encoder.layer.2.attention.self.key_proj.weight", "bert4local.encoder.layer.2.attention.self.key_proj.bias", "bert4local.encoder.layer.2.attention.self.value_proj.weight", "bert4local.encoder.layer.2.attention.self.value_proj.bias", "bert4local.encoder.layer.2.attention.output.dense.weight", "bert4local.encoder.layer.2.attention.output.dense.bias", "bert4local.encoder.layer.2.attention.output.LayerNorm.weight", "bert4local.encoder.layer.2.attention.output.LayerNorm.bias", "bert4local.encoder.layer.2.intermediate.dense.weight", "bert4local.encoder.layer.2.intermediate.dense.bias", "bert4local.encoder.layer.2.output.dense.weight", "bert4local.encoder.layer.2.output.dense.bias", "bert4local.encoder.layer.2.output.LayerNorm.weight", "bert4local.encoder.layer.2.output.LayerNorm.bias", "bert4local.encoder.layer.3.attention.self.query_proj.weight", "bert4local.encoder.layer.3.attention.self.query_proj.bias", "bert4local.encoder.layer.3.attention.self.key_proj.weight", "bert4local.encoder.layer.3.attention.self.key_proj.bias", "bert4local.encoder.layer.3.attention.self.value_proj.weight", "bert4local.encoder.layer.3.attention.self.value_proj.bias", "bert4local.encoder.layer.3.attention.output.dense.weight", "bert4local.encoder.layer.3.attention.output.dense.bias", "bert4local.encoder.layer.3.attention.output.LayerNorm.weight", "bert4local.encoder.layer.3.attention.output.LayerNorm.bias", "bert4local.encoder.layer.3.intermediate.dense.weight", "bert4local.encoder.layer.3.intermediate.dense.bias", "bert4local.encoder.layer.3.output.dense.weight", "bert4local.encoder.layer.3.output.dense.bias", "bert4local.encoder.layer.3.output.LayerNorm.weight", "bert4local.encoder.layer.3.output.LayerNorm.bias", "bert4local.encoder.layer.4.attention.self.query_proj.weight", "bert4local.encoder.layer.4.attention.self.query_proj.bias", "bert4local.encoder.layer.4.attention.self.key_proj.weight", "bert4local.encoder.layer.4.attention.self.key_proj.bias", "bert4local.encoder.layer.4.attention.self.value_proj.weight", "bert4local.encoder.layer.4.attention.self.value_proj.bias", "bert4local.encoder.layer.4.attention.output.dense.weight", "bert4local.encoder.layer.4.attention.output.dense.bias", "bert4local.encoder.layer.4.attention.output.LayerNorm.weight", "bert4local.encoder.layer.4.attention.output.LayerNorm.bias", "bert4local.encoder.layer.4.intermediate.dense.weight", "bert4local.encoder.layer.4.intermediate.dense.bias", "bert4local.encoder.layer.4.output.dense.weight", "bert4local.encoder.layer.4.output.dense.bias", "bert4local.encoder.layer.4.output.LayerNorm.weight", "bert4local.encoder.layer.4.output.LayerNorm.bias", "bert4local.encoder.layer.5.attention.self.query_proj.weight", "bert4local.encoder.layer.5.attention.self.query_proj.bias", "bert4local.encoder.layer.5.attention.self.key_proj.weight", "bert4local.encoder.layer.5.attention.self.key_proj.bias", "bert4local.encoder.layer.5.attention.self.value_proj.weight", "bert4local.encoder.layer.5.attention.self.value_proj.bias", "bert4local.encoder.layer.5.attention.output.dense.weight", "bert4local.encoder.layer.5.attention.output.dense.bias", "bert4local.encoder.layer.5.attention.output.LayerNorm.weight", "bert4local.encoder.layer.5.attention.output.LayerNorm.bias", "bert4local.encoder.layer.5.intermediate.dense.weight", "bert4local.encoder.layer.5.intermediate.dense.bias", "bert4local.encoder.layer.5.output.dense.weight", "bert4local.encoder.layer.5.output.dense.bias", "bert4local.encoder.layer.5.output.LayerNorm.weight", "bert4local.encoder.layer.5.output.LayerNorm.bias", "bert4local.encoder.layer.6.attention.self.query_proj.weight", "bert4local.encoder.layer.6.attention.self.query_proj.bias", "bert4local.encoder.layer.6.attention.self.key_proj.weight", "bert4local.encoder.layer.6.attention.self.key_proj.bias", "bert4local.encoder.layer.6.attention.self.value_proj.weight", "bert4local.encoder.layer.6.attention.self.value_proj.bias", "bert4local.encoder.layer.6.attention.output.dense.weight", "bert4local.encoder.layer.6.attention.output.dense.bias", "bert4local.encoder.layer.6.attention.output.LayerNorm.weight", "bert4local.encoder.layer.6.attention.output.LayerNorm.bias", "bert4local.encoder.layer.6.intermediate.dense.weight", "bert4local.encoder.layer.6.intermediate.dense.bias", "bert4local.encoder.layer.6.output.dense.weight", "bert4local.encoder.layer.6.output.dense.bias", "bert4local.encoder.layer.6.output.LayerNorm.weight", "bert4local.encoder.layer.6.output.LayerNorm.bias", "bert4local.encoder.layer.7.attention.self.query_proj.weight", "bert4local.encoder.layer.7.attention.self.query_proj.bias", "bert4local.encoder.layer.7.attention.self.key_proj.weight", "bert4local.encoder.layer.7.attention.self.key_proj.bias", "bert4local.encoder.layer.7.attention.self.value_proj.weight", "bert4local.encoder.layer.7.attention.self.value_proj.bias", "bert4local.encoder.layer.7.attention.output.dense.weight", "bert4local.encoder.layer.7.attention.output.dense.bias", "bert4local.encoder.layer.7.attention.output.LayerNorm.weight", "bert4local.encoder.layer.7.attention.output.LayerNorm.bias", "bert4local.encoder.layer.7.intermediate.dense.weight", "bert4local.encoder.layer.7.intermediate.dense.bias", "bert4local.encoder.layer.7.output.dense.weight", "bert4local.encoder.layer.7.output.dense.bias", "bert4local.encoder.layer.7.output.LayerNorm.weight", "bert4local.encoder.layer.7.output.LayerNorm.bias", "bert4local.encoder.layer.8.attention.self.query_proj.weight", "bert4local.encoder.layer.8.attention.self.query_proj.bias", "bert4local.encoder.layer.8.attention.self.key_proj.weight", "bert4local.encoder.layer.8.attention.self.key_proj.bias", "bert4local.encoder.layer.8.attention.self.value_proj.weight", "bert4local.encoder.layer.8.attention.self.value_proj.bias", "bert4local.encoder.layer.8.attention.output.dense.weight", "bert4local.encoder.layer.8.attention.output.dense.bias", "bert4local.encoder.layer.8.attention.output.LayerNorm.weight", "bert4local.encoder.layer.8.attention.output.LayerNorm.bias", "bert4local.encoder.layer.8.intermediate.dense.weight", "bert4local.encoder.layer.8.intermediate.dense.bias", "bert4local.encoder.layer.8.output.dense.weight", "bert4local.encoder.layer.8.output.dense.bias", "bert4local.encoder.layer.8.output.LayerNorm.weight", "bert4local.encoder.layer.8.output.LayerNorm.bias", "bert4local.encoder.layer.9.attention.self.query_proj.weight", "bert4local.encoder.layer.9.attention.self.query_proj.bias", "bert4local.encoder.layer.9.attention.self.key_proj.weight", "bert4local.encoder.layer.9.attention.self.key_proj.bias", "bert4local.encoder.layer.9.attention.self.value_proj.weight", "bert4local.encoder.layer.9.attention.self.value_proj.bias", "bert4local.encoder.layer.9.attention.output.dense.weight", "bert4local.encoder.layer.9.attention.output.dense.bias", "bert4local.encoder.layer.9.attention.output.LayerNorm.weight", "bert4local.encoder.layer.9.attention.output.LayerNorm.bias", "bert4local.encoder.layer.9.intermediate.dense.weight", "bert4local.encoder.layer.9.intermediate.dense.bias", "bert4local.encoder.layer.9.output.dense.weight", "bert4local.encoder.layer.9.output.dense.bias", "bert4local.encoder.layer.9.output.LayerNorm.weight", "bert4local.encoder.layer.9.output.LayerNorm.bias", "bert4local.encoder.layer.10.attention.self.query_proj.weight", "bert4local.encoder.layer.10.attention.self.query_proj.bias", "bert4local.encoder.layer.10.attention.self.key_proj.weight", "bert4local.encoder.layer.10.attention.self.key_proj.bias", "bert4local.encoder.layer.10.attention.self.value_proj.weight", "bert4local.encoder.layer.10.attention.self.value_proj.bias", "bert4local.encoder.layer.10.attention.output.dense.weight", "bert4local.encoder.layer.10.attention.output.dense.bias", "bert4local.encoder.layer.10.attention.output.LayerNorm.weight", "bert4local.encoder.layer.10.attention.output.LayerNorm.bias", "bert4local.encoder.layer.10.intermediate.dense.weight", "bert4local.encoder.layer.10.intermediate.dense.bias", "bert4local.encoder.layer.10.output.dense.weight", "bert4local.encoder.layer.10.output.dense.bias", "bert4local.encoder.layer.10.output.LayerNorm.weight", "bert4local.encoder.layer.10.output.LayerNorm.bias", "bert4local.encoder.layer.11.attention.self.query_proj.weight", "bert4local.encoder.layer.11.attention.self.query_proj.bias", "bert4local.encoder.layer.11.attention.self.key_proj.weight", "bert4local.encoder.layer.11.attention.self.key_proj.bias", "bert4local.encoder.layer.11.attention.self.value_proj.weight", "bert4local.encoder.layer.11.attention.self.value_proj.bias", "bert4local.encoder.layer.11.attention.output.dense.weight", "bert4local.encoder.layer.11.attention.output.dense.bias", "bert4local.encoder.layer.11.attention.output.LayerNorm.weight", "bert4local.encoder.layer.11.attention.output.LayerNorm.bias", "bert4local.encoder.layer.11.intermediate.dense.weight", "bert4local.encoder.layer.11.intermediate.dense.bias", "bert4local.encoder.layer.11.output.dense.weight", "bert4local.encoder.layer.11.output.dense.bias", "bert4local.encoder.layer.11.output.LayerNorm.weight", "bert4local.encoder.layer.11.output.LayerNorm.bias", "bert4local.encoder.rel_embeddings.weight", "bert4local.encoder.LayerNorm.weight", "bert4local.encoder.LayerNorm.bias", "SA1.encoder.0.SA.query.weight", "SA1.encoder.0.SA.query.bias", "SA1.encoder.0.SA.key.weight", "SA1.encoder.0.SA.key.bias", "SA1.encoder.0.SA.value.weight", "SA1.encoder.0.SA.value.bias", "SA2.encoder.0.SA.query.weight", "SA2.encoder.0.SA.query.bias", "SA2.encoder.0.SA.key.weight", "SA2.encoder.0.SA.key.bias", "SA2.encoder.0.SA.value.weight", "SA2.encoder.0.SA.value.bias", "linear_double.weight", "linear_double.bias", "linear_triple.weight", "linear_triple.bias", "pooler.dense.weight", "pooler.dense.bias", "classifier.weight", "classifier.bias".
Unexpected key(s) in state_dict: "models.0.bert4global.embeddings.position_ids", "models.0.bert4global.embeddings.word_embeddings.weight", "models.0.bert4global.embeddings.LayerNorm.weight", "models.0.bert4global.embeddings.LayerNorm.bias", "models.0.bert4global.encoder.layer.0.attention.self.query_proj.weight", "models.0.bert4global.encoder.layer.0.attention.self.query_proj.bias", "models.0.bert4global.encoder.layer.0.attention.self.key_proj.weight", "models.0.bert4global.encoder.layer.0.attention.self.key_proj.bias", "models.0.bert4global.encoder.layer.0.attention.self.value_proj.weight", "models.0.bert4global.encoder.layer.0.attention.self.value_proj.bias", "models.0.bert4global.encoder.layer.0.attention.output.dense.weight", "models.0.bert4global.encoder.layer.0.attention.output.dense.bias", "models.0.bert4global.encoder.layer.0.attention.output.LayerNorm.weight", "models.0.bert4global.encoder.layer.0.attention.output.LayerNorm.bias", "models.0.bert4global.encoder.layer.0.intermediate.dense.weight", "models.0.bert4global.encoder.layer.0.intermediate.dense.bias", "models.0.bert4global.encoder.layer.0.output.dense.weight", "models.0.bert4global.encoder.layer.0.output.dense.bias", "models.0.bert4global.encoder.layer.0.output.LayerNorm.weight", "models.0.bert4global.encoder.layer.0.output.LayerNorm.bias", "models.0.bert4global.encoder.layer.1.attention.self.query_proj.weight", "models.0.bert4global.encoder.layer.1.attention.self.query_proj.bias", "models.0.bert4global.encoder.layer.1.attention.self.key_proj.weight", "models.0.bert4global.encoder.layer.1.attention.self.key_proj.bias", "models.0.bert4global.encoder.layer.1.attention.self.value_proj.weight", "models.0.bert4global.encoder.layer.1.attention.self.value_proj.bias", "models.0.bert4global.encoder.layer.1.attention.output.dense.weight", "models.0.bert4global.encoder.layer.1.attention.output.dense.bias", "models.0.bert4global.encoder.layer.1.attention.output.LayerNorm.weight", "models.0.bert4global.encoder.layer.1.attention.output.LayerNorm.bias", "models.0.bert4global.encoder.layer.1.intermediate.dense.weight", "models.0.bert4global.encoder.layer.1.intermediate.dense.bias", "models.0.bert4global.encoder.layer.1.output.dense.weight", "models.0.bert4global.encoder.layer.1.output.dense.bias", "models.0.bert4global.encoder.layer.1.output.LayerNorm.weight", "models.0.bert4global.encoder.layer.1.output.LayerNorm.bias", "models.0.bert4global.encoder.layer.2.attention.self.query_proj.weight", "models.0.bert4global.encoder.layer.2.attention.self.query_proj.bias", "models.0.bert4global.encoder.layer.2.attention.self.key_proj.weight", "models.0.bert4global.encoder.layer.2.attention.self.key_proj.bias", "models.0.bert4global.encoder.layer.2.attention.self.value_proj.weight", "models.0.bert4global.encoder.layer.2.attention.self.value_proj.bias", "models.0.bert4global.encoder.layer.2.attention.output.dense.weight", "models.0.bert4global.encoder.layer.2.attention.output.dense.bias", "models.0.bert4global.encoder.layer.2.attention.output.LayerNorm.weight", "models.0.bert4global.encoder.layer.2.attention.output.LayerNorm.bias", "models.0.bert4global.encoder.layer.2.intermediate.dense.weight", "models.0.bert4global.encoder.layer.2.intermediate.dense.bias", "models.0.bert4global.encoder.layer.2.output.dense.weight", "models.0.bert4global.encoder.layer.2.output.dense.bias", "models.0.bert4global.encoder.layer.2.output.LayerNorm.weight", "models.0.bert4global.encoder.layer.2.output.LayerNorm.bias", "models.0.bert4global.encoder.layer.3.attention.self.query_proj.weight", "models.0.bert4global.encoder.layer.3.attention.self.query_proj.bias", "models.0.bert4global.encoder.layer.3.attention.self.key_proj.weight", "models.0.bert4global.encoder.layer.3.attention.self.key_proj.bias", "models.0.bert4global.encoder.layer.3.attention.self.value_proj.weight", "models.0.bert4global.encoder.layer.3.attention.self.value_proj.bias", "models.0.bert4global.encoder.layer.3.attention.output.dense.weight", "models.0.bert4global.encoder.layer.3.attention.output.dense.bias", "models.0.bert4global.encoder.layer.3.attention.output.LayerNorm.weight", "models.0.bert4global.encoder.layer.3.attention.output.LayerNorm.bias", "models.0.bert4global.encoder.layer.3.intermediate.dense.weight", "models.0.bert4global.encoder.layer.3.intermediate.dense.bias", "models.0.bert4global.encoder.layer.3.output.dense.weight", "models.0.bert4global.encoder.layer.3.output.dense.bias", "models.0.bert4global.encoder.layer.3.output.LayerNorm.weight", "models.0.bert4global.encoder.layer.3.output.LayerNorm.bias", "models.0.bert4global.encoder.layer.4.attention.self.query_proj.weight", "models.0.bert4global.encoder.layer.4.attention.self.query_proj.bias", "models.0.bert4global.encoder.layer.4.attention.self.key_proj.weight", "models.0.bert4global.encoder.layer.4.attention.self.key_proj.bias", "models.0.bert4global.encoder.layer.4.attention.self.value_proj.weight", "models.0.bert4global.encoder.layer.4.attention.self.value_proj.bias", "models.0.bert4global.encoder.layer.4.attention.output.dense.weight", "models.0.bert4global.encoder.layer.4.attention.output.dense.bias", "models.0.bert4global.encoder.layer.4.attention.output.LayerNorm.weight", "models.0.bert4global.encoder.layer.4.attention.output.LayerNorm.bias", "models.0.bert4global.encoder.layer.4.intermediate.dense.weight", "models.0.bert4global.encoder.layer.4.intermediate.dense.bias", "models.0.bert4global.encoder.layer.4.output.dense.weight", "models.0.bert4global.encoder.layer.4.output.dense.bias", "models.0.bert4global.encoder.layer.4.output.LayerNorm.weight", "models.0.bert4global.encoder.layer.4.output.LayerNorm.bias", "models.0.bert4global.encoder.layer.5.attention.self.query_proj.weight", "models.0.bert4global.encoder.layer.5.attention.self.query_proj.bias", "models.0.bert4global.encoder.layer.5.attention.self.key_proj.weight", "models.0.bert4global.encoder.layer.5.attention.self.key_proj.bias", "models.0.bert4global.encoder.layer.5.attention.self.value_proj.weight", "models.0.bert4global.encoder.layer.5.attention.self.value_proj.bias", "models.0.bert4global.encoder.layer.5.attention.output.dense.weight", "models.0.bert4global.encoder.layer.5.attention.output.dense.bias", "models.0.bert4global.encoder.layer.5.attention.output.LayerNorm.weight", "models.0.bert4global.encoder.layer.5.attention.output.LayerNorm.bias", "models.0.bert4global.encoder.layer.5.intermediate.dense.weight", "models.0.bert4global.encoder.layer.5.intermediate.dense.bias", "models.0.bert4global.encoder.layer.5.output.dense.weight", "models.0.bert4global.encoder.layer.5.output.dense.bias", "models.0.bert4global.encoder.layer.5.output.LayerNorm.weight", "models.0.bert4global.encoder.layer.5.output.LayerNorm.bias", "models.0.bert4global.encoder.layer.6.attention.self.query_proj.weight", "models.0.bert4global.encoder.layer.6.attention.self.query_proj.bias", "models.0.bert4global.encoder.layer.6.attention.self.key_proj.weight", "models.0.bert4global.encoder.layer.6.attention.self.key_proj.bias", "models.0.bert4global.encoder.layer.6.attention.self.value_proj.weight", "models.0.bert4global.encoder.layer.6.attention.self.value_proj.bias", "models.0.bert4global.encoder.layer.6.attention.output.dense.weight", "models.0.bert4global.encoder.layer.6.attention.output.dense.bias", "models.0.bert4global.encoder.layer.6.attention.output.LayerNorm.weight", "models.0.bert4global.encoder.layer.6.attention.output.LayerNorm.bias", "models.0.bert4global.encoder.layer.6.intermediate.dense.weight", "models.0.bert4global.encoder.layer.6.intermediate.dense.bias", "models.0.bert4global.encoder.layer.6.output.dense.weight", "models.0.bert4global.encoder.layer.6.output.dense.bias", "models.0.bert4global.encoder.layer.6.output.LayerNorm.weight", "models.0.bert4global.encoder.layer.6.output.LayerNorm.bias", "models.0.bert4global.encoder.layer.7.attention.self.query_proj.weight", "models.0.bert4global.encoder.layer.7.attention.self.query_proj.bias", "models.0.bert4global.encoder.layer.7.attention.self.key_proj.weight", "models.0.bert4global.encoder.layer.7.attention.self.key_proj.bias", "models.0.bert4global.encoder.layer.7.attention.self.value_proj.weight", "models.0.bert4global.encoder.layer.7.attention.self.value_proj.bias", "models.0.bert4global.encoder.layer.7.attention.output.dense.weight", "models.0.bert4global.encoder.layer.7.attention.output.dense.bias", "models.0.bert4global.encoder.layer.7.attention.output.LayerNorm.weight", "models.0.bert4global.encoder.layer.7.attention.output.LayerNorm.bias", "models.0.bert4global.encoder.layer.7.intermediate.dense.weight", "models.0.bert4global.encoder.layer.7.intermediate.dense.bias", "models.0.bert4global.encoder.layer.7.output.dense.weight", "models.0.bert4global.encoder.layer.7.output.dense.bias", "models.0.bert4global.encoder.layer.7.output.LayerNorm.weight", "models.0.bert4global.encoder.layer.7.output.LayerNorm.bias", "models.0.bert4global.encoder.layer.8.attention.self.query_proj.weight", "models.0.bert4global.encoder.layer.8.attention.self.query_proj.bias", "models.0.bert4global.encoder.layer.8.attention.self.key_proj.weight", "models.0.bert4global.encoder.layer.8.attention.self.key_proj.bias", "models.0.bert4global.encoder.layer.8.attention.self.value_proj.weight", "models.0.bert4global.encoder.layer.8.attention.self.value_proj.bias", "models.0.bert4global.encoder.layer.8.attention.output.dense.weight", "models.0.bert4global.encoder.layer.8.attention.output.dense.bias", "models.0.bert4global.encoder.layer.8.attention.output.LayerNorm.weight", "models.0.bert4global.encoder.layer.8.attention.output.LayerNorm.bias", "models.0.bert4global.encoder.layer.8.intermediate.dense.weight", "models.0.bert4global.encoder.layer.8.intermediate.dense.bias", "models.0.bert4global.encoder.layer.8.output.dense.weight", "models.0.bert4global.encoder.layer.8.output.dense.bias", "models.0.bert4global.encoder.layer.8.output.LayerNorm.weight", "models.0.bert4global.encoder.layer.8.output.LayerNorm.bias", "models.0.bert4global.encoder.layer.9.attention.self.query_proj.weight", "models.0.bert4global.encoder.layer.9.attention.self.query_proj.bias", "models.0.bert4global.encoder.layer.9.attention.self.key_proj.weight", "models.0.bert4global.encoder.layer.9.attention.self.key_proj.bias", "models.0.bert4global.encoder.layer.9.attention.self.value_proj.weight", "models.0.bert4global.encoder.layer.9.attention.self.value_proj.bias", "models.0.bert4global.encoder.layer.9.attention.output.dense.weight", "models.0.bert4global.encoder.layer.9.attention.output.dense.bias", "models.0.bert4global.encoder.layer.9.attention.output.LayerNorm.weight", "models.0.bert4global.encoder.layer.9.attention.output.LayerNorm.bias", "models.0.bert4global.encoder.layer.9.intermediate.dense.weight", "models.0.bert4global.encoder.layer.9.intermediate.dense.bias", "models.0.bert4global.encoder.layer.9.output.dense.weight", "models.0.bert4global.encoder.layer.9.output.dense.bias", "models.0.bert4global.encoder.layer.9.output.LayerNorm.weight", "models.0.bert4global.encoder.layer.9.output.LayerNorm.bias", "models.0.bert4global.encoder.layer.10.attention.self.query_proj.weight", "models.0.bert4global.encoder.layer.10.attention.self.query_proj.bias", "models.0.bert4global.encoder.layer.10.attention.self.key_proj.weight", "models.0.bert4global.encoder.layer.10.attention.self.key_proj.bias", "models.0.bert4global.encoder.layer.10.attention.self.value_proj.weight", "models.0.bert4global.encoder.layer.10.attention.self.value_proj.bias", "models.0.bert4global.encoder.layer.10.attention.output.dense.weight", "models.0.bert4global.encoder.layer.10.attention.output.dense.bias", "models.0.bert4global.encoder.layer.10.attention.output.LayerNorm.weight", "models.0.bert4global.encoder.layer.10.attention.output.LayerNorm.bias", "models.0.bert4global.encoder.layer.10.intermediate.dense.weight", "models.0.bert4global.encoder.layer.10.intermediate.dense.bias", "models.0.bert4global.encoder.layer.10.output.dense.weight", "models.0.bert4global.encoder.layer.10.output.dense.bias", "models.0.bert4global.encoder.layer.10.output.LayerNorm.weight", "models.0.bert4global.encoder.layer.10.output.LayerNorm.bias", "models.0.bert4global.encoder.layer.11.attention.self.query_proj.weight", "models.0.bert4global.encoder.layer.11.attention.self.query_proj.bias", "models.0.bert4global.encoder.layer.11.attention.self.key_proj.weight", "models.0.bert4global.encoder.layer.11.attention.self.key_proj.bias", "models.0.bert4global.encoder.layer.11.attention.self.value_proj.weight", "models.0.bert4global.encoder.layer.11.attention.self.value_proj.bias", "models.0.bert4global.encoder.layer.11.attention.output.dense.weight", "models.0.bert4global.encoder.layer.11.attention.output.dense.bias", "models.0.bert4global.encoder.layer.11.attention.output.LayerNorm.weight", "models.0.bert4global.encoder.layer.11.attention.output.LayerNorm.bias", "models.0.bert4global.encoder.layer.11.intermediate.dense.weight", "models.0.bert4global.encoder.layer.11.intermediate.dense.bias", "models.0.bert4global.encoder.layer.11.output.dense.weight", "models.0.bert4global.encoder.layer.11.output.dense.bias", "models.0.bert4global.encoder.layer.11.output.LayerNorm.weight", "models.0.bert4global.encoder.layer.11.output.LayerNorm.bias", "models.0.bert4global.encoder.rel_embeddings.weight", "models.0.bert4global.encoder.LayerNorm.weight", "models.0.bert4global.encoder.LayerNorm.bias", "models.0.post_encoder.encoder.0.SA.query.weight", "models.0.post_encoder.encoder.0.SA.query.bias", "models.0.post_encoder.encoder.0.SA.key.weight", "models.0.post_encoder.encoder.0.SA.key.bias", "models.0.post_encoder.encoder.0.SA.value.weight", "models.0.post_encoder.encoder.0.SA.value.bias", "models.0.post_encoder_.encoder.0.SA.query.weight", "models.0.post_encoder_.encoder.0.SA.query.bias", "models.0.post_encoder_.encoder.0.SA.key.weight", "models.0.post_encoder_.encoder.0.SA.key.bias", "models.0.post_encoder_.encoder.0.SA.value.weight", "models.0.post_encoder_.encoder.0.SA.value.bias", "models.0.bert_pooler.dense.weight", "models.0.bert_pooler.dense.bias", "models.0.CDW_LSA.eta1", "models.0.CDW_LSA.eta2", "models.0.CDW_LSA.encoder.encoder.0.SA.query.weight", "models.0.CDW_LSA.encoder.encoder.0.SA.query.bias", "models.0.CDW_LSA.encoder.encoder.0.SA.key.weight", "models.0.CDW_LSA.encoder.encoder.0.SA.key.bias", "models.0.CDW_LSA.encoder.encoder.0.SA.value.weight", "models.0.CDW_LSA.encoder.encoder.0.SA.value.bias", "models.0.CDW_LSA.encoder_left.encoder.0.SA.query.weight", "models.0.CDW_LSA.encoder_left.encoder.0.SA.query.bias", "models.0.CDW_LSA.encoder_left.encoder.0.SA.key.weight", "models.0.CDW_LSA.encoder_left.encoder.0.SA.key.bias", "models.0.CDW_LSA.encoder_left.encoder.0.SA.value.weight", "models.0.CDW_LSA.encoder_left.encoder.0.SA.value.bias", "models.0.CDW_LSA.encoder_right.encoder.0.SA.query.weight", "models.0.CDW_LSA.encoder_right.encoder.0.SA.query.bias", "models.0.CDW_LSA.encoder_right.encoder.0.SA.key.weight", "models.0.CDW_LSA.encoder_right.encoder.0.SA.key.bias", "models.0.CDW_LSA.encoder_right.encoder.0.SA.value.weight", "models.0.CDW_LSA.encoder_right.encoder.0.SA.value.bias", "models.0.CDW_LSA.linear_window_3h.weight", "models.0.CDW_LSA.linear_window_3h.bias", "models.0.CDW_LSA.linear_window_2h.weight", "models.0.CDW_LSA.linear_window_2h.bias", "models.0.post_linear.weight", "models.0.post_linear.bias", "models.0.dense.weight", "models.0.dense.bias", "bert.embeddings.position_ids", "bert.embeddings.word_embeddings.weight", "bert.embeddings.LayerNorm.weight", "bert.embeddings.LayerNorm.bias", "bert.encoder.layer.0.attention.self.query_proj.weight", "bert.encoder.layer.0.attention.self.query_proj.bias", "bert.encoder.layer.0.attention.self.key_proj.weight", "bert.encoder.layer.0.attention.self.key_proj.bias", "bert.encoder.layer.0.attention.self.value_proj.weight", "bert.encoder.layer.0.attention.self.value_proj.bias", "bert.encoder.layer.0.attention.output.dense.weight", "bert.encoder.layer.0.attention.output.dense.bias", "bert.encoder.layer.0.attention.output.LayerNorm.weight", "bert.encoder.layer.0.attention.output.LayerNorm.bias", "bert.encoder.layer.0.intermediate.dense.weight", "bert.encoder.layer.0.intermediate.dense.bias", "bert.encoder.layer.0.output.dense.weight", "bert.encoder.layer.0.output.dense.bias", "bert.encoder.layer.0.output.LayerNorm.weight", "bert.encoder.layer.0.output.LayerNorm.bias", "bert.encoder.layer.1.attention.self.query_proj.weight", "bert.encoder.layer.1.attention.self.query_proj.bias", "bert.encoder.layer.1.attention.self.key_proj.weight", "bert.encoder.layer.1.attention.self.key_proj.bias", "bert.encoder.layer.1.attention.self.value_proj.weight", "bert.encoder.layer.1.attention.self.value_proj.bias", "bert.encoder.layer.1.attention.output.dense.weight", "bert.encoder.layer.1.attention.output.dense.bias", "bert.encoder.layer.1.attention.output.LayerNorm.weight", "bert.encoder.layer.1.attention.output.LayerNorm.bias", "bert.encoder.layer.1.intermediate.dense.weight", "bert.encoder.layer.1.intermediate.dense.bias", "bert.encoder.layer.1.output.dense.weight", "bert.encoder.layer.1.output.dense.bias", "bert.encoder.layer.1.output.LayerNorm.weight", "bert.encoder.layer.1.output.LayerNorm.bias", "bert.encoder.layer.2.attention.self.query_proj.weight", "bert.encoder.layer.2.attention.self.query_proj.bias", "bert.encoder.layer.2.attention.self.key_proj.weight", "bert.encoder.layer.2.attention.self.key_proj.bias", "bert.encoder.layer.2.attention.self.value_proj.weight", "bert.encoder.layer.2.attention.self.value_proj.bias", "bert.encoder.layer.2.attention.output.dense.weight", "bert.encoder.layer.2.attention.output.dense.bias", "bert.encoder.layer.2.attention.output.LayerNorm.weight", "bert.encoder.layer.2.attention.output.LayerNorm.bias", "bert.encoder.layer.2.intermediate.dense.weight", "bert.encoder.layer.2.intermediate.dense.bias", "bert.encoder.layer.2.output.dense.weight", "bert.encoder.layer.2.output.dense.bias", "bert.encoder.layer.2.output.LayerNorm.weight", "bert.encoder.layer.2.output.LayerNorm.bias", "bert.encoder.layer.3.attention.self.query_proj.weight", "bert.encoder.layer.3.attention.self.query_proj.bias", "bert.encoder.layer.3.attention.self.key_proj.weight", "bert.encoder.layer.3.attention.self.key_proj.bias", "bert.encoder.layer.3.attention.self.value_proj.weight", "bert.encoder.layer.3.attention.self.value_proj.bias", "bert.encoder.layer.3.attention.output.dense.weight", "bert.encoder.layer.3.attention.output.dense.bias", "bert.encoder.layer.3.attention.output.LayerNorm.weight", "bert.encoder.layer.3.attention.output.LayerNorm.bias", "bert.encoder.layer.3.intermediate.dense.weight", "bert.encoder.layer.3.intermediate.dense.bias", "bert.encoder.layer.3.output.dense.weight", "bert.encoder.layer.3.output.dense.bias", "bert.encoder.layer.3.output.LayerNorm.weight", "bert.encoder.layer.3.output.LayerNorm.bias", "bert.encoder.layer.4.attention.self.query_proj.weight", "bert.encoder.layer.4.attention.self.query_proj.bias", "bert.encoder.layer.4.attention.self.key_proj.weight", "bert.encoder.layer.4.attention.self.key_proj.bias", "bert.encoder.layer.4.attention.self.value_proj.weight", "bert.encoder.layer.4.attention.self.value_proj.bias", "bert.encoder.layer.4.attention.output.dense.weight", "bert.encoder.layer.4.attention.output.dense.bias", "bert.encoder.layer.4.attention.output.LayerNorm.weight", "bert.encoder.layer.4.attention.output.LayerNorm.bias", "bert.encoder.layer.4.intermediate.dense.weight", "bert.encoder.layer.4.intermediate.dense.bias", "bert.encoder.layer.4.output.dense.weight", "bert.encoder.layer.4.output.dense.bias", "bert.encoder.layer.4.output.LayerNorm.weight", "bert.encoder.layer.4.output.LayerNorm.bias", "bert.encoder.layer.5.attention.self.query_proj.weight", "bert.encoder.layer.5.attention.self.query_proj.bias", "bert.encoder.layer.5.attention.self.key_proj.weight", "bert.encoder.layer.5.attention.self.key_proj.bias", "bert.encoder.layer.5.attention.self.value_proj.weight", "bert.encoder.layer.5.attention.self.value_proj.bias", "bert.encoder.layer.5.attention.output.dense.weight", "bert.encoder.layer.5.attention.output.dense.bias", "bert.encoder.layer.5.attention.output.LayerNorm.weight", "bert.encoder.layer.5.attention.output.LayerNorm.bias", "bert.encoder.layer.5.intermediate.dense.weight", "bert.encoder.layer.5.intermediate.dense.bias", "bert.encoder.layer.5.output.dense.weight", "bert.encoder.layer.5.output.dense.bias", "bert.encoder.layer.5.output.LayerNorm.weight", "bert.encoder.layer.5.output.LayerNorm.bias", "bert.encoder.layer.6.attention.self.query_proj.weight", "bert.encoder.layer.6.attention.self.query_proj.bias", "bert.encoder.layer.6.attention.self.key_proj.weight", "bert.encoder.layer.6.attention.self.key_proj.bias", "bert.encoder.layer.6.attention.self.value_proj.weight", "bert.encoder.layer.6.attention.self.value_proj.bias", "bert.encoder.layer.6.attention.output.dense.weight", "bert.encoder.layer.6.attention.output.dense.bias", "bert.encoder.layer.6.attention.output.LayerNorm.weight", "bert.encoder.layer.6.attention.output.LayerNorm.bias", "bert.encoder.layer.6.intermediate.dense.weight", "bert.encoder.layer.6.intermediate.dense.bias", "bert.encoder.layer.6.output.dense.weight", "bert.encoder.layer.6.output.dense.bias", "bert.encoder.layer.6.output.LayerNorm.weight", "bert.encoder.layer.6.output.LayerNorm.bias", "bert.encoder.layer.7.attention.self.query_proj.weight", "bert.encoder.layer.7.attention.self.query_proj.bias", "bert.encoder.layer.7.attention.self.key_proj.weight", "bert.encoder.layer.7.attention.self.key_proj.bias", "bert.encoder.layer.7.attention.self.value_proj.weight", "bert.encoder.layer.7.attention.self.value_proj.bias", "bert.encoder.layer.7.attention.output.dense.weight", "bert.encoder.layer.7.attention.output.dense.bias", "bert.encoder.layer.7.attention.output.LayerNorm.weight", "bert.encoder.layer.7.attention.output.LayerNorm.bias", "bert.encoder.layer.7.intermediate.dense.weight", "bert.encoder.layer.7.intermediate.dense.bias", "bert.encoder.layer.7.output.dense.weight", "bert.encoder.layer.7.output.dense.bias", "bert.encoder.layer.7.output.LayerNorm.weight", "bert.encoder.layer.7.output.LayerNorm.bias", "bert.encoder.layer.8.attention.self.query_proj.weight", "bert.encoder.layer.8.attention.self.query_proj.bias", "bert.encoder.layer.8.attention.self.key_proj.weight", "bert.encoder.layer.8.attention.self.key_proj.bias", "bert.encoder.layer.8.attention.self.value_proj.weight", "bert.encoder.layer.8.attention.self.value_proj.bias", "bert.encoder.layer.8.attention.output.dense.weight", "bert.encoder.layer.8.attention.output.dense.bias", "bert.encoder.layer.8.attention.output.LayerNorm.weight", "bert.encoder.layer.8.attention.output.LayerNorm.bias", "bert.encoder.layer.8.intermediate.dense.weight", "bert.encoder.layer.8.intermediate.dense.bias", "bert.encoder.layer.8.output.dense.weight", "bert.encoder.layer.8.output.dense.bias", "bert.encoder.layer.8.output.LayerNorm.weight", "bert.encoder.layer.8.output.LayerNorm.bias", "bert.encoder.layer.9.attention.self.query_proj.weight", "bert.encoder.layer.9.attention.self.query_proj.bias", "bert.encoder.layer.9.attention.self.key_proj.weight", "bert.encoder.layer.9.attention.self.key_proj.bias", "bert.encoder.layer.9.attention.self.value_proj.weight", "bert.encoder.layer.9.attention.self.value_proj.bias", "bert.encoder.layer.9.attention.output.dense.weight", "bert.encoder.layer.9.attention.output.dense.bias", "bert.encoder.layer.9.attention.output.LayerNorm.weight", "bert.encoder.layer.9.attention.output.LayerNorm.bias", "bert.encoder.layer.9.intermediate.dense.weight", "bert.encoder.layer.9.intermediate.dense.bias", "bert.encoder.layer.9.output.dense.weight", "bert.encoder.layer.9.output.dense.bias", "bert.encoder.layer.9.output.LayerNorm.weight", "bert.encoder.layer.9.output.LayerNorm.bias", "bert.encoder.layer.10.attention.self.query_proj.weight", "bert.encoder.layer.10.attention.self.query_proj.bias", "bert.encoder.layer.10.attention.self.key_proj.weight", "bert.encoder.layer.10.attention.self.key_proj.bias", "bert.encoder.layer.10.attention.self.value_proj.weight", "bert.encoder.layer.10.attention.self.value_proj.bias", "bert.encoder.layer.10.attention.output.dense.weight", "bert.encoder.layer.10.attention.output.dense.bias", "bert.encoder.layer.10.attention.output.LayerNorm.weight", "bert.encoder.layer.10.attention.output.LayerNorm.bias", "bert.encoder.layer.10.intermediate.dense.weight", "bert.encoder.layer.10.intermediate.dense.bias", "bert.encoder.layer.10.output.dense.weight", "bert.encoder.layer.10.output.dense.bias", "bert.encoder.layer.10.output.LayerNorm.weight", "bert.encoder.layer.10.output.LayerNorm.bias", "bert.encoder.layer.11.attention.self.query_proj.weight", "bert.encoder.layer.11.attention.self.query_proj.bias", "bert.encoder.layer.11.attention.self.key_proj.weight", "bert.encoder.layer.11.attention.self.key_proj.bias", "bert.encoder.layer.11.attention.self.value_proj.weight", "bert.encoder.layer.11.attention.self.value_proj.bias", "bert.encoder.layer.11.attention.output.dense.weight", "bert.encoder.layer.11.attention.output.dense.bias", "bert.encoder.layer.11.attention.output.LayerNorm.weight", "bert.encoder.layer.11.attention.output.LayerNorm.bias", "bert.encoder.layer.11.intermediate.dense.weight", "bert.encoder.layer.11.intermediate.dense.bias", "bert.encoder.layer.11.output.dense.weight", "bert.encoder.layer.11.output.dense.bias", "bert.encoder.layer.11.output.LayerNorm.weight", "bert.encoder.layer.11.output.LayerNorm.bias", "bert.encoder.rel_embeddings.weight", "bert.encoder.LayerNorm.weight", "bert.encoder.LayerNorm.bias".
size mismatch for dense.weight: copying a param with shape torch.Size([3, 3]) from checkpoint, the shape in current model is torch.Size([3, 768]).

[yangheng95]

remove the from_checkpoint arg in the trainer will fix this error

[kirankm]

Hi @yangheng95 thanks, that solved the problem. I have 2 follow up questions though

1. In case I want to use a pretrained model. What is the option
2. In case I want to continue training from a checkpoint, what is the option

[yangheng95]

The pretrained checkpoint are based on FAST_LCF model that are not the same with your selected model, so this error occurs. Actually, It is not encourged to resume from a checkpoint if you want to get best performance, it is only designed to accelerate the training process.