RuntimeError: unexpected EOF, expected 1056959984 more bytes. The file might be corrupted.

[nongbaoting]

Hi, I would like to use your excellent codes, but I encountered an error!

 CUDA_VISIBLE_DEVICES=3 python bin/recommend.py MNHDQEFDPPKVYPPVPAEKRKPIRVLSLFDGIATGLLVLK                                                                                                                                         
Traceback (most recent call last):
  File "/training/nong/app/miniconda3/envs/efficient-evolution/lib/python3.9/site-packages/esm/pretrained.py", line 33, in load_hub_workaround
    data = torch.hub.load_state_dict_from_url(url, progress=False, map_location="cpu")
  File "/training/nong/app/miniconda3/envs/efficient-evolution/lib/python3.9/site-packages/torch/hub.py", line 731, in load_state_dict_from_url
    return torch.load(cached_file, map_location=map_location)
  File "/training/nong/app/miniconda3/envs/efficient-evolution/lib/python3.9/site-packages/torch/serialization.py", line 713, in load
    return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args)
  File "/training/nong/app/miniconda3/envs/efficient-evolution/lib/python3.9/site-packages/torch/serialization.py", line 938, in _legacy_load
    typed_storage._storage._set_from_file(
RuntimeError: unexpected EOF, expected 1056959984 more bytes. The file might be corrupted.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/training/nong/app/efficient-evolution/bin/recommend.py", line 39, in <module>
    mutations_models = reconstruct_multi_models(
  File "/training/nong/app/efficient-evolution/bin/amis.py", line 221, in reconstruct_multi_models
    model = get_model_name(model_name)
  File "/training/nong/app/efficient-evolution/bin/amis.py", line 39, in get_model_name
    model = FBModel(
  File "/training/nong/app/efficient-evolution/bin/fb_model.py", line 10, in __init__
    model, alphabet = pretrained.load_model_and_alphabet(name)
  File "/training/nong/app/miniconda3/envs/efficient-evolution/lib/python3.9/site-packages/esm/pretrained.py", line 28, in load_model_and_alphabet
    return load_model_and_alphabet_hub(model_name)
  File "/training/nong/app/miniconda3/envs/efficient-evolution/lib/python3.9/site-packages/esm/pretrained.py", line 63, in load_model_and_alphabet_hub
    model_data, regression_data = _download_model_and_regression_data(model_name)
  File "/training/nong/app/miniconda3/envs/efficient-evolution/lib/python3.9/site-packages/esm/pretrained.py", line 54, in _download_model_and_regression_data
    model_data = load_hub_workaround(url)
  File "/training/nong/app/miniconda3/envs/efficient-evolution/lib/python3.9/site-packages/esm/pretrained.py", line 37, in load_hub_workaround
    data = torch.load(
  File "/training/nong/app/miniconda3/envs/efficient-evolution/lib/python3.9/site-packages/torch/serialization.py", line 713, in load
    return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args)
  File "/training/nong/app/miniconda3/envs/efficient-evolution/lib/python3.9/site-packages/torch/serialization.py", line 938, in _legacy_load
    typed_storage._storage._set_from_file(
RuntimeError: unexpected EOF, expected 1056959984 more bytes. The file might be corrupted.

[brianhie]

I am not sure what is going on here, though you should probably check if your computer has sufficient space to download the models (they do take up a lot of disk space, around 50 GB). E.g., see here: huggingface/transformers#1491