Pytorch model download

[yuanxion]

HTTP Error 403

• Error

urllib.error.HTTPError: HTTP Error 403: rate limit exceeded

Traceback (most recent call last):
  File "demo-diffusion-prompt.py", line 26, in <module>
    loss = diffusion(videos, cond = text)
  File "/home/intel/anaconda3/envs/t2v/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1194, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/intel/anaconda3/envs/t2v/lib/python3.8/site-packages/video_diffusion_pytorch/video_diffusion_pytorch.py", line 743, in forward
    return self.p_losses(x, t, *args, **kwargs)
  File "/home/intel/anaconda3/envs/t2v/lib/python3.8/site-packages/video_diffusion_pytorch/video_diffusion_pytorch.py", line 724, in p_losses
    cond = bert_embed(tokenize(cond), return_cls_repr = self.text_use_bert_cls)
  File "/home/intel/anaconda3/envs/t2v/lib/python3.8/site-packages/video_diffusion_pytorch/text.py", line 34, in tokenize
    tokenizer = get_tokenizer()
  File "/home/intel/anaconda3/envs/t2v/lib/python3.8/site-packages/video_diffusion_pytorch/text.py", line 16, in get_tokenizer
    TOKENIZER = torch.hub.load('huggingface/pytorch-transformers', 'tokenizer', 'bert-base-cased')
  File "/home/intel/anaconda3/envs/t2v/lib/python3.8/site-packages/torch/hub.py", line 539, in load
    repo_or_dir = _get_cache_or_reload(repo_or_dir, force_reload, trust_repo, "load",
  File "/home/intel/anaconda3/envs/t2v/lib/python3.8/site-packages/torch/hub.py", line 203, in _get_cache_or_reload
    _validate_not_a_forked_repo(repo_owner, repo_name, ref)
  File "/home/intel/anaconda3/envs/t2v/lib/python3.8/site-packages/torch/hub.py", line 162, in _validate_not_a_forked_repo
    response = json.loads(_read_url(Request(url, headers=headers)))
  File "/home/intel/anaconda3/envs/t2v/lib/python3.8/site-packages/torch/hub.py", line 145, in _read_url
    with urlopen(url) as r:
  File "/home/intel/anaconda3/envs/t2v/lib/python3.8/urllib/request.py", line 222, in urlopen
    return opener.open(url, data, timeout)
  File "/home/intel/anaconda3/envs/t2v/lib/python3.8/urllib/request.py", line 531, in open
    response = meth(req, response)
  File "/home/intel/anaconda3/envs/t2v/lib/python3.8/urllib/request.py", line 640, in http_response
    response = self.parent.error(
  File "/home/intel/anaconda3/envs/t2v/lib/python3.8/urllib/request.py", line 569, in error
    return self._call_chain(*args)
  File "/home/intel/anaconda3/envs/t2v/lib/python3.8/urllib/request.py", line 502, in _call_chain
    result = func(*args)
  File "/home/intel/anaconda3/envs/t2v/lib/python3.8/urllib/request.py", line 649, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 403: rate limit exceeded

• Solution

+ torch.hub._validate_not_a_forked_repo=lambda a,b,c: True
model = torch.hub.load('pytorch/vision:v0.9.0', 'resnet18', pretrained=True)

[yuanxion]

HTTP Error 443

• Error

requests.exceptions.SSLError: HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /bert-base-cased/resolve/main/vocab.txt (Caused by SSLError(SSLEOFError(8, 'EOF occurred in violation of protocol (_ssl.c:1131)')))

> /home/intel/yuan/workspace/poc/t2v/video-diffusion-pytorch/test-download-huggingface-bert.py(5)<module>()
-> tokenizer = torch.hub.load('huggingface/pytorch-transformers', 'tokenizer', 'bert-base-cased')
(Pdb) 
Using cache found in /home/intel/.cache/torch/hub/huggingface_pytorch-transformers_main
2023-08-31 15:05:12.551064: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
2023-08-31 15:05:12.575219: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 AVX_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2023-08-31 15:05:13.446573: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
requests.exceptions.SSLError: HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /bert-base-cased/resolve/main/vocab.txt (Caused by SSLError(SSLEOFError(8, 'EOF occurred in violation of protocol (_ssl.c:1131)')))

• Solution

git init
git lfs install
git clone https://huggingface.co/bert-base-cased

[yuanxion]

Models/files cache

Download models/config files and cache them for local usage.
https://zhuanlan.zhihu.com/p/475260268

• git lfs install
  https://blog.csdn.net/flyingluohaipeng/article/details/130788293

git clone https://huggingface.co/dreamlike-art/dreamlike-photoreal-2.0
mkdir dreamlike-art
mv dreamlike-photoreal-2.0/ dreamlike-art/

• fusing/stable-diffusion-v1-5-controlnet-openpose
  diffusion_pytorch_model.bin can be downloaded from https://huggingface.co/fusing/stable-diffusion-v1-5-controlnet-openpose

• runwayml/stable-diffusion-v1-5
  https://huggingface.co/runwayml/stable-diffusion-v1-5/

• huggingface_hub snapshot_download

requests.exceptions.SSLError: HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /api/models/dreamlike-art/dreamlike-photoreal-2.0/revision/main (Caused by SSLError(SSLEOFError(8, 'EOF occurred in violation of protocol (_ssl.c:1131)')))

Add import os; os.environ['CURL_CA_BUNDLE'] = '' to try downloading again, or try many times.

from huggingface_hub import snapshot_download
import os
os.environ['CURL_CA_BUNDLE'] = ''
snapshot_download(repo_id="dreamlike-art/dreamlike-photoreal-2.0")