Download Failure for Specific camera at Specific Hour Range.

[SebDominguez]

Strange issue here.

The command :
docker run -e TZ="Europe/Madrid" --volume /Users/sebastien/unifi/download:/downloads unifitoolbox/protect-archiver download /downloads --username="user" --password="password" --address="10.0.0.1" --start="2024-08-29 15:00:00" --end="2024-08-29 16:00:00" --cameras="camera"

The output:

Getting camera list
Successfully retrieved data from https://10.0.0.1:443/proxy/protect/api/cameras
Cameras found:
- Entree Nord (mac)
- Jardin Est (mac)
- Portillon Est (mac)
- Dépendance (mac)
- Portail (mac)
- Piscine Oest (mac)
- Portillon Nord (mac)
- Terrasse Ouest (mac)
- Dépendance  (mac)
- Salle du serveur (mac)
- G3 Instant (mac)
Downloading footage for camera 'Portail' (63452b83013ebd038700bcff)
Downloading video files between 2024-08-29 15:00:00 and 2024-08-29 16:00:00 from 'https://10.0.0.1:443/proxy/protect/api/video/export' for camera Portail
Downloading video for time range 2024-08-29 15:00:00 - 2024-08-29 15:59:59.999000 to /downloads/2024/08/29/Portail (bcff)/Portail (bcff) - 2024-08-29 - 15.00.00.mp4
Download failed: HTTPSConnectionPool(host='10.0.0.1', port=443): Read timed out. (read timeout=60.0)
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 537, in _make_request
    response = conn.getresponse()
  File "/usr/local/lib/python3.8/site-packages/urllib3/connection.py", line 466, in getresponse
    httplib_response = super().getresponse()
  File "/usr/local/lib/python3.8/http/client.py", line 1348, in getresponse
    response.begin()
  File "/usr/local/lib/python3.8/http/client.py", line 316, in begin
    version, status, reason = self._read_status()
  File "/usr/local/lib/python3.8/http/client.py", line 277, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File "/usr/local/lib/python3.8/socket.py", line 669, in readinto
    return self._sock.recv_into(b)
  File "/usr/local/lib/python3.8/ssl.py", line 1274, in recv_into
    return self.read(nbytes, buffer)
  File "/usr/local/lib/python3.8/ssl.py", line 1132, in read
    return self._sslobj.read(len, buffer)
socket.timeout: The read operation timed out

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/requests/adapters.py", line 589, in send
    resp = conn.urlopen(
  File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 847, in urlopen
    retries = retries.increment(
  File "/usr/local/lib/python3.8/site-packages/urllib3/util/retry.py", line 470, in increment
    raise reraise(type(error), error, _stacktrace)
  File "/usr/local/lib/python3.8/site-packages/urllib3/util/util.py", line 39, in reraise
    raise value
  File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 793, in urlopen
    response = self._make_request(
  File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 539, in _make_request
    self._raise_timeout(err=e, url=url, timeout_value=read_timeout)
  File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 370, in _raise_timeout
    raise ReadTimeoutError(
urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='10.0.0.1', port=443): Read timed out. (read timeout=60.0)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/protect_archiver/downloader/download_file.py", line 36, in download_file
    requests.get(
  File "/usr/local/lib/python3.8/site-packages/requests/api.py", line 73, in get
    return request("get", url, params=params, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/requests/api.py", line 59, in request
    return session.request(method=method, url=url, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/requests/sessions.py", line 589, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.8/site-packages/requests/sessions.py", line 703, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/requests/adapters.py", line 635, in send
    raise ReadTimeout(e, request=request)
requests.exceptions.ReadTimeout: HTTPSConnectionPool(host='10.0.0.1', port=443): Read timed out. (read timeout=60.0)
Retrying in 3 second(s)...
Download failed: HTTPSConnectionPool(host='10.0.0.1', port=443): Read timed out. (read timeout=60.0)
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 537, in _make_request
    response = conn.getresponse()
  File "/usr/local/lib/python3.8/site-packages/urllib3/connection.py", line 466, in getresponse
    httplib_response = super().getresponse()
  File "/usr/local/lib/python3.8/http/client.py", line 1348, in getresponse
    response.begin()
  File "/usr/local/lib/python3.8/http/client.py", line 316, in begin
    version, status, reason = self._read_status()
  File "/usr/local/lib/python3.8/http/client.py", line 277, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File "/usr/local/lib/python3.8/socket.py", line 669, in readinto
    return self._sock.recv_into(b)
  File "/usr/local/lib/python3.8/ssl.py", line 1274, in recv_into
    return self.read(nbytes, buffer)
  File "/usr/local/lib/python3.8/ssl.py", line 1132, in read
    return self._sslobj.read(len, buffer)
socket.timeout: The read operation timed out

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/requests/adapters.py", line 589, in send
    resp = conn.urlopen(
  File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 847, in urlopen
    retries = retries.increment(
  File "/usr/local/lib/python3.8/site-packages/urllib3/util/retry.py", line 470, in increment
    raise reraise(type(error), error, _stacktrace)
  File "/usr/local/lib/python3.8/site-packages/urllib3/util/util.py", line 39, in reraise
    raise value
  File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 793, in urlopen
    response = self._make_request(
  File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 539, in _make_request
    self._raise_timeout(err=e, url=url, timeout_value=read_timeout)
  File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 370, in _raise_timeout
    raise ReadTimeoutError(
urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='10.0.0.1', port=443): Read timed out. (read timeout=60.0)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/protect_archiver/downloader/download_file.py", line 36, in download_file
    requests.get(
  File "/usr/local/lib/python3.8/site-packages/requests/api.py", line 73, in get
    return request("get", url, params=params, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/requests/api.py", line 59, in request
    return session.request(method=method, url=url, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/requests/sessions.py", line 589, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.8/site-packages/requests/sessions.py", line 703, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/requests/adapters.py", line 635, in send
    raise ReadTimeout(e, request=request)
requests.exceptions.ReadTimeout: HTTPSConnectionPool(host='10.0.0.1', port=443): Read timed out. (read timeout=60.0)
Retrying in 3 second(s)...

However if I tweak the hour range:
docker run -e TZ="Europe/Madrid" --volume /Users/sebastien/unifi/download:/downloads unifitoolbox/protect-archiver download /downloads --username="user" --password="password" --address="10.0.0.1" --start="2024-08-29 16:00:00" --end="2024-08-29 17:00:00" --cameras="camera"

The output:

Getting camera list
Successfully retrieved data from https://10.0.0.1:443/proxy/protect/api/cameras
Cameras found:
- Entree Nord (609aad540063fd03870003eb)
- Jardin Est (609aad5400e9fd03870003ef)
- Portillon Est (609aad5400c7fd03870003ee)
- Dépendance (609aad54013cfd03870003f2)
- Portail (63452b83013ebd038700bcff)
- Piscine Oest (609aad5400aafd03870003ed)
- Portillon Nord (609aad54010afd03870003f0)
- Terrasse Ouest (609aad54008ffd03870003ec)
- Dépendance  (6558b962004e4c03e4036cdd)
- Salle du serveur (6194facb03957f0387004da9)
- G3 Instant (658c1f870243e503e40294ce)
Downloading video files between 2024-08-29 16:00:00 and 2024-08-29 17:00:00 from 'https://10.0.0.1:443/proxy/protect/api/video/export' for camera Portail
Downloading footage for camera 'Portail' (63452b83013ebd038700bcff)
Downloading video for time range 2024-08-29 16:00:00 - 2024-08-29 16:59:59.999000 to /downloads/2024/08/29/Portail (bcff)/Portail (bcff) - 2024-08-29 - 16.00.00.mp4
Download successful after 47s (532.0 mb, 11.0 mbps)
1 files downloaded (532.0 mb), 0 files skipped, 0 files failed, 1 files total

Strange isn't it?

[danielfernau]

Hi there, thanks for reaching out and sorry for the late reply!

urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='10.0.0.1', port=443): Read timed out. (read timeout=60.0) looks to me like there's an issue internal to the Protect Application, not the downloader tool, caused by a very specific time range selection. Will try to look into it when building a newer version of the tool, so leaving this issue open for now.

[jforeman]

Seeing similar behavior. Protect 5.0.51. Also seeing other problems (will open a new issue).

I'm available to debug.

[danielfernau]

@jforeman - Thanks for the additional info. Will look into it as soon as my spare time allows.