client.py doesn't retry a connection on a requests.exceptions.ConnectionError "RemoteDisconnected" #1885
Description
Description of the problem, including code/CLI snippet
On an unstable gitlab instance, sometime the instance randomly close the connection without a response, this case is not handled by python-gitlab.
Expected Behavior
Because a random connection close raise a requests.exceptions.ConnectionError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response')) this bypass the whole retry mechanism in gitlab/client.py while using it would have solved the issue transparently.
That would likely be a try/catch to add here or something similar
python-gitlab/gitlab/client.py
Lines 680 to 690 in 5beda3b
Actual Behavior
Here is a classical traceback for this error:
Traceback (most recent call last):
File "/home/psycojoker/.virtualenvs/cube-doctor/bin/cube-doctor", line 11, in <module>
load_entry_point('cube-doctor', 'console_scripts', 'cube-doctor')()
File "/home/psycojoker/code/yunohost/ynh-dev/logilab/cube-doctor/cube_doctor/doctor_hg.py", line 169, in main
COMMANDS[args.command].workflow(
File "/home/psycojoker/code/yunohost/ynh-dev/logilab/cube-doctor/cube_doctor/__init__.py", line 445, in wrap
return self._workflow(*args, **kwargs)
File "/home/psycojoker/code/yunohost/ynh-dev/logilab/cube-doctor/cube_doctor/__init__.py", line 594, in workflow
metadata = next(modifications)
File "/home/psycojoker/code/yunohost/ynh-dev/logilab/cube-doctor/cube_doctor/transforms/fix_deprecated_warnings.py", line 267, in modify_code
warnings = self.get_warnings(cube)
File "/home/psycojoker/code/yunohost/ynh-dev/logilab/cube-doctor/cube_doctor/transforms/fix_deprecated_warnings.py", line 285, in get_warnings
for pipeline in cube.pipelines.list(
File "/home/psycojoker/.virtualenvs/cube-doctor/lib/python3.8/site-packages/gitlab/exceptions.py", line 304, in wrapped_f
return f(*args, **kwargs)
File "/home/psycojoker/.virtualenvs/cube-doctor/lib/python3.8/site-packages/gitlab/mixins.py", line 242, in list
obj = self.gitlab.http_list(path, **data)
File "/home/psycojoker/.virtualenvs/cube-doctor/lib/python3.8/site-packages/gitlab/client.py", line 722, in http_list
return list(GitlabList(self, url, query_data, get_next=False, **kwargs))
File "/home/psycojoker/.virtualenvs/cube-doctor/lib/python3.8/site-packages/gitlab/client.py", line 881, in __init__
self._query(url, query_data, **self._kwargs)
File "/home/psycojoker/.virtualenvs/cube-doctor/lib/python3.8/site-packages/gitlab/client.py", line 891, in _query
result = self._gl.http_request("get", url, query_data=query_data, **kwargs)
File "/home/psycojoker/.virtualenvs/cube-doctor/lib/python3.8/site-packages/gitlab/client.py", line 594, in http_request
result = self.session.send(prepped, timeout=timeout, **settings)
File "/home/psycojoker/.virtualenvs/cube-doctor/lib/python3.8/site-packages/requests/sessions.py", line 643, in send
r = adapter.send(request, **kwargs)
File "/home/psycojoker/.virtualenvs/cube-doctor/lib/python3.8/site-packages/requests/adapters.py", line 498, in send
raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))
This is the related code:
python-gitlab/gitlab/client.py
Lines 678 to 709 in 5beda3b
Specifications
- python-gitlab version: 2.10.1 but the related code is the same (and it appears also on the CI that pulls the latest version of python-gitlab each time)
- API version you are using (v3/v4): v4
- Gitlab server version (or gitlab.com): https://forge.extranet.logilab.fr/
Thanks for this lib ❤️