Handling connection error issue #1660

[apeksha235]

In this PR, I tried handling the connection error that might occur while using Socrata API. If there's anything else I should consider please let me know. Thank you.

[henrykironde]

Thank you @apeksha235. What test-case scenarios have you considered?

[apeksha235]

@henrykironde I considered two scenarios:

1. When the user is trying to access the datasets from the Socrata API
2. When the user is trying to access datasets from Rdatasets.
   But it seems like the URL added for Rdatasets in the code is invalid, so for now I only made changes for socrata_autocomplete_search, socrata_dataset_info,find_socrata_dataset_by_id which require the Socrata API.
   Please do correct me if I'm wrong.

[henrykironde]

A space (HTTPError,requests.exception (HTTPError, requests.exception

print("Error Occured :", e)
print("Error Occured : ", e) # Add a space after : 

[apeksha235]

@henrykironde I have made the corrections. Please check it. Is there anything else I should consider?

[henrykironde]

@apeksha235 install this branch using pip install . -U
Then turn off the internet connection and run a few commands.
Paste the results here

[apeksha235]

1. Searching Socrata datasets:

2. Getting socrata dataset info by name

@henrykironde This is the output I am getting with no internet connection. If required I can change the output that's displayed.

[henrykironde]

Thank you @apeksha235. To improve this PR, let us start by writing a small program to detect the availability of an internet connection and tell the user the status (available or not available). You can paste the code here.

[apeksha235]

@henrykironde Here's the code:

try:
    response = requests.get(AnyURL, timeout=5)
    response.raise_for_status()
    print("Internet connection is available")
except requests.exceptions.RequestException as e:
    print("Error:", e)
    print("No internet connection")

[henrykironde]

@apeksha235 I need ready to run code

[apeksha235]

@henrykironde sorry my bad, I thought you wanted a general code.

try:
    response = requests.get('http://api.us.socrata.com/api/catalog/v1', timeout=5)
    response.raise_for_status()
    print("Internet connection is available")
except requests.exceptions.RequestException as e:
    print("Error:", e)
    print("No internet connection")

This is for the Socrata API request

[henrykironde]

I will try to review the PR this week.

[henrykironde]

This is still giving an error

(retriever3.8) ➜  retriever git:(test) retriever ls 
Traceback (most recent call last):
  File "/opt/miniconda3/envs/retriever3.8/lib/python3.8/urllib/request.py", line 1354, in do_open
    h.request(req.get_method(), req.selector, req.data, headers,
  File "/opt/miniconda3/envs/retriever3.8/lib/python3.8/http/client.py", line 1256, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/opt/miniconda3/envs/retriever3.8/lib/python3.8/http/client.py", line 1302, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/opt/miniconda3/envs/retriever3.8/lib/python3.8/http/client.py", line 1251, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/opt/miniconda3/envs/retriever3.8/lib/python3.8/http/client.py", line 1011, in _send_output
    self.send(msg)
  File "/opt/miniconda3/envs/retriever3.8/lib/python3.8/http/client.py", line 951, in send
    self.connect()
  File "/opt/miniconda3/envs/retriever3.8/lib/python3.8/http/client.py", line 1418, in connect
    super().connect()
  File "/opt/miniconda3/envs/retriever3.8/lib/python3.8/http/client.py", line 922, in connect
    self.sock = self._create_connection(
  File "/opt/miniconda3/envs/retriever3.8/lib/python3.8/socket.py", line 787, in create_connection
    for res in getaddrinfo(host, port, 0, SOCK_STREAM):
  File "/opt/miniconda3/envs/retriever3.8/lib/python3.8/socket.py", line 918, in getaddrinfo
    for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno 8] nodename nor servname provided, or not known

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/miniconda3/envs/retriever3.8/bin/retriever", line 5, in <module>
    from retriever.__main__ import main
  File "/opt/miniconda3/envs/retriever3.8/lib/python3.8/site-packages/retriever/__main__.py", line 18, in <module>
    from retriever.lib.get_opts import parser
  File "/opt/miniconda3/envs/retriever3.8/lib/python3.8/site-packages/retriever/lib/get_opts.py", line 33, in <module>
    script_list.extend(get_rdataset_names())
  File "/opt/miniconda3/envs/retriever3.8/lib/python3.8/site-packages/retriever/lib/rdatasets.py", line 185, in get_rdataset_names
    update_rdataset_catalog()
  File "/opt/miniconda3/envs/retriever3.8/lib/python3.8/site-packages/retriever/lib/rdatasets.py", line 17, in update_rdataset_catalog
    df = pd.read_csv(RDATASETS_URL)
  File "/opt/miniconda3/envs/retriever3.8/lib/python3.8/site-packages/pandas/util/_decorators.py", line 311, in wrapper
    return func(*args, **kwargs)
  File "/opt/miniconda3/envs/retriever3.8/lib/python3.8/site-packages/pandas/io/parsers/readers.py", line 680, in read_csv
    return _read(filepath_or_buffer, kwds)
  File "/opt/miniconda3/envs/retriever3.8/lib/python3.8/site-packages/pandas/io/parsers/readers.py", line 575, in _read
    parser = TextFileReader(filepath_or_buffer, **kwds)
  File "/opt/miniconda3/envs/retriever3.8/lib/python3.8/site-packages/pandas/io/parsers/readers.py", line 934, in __init__
    self._engine = self._make_engine(f, self.engine)
  File "/opt/miniconda3/envs/retriever3.8/lib/python3.8/site-packages/pandas/io/parsers/readers.py", line 1218, in _make_engine
    self.handles = get_handle(  # type: ignore[call-overload]
  File "/opt/miniconda3/envs/retriever3.8/lib/python3.8/site-packages/pandas/io/common.py", line 667, in get_handle
    ioargs = _get_filepath_or_buffer(
  File "/opt/miniconda3/envs/retriever3.8/lib/python3.8/site-packages/pandas/io/common.py", line 336, in _get_filepath_or_buffer
    with urlopen(req_info) as req:
  File "/opt/miniconda3/envs/retriever3.8/lib/python3.8/site-packages/pandas/io/common.py", line 236, in urlopen
    return urllib.request.urlopen(*args, **kwargs)
  File "/opt/miniconda3/envs/retriever3.8/lib/python3.8/urllib/request.py", line 222, in urlopen
    return opener.open(url, data, timeout)
  File "/opt/miniconda3/envs/retriever3.8/lib/python3.8/urllib/request.py", line 525, in open
    response = self._open(req, data)
  File "/opt/miniconda3/envs/retriever3.8/lib/python3.8/urllib/request.py", line 542, in _open
    result = self._call_chain(self.handle_open, protocol, protocol +
  File "/opt/miniconda3/envs/retriever3.8/lib/python3.8/urllib/request.py", line 502, in _call_chain
    result = func(*args)
  File "/opt/miniconda3/envs/retriever3.8/lib/python3.8/urllib/request.py", line 1397, in https_open
    return self.do_open(http.client.HTTPSConnection, req,
  File "/opt/miniconda3/envs/retriever3.8/lib/python3.8/urllib/request.py", line 1357, in do_open
    raise URLError(err)
urllib.error.URLError: <urlopen error [Errno 8] nodename nor servname provided, or not known>

[apeksha235]

@henrykironde I am not facing this issue, but I'll certainly check why this is happening.

[henrykironde]

You have to put off your internet connection