edward/geocode
1
0
Fork 0
Code Issues 4 Pull requests Packages Projects Releases Wiki Activity

Retry failed Wikidata SPARQL queries #11

New issue
Closed
opened 2023-11-26 10:38:47 +00:00 by edward · 1 comment
edward commented 2023-11-26 10:38:47 +00:00
Owner
Copy link

Sometimes a call to the WDQS fails with a 500 error. We should retry when this happens.

Sometimes a call to the WDQS fails with a 500 error. We should retry when this happens.
edward commented 2023-12-03 16:24:14 +00:00
Author
Owner
Copy link

Example traceback

    Message type:       ERROR
    Location:           /usr/lib/python3/dist-packages/flask/app.py:1744
    Module:             app
    Function:           log_exception
    Time:               2023-12-02 08:00:15,556
    GET args:           ImmutableMultiDict([('lat', '56.34462430949934'), ('lon', '-3.0971737818656364')])
    URL:                https://edwardbetts.com/geocode/?lat=56.34462430949934&lon=-3.0971737818656364

    Message:

    Exception on / [GET]
   
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/requests/models.py", line 971, in json
    return complexjson.loads(self.text, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/simplejson/__init__.py", line 514, in loads
    return _default_decoder.decode(s)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/simplejson/decoder.py", line 386, in decode
    obj, end = self.raw_decode(s)
               ^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/simplejson/decoder.py", line 416, in raw_decode
    return self.scan_once(s, idx=_w(s, idx).end())
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
simplejson.errors.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/edward/src/2019/uk-geocode/geocode/wikidata.py", line 88, in wdqs
    return typing.cast(list[Row], r.json()["results"]["bindings"])
                                  ^^^^^^^^
  File "/usr/lib/python3/dist-packages/requests/models.py", line 975, in json
    raise RequestsJSONDecodeError(e.msg, e.doc, e.pos)
requests.exceptions.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/flask/app.py", line 2529, in wsgi_app
    response = self.full_dispatch_request()
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/flask/app.py", line 1825, in full_dispatch_request
    rv = self.handle_user_exception(e)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/flask/app.py", line 1823, in full_dispatch_request
    rv = self.dispatch_request()
         ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/flask/app.py", line 1799, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/edward/src/2019/uk-geocode/lookup.py", line 206, in index
  File "/home/edward/src/2019/uk-geocode/lookup.py", line 61, in lat_lon_to_wikidata
    north, west = 53.7984, -2.7296
               ^^^^^^^^^^^^^^^^^^^^
  File "/home/edward/src/2019/uk-geocode/geocode/wikidata.py", line 153, in lookup_scottish_parish_in_wikidata
    return wdqs(render_template("sparql/scottish_parish.sparql", code=code))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/edward/src/2019/uk-geocode/geocode/wikidata.py", line 90, in wdqs
    raise QueryError(query, r)
geocode.wikidata.QueryError: ('SELECT ?item ?itemLabel ?commonsSiteLink ?commonsCat WHERE {\n  ?item wdt:P528 "186" .\n  ?item wdt:P31 wd:Q5124673 .\n  OPTIONAL { ?commonsSiteLink schema:about ?item ;\n             schema:isPartOf <https://commons.wikimedia.org/> }\n  OPTIONAL { ?item wdt:P373 ?commonsCat }\n  SERVICE wikibase:label { bd:serviceParam wikibase:language "[AUTO_LANGUAGE],en". }\n}', <Response [429]>)
Example traceback ``` Message type: ERROR Location: /usr/lib/python3/dist-packages/flask/app.py:1744 Module: app Function: log_exception Time: 2023-12-02 08:00:15,556 GET args: ImmutableMultiDict([('lat', '56.34462430949934'), ('lon', '-3.0971737818656364')]) URL: https://edwardbetts.com/geocode/?lat=56.34462430949934&lon=-3.0971737818656364 Message: Exception on / [GET] Traceback (most recent call last): File "/usr/lib/python3/dist-packages/requests/models.py", line 971, in json return complexjson.loads(self.text, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/simplejson/__init__.py", line 514, in loads return _default_decoder.decode(s) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/simplejson/decoder.py", line 386, in decode obj, end = self.raw_decode(s) ^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/simplejson/decoder.py", line 416, in raw_decode return self.scan_once(s, idx=_w(s, idx).end()) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ simplejson.errors.JSONDecodeError: Expecting value: line 1 column 1 (char 0) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/edward/src/2019/uk-geocode/geocode/wikidata.py", line 88, in wdqs return typing.cast(list[Row], r.json()["results"]["bindings"]) ^^^^^^^^ File "/usr/lib/python3/dist-packages/requests/models.py", line 975, in json raise RequestsJSONDecodeError(e.msg, e.doc, e.pos) requests.exceptions.JSONDecodeError: Expecting value: line 1 column 1 (char 0) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/lib/python3/dist-packages/flask/app.py", line 2529, in wsgi_app response = self.full_dispatch_request() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/flask/app.py", line 1825, in full_dispatch_request rv = self.handle_user_exception(e) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/flask/app.py", line 1823, in full_dispatch_request rv = self.dispatch_request() ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/flask/app.py", line 1799, in dispatch_request return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/edward/src/2019/uk-geocode/lookup.py", line 206, in index File "/home/edward/src/2019/uk-geocode/lookup.py", line 61, in lat_lon_to_wikidata north, west = 53.7984, -2.7296 ^^^^^^^^^^^^^^^^^^^^ File "/home/edward/src/2019/uk-geocode/geocode/wikidata.py", line 153, in lookup_scottish_parish_in_wikidata return wdqs(render_template("sparql/scottish_parish.sparql", code=code)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/edward/src/2019/uk-geocode/geocode/wikidata.py", line 90, in wdqs raise QueryError(query, r) geocode.wikidata.QueryError: ('SELECT ?item ?itemLabel ?commonsSiteLink ?commonsCat WHERE {\n ?item wdt:P528 "186" .\n ?item wdt:P31 wd:Q5124673 .\n OPTIONAL { ?commonsSiteLink schema:about ?item ;\n schema:isPartOf <https://commons.wikimedia.org/> }\n OPTIONAL { ?item wdt:P373 ?commonsCat }\n SERVICE wikibase:label { bd:serviceParam wikibase:language "[AUTO_LANGUAGE],en". }\n}', <Response [429]>) ```
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
master
No results found.
Labels
Clear labels
No items
No labels
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: edward/geocode#11
No description provided.
Delete branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?