Extend jobsub_api for getting completed job information via lens/landscape

[marcmengel]

@retzkek mentioned:

One can query the landscape job API (what’s used to populate job status and information in a number of fifemon dashboards), e.g:

>> requests.post(LENS_URL,json={'query':'query{submission(id:"78637685.0@jobsub01.fnal.gov"){id idle running completed}}'}).json()
{'data': {'submission': {'id': '78637685@jobsub01.fnal.gov', 'idle': 874, 'running': 581, 'completed': 3194}}}>

the API docs are in the GraphQL playground (button in the upper-left corner), which is also useful for coming up with your query. In GraphQL you explicitly request the data you want it to return.
caveats: updates can be delayed by a few minutes, and support is best-effort only.

We might consider adding a "lens()" call to the SubmittedJob objects to get data that way... it could update the job data and return a dictionary very much like q_long().

[retzkek]

When I discussed this with Ray he mentioned that he was concerned about cases when jobsub_q (in the past) would sometimes incorrectly return an empty set; Lens should return an error if it can't find data for a submission, but we need to make sure that is propagated through the API.

>>> pprint(requests.post(LENS_URL,json={'query':'query{submission(id:"99999.0@jobsub01.fnal.gov"){id idle running completed}}'}).json())
{'data': {'submission': {'completed': 0,
                         'id': '99999@jobsub01.fnal.gov',
                         'idle': 0,
                         'running': 0}},
 'errors': [{'message': 'unable to get states for 99999@jobsub01.fnal.gov',
             'path': ['submission']}]}

That's only when using the submission() or job() query to get a specific submission/job status; requesting all "active" submissions() could still result in an empty set without an error.