py2http2py

[thorwhalen]

The functions case

Simply put, we want to be able, given a list funcs of callables, be able to

w_funcs = http2py(py2http(funcs))
assert w_funcs == funcs 

get another list w_funcs of callables that are equivalent to the original funcs list.
The difference being that the functions of func_ are actually calling a web-service that itself calls the original functions of funcs.

Equivalent in what sense?

Well, to be defined, and possibly configured. But ideally we'd like to get as close to possible to a state where a user of the functions difference between using funcs or w_funcs (besides perhaps speed). This means of course, same inputs for the same outputs, but also same signature and input/output types.

So of course the "simply put" code above is just for illustration. In reality,

The first line will look more like:

web_service = py2http.mk_web_service(funcs, **ws_configs)
specs = web_service.openapi_specs
w_funcs = mk_request_funcs.from_openapi_specs(specs, **py_configs)

# equivalence tests ############################################
inputs_for_func = ...  # a map from functions of func to an [(args, kwargs),...] list of valid inputs
with run_server(web_service, wait_before_entering=2):
    for f, ff in zip(funcs, w_funcs):
        assert signature(f) == signature(ff)
        for args, kwargs in inputs_for_func[f]:
            f_output = f(*args, **kwargs)
            ff_output = ff(*args, **kwargs)
            assert f_output == ff_output

The general case

Now let's look at the general case. That is;

w_obj= http2py(py2http(obj))
assert w_obj == obj 

It's the same as above except we have obj instead of funcs. obj can be any python object.
Again, what needs to be defined here is what the equivalence (not "equality") == means?
What it means will depend on what we need it to mean.
We have more than that though: We know that what ever it means, the only thing that really matters, pragmatically, is behavior, and the behavior of a python object is completely defined by its methods (yes -- even data only shares its contents via methods (think of __iter__, __getitem__, etc.).

So we're back to needing equivalence over functions, which is why we started with that.
Still, let's make our general case test a bit more fleshed out.

w_obj= http2py(
    py2http(obj, py2http_configs),
    http2py_configs
)
assert are_equivalent(w_obj, obj)

Here, py2http_configs could contain information like what methods of the object we actually want to expose (and how, under what name, etc.), http2py_configs could contain particulars of how we need to wrap the py2http "http service" object to get a python object. Finally, are_equivalent can be defined structurally, or simply running some methods on some data, and checking if the datas returned by the methods are equal (or, really... equivalent in some sense, yet again).

One problem that needs to be kept in mind is the fact that during the py -> http -> py loop we have a py_data -> json_data -> py_data loop, so codecs will need to be defined. See py2json.

Also strongly related to this is the Mapping python collections builtin methods to REST API patterns discussion.

[thorwhalen]

Ideas code to get back the mapping object (store) from the http service. I.e. proper py2http2py py_obj -> http_service -> py_obj, for any python objects.
Here, I had two routes talking to a backend store: __iter__ and __getitem__, and wanted to reconstruct a full Mapping.
One of the things I had to do (and tried to do it in a generalizable way) is to make a "local" __len__ and __contains__ so that I could have a proper Mapping.

Need to figure out a much nicer/simpler way to do this, and one that is applicable to the more general case.

The code below is too particular (and convoluted).

from typing import Iterator, Mapping, Union, Iterable, Tuple, Any
from dataclasses import dataclass
from i2 import Sig
from http2py import HttpClient


OpenApiSpecs = Union[str, Mapping, Iterable[Tuple[str, Any]]]
dict_to_tuple = lambda d: tuple(d.items())

DFLT_OPENAPI_SPECS = dict_to_tuple({'url': 'http://20.219.8.178:8080/openapi'})

# TODO: Change this to default to some attribute of `openapi_specs` instead
DFLT_METHOD_CLIENT_ATTR = dict_to_tuple({
    '__iter__': '__iter__',
    '__getitem__': '__getitem__',
})

# TODO: Deal with signature and name of _func
def add_self(func, *, func_sig=None, name=None):
    def _func(self, *args, **kwargs):
        return func(*args, **kwargs)
    _func.__name__ = name or func.__name__
    func_sig = func_sig or Sig(func)
    _func.__signature__ = Sig('self') + func_sig
    return _func

class FallbackMethods:
    def __len__(self):
        count = 0
        for k in self:
            count += 1
        return count
    
    def __contains__(self, k):
        found = False
        for key in self:
            if key == k:
                found = True
                break
        return found

    def __getitem__(self, k):
        raise NotImplementedError('__getitem__ is not implemented')
    
    def __iter__(self, k):
        raise NotImplementedError('__iter__ is not implemented')
    
def method_func(store_client, method_name, method_client_attr):
    attr_name = method_client_attr.get(method_name, None)
    if attr_name:
        func = _temp_hack_to_deal_with_bad_sig(partial(store_client, _attr_name=attr_name))
        return add_self(func, name=method_name)
    else:
        fallback_function = getattr(FallbackMethods, method_name)
        return add_self(fallback_function, name=method_name)
    
def _temp_hack_to_deal_with_bad_sig(func):
    import i2
    sig = Sig(func)

    @sig
    def _func(*args, **kwargs):
        return i2.call_forgivingly(func, *args, **kwargs)
    return _func

@dataclass
class RemoteStore(Mapping):
    store_attr: str
    openapi_specs: OpenApiSpecs = DFLT_OPENAPI_SPECS
    client_factory: type =  HttpClient
    method_client_attr: Union[Mapping, Iterable] = DFLT_METHOD_CLIENT_ATTR  # TODO: Change this so that if not given explicitly, defaults to some attribute of `openapi_specs` instead

    def __post_init__(self):
        if isinstance(self.openapi_specs, str):
            url = self.openapi_specs
            self.openapi_specs = (('url', url))
        self.openapi_specs = dict(self.openapi_specs)
        self.method_client_attr = dict(self.method_client_attr)
        self.client = self.client_factory(**self.openapi_specs)
        self.store_client = getattr(self.client, self.store_attr)
        for method_name in ['__iter__', '__getitem__', '__len__', '__contains__']:
            setattr(self, '_' + method_name, method_func(self.store_client, method_name, self.method_client_attr))

    def __iter__(self) -> Iterator:
        return iter(self.___iter__(self))
    
    def __getitem__(self, k):
        return self.___getitem__(self, k)
    
    def __len__(self) -> int:
        return self.___len__(self)
    
    def __contains__(self, k) -> bool:
        return self.___contains__(self, k)

remote_store = RemoteStore('dag_store')