Feature request: make HTTP library pluggable

[tommorris]

I was looking through the code for mf2py the other day and realised we boil requests in as a dependency. httpx exists and it's pretty good (async, http2, etc). Might be useful if we could make it support httpx as well as requests and reconsider requests as default. There are a few differences that are documented.

[Ousret]

Hello there,

Sorry to barge in,
There's an alternative you may be interested in. https://github.com/jawah/niquests

This is a drop-in-worthy replacement for Requests that ships with modern capabilities and allows you a painless transition.
I can answer any concerns you may have.

[angelogladding]

The singular instance is here:

mf2py/mf2py/parser.py

Lines 129 to 143 in 8f72a81

	data = requests.get(
	self.__url__,
	headers={
	"User-Agent": self.useragent,
	},
	)
	# update to final URL after redirects
	self.__url__ = data.url
	# HACK: check for character encodings and use 'correct' data
	if "charset" in data.headers.get("content-type", ""):
	doc = data.text
	else:
	doc = data.content

requests' stability makes for a sensible default.

Here's what you'd currently have to use to leverage a different library:

import httpx
url = "https://example.com"
mf2py.parse(httpx.get(url), url=url)

Here's what we could do:

import httpx
mf2py.parse(url="https://example.com", url_fetcher=httpx)

This would work with anything that emulates the requests API -- specifically any namespace/object that has a .get() function/method:

import niquests
mf2py.parse(url="https://example.com", url_fetcher=niquests)

This would also enable session support which is something I've wanted before and may prove to be more useful as protected posts proliferate:

import requests
session = requests.Session()
session.auth = ('user', 'pass')
mf2py.parse(url="https://example.com", url_fetcher=session)

Choices for the keyword argument are url_fetcher, url_getter, user_agent, agent, ...