dependencies: replace requests with httpx

Rather than a suggestion or even proposal, I'm opening this thread merely as an early discussion for a potential future replacement of Streamlink's requests dependency in favor of httpx.

httpx is a new-ish replacement-library for requests with an (almost) idential API and it offers more features like HTTP/2 for example and it (optionally) also allows writing more modern code like asynchronous HTTP requests. There were attempts in the past to provide asynchronous APIs for requests via requests-async, but that project was abandoned and its authors continued with the httpx project.

requests may be depended upon by hundred thousands or even millions of projects in the Python ecosystem, but it has become very stale in terms of upgrades and features, and some of its optional dependencies like PySocks (which we're defining directly here for some reason) have been abandoned since years.

httpx uses the BSD 3-clause license and has a completely different set of dependencies compared to requests, with a much better and clear distinction between the purposes and features of its own project and its dependencies, unlike requests and urllib3, which are sometimes difficult to understand and debug. requests and urllib3 are also struggling with their future 3.0.0 and 2.0.0 releases respectively due to their vast usages and thus stability guarantees, meaning that there's a ton of outdated stuff like py2 compatibility for example still in there, among other things.

A stable 1.0.0 release of httpx hasn't been published yet though, so even if there were a strong consensus of everyone here and even if it were easy to replace, it wouldn't make sense just yet.

Talking about ease of replacement, since both projects share an almost identical API, replacing it as the base class of Streamlink's HTTPSession should be fairly simple. However, over the years, we've added lots of compatibility bugfixes and monkeypatches to requests and urllib3 (1, 2), which means that this would need to be checked and evaluated first.

Then there's also the part where both projects differ. Most of it should be fine though and would only require a change in the HTTPSession class itself, and not in any actual http session usages like stream implementations or plugins, etc. For users of Streamlink, I don't think there would be any major changes in terms of features supported by requests directly, like TLS/SSL or proxy stuff via env vars, etc. Streamlink also packages a fork of a requests FileAdapter for the file:// protocol, but that should be fine for the most part, and if not, should be easy to replace or properly rewrite.

Another replacement would be requests-mock in Streamlink's tests which would need to be figured out as well. A substitution exists in the form of RESPX.

Packaging-wise, the first httpx releases should be available since 2020 in non-monolithic package repositories, e.g.:

• https://packages.debian.org/bullseye/python3-httpx
• https://archlinux.org/packages/community/any/python-httpx/
• https://src.fedoraproject.org/rpms/python-httpx
• https://packages.gentoo.org/packages/dev-python/httpx
• https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/python-modules/httpx/default.nix
• https://build.opensuse.org/package/show/devel:languages:python/python-httpx
• https://github.com/void-linux/void-packages/tree/master/srcpkgs/python3-httpx
• https://pkgsrc.se/www/py-httpx

The ones where I couldn't find httpx were homebrew and solus.

The main question however is how reliable httpx will be in the future. Since it appears well maintained and looks like it has some momentum in terms of development and adoption by other projects, this doesn't look bad at all.