http: deal with the lack of http proxy support in node.js core.

[martinheidegger]

This is a follow-up on #8381.

Summary:
Node's http module does not support http proxies by default. This is a unfortunate situation as libraries don't use packages for http that support http proxies and as a result quite a few tools in the Node ecosystem lack the support for http proxies (i.e. through environment variables).

It would be helpful if Node.js could support user proxies and if someone could take the time to implement it, I think the following plan could help get us there:

1. Old versions of Node.js will never gain http proxy support so it would be important to have a recommended way how to add proxy support to user-land libraries to work on the old versions of Node.js. A guide on "how to support proxies" (maybe combined with a package) seems like a minimum requirement to fix this issue which will help no matter how this topic progresses. (Anyone can start with this, no need to wait for approval, help dearly welcome).
2. Create a PR to the Node.js docs in which it clearly explains that proxies are the responsibility of user-land and reference the recommended implementation/article in 1. (Just reflecting the current state of affairs)
3. Create a new discussion on to how the package used in 1. could become part of Node.js core in order to reduce the implementation cost for current and new packages.
4. Discuss the possibility of creating a Node flag that enables the behavior of 3. in case someone is stuck with user-land code that doesn't implement proxies.

[mscdex]

I'm not keen on adding HTTP proxy support in core, especially because it could be a slippery slope (e.g. people may ask to also add support for SOCKS, etc.). IMHO this is easily done in userland with a custom http.Agent. If you want it to be more automatic for all HTTP requests, you can just overwrite the http.globalAgent with your custom implementation.

[martinheidegger]

@mscdex The point of this issue is to bikeshed this particular discussion until point 3.

[Fishrock123]

Old versions of Node.js will never gain http support so it would be (...)

Did you mean "never gain http proxy support"?

[martinheidegger]

#15620 (comment) Changed

[jackjocross]

I'm not sure this is the kind of implementation that people had in mind, but after reading through #8381 I wrote a tiny lib env-proxy-agent that just wraps proxy-from-env and proxy-agent.

I wasn't able to come up with a solution that could overwrite http.globalAgent as I think NO_PROXY needs to be matched against each request individually. This is probably due to my own lack of knowledge about http.Agent but if anyone has guidance on the issue that would be great.

[stevenvachon]

@crosscompile why write a separate project when you could've instead fulfilled TooTallNate/node-proxy-agent#11

[jackjocross]

@stevenvachon thanks for pointing that out, I did not see that open issue on node-proxy-agent.

I pushed up some changes to my fork of node-proxy-agent that I'm hoping to finish up soon, will follow up in TooTallNate/node-proxy-agent#11 as it's more relevant there.

[chrmarti]

Overwriting http.globalAgent as suggest above does not work (#9057). One has to pass the agent option to each and every request.