initial base_url companion support + proxy companion

[unixfox]

Explained what I want to implement: iv-org/invidious-companion#118 (comment)

And support proxying the requests to invidious companion from invidious: #5215 in order to ease the deployment of invidious + companion.

Fixes #5215

TODO

• Allow the ability to omit "public_url" in "invidious_companion" config for activating the built-in proxy in invidious.
• Fix the built-in proxy
• Re-review if the way of accessing the chosen invidious_companion created from the COMPANION_POOL is a correct way.

[syeopite]

This is leaking a client

Suggested change

client = make_client(companion.private_url, use_http_proxy: false)

[syeopite]

JSON.parse can actually accept an IO as an argument. It'll be more efficient to stream the request and parse the JSON as data comes in.

You should be able to do something like this:

wrapper.client.post("#{companion_base_url}#{endpoint}", headers: headers, body: data.to_json) do | response |
  response_body = JSON.parse(response.body_io)
end

[syeopite]

This is duplicating the error message twice when it'd be appended to the InfoException below

Suggested change

"Error while communicating with Invidious companion: " \

[syeopite]

Suggested change

	@client = HTTP::Client.new(companion.private_url)
	@client = make_client(companion.private_url, use_http_proxy: false)

[syeopite]

Please add the Accept-Encoding header to each companion request as so you aren't decompressing the body. Invidious should just directly proxy the compressed response from companion to the end user.

https://crystal-lang.org/api/1.16.3/HTTP/Client.html

If compress [HTTP::Client instance property] isn't set to false, and no Accept-Encoding header is explicitly specified, an HTTP::Client will add an "Accept-Encoding": "gzip, deflate" header, and automatically decompress the response body/body_io.

[syeopite]

Replaced with invidious_companion_urls

Suggested change

invidious_companion = CONFIG.invidious_companion.sample

[syeopite]

Suggested change

	wrapper.client.close
	wrapper.close

[syeopite]

Suggested change

client = make_client(companion.private_url, use_http_proxy: false)

[syeopite]

I don't think there's a reason for this to be a class.

Suggested change

	class CompanionWrapper
	struct CompanionWrapper

[syeopite]

Do you think you can also document this type? It wasn't immediately obvious to me what this wrapper was used for. Maybe something like:

# Packages a `HTTP::Client` to an invidious-companion instance alongside the configuration for that instance.
#  
# This is used as the resource for the `CompanionPool` as to allow the ability to
# query companion instances hosted on a subpath
struct CompanionWrapper

[syeopite]

Debug puts

Suggested change

puts "Using companion: #{wrapper.companion.private_url}"