Allow to provide an alternative CKAN host

[amercader]

The xloader currently uses CKAN URLs for two things:

• Download resources of type "upload" (eg https://myckan.site.org/dataset/{id}/resource/{id}/download)
• Ping a xloader_hook action via a POST request. This was a surprise, as I was under the impression that xloader did not communicate via HTTP with the CKAN API.

In any case, both these requests use the URL built with ckan.site_url, which is the public facing URL. This won't work on scenarios where the xloader has no access to it, for instance when running in a separate container in a docker compose setup. We need to introduce support for a new config option similar to ckan.datapusher.callback_url_base which will allow xloader to call e.g. http://ckan:5000/dataset/{id}/resource/{id}/download instead of https://myckan.site.org/dataset/{id}/resource/{id}/download

• Add new ckanext.xloader.callback_url_base option to the config declaration
• To modify the download URL, the new config option needs to be used here to replace the public host with the value of the config option
• The API endpoint is passed as result_url here, so that's the place to replace the host as well if the config option is present

[ThrawnCA]

We need to introduce support for a new config option similar to ckan.datapusher.callback_url_base which will allow xloader to call e.g. http://ckan:5000/dataset/{id}/resource/{id}/download instead of https://myckan.site.org/dataset/{id}/resource/{id}/download

I'm a little confused about why the site_url would point to myckan.site.org if that URL doesn't work.

[wardi]

There can be network restrictions or reverse proxies that in prevent accessing "outside" resources from the web server.

A config option like this lets admins provide an alternate internal name/address that can be used to connect to ckan.

[ThrawnCA]

There can be network restrictions or reverse proxies that in prevent accessing "outside" resources from the web server.

Wouldn't that make the XLoader essentially unusable? I suppose it could still parse uploaded files, just not links.

[duttonw]

ckan/ckan-docker-base#76 #245