Heatmaps now giving 403 error

[w-biggs]

Not sure if this is actually a bug with the extension or (hopefully not) a policy change on the Strava side but the authentication no longer seems to be working for me. When I copy the TMS url into JOSM, the tiles all show HTTP Error 403.

[zekefarwell]

I was made aware of a url change on Strava's side from this post on the OSM forum: https://community.openstreetmap.org/t/new-strava-heatmap-extension-for-id/100544/27

I haven't looked into it yet, but it sounds like the urls this extension generates will no longer work. Hopefully it will be possible to update to the new format with minimal fuss. I plan to have a look at the issue soon.

[cbueth]

I can reproduce your issue.

The latest update to the Strava article on the OSM Wiki notes "Strava [is deploying] new URLs with enhanced security features", see
https://wiki.openstreetmap.org/w/index.php?title=Strava&oldid=2828749#Global_Heatmap, and as
as @zekefarwell also just noted. I assume they are doing this to keep users on their site.
In contrast to whb's comment, there is a further cookie needed.

Since I have not worked with this tile server until today, perhaps someone with more experience can provide input. Here are some observations: When communicating with the initial server https://heatmap-external-a.strava.com/tiles-auth/all/mobileblue/15/22155/18212.png, I receive the error code InvalidKey with the message Unknown Key.

Looking into the network traffic on the global heatmap website, tiles are loaded from another server, namely https://content-a.strava.com. For the server to serve the tiles, not just CloudFront-Key-Pair-Id, CloudFront-Policy, and CloudFront-Signature were needed, but also the cookie _strava_idcf.
When the cookies are passed in the headers, the tile is served. Using curl:

curl 'https://content-a.strava.com/identified/globalheat/sport_Run/mobileblue/15/22155/18212.png' \
  -H 'Accept-Encoding: gzip, deflate, br, zstd' \
  -H 'Cookie: CloudFront-Key-Pair-Id=...; CloudFront-Policy=...; CloudFront-Signature=...; _strava_idcf=...;' \
  --output test.png

Since applications typically use a single URL for TMS communication, it would be beneficial to include all parameters in the query string, as it is possible with the three CloudFront parameters. However, content-a.strava.com appears to process the three parameters, as it returns error messages when they are not provided. When passing all three, the empty response is given. Adding the _strava_idcf does not solve the problem. If your application can pass the headers for the request, this will resolve the issue.
Another workaround could be setting up a proxy, but in my opinion, the complexity outweighs the benefits.

[zekefarwell]

Thanks for the info @cbueth. Discussion in julcnx/id-strava-heatmap-extension#6 seems to also suggest that the cookie is required. This would be rather unfortunate as I'm not aware of a way to easily make JOSM add a cookie to imagery requests. Maybe it is possible, but it would beyond the scope of this browser extension anyway. In #11 we had some input from a Strava team member expressing interest in having this extension remain usable. @kafkersmash, are you still with Strava and can you provide us any insight about these recent url changes?

[w-biggs]

Thanks all for the info! I filed an issue with Strava Metro earlier today about this issue and just added to that thread with a link to this GitHub issue and some of these new details; if they get back to me with any info I'll pass that on here.

I also filed a JOSM ticket to add the ability to include cookies in imagery requests, though I'm hot holding my breath for this to actually happen since this is a niche issue: https://josm.openstreetmap.de/ticket/24237

[zekefarwell]

Looks like it should be possible to pass a cookie param to JOSM via remote control: https://josm.openstreetmap.de/wiki/Help/RemoteControlCommands#imagery
If this works as expected it should be possible to get the click to open in JOSM feature working again.

I'm not seeing a way to pass a cookie param to iD with a custom background layer: https://github.com/openstreetmap/iD/blob/develop/API.md#url-parameters
So the click to open in iD feature may need to be removed. This is probably ok though since https://github.com/cmoffroad/id-strava-heatmap-extension now provides a better experience for iD users anyway.

The manual copy/paste option also may as well be removed since the JOSM Add Imagery URL UI doesn't provide a place to put cookie information.

[littlebtc]

I can confirm that using the cookies query parameters in the remote control work, like:

http://localhost:8111/imagery?title=Strava&type=tms&cookies=CloudFront-Key-Pair-Id=XXXXXXX; CloudFront-Policy=eyJXXXXXXX; CloudFront-Signature=XXXXXXX; _strava_idcf=eyXXXXXXXX;&url=https://content-a.strava.com/identified/globalheat/all/mobileblue/{zoom}/{x}/{y}.png

Replace cookies with the actual cookies sent to the Strava tile server with at least four keys listed above, especially _strava_idcf which is recently added.

And it may not work if you do not put cookies before url; those query parameters will work either with or without encodeURIComponent-encoded. Also, julcnx/id-strava-heatmap-extension#6 (comment) says that only content-a is available (not b or c), so something like {switch:a,b,c} is not required for now.

Also it seems that _strava_idcf is a JWT valid for 24 hours, I don't know whether the tile can be still loaded if that expires.

[w-biggs]

The other Cloudfront cookies expire on a regular basis (not sure how often, but not much longer than a few days) so having to refresh the tokens every day isn't too big of a hurdle IMO. I can deal with that.

[kafkersmash]

Hey team - thanks for the tag. Looking in to this now.

[secutanudu]

Is there any update on this? Any workaround to get Strava heatmaps into JOSM in the meantime?

[hungerburg]

Constructing a remote call to JOSM like specified above #19 (comment) by @littlebtc indeed loads the requested Strava heatmap into the editor:

http://localhost:8111/imagery?title=Strava All 15&type=tms&max_zoom=15&cookies=CloudFront-Key-Pair-Id=XXXXXXX; CloudFront-Policy=eyJXXXXXXX; CloudFront-Signature=XXXXXXX; _strava_idcf=eyXXXXXXXX;&url=https://content-a.strava.com/identified/globalheat/all/hot/{zoom}/{x}/{y}.png

A bit of a nuisance to copy/paste the missing values manually. The single click much more convenient! As said, it expires quite sooner than the previous method, less than 24 hours even? max_zoom at 15 useful to prevent a large number of 404s?

[NathanHowell]

#20 has a rough implementation of #19 (comment) the Copy & Paste text box no longer makes sense since it doesn't include the cookies, and iD support is (still) broken.

[Nekzuris]

iD supports it perfectly with this extension: https://github.com/cmoffroad/id-strava-heatmap-extension

[hungerburg]

I got bored of copying Strava cookies manually from firefox developer tools into a text editor to create the JOSM remote call, so I did download the code from @NathanHowell fork and did as https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions/Your_first_WebExtension#installing - Voilà I can load Strava layer from a click again :)

PS: The copy/paste box I never used.

[whymoo]

Downloaded the source as a zip and did load unpacked in edge, getting this error when I try to hit the button "Unknown error
Couldn't build url. Check console for errors." In the console I get this error:
TypeError: cookies.entries is not a function
at buildJosmUrl (content.js:97:35)
at setModalHtmlSuccess (content.js:112:28)
at HTMLButtonElement.openModalDialog (content.js:71:13)

[zekefarwell]

It only works in Firefox so far. See: #20 (review)