Automated emails should have working unsubscribe link

[MrBartusek]

Hey, I constantly get Automated Best Practices Badge Reminder e-mails from this project, and the unsubscribe link doesn't seem to work correctly. It redirects me to a page that looks something like this:

1. The supposed unsubscribe functionality is a small checkbox marked as advanced with a wall of text next to it - why? It's extremely hard to find.
2. This checkbox doesn't work if I am not logged in. Why would I want to log in to unsubscribe from emails?

This behaviour is completely unacceptable. I've just marked these emails as spam.

[david-a-wheeler]

We have to have people log in to modify this value, because otherwise anyone could control the settings of anyone else's projects. The application is a web application and it currently has no way to receive emails, it can only send them.

Alternatively: Feel free to simply email me at dwheeler (at) linuxfoundation (dot) org. I'm happy to stop sending reminder emails for that project.

Our preferred way to stop getting the reminder emails is to get a passing badge :-). These emails are only sent to people who have started but haven't completed a passing badge.

[david-a-wheeler]

These project-wide settings could be moved to their own page, then at least it'd be easy to find on that page.

[MrBartusek]

@david-a-wheeler

We have to have people log in to modify this value, because otherwise anyone could control the settings of anyone else's projects. The application is a web application and it currently has no way to receive emails, it can only send them.

I would suggest an implementation using regular tokens or JWTs. That would depend if you have some means to store state or not. You can e-mail the unsubscribe link with a special token that would automatically uncheck this option. This is a solved problem, and works this way on any other service. You can check on your own inbox that this is the way most of the services operate.

Alternatively: Feel free to simply email me at dwheeler (at) linuxfoundation (dot) org. I'm happy to stop sending reminder emails for that project.

I hope it's not the official unsubscribe method 😬 As for me, I've marked them as spam so it won't bother me anymore :)

[david-a-wheeler]

State is stored in the web browser already, so in many cases the URL leads them directly to how to change the setting. I'm very hesitant to send tokens/JWTs in the reminder email. If someone expressly says "I've lost my password send me an email" that's fine. However, some email systems don't even encrypt hop-by-hop; quietly sending a login token that way without user initiation would create a vulnerability that the user won't be aware of.

No, sending email to me isn't the official unsubscribe method :-). I'm just trying to be helpful. I don't like spam either.

Getting emails to the app is harder than you might think in this setup :-(. It used to be easier to configure email sending and receiving than it is today.

As a first step, I think we should move the "generic" settings to their own page. Then it won't be hiding in a sea of options. Iinstead, it'd be "don't send emails" and "add/remove authorized people". That would make it TRIVIAL to see the option.

Thanks.

[MrBartusek]

@david-a-wheeler

State is stored in the web browser already, so in many cases the URL leads them directly to how to change the setting.

In some cases - not really "many" cases. You are just disallowing unsubscribing from emails without log-in for most cases, it's a horrible horrible practice.

I'm very hesitant to send tokens/JWTs in the reminder email. If someone expressly says "I've lost my password send me an email" that's fine. However, some email systems don't even encrypt hop-by-hop; quietly sending a login token that way without user initiation would create a vulnerability that the user won't be aware of.

I've never stated to send a "login token" via email. I've explicitly stated that it should be the unsubscribe link. I really suggest checking your own personal inbox; it's a common practice everywhere.

As a first step, I think we should move the "generic" settings to their own page. Then it won't be hiding in a sea of options. Iinstead, it'd be "don't send emails" and "add/remove authorised people". That would make it TRIVIAL to see the option.

It won't resolve the problem, still would require sign-in. You could put a giant glowing button on the middle of the screen, and if it requires login, it's not a correctly implemented unsubscribe link.

I would suggest revising your decision to close this issue. This is a problem in terms of user experience, email deliverability and legality. GDPR, for example, requires each email sent to the EU, to have an unsubscribe link that doesn't require login. I'm positive there are other USA and UK laws that you can apply here. Current implementation is very likely illegal. If you don't plan to do anything here, I would at least suggest blocking emails to the EU and possibly other countries to avoid fines.

[MrBartusek]

Getting emails to the app is harder than you might think in this setup :-(. It used to be easier to configure email sending and receiving than it is today.

Not revelant, nobody does that and i've never suggested that

[david-a-wheeler]

Ah, now I see what you're getting at. Thank you. I like the idea of a specialized unsubscribe link that is NOT the same as a login.

Yes, I think we should implement this. This approach means we would not need to send any login info via email, and in fact, the app doesn't need to be able to read emails at all (reading emails turns out to be a problem for our environment). We could send a one-time "goodbye" email when they are removed, just in case someone maliciously sends such a response. Currently we track this information per-project, not per-person, since someone might want reports on one and not another, but that should not be a problem.

We do have an unsubscribe link of course! It just doesn't currently work the way you want it to work. I suspect the current system is enough to comply with the GDPR, though I'm not a lawyer. But the goal isn't only to comply with the GDPR. I want to exceed GDPR requirements where it's reasonable to do so.

But you're right, this approach would work much better than what we do currently.

I'll reopen this.

[david-a-wheeler]

We already support URLs of the form https://www.bestpractices.dev/projects/PROJECTNUMBER/badge and https://www.bestpractices.dev/LOCALE/projects/PROJECTNUMBER/LEVEL among others. We could add a new URL format specifically for unsubscribing.

We probably want to confirm that someone wants to stop notifications. We support many locales, so the URL should probably include the locale so we present the locale best understood by the user.

Here's one approach: We could support a new URL format https://www.bestpractices.dev/LOCALE/projects/PROJECTNUMBER/stop_reminders?auth=RANDOMID where RANDOMID is some project-unique long cryptographically random value that must match to be accepted. Given this:

• A GET could reply a message in this locale asking if the user really wants to stop reminders, with a "confirm" button. The "confirm" button would need to pass on the auth value as a POST request.
• A POST would stop reminders.

This would create special cases where this special "level" does something quite different. We could pin that to "level 0" but that seems odd since it's not pinned to a level. We could omit the locale, but then we have to figure out the locale by looking up info on the user, which would leak "preferred locale" information about the owner to others.

Comments welcome.

[david-a-wheeler]

Note: It's important that a mere "GET" request not actually change state (e.g., disable emails). Some systems "preload" URLs mentioned in emails by GETting data from those URLs (e.g., to check if their content is malicious). That won't be a problem with the approach I described; a GET will simply show information to be confirmed, and only an express POST request from a user from the web form would actually disable reminders.

[TonyLHansen]

Most of what you wrote seems reasonable. However, I'm not sure how you plan on creating RANDOMID or storing it for subsequent comparison. Should RANDOMID instead be a salted hash or encrypted value that is provably associated with the project?

[MrBartusek]

@TonyLHansen You can create a random crypto-safe string and store it in the database, with relation to the user. Then, if it matches, it disables e-mails in all projects connected with that user. Table could be really as simple as id, user_id, expires_at, created_at, Then you just generate this token for each email sent and periodically pruge this table

[david-a-wheeler]

I was planning to store the random value in the database. My plan is to associate this with the project, not the user, since users may want to hear reminders about one and not the other. I'm thinking we only need 1 more column in the "projects" table. Only the owner receives this reminder email.

[david-a-wheeler]

We certainly could associate this information per-user instead of per-project. Currently we do this per-project. Per-user might be better in case we add other kinds of reminders. And granted, it's a more common approach.

In that case, I'd recommend a URL of this form: https://www.bestpractices.dev/LOCALE/users/USERNUMBER/unsubscribe?auth=RANDOMID

A GET on that resource would ask for confirmation, a POST on it would disable it. To re-enable, the user would have to log in, where they could directly control "subscription". This requires a new column on users, "subscribed", similar to what we already do on projects. The obvious way to implement authentication would be to add one additional column on the users' name ("unsubscribe_token" or some such). An alternative is to create a keyed secret using the user ID, a constant like "subscribe" and an internal secret - we could then verify it's valid. I guess the keyed secret approach might be better, it'd eliminate a lot of unnecessary data storage.

We might also want to verify that the date was in a reasonable time; it's best to have a limited time for any authentication (say, 3 weeks). The URL could include &requesttime=TIME, and the keyed secret could verify that time was the one used and was within the expected timeframe.

Again, when this happens, a final email would be sent. That way, if the user didn't do it, they'll receive a notification about it.

One problem is that we currently store this information per-project, not per-user. Most users don't unsubscribe, so we could walk the database to find all the unsubscriptions, then unsubscribe any corresponding user. I would do this in a phased approach, keeping the old (now unused) data until we're sure the new system works.