Email delivery using TLS to the gmails and hotmails of this world

[akleinloog]

I've been trying to set up outbound email delivery as secure as I can have it.
This has been challenging when mail is to be delivered to the big ones, like gmail and hotmail.
Delivery fails for these, with an error like 'STARTTLS not advertised by host'. I noticed delivery is always tried on port 25.

As far as I can see, my certificates are set up correctly and the outgoing traffic is working fine. Delivery does work for some lesser known domains, so I doubt that it is my server config.

When I disable TLS for these outgoing domains, the emails get delivered. I'm just hoping there is a better way.

What I could come up with so far, are these options:

1. Create a TLS strategy that tries with TLS on the first attempt, and disables TLS on subsequent attempts if the first one fails (I tried following the example in https://stalw.art/docs/mta/outbound/tls but that didn't work, it kept repeating with TLS. Maybe the last_error has a different value, but I'm not sure how to figure out what it was. Instead of checking for a specific error, I could check only if retry_num > 0)

2. Create a failover routing strategy that tries the normal mx route first and if that fails, use a relay service for delivery.
   (maybe combine it with the one above, so it becomes a 3 step process)

3. Create an exception for every domains that has this issue and disable TLS when delivering to them (figuring it out one error at the time )

It is my understanding that these big ones should accept TLS when using a different port, like 587 or 465, but I haven't found a way to do this. It seems mx routing always uses port 25 by default. Is there any way to influence this or am I missing something?

Does anyone have a better idea or is it just the state of affairs when it comes to email delivery?
Also if you think I might have missed anything, please point it out!

[akleinloog]

So I tried the first option with the following config:

[queue.strategy]
tls = [ { if = "retry_num > 0", then = "'disabled'" },
        { else = "'enabled'" } ]

[queue.tls.disabled]
starttls = "disable"
mta-sts = "disable"
dane = "disable"

[queue.tls.enabled]
allow-invalid-certs = false
starttls = "require"
mta-sts = "optional"
dane = "optional"

Then sending an email to a gmail address gave the following errors on the first attempt:

• Error fetching TLSA record
  details = "DNSSEC Negative Record Response for _25._tcp.gmail-smtp-in.l.google.com. IN TLSA, Bogus"

• STARTTLS unavailable
  details = "STARTTLS was not advertised by host" (remote port was 25)

And the same errors on the second and third attempt, so this didn't work. It seems the if = "retry_num > 0" is getting ignored.
Anyone can see a reason why?

[akleinloog]

Tried a slightly different version of this, with startles = optional:

[queue.strategy]
tls = [ { if = "retry_num > 0", then = "'disable'" },
        { else = "'optional'" } ]

[queue.tls.disable]
allow-invalid-certs = true
starttls = "disable"
mta-sts = "disable"
dane = "disable"

[queue.tls.optional]
allow-invalid-certs = false
starttls = "optional"
mta-sts = "optional"
dane = "optional"

But the result is the same. 3 failures with the same "STARTTLS was not advertised by host" error. So what is the meaning of "optional" if it fails when it is not offered?

Google's AI Overview offered this:
STARTTLS was likely not advertised by gmail-smtp-in.l.google.com because, as a Mail Transfer Agent (MTA) receiving mail for the Gmail domain, it primarily uses opportunistic TLS (or no encryption) for inter-server email delivery, and does not perform client authentication that would necessitate the STARTTLS command. You should be using the smtp.gmail.com server with a TLS-enabled port (like 587 or 465) for sending mail from an email client, as this is the correct endpoint for mail submission and authentication, not the MX servers.

I am guessing this point is towards email clients -> they should be using 587 or 465, not other mail servers.
When asking if gmail support TLS when receiving mail from other servers, it offered:
Yes, Gmail supports TLS for receiving emails, by default attempting to use secure TLS connections and supporting TLS versions 1.2 and 1.3. A secure, encrypted connection for incoming mail requires both Gmail and the sending server to support TLS. While Gmail defaults to attempting TLS, if the sending server doesn't support it, the connection will still occur but won't be secure

When asking what port other mail servers should use to deliver mails to gmail's mx servers, it offered:
Other mail servers should use port 25 to deliver email to Gmail's MX (Mail Exchanger) servers for standard server-to-server email transfer. While other secure ports like 465 (implicit TLS) and 587 (STARTTLS) are used for submitting email from clients, port 25 remains the standard for the actual delivery of mail between Mail Transfer Agents (MTAs)

So it seems that port 25 is the right port, and TLS should be supported, but STARTTLS should not be used. How do you configure stalwart to do this?

[akleinloog]

Then tried option 3 with the following config:

[queue.strategy]
tls = [ { if = "eq_ignore_case('gmail.com', rcpt_domain)", then = "'no-tls'" },
        { if = "eq_ignore_case('hotmail.com', rcpt_domain)", then = "'no-tls'" },
        { if = "eq_ignore_case('icloud.com', rcpt_domain)", then = "'no-tls'" },
        { else = "'default'" } ]

[queue.tls.no-tls]
starttls = "disable"
mta-sts = "disable"
dane = "disable"

[queue.tls.default]
allow-invalid-certs = false
starttls = "require"
mta-sts = "optional"
dane = "optional"

Mails to all three domains were delivered successfully. But of course, sent over the wire in plain-text, not something really desirable.

Anyone has any ideas how to do this right?

[akleinloog]

So I also tried option two with fallback, like this:

[queue.strategy]
route = [ { if = "is_local_domain('', rcpt_domain)", then = "'local'" }, 
          { if = "retry_num > 0", then = "'relay'" }, 
          { else = "'mx'" } ]

[queue.route.local]
type = "local"

[queue.route.mx]
type = "mx"

[queue.route.relay]
type = "relay"
address = "<RELAY_SERVICE>"
port = 587
protocol = "smtp"

[queue.route.relay.auth]
username = "<USER_NAME>"
secret = "<SECRET>"

[queue.route.relay.tls]
implicit = false

[queue.strategy]
tls = "'default'"

[queue.tls.default]
allow-invalid-certs = false
starttls = "require"
mta-sts = "optional"
dane = "optional"

And the relay does not get triggered, it keeps repeating the same error "STARTTLS unavailable".
Somehow, that error does not cause "retry_num > 0" to become true.
Is there something wrong with my config or is this a bug?

[akleinloog]

As a temporary workaround, I will use the following config which seems to work for the recipient emails I've tested it with:

[queue.strategy]
route = [ { if = "is_local_domain('', rcpt_domain)", then = "'local'" }, 
          { else = "'mx'" } ]

[queue.route.local]
type = "local"

[queue.route.mx]
type = "mx"

[queue.strategy]
tls = "'default'"

[queue.tls.default]
allow-invalid-certs = false
starttls = "disable"
mta-sts = "optional"
dane = "optional"

If that gives a lot of other errors, then I'll switch to always using a relay service.

I really hope for a better solution, and to understand why the "retry_num > 0" doesn't work.
Thanks in advance for any hints or pointers .

[mdecimus]

Perhaps the errors you're getting are permanent and your message is not being retried at all?
Check your logs after the first failed delivery attempt.
Also, there shouldn't be any issues delivering to Google or MSN servers, both providers have TLS ciphers supported by Stalwart,

[akleinloog]

Thank you for your reply!

I tried carefully, one message at the time. From the logs I saw that the messages were being retried, they also stayed on the message queue. The retries failed with the same pattern - STARTTLS disabled, but somehow didn't trigger the "retry_num > 0" rule.
I have been changing the rules in the toml config file, and restarting the server after changes. Could it be that somehow existing rules in the database conflict with the ones in the config file? I figured it was ok to do this, as changing the starttls setting does take effect.

Since setting starttls = "disable", delivery has been smooth.

Checking again today, I noticed that I have 2 DMARC reports from google.
They mentioned a DKIM result with 202508e Fail and an SPF result with Unspecified SoftFail.
I also got 2 TLS Reports with MTA-STS successes, so that's looking good.

Those DKIM and SPF results do hint at something being wrong with the related DNS records, so I'll go double check those.

[mdecimus]

Could it be that somehow existing rules in the database conflict with the ones in the config file?

Yes, read about local vs database settings. If you use the webadmin, do not add settings manually to the TOML otherwise you'll have encounter issues