Error "remote delivery failed: corrupted queue file" with RT submission

[mslehto]

Hello, I am trying to setup DragonFly Mail Agent from FreeBSD 13.1 base to
work with RT (Request Tracker ticket handling software).

Sometimes error "remote delivery failed: corrupted queue file" is logged
and then mail is bounced.

From remote server log (and TCP port capture) it looks like DMA
disconnects after DATA before remote acknowledgement.
Here is what remote server gets in Postfix log:
postfix/smtpd[3592]: lost connection after DATA (873 bytes)

Currently my RT setup submits automated emails to DMA with options -bq.
At this stage local queuing, before remote delivery, seem to happen without logged errors.

When I later manually flush DMA queue with option -q then syslog shows:

Sep 21 22:05:11 rt dma[5697f.801c48000][72412]: remote delivery failed: corrupted queue file
Sep 21 22:05:11 rt dma[5697f.801c48000][72412]: delivery failed, bouncing as 575a0

(I also tried, with same results, DMA from FreeBSD binary pkg, version string is 0.13.20220128_1,1)

[mslehto]

Fortunately I was able to extract failing raw queue file from /var/spool/dma, attached M5697f.801c48000.txt

M5697f.801c48000.txt

[mslehto]

I don't know too much about queue files or content transfer encoding, but when comparing failing file
to working queue file I notice that one obvious difference is last byte.
Working raw queue file ends in byte 0x0A while failing file ends in actual mail content byte.

[mslehto]

I notice that near log statement in code[1] regarding corrupted queue file
there is some some \n newline checking so that's why 0x0A in raw queue file seems interesting.

[1] https://cgit.freebsd.org/src/tree/contrib/dma/net.c?h=releng/13.1#n596

[mslehto]

Also, open issue #18[2] and merge request #49[3] is somehow related to newline handling and
contains interesting discussion.

[2] #18
[3] #49 (comment)

[corecode]

So the question is, why is there a newline missing in the queue file?

[mslehto]

I think so yes, I guess that newline separator in queue file is part of the problem.

I do not understand whether "corrupted queue file" is something the caller[1] of DMA causes.
I don't think it is unreasonable to expect that DMA would not create corrupted queue file for itself,
or maybe I am missing something in my usage or debug approach?

[1] When sending mail, RT calls "/usr/libexec/dma" with arguments "-bq -t"

[corecode]

The code in readmail() doesn't have any obvious route that would leave the queue file without newline at the end, even if the submission ends without a newline. But it's happening, so: where, how? I guess you could add logging statements that wind up in syslog.

[mslehto]

Thanks, I added some logging statement and built DMA... ...and couldn't reproduce anymore.

Turns out that I was not entirely using updated version (v0.13) from pkg.
Submission (-bq) was always done with older version (v0.11) from base system.

When using FreeBSD pkg v0.13 also for submission, then later queue flushing is succesful.

Summary: it seems that DMA v0.11 from FreeBSD 13.1 base creates corrupted queue files on submission. Sorry for the noise I'll report this to FreeBSD.

[corecode]

Thanks!

[mslehto]

( downstream bug report is here: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=266629 )