Bash email forwarding script

[Kariton]

Hey ho,

After some try and error I got a working bash script to forward system emails to this API.
My primary problem is that munpack is not supplied within the default repositorys (or EPEL) for RHEL 8.

After some googeling I found a extremely helpful post that got me started.

So here it is.
My little dirty way to redirect mails (even with multiple attachments) to Signal.

WARNING:
it is quiet easy to break or maybe even misuse this script with a preperated mail.
it may be possible to trigger a code execution from within the emails

What it does
This script may break your system.

• Extract attachments and message body of emails.
• Mail receive / Signal Message sending follows this basic flow:
  • remove most information from mail header
  • no attachment -> send email body
  • one attachment -> send email body with attachment
  • N attachments -> send email body with first found attachment
    send every other attachment afterwards
  • attachments bigger then 100MB (Signal limit) will be moved to ${large_attachment_dir} and stay there until manual deletion.
    the ${message_recipient} will receive a information about that.

What it NOT does

• track or handle any POST errors (send to API).

Known problems

• there might be a security flaw
• I get random names for my received files.
• Messed up file extentions. for example: *.tar.gz results in *.gz only tested with *.jpg and *.tar.gz
  (is this another issue worth? Set file names of attachments #136)

Usage
Alias /etc/aliases

mail2signal: |/usr/local/bin/mail2signal.sh

Postfix /etc/postfix/virtual

signal@example.com mail2signal@localhost

Script /usr/local/bin/mail2signal.sh

Version 1

Requirements
(at least for me)
jq

#!/bin/bash

# message_server="https://signal.example.com/v2/send"
# message_sender='+987654321'
#
# message_recipient='"+123456", "+789456", "+456789"'
# message_recipient='"+123456"'

# where to save attachments that are to large for Signal
large_attachment_dir="/srv/mail2signal"


### --- ### --- ### --- ###

# create unique folder per mail
tmp="$(mktemp -d /tmp/XXXXXXXXXX)"

timestamp=$(date +20%y-%m-%d_%H-%M-%S)
raw_mail="${tmp}/raw_mail"
raw_message="${tmp}/raw_message"
message="${tmp}/message"
attachment_dir="${tmp}/attachments"
raw_attachment="${tmp}/tmp_attachment"
base64_attachment="${tmp}/base64_attachment"

# save mail
cat > "${raw_mail}"

# greate directorys
mkdir -p ${large_attachment_dir}
mkdir -p ${attachment_dir}

# count attachments
nattachment=$(grep -i 'content-disposition: attachment;' -A 1 ${raw_mail} | grep -ci 'filename=')

# extract attachments
##################################################################################################################################
##                                                                                                                              ##
## SPECIAL THANKS TO ANDY - He posted a Bash native solution, after some minor tweaks it does the trick                         ##
## > https://stackoverflow.com/questions/28741486/what-is-the-alternative-of-munpackmime-encoded-mail-message/29386144#29386144 ##
##                                                                                                                              ##
## (munpack is not available on RHEL 7/8 and deviates -> https://access.redhat.com/articles/6543)                               ##
##################################################################################################################################
i=1
while [ ${i} -le ${nattachment} ];do
    #get filename of 1. attachments in remaining $tmpfile --> option -m 1 in grep!
    filename=$(grep -ni 'content-disposition: attachment;' ${raw_mail} -m ${i} -A 1 | tail -n 1 | cut -d ';' -f 2 | cut -d '=' -f 2 | sed 's/"//g')

    #get startline of 1. attachments in remaining $tmpfile--> option -m 1 in grep
    startline=$(grep -ni 'content-transfer-encoding: base64' ${raw_mail} -m ${i} | tail -n 1 | cut -d : -f 1)

    #attachment starts after a blank line
    let startline=${startline}+1

    #keep tail in $tmpfile
    tail -n +${startline} ${raw_mail} > ${raw_attachment}

    #get endline of the 1. attachment in remaining $tmpfile
    #attachment ends with a blank line and a string that begins with '--'
    endline=$(grep -n '^--' ${raw_attachment} -m ${i} | tail -n 1 | cut -d ':' -f 1)
    let endline=${endline}-1

    #keep head and decode attachment with base64 via openssl.
    head -n ${endline} ${raw_attachment} > ${base64_attachment}
    openssl enc -d -base64 -in ${base64_attachment} -out ${attachment_dir}/${timestamp}_${i}_${filename}

    # keep tail for further attachemnts
    let endline=${endline}+2
    tail -n +${endline} ${raw_mail} > ${raw_attachment}

    let i=${i}+1
done

# create message
# cleanup HEADER
sed '1d;0,/^Return-Path: .*/{/^Return-Path: .*/d;};0,/^Delivered-To: .*/{/^Delivered-To: .*/d;};0,/^.*id .*;.*/{/^.*id .*;.*/d;};0,/^.*by .*/{/^.*by .*/d;};0,/^.*for .*/{/^.*for .*/d;};0,/^To: .*/{/^To: .*/d;};0,/^User-Agent: .*/{/^User-Agent: .*/d;};0,/^MIME-Version: .*/{/^MIME-Version: .*/d;};0,/^Content-Type: .*/{/^Content-Type: .*/d;};0,/^.*boundary=.*/{/^.*boundary=.*/d;};0,/^Message-Id: .*/{/^Message-Id: .*/d;};' "${raw_mail}" > "${raw_message}"

# remove metadata fróm attachments
if [ ${nattachment} -gt 0 ]; then
    # remove MIME header - TOP of mail body
    remove_startline=$(grep -ni 'This is a multi-part message in MIME format.' ${raw_message} -m 1 | cut -d : -f 1)
    remove_endline=$(grep -ni 'Content-Disposition: inline' ${raw_message} -m 1 -A 1 | tail -n 1 | cut -d - -f 1)
    sed -i "${remove_startline},${remove_endline}d" "${raw_message}"

    # remove MIME attachments from message
    endline=$(grep -ni 'content-disposition: attachment;' "${raw_message}" -m 1 -B 4 -A1 | head -n 1 | cut -d - -f 1)
    sed -n "1,$(( ${endline} - 1 )) p; ${endline} q" "${raw_message}" > ${message}
else
    mv "${raw_message}" ${message}
fi
message_text="$(jq -R -s '.' < ${message})"

# send message
# if attachments exists
if [ "$(ls -A ${attachment_dir})" ]; then
    # send all attachments but only one message
    if [ "$( find ${attachment_dir} -maxdepth 1 -type f | wc -l)" -gt 1 ]; then
        i=1
        for file in $(find ${attachment_dir} -maxdepth 1 -type f); do
            file_size="$(wc -c ${file} | awk '{print $1}')"
            # check size of attachment - more then 100MB / 13110000 bytes is not allowd. in that case recipients will be informed
            if [ "${file_size}" -lt 13110000 ]; then
                base64_file="$(base64 -w 0 < ${file})"
                case ${i} in
                    1)
                         curl "${message_server}" -x "" \
                         -H "Accept: application/json" \
                         -H "Content-Type:application/json" \
                         --data @<(cat <<EOF
{
  "number": "${message_sender}",
  "message": ${message_text},
  "base64_attachments": [
    "${base64_file}"
  ],
  "recipients": [
    ${message_recipient}
  ]
}
EOF
) &> /dev/null
                         let i=${i}+1
                         ;;
                    *)
                         curl "${message_server}" -x "" \
                         -H "Accept: application/json" \
                         -H "Content-Type:application/json" \
                         --data @<(cat <<EOF
{
  "number": "${message_sender}",
  "base64_attachments": [
    "${base64_file}"
  ],
  "recipients": [
    ${message_recipient}
  ]
}
EOF
) &> /dev/null
                         let i=${i}+1
                         ;;
                    esac
            else
                # filesize over 100MB / 13110000 bytes
                mkdir -p ${large_attachment_dir}${file%/*}
                mv ${file} ${large_attachment_dir}${file}
                mv ${message} > ${message}_e_${i}
                echo -e "\n\nERROR: Attachment too large!\n'${file##*/}' is larger then Signals maximum supported filesize.\n\nFilesize: ${file_size} bytes\nMaximum allowed Filesize: 13110000 bytes\n\nThe File was moved to: '$(hostname -f):${large_attachment_dir}${file}'" >> ${message}_e_${i}
                e_message_text="$(jq -R -s '.' < ${message}_e_${i})"
                curl "${message_server}" -x "" \
                -H "Accept: application/json" \
                -H "Content-Type:application/json" \
                --data @<(cat <<EOF
{
  "number": "${message_sender}",
  "message": ${e_message_text},
  "recipients": [
    ${message_recipient}
  ]
}
EOF
) &> /dev/null
                let i=${i}+1
            fi
        done
    else
        # there is just one attachment
        attachment="$(find ${tmp}/attachments/ -maxdepth 1 -type f)"
        base64_file="$(base64 -w 0 < ${attachment})"
        curl "${message_server}" -x "" \
            -H "Accept: application/json" \
            -H "Content-Type:application/json" \
            --data @<(cat <<EOF
{
  "number": "${message_sender}",
  "message": ${message_text},
  "base64_attachments": [
    "${base64_file}"
  ],
  "recipients": [
    ${message_recipient}
  ]
}
EOF
) &> /dev/null
    fi
# no attachment - just send message
else
    curl "${message_server}" -x "" \
    -H "Accept: application/json" \
    -H "Content-Type:application/json" \
    --data @<(cat <<EOF
{
  "number": "${message_sender}",
  "message": ${message_text},
  "recipients": [
    ${message_recipient}
  ]
}
EOF
) &> /dev/null
fi

# delete everything
rm -rf "${tmp}"

Requirements
(at least for me)
jq
ripmime

#!/bin/bash
export LC_ALL="C"
export LC_NUMERIC="C"

# message_server="https://signal.example.com/v2/send"
# message_sender='+987654321'
#
# message_recipient='"+123456", "+789456", "+456789"'
# message_recipient='"+123456"'
# message_recipient='"group.MV4GC3LQNRSWO4TPOVYHG5DSNFXGO==="'

# where to save attachments that are to large for signal
large_attachment_dir="/srv/mail2signal"

# curl options
curl_opts=(
  --raw
  --retry 12
  --retry-max-time 60
)


### --- ### --- ### --- ###

# create unique folder per mail
tmp="$(mktemp -d /tmp/XXXXXXXXXX)"
raw_mail="${tmp}/mail"
header="${tmp}/header"

# save mail
cat > "${raw_mail}"

# extract MIME
ripmime -q --name-by-type -i "${raw_mail}" -e "${header##*/}" -d "${tmp}"

# get plain message
message_plain="$(cat $(find "${tmp}" -type f -name "text-plain[[:digit:]]"))"

# add some MIME e-Mail header to message text
## get MIME header
## remove unnecessary information
message_header="$(
    sed '/^$/q' "${header}" | \
    sed -e '1d;0,/^Return-Path: .*/{/^Return-Path: .*/d;};' \
        -e '1d;0,/^Delivered-To: .*/{/^Delivered-To: .*/d;};' \
        -e '1d;0,/^.*id .*;.*/{/^.*id .*;.*/d;};' \
        -e '1d;0,/^.*by .*/{/^.*by .*/d;};' \
        -e '1d;0,/^.*for .*/{/^.*for .*/d;};' \
        -e '1d;0,/^To: .*/{/^To: .*/d;};' \
        -e '1d;0,/^User-Agent: .*/{/^User-Agent: .*/d;};' \
        -e '1d;0,/^MIME-Version: .*/{/^MIME-Version: .*/d;};' \
        -e '1d;0,/^Content-Type: .*/{/^Content-Type: .*/d;};' \
        -e '1d;0,/^.*boundary=.*/{/^.*boundary=.*/d;};' \
        -e '1d;0,/^Message-Id: .*/{/^Message-Id: .*/d;};' \
        -e '1d;0,/^Message-ID: .*/{/^Message-ID: .*/d;};' \
        -e '1d;0,/^Content-Transfer-Encoding: .*/{/^Content-Transfer-Encoding: .*/d;};'
)"

## insert cleaned header at message start
message="${message_header}

${message_plain}"

# set json message text
message_text="$(jq -R -s '.' <<< "${message}")"

# path to  attachments
attachments="$(find "${tmp}" -type f \( ! -name "${raw_mail##*/}" ! -name "${header##*/}" ! -name "multipart-mixed[[:digit:]]" ! -name "multipart-alternative[[:digit:]]" ! -name "text-plain[[:digit:]]" \) -print)"
# count attachments
nattachments="$(printf "%s" "${attachments}" | grep -c "^")"

# send message
## if attachments exists
if [ "${nattachments}" -ne 0 ]; then
    ## send all attachments but only one message
    if [ "${nattachments}" -gt 1 ]; then
        i=1
        while IFS= read -r file; do
            file_size="$(wc -c "${file}" | awk '{print $1}')"
            ## check size of attachment - more then 100MB / 13110000 bytes is not allowd. in that case recipients will be informed
            #if [ "${file_size}" -lt 13110000 ]; then
            if [ "${file_size}" -lt 13110000 ]; then
                base64_file="$(base64 -w 0 < "${file}")"
                case ${i} in
                    1)
                         curl "${curl_opts[@]}" \
                             "${message_server}" \
                             -H "Accept: application/json" \
                             -H "Content-Type:application/json" \
                             --data @<(cat <<EOF
{
  "number": "${message_sender}",
  "message": ${message_text},
  "base64_attachments": [
    "${base64_file}"
  ],
  "recipients": [
    ${message_recipient}
  ]
}
EOF
) &> /dev/null
                         let i=${i}+1
                         ;;
                    *)
                         curl "${curl_opts[@]}" \
                             "${message_server}" \
                             -H "Accept: application/json" \
                             -H "Content-Type:application/json" \
                             --data @<(cat <<EOF
{
  "number": "${message_sender}",
  "base64_attachments": [
    "${base64_file}"
  ],
  "recipients": [
    ${message_recipient}
  ]
}
EOF
) &> /dev/null
                         let i=${i}+1
                         ;;
                    esac
            else
                ## filesize over 100MB / 13110000 bytes
                mkdir -p "${large_attachment_dir}${file%/*}"
                mv "${file}" "${large_attachment_dir}${file}"
                printf "%s" "${message}" > "${tmp}/error_mail_${i}"
                printf "\n\nERROR: Attachment too large! \n'%s' is larger then Signals maximum supported filesize.\n\nFilesize: %s bytes\nMaximum allowed Filesize: 13110000 bytes\n\nThe File was moved to: '%s:%s'" "${file##*/}" "${file_size}" "$(hostname -f)" "${large_attachment_dir}${file}" >> "${tmp}/error_mail_${i}"

                error_message_plain="$(cat "${tmp}/error_mail_${i}")"
                error_message="${message_header}

${message_plain}"
                error_message_text="$(jq -R -s '.' <<< "${error_message}")"
                curl "${curl_opts[@]}" \
                    "${message_server}" \
                    -H "Accept: application/json" \
                    -H "Content-Type:application/json" \
                    --data @<(cat <<EOF
{
  "number": "${message_sender}",
  "message": ${error_message_text},
  "recipients": [
    ${message_recipient}
  ]
}
EOF
) &> /dev/null
                let i=${i}+1
            fi
        done <<< "${attachments}"
    else
        ## there is just one attachment
        file_size="$(wc -c "${attachments}" | awk '{print $1}')"
        ## check size of attachment - more then 100MB / 13110000 bytes is not allowd. in that case recipients will be informed
        #if [ "${file_size}" -lt 13110000 ]; then
        if [ "${file_size}" -lt 13110000 ]; then
            base64_file="$(base64 -w 0 < "${attachments}" )"
            curl "${curl_opts[@]}" \
                "${message_server}" \
                -H "Accept: application/json" \
                -H "Content-Type:application/json" \
                --data @<(cat <<EOF
{
  "number": "${message_sender}",
  "message": ${message_text},
  "base64_attachments": [
    "${base64_file}"
  ],
  "recipients": [
    ${message_recipient}
  ]
}
EOF
) &> /dev/null
        else
            ## filesize over 100MB / 13110000 bytes
            mkdir -p "${large_attachment_dir}${attachments%/*}"
            mv "${attachments}" "${large_attachment_dir}${attachments}"
            printf "%s" "${message}" > "${tmp}/error_mail"
            printf "\n\nERROR: Attachment too large! \n'%s' is larger then Signals maximum supported filesize.\n\nFilesize: %s bytes\nMaximum allowed Filesize: 13110000 bytes\n\nThe File was moved to: '%s:%s'" "${attachments##*/}" "${file_size}" "$(hostname -f)" "${large_attachment_dir}${attachments}" >> "${tmp}/error_mail"

            error_message_plain="$(cat "${tmp}/error_mail")"
            error_message="${message_header}

${error_message_plain}"
            error_message_text="$(jq -R -s '.' <<< "${error_message}" )"
            curl "${curl_opts[@]}" \
                "${message_server}" \
                -H "Accept: application/json" \
                -H "Content-Type:application/json" \
                --data @<(cat <<EOF
{
  "number": "${message_sender}",
  "message": ${error_message_text},
  "recipients": [
    ${message_recipient}
  ]
}
EOF
) &> /dev/null
        fi
    fi
## no attachment - just send message
else
    curl "${curl_opts[@]}"\
        "${message_server}" \
        -H "Accept: application/json" \
        -H "Content-Type:application/json" \
        --data @<(cat <<EOF
{
  "number": "${message_sender}",
  "message": ${message_text},
  "recipients": [
    ${message_recipient}
  ]
}
EOF
) &> /dev/null
fi

# delete everything
rm -rf "${tmp}"

Test
Without attachment

echo "Test Body" | mail -s "Test Subject" mail2signal@localhost

With attachment ${TESTFILE}

echo "Test Body" | mail -s "Test Subject" -a ${TESTFILE} mail2signal@localhost

From remote system

echo "Test Body" | mail -s "Test Subject" -a ${TESTFILE} signal@example.com

Any idea or improvement is very welcome.
At some point I might redesign this script with some improvements and an eye on proper coding style.
Or may be able to write a python replacement.
But for now it does the trick for me.

EDIT1: add script requirements
EDIT2: script version 2 which uses ripmime instead of grep / tail / head / sed

[Kariton]

SELinux
I'm by no means an SELinux expert!
But i managed to generate a working ruleset to keep SELinux in enforcing mode.

Save script in /usr/libexec/postfix/ instead of /usr/local/bin/
Set system_u:object_r:postfix_exec_t:s0 as security context.

ruleset
create the following files:
mail2signal.pp

||SE Linux Module
mail2signal1.0@fileexecute_no_tranobject_r@@@@@postfix_exec_t@postfix_local_t@@@@@@@@@@@@@@@@@@@@@@@@@@filobject_rpostfix_exec_tpostfix_local_t

mail2signal.te

module mail2signal 1.0;

require {
        type postfix_local_t;
        type postfix_exec_t;
        class file execute_no_trans;
}

#============= postfix_local_t ==============
allow postfix_local_t postfix_exec_t:file execute_no_trans;

mail2signal_curl.pp

||SE Linux Module
mail2signal_curl1.0lnk_filereaobject_r@@@@@tmpfs_t@postfix_local_t@@@@@@@@@@@@@@@@@@@@@@@@@lnk_filobject_rtmpfs_tpostfix_local_t

mail2signal_curl.te

module mail2signal_curl 1.0;

require {
        type postfix_local_t;
        type tmpfs_t;
        class lnk_file read;
}

#============= postfix_local_t ==============
allow postfix_local_t tmpfs_t:lnk_file read;

install them

semodule -i mail2signal.pp
semodule -i mail2signal_curl.pp

[bbernhard]

very nice - many thanks for sharing!

[Kariton]

I updated the script which now uses ripmime instead of grep / tail / head / sed