Commit graph

49 commits

Author SHA1 Message Date
Thomas Gleixner
759dd19c9e remail: Simplify send_mail()
1) Use EmailMessage

   EmailMessage with the default policy handles header encoding correctly by
   default and uses '\n' line separators, which works for both as_string() and
   smtplib.send_message() correctly. The latter flattens the message with the
   RFC conform '\r\n' line separators unconditionally.

   Aside of that EmailMessage provides defects reporting which is useful to
   ensure that the outgoing mails are correct. A check for that will be added
   later

2) Simplify header handling

   Remove all existing headers from the cloned message first and add those
   which are required either as new headers or copied from the original
   message which was handed in to send_mail()

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2023-06-20 23:43:23 +02:00
Thomas Gleixner
79aa834870 remail: Unify send_mail()
No need for two functions.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2023-06-20 23:43:23 +02:00
Thomas Gleixner
c3b13e47f7 remail: Simplify msg_set_header()
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2023-06-20 23:43:23 +02:00
Thomas Gleixner
88f8e48b13 remail: Use email.utils.get_addresses()
Replace more historical homebrewn parsing

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2023-06-20 23:43:23 +02:00
Thomas Gleixner
893b9510db remail: Use email.utils.parseaddr()
Manual parsing is error prone.

Suggested-by: Konstantin Ryabitsev <konstantin@linuxfoundation.org>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2023-06-20 23:43:23 +02:00
Thomas Gleixner
c8ac6b5a08 remail: Use existing policy for pipe
No need for a new one.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2023-06-20 23:43:23 +02:00
Thomas Gleixner
2cee74a08b remail: Remove unused functions and debug leftovers
Spring cleaning.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2023-06-20 23:43:23 +02:00
Thomas Gleixner
bab35c35b6 remail: v13
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2020-11-05 01:03:46 +01:00
Thomas Gleixner
e2626e1dca remail: Fix the from mangling
The comment in mangle_from() says that if there is not a real name then the
email address is mangled by replacing @ with _at_. The comment is great
just the implementation is not doing so.

Add the replacement functionality.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2020-11-05 01:03:46 +01:00
Thomas Gleixner
576165a654 remail: v0.12
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2020-11-01 22:19:22 +01:00
Thomas Gleixner
46f6145ac5 remail: Allow optional transport based security
Add an "encryption" option 'use_transport' which does not bother with
encryption and just relies on transport security.

For admins this makes sense as none of the admin messages is really
confidential.

This is also a valid option for a subscriber and makes some sense in
scenarios where the mail provider manages the subscriber key (sic!)
and does server side decryption. Think twice before using this.

Requested-by: Konstantin Ryabitsev <konstantin@linuxfoundation.org>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Tested-by: Konstantin Ryabitsev <konstantin@linuxfoundation.org>
Reviewed-by: Konstantin Ryabitsev <konstantin@linuxfoundation.org>
2020-11-01 22:19:22 +01:00
Thomas Gleixner
1e490249a7 remail: v0.11
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2020-09-19 01:53:54 +02:00
Thomas Gleixner
05ac7bb6ac remail/pipe: Add command line option to handle config updates
When using pipe mode configuration updates are not handled until the next
mail is sent to the list. That means that e.g. welcome mails are delayed.

Add a command line option to allow the invocation of remail_pipe just to
deal with configuration updates without trying to process mail from stdin.

The update handling is serialized against concurrent incoming mail via the
folder lock, so no extra serialization is required. Whichever comes first
handles it.

Reported-by: Konstantin Ryabitsev <konstantin@linuxfoundation.org>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Tested-by: Konstantin Ryabitsev <konstantin@linuxfoundation.org>
2020-09-19 01:43:07 +02:00
Thomas Gleixner
3237b7ee67 remail: v0.10
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2020-09-04 10:32:42 +02:00
Thomas Gleixner
cf0f7bc5fb remail/smime: Store sender information
For contact points (open list) it's required to collect the sender
information, email address and if available the S/MIME certificate which is
contained in the signature.

Set the encryption method info and if the mail is signed store the S/MIME
certificate.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2020-09-04 10:32:42 +02:00
Thomas Gleixner
cfe274374d remail/gpg: Store sender information
For contact points (open list) it's required to collect the sender
information, email address and if available the GPG key which is contained
in the signature.

Set the encryption method info and if the mail is signed store the GPG key.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2020-09-04 10:32:42 +02:00
Thomas Gleixner
3407aa262f remail: Retrieve sender information
For open lists and especially contact points the 'From' mangling is
suboptimal as the senders email address is not contained in the mail
itself. Due to re-encryption a eventual signature is not longer intact
which means that the GPG key or the S/MIME certificate which are embedded
into the signature are not transported either.

Add infrastructure to collect sender information including key/certificate
if available and attach it to the mail. The first attachment contains
sender information and the second one if available contains the key or the
certificate.

The information is only stored when the config switch is enabled and the
sender is not subscribed to the list.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2020-09-04 10:32:42 +02:00
Thomas Gleixner
ea361f973c remail/smime: Use certificate embedded in signature for verification
An open list does not have the certificate of senders and for signature
verification there is no requirement to have the certificate on disk.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2020-09-04 10:32:42 +02:00
Thomas Gleixner
43a0b5cb47 remail/remaild: Fix failure message
While handling an exception in process_msg() the handler triggers another
one by trying to print a undefined variable (mailfile).

Hand the mailfile name into the function. Use 'pipe input' for pipe mode.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2020-09-04 10:32:42 +02:00
Thomas Gleixner
7d69dec9ae v0.9
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2020-03-11 13:17:15 +01:00
Thomas Gleixner
7e4cace1e2 remail/mail: Handle dwmw2s magic evo plugin
dwmw2's evolution plugin which seems to workaround ms-exchange oddities has
another interesting format not handled by remail yet:

multipart/mixed with:
  - empty text/plain section
  - application/pgp-encrypted
  - application/octed-stream

Check for this with at least some sanity checks and morph it into a regular
multipart/encrypted message so decrypt can handle it without adding more
mess to that part.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2020-03-11 12:10:07 +01:00
Thomas Gleixner
76c07f6d49 remail/mail: Add missing encrypted.asc filename
Assuming that there is a limited number of possible filenames is just
wrong. Add yet another one.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2020-03-11 12:10:06 +01:00
Thomas Gleixner
4ae83f4389 remail: v0.8
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2020-03-04 09:42:35 +01:00
Andreas Rammhold
d17b1394e9 remaild: Also notify on IN_MOVED_TO events
Some MTAs move mails into the maildir after constructing them in some
temporary path.

Signed-off-by: Andreas Rammhold <andreas@rammhold.de>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2020-03-04 09:42:35 +01:00
Andreas Rammhold
8e6e7c2cc5 config: Introduce an enabled flag for S/MIME
This allows setups where there is no S/MIME. In some scenarios using just
GPG is fine and S/MIME might even be discouraged. Previously this required
to provide a dummy S/MIME key just to make remail happy. With this new flag
there is no need for that key if S/MIME is not required for the list.

Signed-off-by: Andreas Rammhold <andreas@rammhold.de>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2020-03-04 09:42:35 +01:00
Andreas Rammhold
bc2b62bdda remail: Relax the mail validation so new gTLDs have a chance to pass
These days it is common to see mail addresses from the newer TLDs that do
not always fit in two to three chracters. These are not covered by the
basic mail address validation regex.

Relax the requirement from two or three characters to just two or more
characters to covers these.

Signed-off-by: Andreas Rammhold <andreas@rammhold.de>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2020-03-04 09:42:35 +01:00
Andreas Rammhold
79d26fccfe config: Pass the incorrect mail address instead of the config path
Previously remail just logged the path of the attribute (e.g.
base.lists.demo.listaccount.addr) instead of the actually mail address that
wasn't valid.

Signed-off-by: Andreas Rammhold <andreas@rammhold.de>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2020-03-04 09:42:35 +01:00
Andreas Rammhold
a031afbb25 config: Throw correct exception when address isn't valid
Signed-off-by: Andreas Rammhold <andreas@rammhold.de>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2020-03-04 09:42:35 +01:00
Thomas Gleixner
342d696b2b remail/lists: Emit warning text instead of 'txt'
'txt' is pretty useless in syslog and admin reports.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2020-02-20 12:42:50 +01:00
Thomas Gleixner
fa1e6fc47e remail: v0.7
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2020-02-19 01:14:51 +01:00
Thomas Gleixner
fc918a9bef remail/gpg: Use the lists private key for signing
Konstantin reported that outgoing mail from a mailing list is signed with
the default private key found in the private keyring.

That's caused by just handing boolen True into the 'sign' argument of
gpg_encrypt() while the documentation clearly says:

  sign (defaults to None)
    Either the Boolean value True, or the fingerprint of a key which is
    used to sign the encrypted data. If True is specified, the default key
    is used for signing. When not specified, the data is not signed.

Hand the list account fingerprint in if signing is enabled in the
configuration.

Reported-by: Konstantin Ryabitsev <konstantin@linuxfoundation.org>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Tested-by: Konstantin Ryabitsev <konstantin@linuxfoundation.org>
Reviewed-by: Konstantin Ryabitsev <konstantin@linuxfoundation.org>
2020-02-19 01:14:51 +01:00
Thomas Gleixner
9b0ec00c8f remail: v0.6
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2020-02-17 20:04:58 +01:00
Thomas Gleixner
1920962d97 Documentation: Add mailinglist info
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2020-02-17 20:04:23 +01:00
Andreas Rammhold
6bcb72f32e ruamel/config: Use ruamel.yaml instead of yaml
This removes the (undocumented) dependency on PyYAML and just uses
ruamel.yaml that is already a dependency.

Signed-off-by: Andreas Rammhold <andreas@rammhold.de>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2020-02-17 19:50:26 +01:00
Thomas Gleixner
231d9f0bdb remail: v0.5
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2020-01-09 19:53:44 +01:00
Konstantin Ryabitsev
5e8939d9a0 remail: Make sure list configs inherit global gpg/smime
Per-list config dictionaries were always instantiated with default gpg
configuration because global gpg/smime settings were being ignored. This
change makes sure that smime and gpg options set on the global level are
inherited by list-specific configurations.

Signed-off-by: Konstantin Ryabitsev <konstantin@linuxfoundation.org>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2020-01-09 19:52:22 +01:00
Konstantin Ryabitsev
18a16b08fb remail: Fix default list-id value to conform to RFC2919
According to RFC2919, List-Id header should be in the form of a hostname
value enclosed inside angle brackets. This change does two things:

1. Fixes the default to be the list address with "@" replaced by a "."
2. Allows setting custom list-id values inside remail.yaml
3. Documents the "listid" optional setting in the manpage

Signed-off-by: Konstantin Ryabitsev <konstantin@linuxfoundation.org>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Link: https://tools.ietf.org/html/rfc2919
2020-01-09 19:52:22 +01:00
Konstantin Ryabitsev
0eecc8bd2d remail: Add requirements.txt file
Using a requirements.txt file makes it easier to install dependencies
via the pip python package manager:

    pip install -r requirements.txt

This is particularly handy for testing when running via virtualenv.

Signed-off-by: Konstantin Ryabitsev <konstantin@linuxfoundation.org>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2020-01-09 19:52:22 +01:00
Thomas Gleixner
e3a2f4bc1d remail/remaild: Use correct path for frozen mails and update log message
Store the frozen mails in maildir.frozen/new and not in the base
directory. Move the log message to the actual frozen handling function and
provide the correct file pathes in the log.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2019-11-14 15:49:35 +01:00
Thomas Gleixner
d7b8db9b96 remail/mail: Handle Outlook GPG plugin proper
The Outlook GPG plugin works in interesting two variants:

  1) msg.asc or msc.gpg provided as a plain attachement
     without PGP envelope

  2) GpgOL_MIME_structure.txt contains a fully enveloped
     PGP payload with the proper headers.

Of course everything can be base64 encoded and the number of payload
sections is variable as well.

Implement the handling for #2 so it can coexist with the existing
workaround for #1.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2019-11-13 00:07:10 +01:00
Thomas Gleixner
8ada1c09b3 remail/mail: Sanitize incoming headers
Remove CR/LF leftovers which might be in incoming headers before setting
them. Happens when handling the weird GPG Outlook attachments.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2019-11-13 00:07:10 +01:00
Thomas Gleixner
dac7c0af8d remail/pipe: Allow UTF-8 for incoming mails
Similar to mails in files, this needs to be explicitly enabled.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2019-11-13 00:07:10 +01:00
Thomas Gleixner
f8da9d3278 remail/mail: Add yet another outlook magic filename
GpgOL_MIME_structure.txt is missing from the outlook GPG repair list.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2019-11-12 20:50:04 +01:00
Thomas Gleixner
aa10d8c48a remail: version 0.4
Bump the version number.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2019-11-11 22:52:36 +01:00
Konstantin Ryabitsev
1ed9841f76 remail/gpg: Allow specifying path to the gpgbinary to use
We want to be able to support ECC subkeys, which limits us to gnupg
versions 2.2 and above. CentOS-7 ships with gnupg-2.0, which cannot be
easily upgraded to 2.2 due to a slew of potential problems, so we
install the newer version into /opt/gnupg22 and must call it as
/opt/gnupg22/bin/gpg.

Allow specifying gpg binary path to use instead of the default "gpg" in
$PATH.

Signed-off-by: Konstantin Ryabitsev <konstantin@linuxfoundation.org>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2019-11-11 22:49:35 +01:00
Thomas Gleixner
323b8cda64 remail: Add remail_pipe script
Add a handle pipe function to the remailer and a pipe script for handling
mail in a MTA delivery path.

Requested-by: Konstantin Ryabitsev <konstantin@linuxfoundation.org>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2019-11-01 13:30:25 +01:00
Thomas Gleixner
266221e151 Documentation: Rename configuration man page
The upcoming pipe script will use the same configuration.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2019-11-01 13:29:19 +01:00
Thomas Gleixner
2b43bd7789 remail: Add TODO and a hacky service file
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2019-09-22 22:48:52 +02:00
Thomas Gleixner
60f6698e52 remail: Initial import
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2019-09-22 22:38:59 +02:00