Commit graph

54 commits

Author SHA1 Message Date
Thomas Gleixner 25900eb7c6 remail: v0.14
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2023-06-20 23:43:23 +02:00
Thomas Gleixner dc68e97937 remail: Make From header mangling less convoluted
Due to the requirement to reencrypt the incoming mail, remail must rewrite
the mail headers and create a new email with the 'From:' header being the
list address. So remail implemented a scheme to mangle the original senders
name into the reencrypted mails 'From:' header.

That fell flat on its nose when there was an incoming mail which had no
name part and just consisted of the actual email address. Instead of
differentiating between these cases the authors sleep deprived and grump
laden brain decided to implement 'From:' mangling in the way it is now.

It converts the original sender mail address '[Name]
<mailname@sender.domain>' to:

  'list-name for mailname_at_sender.domain' <list-name@list.domain>

This is hard to read and follow. 

Use the common format:

  'Name via list-name' <list-name@list.domain>

Except for the case where the original 'From:' header is a plain email
address without a name. This will mangle it to:

  'mailname at sender.domain via list-name' <list-name@list.domain>

The 'via list-name' add on is technically not required but is useful to
visually differentiate the name in auto-completion.

Suggested-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2023-06-20 23:43:23 +02:00
Thomas Gleixner 424944ad36 remail: Add X-Original-From header
The 'From:' header mangling makes it unnecessarily hard to figure out the
name and email address of the person who posted to a list.

Save the original sender in the 'X-Original-From:' header.

Suggested-by: Linus Torvalds <torvalds@linux-foundation.org>
Suggested-by: Konstantin Ryabitsev <konstantin@linuxfoundation.org>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Link: https://wiki.ietf.org/group/dmarc/XOriginalFrom
2023-06-20 23:43:23 +02:00
Thomas Gleixner ea4b0b4632 remail: Fix closing boundaries
EmailMessage.defects reports a CloseBoundaryNotFoundDefect on the outgoing
PGP message.

Add the missing closing boundary after the PGP payload section and fix up the
closing boundaries format in msg_strip_.*().

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2023-06-20 23:43:23 +02:00
Thomas Gleixner 2760a2d671 remail: Check outgoing mails for defects
EmailMessage.defects allows to inspect outgoing mails for defects. If found
report them in logging.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2023-06-20 23:43:23 +02:00
Thomas Gleixner 759dd19c9e remail: Simplify send_mail()
1) Use EmailMessage

   EmailMessage with the default policy handles header encoding correctly by
   default and uses '\n' line separators, which works for both as_string() and
   smtplib.send_message() correctly. The latter flattens the message with the
   RFC conform '\r\n' line separators unconditionally.

   Aside of that EmailMessage provides defects reporting which is useful to
   ensure that the outgoing mails are correct. A check for that will be added
   later

2) Simplify header handling

   Remove all existing headers from the cloned message first and add those
   which are required either as new headers or copied from the original
   message which was handed in to send_mail()

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2023-06-20 23:43:23 +02:00
Thomas Gleixner 79aa834870 remail: Unify send_mail()
No need for two functions.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2023-06-20 23:43:23 +02:00
Thomas Gleixner c3b13e47f7 remail: Simplify msg_set_header()
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2023-06-20 23:43:23 +02:00
Thomas Gleixner 88f8e48b13 remail: Use email.utils.get_addresses()
Replace more historical homebrewn parsing

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2023-06-20 23:43:23 +02:00
Thomas Gleixner 893b9510db remail: Use email.utils.parseaddr()
Manual parsing is error prone.

Suggested-by: Konstantin Ryabitsev <konstantin@linuxfoundation.org>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2023-06-20 23:43:23 +02:00
Thomas Gleixner c8ac6b5a08 remail: Use existing policy for pipe
No need for a new one.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2023-06-20 23:43:23 +02:00
Thomas Gleixner 2cee74a08b remail: Remove unused functions and debug leftovers
Spring cleaning.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2023-06-20 23:43:23 +02:00
Thomas Gleixner bab35c35b6 remail: v13
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2020-11-05 01:03:46 +01:00
Thomas Gleixner e2626e1dca remail: Fix the from mangling
The comment in mangle_from() says that if there is not a real name then the
email address is mangled by replacing @ with _at_. The comment is great
just the implementation is not doing so.

Add the replacement functionality.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2020-11-05 01:03:46 +01:00
Thomas Gleixner 576165a654 remail: v0.12
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2020-11-01 22:19:22 +01:00
Thomas Gleixner 46f6145ac5 remail: Allow optional transport based security
Add an "encryption" option 'use_transport' which does not bother with
encryption and just relies on transport security.

For admins this makes sense as none of the admin messages is really
confidential.

This is also a valid option for a subscriber and makes some sense in
scenarios where the mail provider manages the subscriber key (sic!)
and does server side decryption. Think twice before using this.

Requested-by: Konstantin Ryabitsev <konstantin@linuxfoundation.org>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Tested-by: Konstantin Ryabitsev <konstantin@linuxfoundation.org>
Reviewed-by: Konstantin Ryabitsev <konstantin@linuxfoundation.org>
2020-11-01 22:19:22 +01:00
Thomas Gleixner 1e490249a7 remail: v0.11
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2020-09-19 01:53:54 +02:00
Thomas Gleixner 05ac7bb6ac remail/pipe: Add command line option to handle config updates
When using pipe mode configuration updates are not handled until the next
mail is sent to the list. That means that e.g. welcome mails are delayed.

Add a command line option to allow the invocation of remail_pipe just to
deal with configuration updates without trying to process mail from stdin.

The update handling is serialized against concurrent incoming mail via the
folder lock, so no extra serialization is required. Whichever comes first
handles it.

Reported-by: Konstantin Ryabitsev <konstantin@linuxfoundation.org>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Tested-by: Konstantin Ryabitsev <konstantin@linuxfoundation.org>
2020-09-19 01:43:07 +02:00
Thomas Gleixner 3237b7ee67 remail: v0.10
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2020-09-04 10:32:42 +02:00
Thomas Gleixner cf0f7bc5fb remail/smime: Store sender information
For contact points (open list) it's required to collect the sender
information, email address and if available the S/MIME certificate which is
contained in the signature.

Set the encryption method info and if the mail is signed store the S/MIME
certificate.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2020-09-04 10:32:42 +02:00
Thomas Gleixner cfe274374d remail/gpg: Store sender information
For contact points (open list) it's required to collect the sender
information, email address and if available the GPG key which is contained
in the signature.

Set the encryption method info and if the mail is signed store the GPG key.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2020-09-04 10:32:42 +02:00
Thomas Gleixner 3407aa262f remail: Retrieve sender information
For open lists and especially contact points the 'From' mangling is
suboptimal as the senders email address is not contained in the mail
itself. Due to re-encryption a eventual signature is not longer intact
which means that the GPG key or the S/MIME certificate which are embedded
into the signature are not transported either.

Add infrastructure to collect sender information including key/certificate
if available and attach it to the mail. The first attachment contains
sender information and the second one if available contains the key or the
certificate.

The information is only stored when the config switch is enabled and the
sender is not subscribed to the list.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2020-09-04 10:32:42 +02:00
Thomas Gleixner ea361f973c remail/smime: Use certificate embedded in signature for verification
An open list does not have the certificate of senders and for signature
verification there is no requirement to have the certificate on disk.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2020-09-04 10:32:42 +02:00
Thomas Gleixner 43a0b5cb47 remail/remaild: Fix failure message
While handling an exception in process_msg() the handler triggers another
one by trying to print a undefined variable (mailfile).

Hand the mailfile name into the function. Use 'pipe input' for pipe mode.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2020-09-04 10:32:42 +02:00
Thomas Gleixner 7d69dec9ae v0.9
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2020-03-11 13:17:15 +01:00
Thomas Gleixner 7e4cace1e2 remail/mail: Handle dwmw2s magic evo plugin
dwmw2's evolution plugin which seems to workaround ms-exchange oddities has
another interesting format not handled by remail yet:

multipart/mixed with:
  - empty text/plain section
  - application/pgp-encrypted
  - application/octed-stream

Check for this with at least some sanity checks and morph it into a regular
multipart/encrypted message so decrypt can handle it without adding more
mess to that part.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2020-03-11 12:10:07 +01:00
Thomas Gleixner 76c07f6d49 remail/mail: Add missing encrypted.asc filename
Assuming that there is a limited number of possible filenames is just
wrong. Add yet another one.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2020-03-11 12:10:06 +01:00
Thomas Gleixner 4ae83f4389 remail: v0.8
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2020-03-04 09:42:35 +01:00
Andreas Rammhold d17b1394e9 remaild: Also notify on IN_MOVED_TO events
Some MTAs move mails into the maildir after constructing them in some
temporary path.

Signed-off-by: Andreas Rammhold <andreas@rammhold.de>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2020-03-04 09:42:35 +01:00
Andreas Rammhold 8e6e7c2cc5 config: Introduce an enabled flag for S/MIME
This allows setups where there is no S/MIME. In some scenarios using just
GPG is fine and S/MIME might even be discouraged. Previously this required
to provide a dummy S/MIME key just to make remail happy. With this new flag
there is no need for that key if S/MIME is not required for the list.

Signed-off-by: Andreas Rammhold <andreas@rammhold.de>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2020-03-04 09:42:35 +01:00
Andreas Rammhold bc2b62bdda remail: Relax the mail validation so new gTLDs have a chance to pass
These days it is common to see mail addresses from the newer TLDs that do
not always fit in two to three chracters. These are not covered by the
basic mail address validation regex.

Relax the requirement from two or three characters to just two or more
characters to covers these.

Signed-off-by: Andreas Rammhold <andreas@rammhold.de>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2020-03-04 09:42:35 +01:00
Andreas Rammhold 79d26fccfe config: Pass the incorrect mail address instead of the config path
Previously remail just logged the path of the attribute (e.g.
base.lists.demo.listaccount.addr) instead of the actually mail address that
wasn't valid.

Signed-off-by: Andreas Rammhold <andreas@rammhold.de>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2020-03-04 09:42:35 +01:00
Andreas Rammhold a031afbb25 config: Throw correct exception when address isn't valid
Signed-off-by: Andreas Rammhold <andreas@rammhold.de>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2020-03-04 09:42:35 +01:00
Thomas Gleixner 342d696b2b remail/lists: Emit warning text instead of 'txt'
'txt' is pretty useless in syslog and admin reports.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2020-02-20 12:42:50 +01:00
Thomas Gleixner fa1e6fc47e remail: v0.7
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2020-02-19 01:14:51 +01:00
Thomas Gleixner fc918a9bef remail/gpg: Use the lists private key for signing
Konstantin reported that outgoing mail from a mailing list is signed with
the default private key found in the private keyring.

That's caused by just handing boolen True into the 'sign' argument of
gpg_encrypt() while the documentation clearly says:

  sign (defaults to None)
    Either the Boolean value True, or the fingerprint of a key which is
    used to sign the encrypted data. If True is specified, the default key
    is used for signing. When not specified, the data is not signed.

Hand the list account fingerprint in if signing is enabled in the
configuration.

Reported-by: Konstantin Ryabitsev <konstantin@linuxfoundation.org>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Tested-by: Konstantin Ryabitsev <konstantin@linuxfoundation.org>
Reviewed-by: Konstantin Ryabitsev <konstantin@linuxfoundation.org>
2020-02-19 01:14:51 +01:00
Thomas Gleixner 9b0ec00c8f remail: v0.6
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2020-02-17 20:04:58 +01:00
Thomas Gleixner 1920962d97 Documentation: Add mailinglist info
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2020-02-17 20:04:23 +01:00
Andreas Rammhold 6bcb72f32e ruamel/config: Use ruamel.yaml instead of yaml
This removes the (undocumented) dependency on PyYAML and just uses
ruamel.yaml that is already a dependency.

Signed-off-by: Andreas Rammhold <andreas@rammhold.de>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2020-02-17 19:50:26 +01:00
Thomas Gleixner 231d9f0bdb remail: v0.5
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2020-01-09 19:53:44 +01:00
Konstantin Ryabitsev 5e8939d9a0 remail: Make sure list configs inherit global gpg/smime
Per-list config dictionaries were always instantiated with default gpg
configuration because global gpg/smime settings were being ignored. This
change makes sure that smime and gpg options set on the global level are
inherited by list-specific configurations.

Signed-off-by: Konstantin Ryabitsev <konstantin@linuxfoundation.org>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2020-01-09 19:52:22 +01:00
Konstantin Ryabitsev 18a16b08fb remail: Fix default list-id value to conform to RFC2919
According to RFC2919, List-Id header should be in the form of a hostname
value enclosed inside angle brackets. This change does two things:

1. Fixes the default to be the list address with "@" replaced by a "."
2. Allows setting custom list-id values inside remail.yaml
3. Documents the "listid" optional setting in the manpage

Signed-off-by: Konstantin Ryabitsev <konstantin@linuxfoundation.org>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Link: https://tools.ietf.org/html/rfc2919
2020-01-09 19:52:22 +01:00
Konstantin Ryabitsev 0eecc8bd2d remail: Add requirements.txt file
Using a requirements.txt file makes it easier to install dependencies
via the pip python package manager:

    pip install -r requirements.txt

This is particularly handy for testing when running via virtualenv.

Signed-off-by: Konstantin Ryabitsev <konstantin@linuxfoundation.org>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2020-01-09 19:52:22 +01:00
Thomas Gleixner e3a2f4bc1d remail/remaild: Use correct path for frozen mails and update log message
Store the frozen mails in maildir.frozen/new and not in the base
directory. Move the log message to the actual frozen handling function and
provide the correct file pathes in the log.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2019-11-14 15:49:35 +01:00
Thomas Gleixner d7b8db9b96 remail/mail: Handle Outlook GPG plugin proper
The Outlook GPG plugin works in interesting two variants:

  1) msg.asc or msc.gpg provided as a plain attachement
     without PGP envelope

  2) GpgOL_MIME_structure.txt contains a fully enveloped
     PGP payload with the proper headers.

Of course everything can be base64 encoded and the number of payload
sections is variable as well.

Implement the handling for #2 so it can coexist with the existing
workaround for #1.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2019-11-13 00:07:10 +01:00
Thomas Gleixner 8ada1c09b3 remail/mail: Sanitize incoming headers
Remove CR/LF leftovers which might be in incoming headers before setting
them. Happens when handling the weird GPG Outlook attachments.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2019-11-13 00:07:10 +01:00
Thomas Gleixner dac7c0af8d remail/pipe: Allow UTF-8 for incoming mails
Similar to mails in files, this needs to be explicitly enabled.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2019-11-13 00:07:10 +01:00
Thomas Gleixner f8da9d3278 remail/mail: Add yet another outlook magic filename
GpgOL_MIME_structure.txt is missing from the outlook GPG repair list.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2019-11-12 20:50:04 +01:00
Thomas Gleixner aa10d8c48a remail: version 0.4
Bump the version number.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2019-11-11 22:52:36 +01:00
Konstantin Ryabitsev 1ed9841f76 remail/gpg: Allow specifying path to the gpgbinary to use
We want to be able to support ECC subkeys, which limits us to gnupg
versions 2.2 and above. CentOS-7 ships with gnupg-2.0, which cannot be
easily upgraded to 2.2 due to a slew of potential problems, so we
install the newer version into /opt/gnupg22 and must call it as
/opt/gnupg22/bin/gpg.

Allow specifying gpg binary path to use instead of the default "gpg" in
$PATH.

Signed-off-by: Konstantin Ryabitsev <konstantin@linuxfoundation.org>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2019-11-11 22:49:35 +01:00