Heho, after our paper on mail sending configurations some time ago [1], we now glued that together into a self-service site: https://email-security-scans.org/ Even though some of you might have already seen this at the APNIC blog or on other lists, i though i'd still drop it here as well, especially given that it includes a full IPv6 readiness test for mail infra (recursive DNS, authoritative DNS, and v6 mail delivery ;-)). I'd be happy to hear your feedback, especially if things do not work as expected (then, your test ID and ideally stored emails would be really helpful, so i can double check what went wrong). More details below (also see [2] for an overview of the tests we run). Please note that setting up the tests (as we have to configure vhosts for some MTA-STS cases etc.) takes some time on our site. The test-site should periodically reload and provide the status. As we use JS for that part, please reload it manually every few minutes if you block JS. I would also be interested to hear what you think should be included in addition to the current tests; Some of our plans for the future below as well. We already found some interesting bits, like Vultr having lame v6 delegation for their AuthNS servers' domain, making their domain and rDNS non v6-resolvable [3], or mail-in-a-box using relaxed/simple for DKIM, which breaks signature validity on long To: headers [4]. If you want me to block certain domains/IPv(4|6) ranges or ASes for the service to keep your users from using it, please let me know and i will implement it asap. # Overview The site tests a variety of parameters about mail sending (details: [2]), by evaluating mails a user sends us in reply to a message that can be requested on the site. Simply requesting a message on the site does not start any evaluation or measurements, i.e., you have to send a reply-all to a requested message to start the evaluation. After the test, you can also delete the test or download a copy of your data, and if you opted to do so, a copy of the emails as we received them. # Method Users send emails to us (in reply-all to a message we sent, for easier usability).Target MXes on our site are either configured in a fully RFC compliant way, or intentionally misconfigured in a common way (broken DNSSEC for the domain, for example, or a broken DANE record, again, see [2] for an overview). Thereby, we can determine the configuration/support of features by the senders' MXes. # Possible Harm The worst thing that should happen to mail operators is finding specific undeliverable mails (broken DNSSEC if DNSSEC is validated, v6 only DNS if no IPv6 DNS resolution is supported) in their mail queue, or users receiving bounces (see FAQ on the page). So far the system has been tested with several hundred mail-setups, without encountering issues. If you encounter other unintended side-effects, please reach out to abuse@email-security-scans.org or me directly off-list. # Test details Specifically, we are testing: - v4 Mail sending - v6 Mail sending - Greylisting support - Transport encryption (Plaintext/TLS cipher strength and version support, opportunistic encryption) - DANE support outbound, i.e., if you honor DANE - MTA-STS support outbound, i.e., if you honor MTA-STS (incl. whether you prefer DANE over MTA-STS, as it should be) - Sending of TLS reports and whether these are standard compliant, also providing a copy of the received report (Even though so far only three operators delivered TLS-RPT). - DNS resolvers used by the mailserver - Whether the mailers' DNS servers resolve via IPv6 - Whether the mailers' DNS servers validate DNSSEC - If all names relevant for email delivery resolve via IPv6 - If all names relevant for email delivery are DNSSEC signed - Mailer-basics (ehlo=rdns=fcrdns), SPF (from/envelope) - SPF policy size (by retrieving your policy, up to 20 referrals deep) - DKIM (also checking key-sizes, algorithm mismatches and canonicalization issues; These are surprisingly common, i.e., simple being set breaking only in cases not caught by standard dkim testers) - DMARC (policy validity, conformance of sent mails, displaying implicit defaults) - DMARC report deliverability by sending one(1) valid DMARC report for a received email, including authorization of out-of-domain RUA/RUF (also providing a copy of any bounces received) - Full IPv6 readiness (join over three previous IPv6 tests) - Whether any headers that should not be duplicated are duplicated - Whether any mandatory headers are missing - If From/Envelope match # Planned tests - ARC - evaluate outbound spam filtering - evaluate received_from stripping - test for inbound-link parsers - cluster by provider (via MTA sets), not domain - Add DANE checks for senders' MXes - test for host-addresses in SPF prefixes - add MTA-STS check for senders' domains - add test for handling more than 5 MX with only the lowest-prio one(s) being reachable - requesting-domains' MX aspects (null MX, IPv4/6 addr in MX record, non-reachable MX, broken MTA-STS/TLSA, broken DNSSEC, validation of our setups SPF/DKIM (via DNS server logs, v6 resolvability), etc.). With best regards, Tobias [1] https://www.usenix.org/system/files/atc22-holzbauer.pdf [2] https://www.email-security-scans.org/description.php [3] https://twitter.com/chelloway/status/1629999886694318080 [4] https://github.com/mail-in-a-box/mailinabox/issues/2239