Sympa Mailinglist Manager - LinuXia · Sympa basics List creation templates Extract of newsletter...

Sympa Mailinglist Manager

Stefan Hornburg

GP/RW Erlangen 4th March 2020

Stefan Hornburg Sympa Mailinglist Manager GP/RW Erlangen 4th March 2020 1 / 56

Introduction

About Sympa

Mailing list managerAlmost 23 years oldLegacy Perl


Introduction

Applications

Mailing list managerNewsletterMarketing


Introduction Users

Users

UniversitiesRiseup (riseup.net)

20 thousand lists3.3 million users4.8 million subscriptions


https://riseup.net/

Introduction Customers

Customers

HTWK LeipzigOpen Hardware Repository (CERN)Shop for riderseShop (US government)Social Security Law (USA)


Introduction Competition

Email Marketing


Introduction Motivation

Motivation


Introduction Motivation

Motivation

PrivacyInfrastructureOpen SourceFlexibilityCosts


Introduction Flexibility

Flexibility

ConfigurationTemplatesScenariosPlugins


Sympa basics

Sympa basics

List typesList creation templatesScenarios


Sympa basics List types

List types

Public discussion mailing listNewsletterConfidential list...


Sympa basics List creation templates

Template files

config.tt2 (Configuration)comment.tt2 (Description)


Sympa basics List creation templates

Extract of newsletter configuration

visibility noconceal

send newsletter

process_archive on

archiveweb_access publicmail_access owner

subscribe open

unsubscribe open,notify


Sympa basics Scenarios

Incoming email for newletters

# send.newsletter scenario

title.gettext Newsletter, restricted to moderators

is_editor([listname],[sender])smtp,dkim,smime,md5 -> do_it

true()smtp,dkim,smime,md5 -> reject(reason='send_editor')


Sympa basics Scenarios

Authentication methods

smtp From: address in emailmd Web interface authenticationdkim Valid DKIM signaturesmime Valid S/MIME signature


Sympa in production

Sympa in production

HTWK LeipzigSympa university wide


Sympa in production Installation and requirements

Installation

Sympa 6.2.16PostgreSQL 9.4PostfixNginx


Sympa in production Installation and requirements

Requirements

Migration from MajordomoAuthentication AD/LDAPAutomated mailing listsListmastersAdaption of CD


Sympa in production Authentication AD/LDAP

Authentication AD/LDAP

LDAP only


Sympa in production Authentication AD/LDAP

Authentication AD/LDAP

ldapregexp .*host ldap.htwk-leipzig.de:636bind_dn [email protected]_password nevairbesuffix dc=htwk-leipzig,dc=deget_dn_by_uid_filter (uid=[sender])get_dn_by_email_filter(&(objectcategory=person)(mail=[sender])(!(userAccountControl:1.2.840.113556.1.4.803:=2)))

email_attribute mailscope subuse_tls ldapsca_file /etc/ssl/certs/htwk-chain.pemca_verify required


Sympa in production Automated mailing lists

Automated mailing lists

LDAP queriesList templates



LDAP query for specific seminar groups

(&(objectcategory=person)(extensionAttribute3=08FOO15)(mail=*)(!(userAccountControl:1.2.840.113556.1.4.803:=2)))



Get all seminar groups

(&(objectcategory=person)(extensionAttribute3=*)(mail=*)(!(userAccountControl:1.2.840.113556.1.4.803:=2)))



LDAP with Perl

Net::LDAPNet::LDAP::Control



LDAP with Perl

# apply page and sort controls$page = Net::LDAP::Control::Paged->new( size => 500 );$sort = Net::LDAP::Control::Sort->new( order => "cn" );



Adjust list names

# replace / and ä ö ü ßuse Encode 'decode';use Lingua::DE::ASCII 'to_ascii';

$group =~ s/\//_/g;$group = lc( to_ascii( decode( 'UTF-8', $attribute ) ) );



List creation

List directoryList from templateSync subscribersUpdate email aliases


Sympa in production Adaption of CD

Adaption of CD

Title and Logologo_html_definition<a title="HTWK home" href="http://www.htwk-leipzig.de/"><img alt="HTWK logo" src="/sympa/icons/htwk-logo.svg"></a>CSS templateweb_tt2/css.tt2Header templateweb_tt2/header.tt2

Sympa in production Problems

Synchronisation problem

lots of lists at the same timeLDAP/AD server error



Synchronisation problem

ttl 3 hoursrandom 0-59 minutes



Caching problem

stale dataowners, moderatorspermissions


Integration

Integration

AuthenticationData Sources


Integration Authentication

Authentication

LDAPCASSSOTLSShibbolet


Integration Data Sources

Data Sources

LDAPSQLSympa ListFile



Data Sources

SubscribersModeratorsOwners



Data Sources

Update in regular intervals (ttl, distribution_ttl)Marker for included recordsUser can unsubscribe



Data Sources

include_file /home/sympa/subscribers.txt

[email protected] Test [email protected] Foo [email protected]


Features

Features

PersonalizationProtectionHierarchyScenariosPlugins


Features Personalization

Enable personalization

Web interface: Sending/receiving setupList config: merge_feature on


Example

Welcome to our great workshop, [% user.gecos %]!

Example

Welcome to our great workshop, Stefan Hornburg!


Add custom attributes

custom_attributeid levelname Your game leveltype integer


Welcome to our great workshop, [% user.gecos %]

Your game level is [% user.custom_attribute.level.value %].

Welcome to our great workshop, Stefan Hornburg!

Your game level is 25.

Features Protection

Protection

DKIMDMARC / ARCS/MIME


Features Hierarchy

Hierarchy

ServerVirtual hosts (Robots)List


Features Plugins

Plugins

Template pluginsModlist/Whitelist pluginDistribution notification pluginmessage_hookpost_archive DistributionNotification


Challenges

Challenges

Reverse DNSSPF/DKIMBouncesAddress verificationLog parsing (Sisimai)Blocking


Challenges Vetting email addresses

Vetting email addresses

my $validator = Email::Valid->new(-mxcheck => 1,

);

unless ($validator->address($email)) {warn "Email $email invalid: " .

$validator->details . "\n";}


Future Projects

Projects

REST / GraphQL APIContainerAnsibleArchive search


Future API

API

SOAP APIREST APIGraphQL APIDBIx::Class schema


Future Container

Container

SympaDatabase (PostgreSQL, MySQL, ...)MTA / IMAP / getmailWeb frontend


Future Ansible

Ansible

ProvisioningTesting


Future Ansible

Archive Search

setup Elasticsearch and index structurewrite indexer that parses the emails in the archivesetup very simple website for searchauthentication for website is list membership


Finish Questions

Questions


Finish The end

The end


Sympa Mailinglist Manager - LinuXia · Sympa basics List creation templates Extract of newsletter...

Documents

Transcript of Sympa Mailinglist Manager - LinuXia · Sympa basics List creation templates Extract of newsletter...