Blogs

ISPConfig3 with Mail Quarantine, Roundcube and Amacube plugin

Netuxo recently migrated its mail hosting from an aging server using postfixadmin to a new server which forms part of our wider ISPConfig3 based hosting infrastructure. The default ISPConfig3 installation uses amavisd-new, all administered via the ISPConfig3 interface, which in our case is on a different server (a master-slave setup with one master server and several slave servers). We also wanted to provide the option to quarantine spam and virus emails, as several clients had expressed their fear of losing that important email because it could just get bounced (and they wouldn't know where this important email would come from, so whitelisting wasn't an option). It took us a while to figure out a good way of doing this, and this blog post describes what we did.

Our setup

Our setup is as follows:

  • A ISPConfig3 master server on server A. This server also provides webmail services using Roundcube.
  • A mailserver (server B), with ISPConfig3 as slave to the master on server A.
  • The ISPConfig3 Roundcube plugins, which allow users to control many aspects of their email account (such as vacation reply, forwarders, spam policy) from Roundcube.
  • A badly hacked version of the Amacube Roundcube plugin to access the mail quarantine. We did not want users to change their amavis settings using this plugin, as this is dealt with via the ISPConfig3 plugins.
  • For us as system admins a version of Mailzu to access the site or server quarantine.

Installation and configuration

We started with the default ISPConfig3 installation on a Debian Stretch server, adding server B as a slave to server A during installation. Server B only provides mail services, and in theory does not even need a web server, but this does not matter here. After setting up server B as mail server, we migrated all mail domains, accounts, aliases, forwarders etc from postfixadmin (this might be the topic for another blog post), so that they exist in ISPConfig3's databases on both, the master and slave servers.

The next step was the installation of the ISPConfig3 Roundcube plugins. The installation is well documented on the github page, so we do not cover it here. The set of plugins make use of the ISPConfig3 API interface and access the master server A. In our case, and following the installation instructions, this worked well out of the box. ISPConfig3 takes then care of replicating the settings to the slave server(s).

Mail Quarantine

We mostly followed the manual by Mathieu at https://uname.pingveno.net to set up mail quarantine. In our case we opted for creating the additional tables for mail quarantine in the Amavis/ISPConfig3 database, mostly because Amacube later gave us a lot of trouble trying to use two different databases. There is no problem with having these additional tables in the ISPConfig3 database on only the slave server B, as ISPConfig3 will just ignore them. Important is that these tables need to be on the server actually running amavisd-new, NOT on the master server (unless the master server happens to be your mail server).

Amacube

Amacube is where it gets complicated, and where we had to start hacking. For a start, Amacube is designed to access the amavis database directly, and in its original form allows a user to manipulate values in the amavis database. This is not what we wanted here, for two reasons:

  • As in our case we have a master->slave setup via ISPConfig3, manipulating the database on one server (server B) would get ISPConfig3 confused. Amacube does not do it the ISPConfig3 way - it just is not designed to do so (and there is nothing wrong with that).
  • Most of that side of the functionality of Amacube is provided by the set of ISPConfig3 Roundcube plugins, so there was no need for Amacube to provide that functionality.

What we were after was Amacube quarantine management, nothing more.

In a first step, we commented out the functionality related to the mail spam filter settings in Amacube. It probably would have been cleaner to do so via an additional config variable which would allow to disable this functionality, but in a first step we went for just commenting it out.

Then we got to the point we were interested in: Managing the Mail Quarantine. Amacube needs access (can be readonly) to the amavis/ISPConfig3 database on server B, which is the mail server. We created a MySQL user which can access this database remotely (remember, we have Roundcube installed on server A). In a first step, we had to adapt the query in Amacube quite a bit to make it work - some field names were different, some fields were in different tables, and some tables have different names in ISPConfig3 compared to Amavis. These fixes got us as far as showing the quarantined emails for exactly the logged in user. So far, so good.

In a second step we added some more to the query to:

  • Get all quarantined email for any alias of that user's email address, or any straight forwarder (account1@domain1.com->account2@domain2.com).
  • Get all quarantined email for a catchall if the user's email is setup as recipient for catchall, but exclude mail to other existing mailboxes.

This added some extra queries (to get aliases, other mailboxes (to exclude), etc), and some more complexity to the query getting the relevant quarantined emails.

The resulting hacked version of Amacube can be downloaded from https://netuxo.co.uk/sites/default/files/amacube-ispconfig-2017-08-07.tar.gz. We would be happy for anyone else to use this version of Amacube and to improve on it.

Mailzu

Mailzu was initially the preferred option for managing the Mail Quarantine, but in the end we opted against it, and to just keep it for "us" to be able to check the Mail Quarantine of the entire server. Mailzu does not seem to be under active development for many years already, and it seems unlikely to get picked up again. The nice thing is that it allows "us" (superusers) to see all quarantined email, but that is not a functionality we would want to provide to anyone else. Mailzu does not have domain admins, which would limit access to all emails from one domain - which would be a nice functionality to have.

More importantly, once we came across Amacube it was clear that from a user perspective the best option was to provide all functionality in one place - Roundcube. This means the quarantined email can easily be accessed from this one place. Although Amacube does not provide the option to preview a quarantined email, we did not consider this so crucial as to justify asking a user to log into a separate interface to manage their quarantined email. So a hacked Amacube it is in the end.

Netuxo is retiring Squirrelmail

How to transfer your addressbook to Roundcube

At the end of November 2015 Netuxo will retire Squirrelmail, which we know some of our clients still use to access their email. We started to provide the much more user friendly and modern Roundcube client a long time ago, and now it is time to retire Squirrelmail as it is no longer being developed very actively.

As some of our clients also use the Squirrelmail addressbook, we'll explain in this blog post how you can export your addressbook from Squirrelmail, and import it into Roundcube. This is a straightforward process and you should not encounter any problems.

So, if you are a Squirrelmail user, please follow the steps explained below BEFORE the end of November 2015. If you do not do so, you will no longer be able to access your Squirrelmail addressbook after that date.

1. Export your addressbook from Squirrelmail

Login to your email using the Squirrelmail client (https://mx.netuxo.co.uk/squirrelmail), and click on the "Addresses" link at the top. This will get you to your addressbook. You will now find some import and export options underneath. You can ignore the import options – if you want to import addresses, please do so into Roundcube only. What you need are the export options - at the very bottom.

You do not need to change any of the defaults here. Just click the button which says “Export to CSV File”. Depending on your browser, you might get different options.

In Firefox, you will be given the options “Open with” or “Save File”. You should save the file on your computer. Again, depending on your browser settings, you may be asked where to save the file (remember where you save it to!), or automatically save it in the default download location on your computer.

You have now successfully exported your Squirrelmail addressbook. The next step is to import the addressbook into Roundcube.

2. Import your addressbook into Roundcube

Now login into Roundcube at https://mx.netuxo.co.uk/roundcube and click on the "Address Book" link at the top. You will probably see in your left sidebar two address books: Personal addresses and Global addresses. The global addresses is relevant for all email accounts in your domain (ie all mailboxes @yourdomain), so if you want the addressbook to be available to all email accounts, you can import it there.

First, select the addressbook you want to import into. This is likely to be empty if you did not use Roundcube before.

You will see the icon for import at the top – it is marked in the screenshot.

When you click on this icon, you will be shown the import screen:

Under “Browse...”, select the file you exported from Squirrelmail. You can again select which addressbook the contacts will be imported into.

Click on “Import”, and the import will start. If you have a large addressbook, this might take a while.

You should finally get a message displaying the number of successfully imported addresses.

And that is it. Simple.

Of course, in the unlikely event you experience any problems, please do not hesitate to contact us.

Additional address book functions in Roundcube

We already mentioned above that Roundcube provides a global addressbook for each domain. This enables you to share your addresses among several email accounts within the same domain.

Roundcube also enables you to add external addressbooks from other servers in carddav format. Netuxo provides a carddav server, which enables you to synchronize your addressbook between Roundcube and other email clients – on your desktop or mobile phone. Please get in touch if you would like to find out more and/or read additional information here: https://mx.netuxo.co.uk/carddav.html

Netuxo luchando contra el spam

Los clientes del alojamiento de correos de Netuxo probablemente ya han notado que hemos reforzado nuestra lucha en contra del spam (correos basura) en los últimos meses. Hemos mejorado tanto los filtros del spam como de los virus, y probablemente han notado que mucho menos correos basura pasan por nuestros filtros.

Pero filtrar los correos spam solamente es una parte del juego y solamente trata de los correos spam recibidos por nuestros servidores. Ahora estamos implementando dos estándares importantes relacionados con la lucha contra el spam, y los dos permiten al servidor recipiente de los correos enviados por nuestros servidores verificar que vienen de un servidor autorizado. Desde agosto estamos implementando SPF (Convenio de Remitentes o en ingles Sender Policy Framework) y DKIM (DomainKeys Identified Mail) en nuestro servidor de correos, e instamos a todos nuestros clientes de alojamiento de correos contactarnos para que podemos implementar los dos estándares para su dominio. En lo siguiente explicamos en breve como funcionan los dos estándares.

Sender Policy Framework (SPF)

SPF (véase https://en.wikipedia.org/wiki/Sender_Policy_Framework) es un mecanismo bastante simple para exponer la falsificación de direcciones del envío en un correo. Permite al servidor de correos que recibe un correo comprobar que el correo fue enviado por un servidor autorizado. Lo hace a través de una entrada de un registro SPF en la zona DNS de un dominio, que incluye:

  • que servidores (por nombre y/o dirección IP) son autorizados para enviar correo de ese dominio
  • que hacer si un servidor recibe un correo que viene de un servidor no incluido en la lista de servidores autorizados.

La mayoría del software de filtración de spam (como spamassassin – que usamos nosotrxs) comprueban el registro SPF de un dominio y si lo existe verifican que el correo recibido viene de un servidor autorizado. Si no, el software anti-spam puede penalizar el correo que aumenta la probabilidad que el correo seria considerada como spam. El mecanismo es simple y desafortunadamente su eficaz es limitado. No obstante ayuda en la lucha contra el spam y es un estándar utilizado mucho.

DomainKeys Identified Mail (DKIM)

DKIM (véase https://es.wikipedia.org/wiki/DomainKeys_Identified_Mail) va mas allá. DKIM permite al servidor de correos recipiente comprobar que un correo recibido de un dominio es autorizado por lxs administradorxs de este dominio y que el correo (incluyendo los adjuntos) no fue modificado en el camino. Una firma digital incluido en el mensaje se puede verificar por el recipiente utilizando la clave publica publicada en el DNS.

DKIM consiste en dos partes. El servidor de correo remitente (nuestro servidor smtp a mx.netuxo.co.uk) firma el correo enviado (el mensaje completo incluyendo adjuntos) de un dominio usando una clave privada especifica para cada dominio. Este firma de DKIM se incluye en un campo de la cabecera del mensaje. El servidor recipiente consulta la clave publica del dominio en el registro DKIM del DNS del dominio, y la utiliza para verificar la firma del mensaje. De esta manera DKIM permite que el correo viene de un servidor autorizado (por que es firmado) y que no fue modificado en el camino (por que se puede verificar la firma).

Como con SPF es necesario un registro DNS del dominio que incluye la clave publica del dominio.

A vosotrxs

En el momento solamente estamos firmando nuestros propios correos con DKIM, y hemos publicado los registros DNS de SPF y DKIM para netuxo.co.uk y netuxo.com. No obstante, nos gustaría firmar también los correos suyos enviados por nuestros servidores. Para que podemos hacerlos, necesitamos vuestra ayuda:

  • Si también gestionamos vuestro dominio podemos añadir los registros DNS necesarios. No obstante, necesitamos que vosotrxs nos dicen si también envían correos a través de otros servidores, y si si de que servidores, para que podemos crear un registro SPF adecuado. Para DKIM podemos crear las claves y publicar el registro DNS.;
  • Si vosotrxs gestionan el dominio por su mismo y han configurado mx.netuxo.co.uk como su registro MX (que significa que nosotros alojamos su correo) vosotrxs tienen que añadir los registros DNS para SPF y DKIM. Podemos ayudarle y darles los detalles del registro DKIM (que incluyen la clave publica), y podemos asistirles en la creación de su registro SPF y en como añadir los registros a su DNS. No obstante, por que eso depende mucho de quien es el registrador de su dominio, y si su registrador provee soporte para los registros DNS requeridos.

Según Kaspersky Lab, en el primer trimestre de 2015 de todos los correos 59,2% fueron considerados spam (véase https://securelist.com/analysis/quarterly-spam-reports/69932/spam-and-phishing-in-the-first-quarter-of-2015/). Esta cifra es mucho menos que las estimaciones de mas que 80% de hace 5 años, pero todavía es mas que solamente una molestia y además ánade mucho estrés a los sistemas de correos. La lucha contra el spam necesita continuamente nuestros esfuerzos. Contactan a nosotrxs ...

Dejandose volar y mirando hacia el futuro

Este verano estuvimos trabajando en unos proyectos con los que nos hemos sentido muy cómodos. Unos de los proyectos es una nueva pagina que se llama “Lost in Samsara” (véase su entrada en nuestro porfolio y visita su pagina aquí (enlace externo)). Él proyecto junta oportunidades para el comercio justo de la base, una “tienda” local y una trueque de productos y servicios (“The Wheel”). Las fundadoras Alessia y Marvi dicen que tuvieron que “enfocarse en lo que realmente importa a nosotras y tener el coraje a cambiar lo que no nos gusta. Al mismo tiempo vemos este proyecto como nuestro camino para romper el circulo samsara de una producción insostenible, una producción que causa sufrimiento tanto para la gente como para el medio ambiente. Nos gustaría empezar a actuar de una manera distinta para obtener resultados distintos y 'Lost in Samsara' es nuestra contribución a un mundo mas sostenible y justo.”

La pagina fue lanzada hace unas semanas, y de momento hay una campana de microfinanciación durante los próximos 40 días, y si puedes apoyar a las mujeres en su misión de continuar el desarrollo del proyecto puedes visitar su pagina de Indiegogo – detalles mas abajo. Mas información e inscripción para The Wheel (el trueque comunitario) en su pagina web.

Mirando hacia el futuro

Con Drupal 8 en el horizonte cercano una de las cosas que hemos hecho recientemente son discusiones iniciales con unos de nuestros clientes de Drupal 6 sobre el “fin de la vida” de esta versión y las opciones y oportunidades de la migración y/o de un nuevo desarrollo que serán disponibles con la publicación anticipada desde mucho tiempo de D8. En las semanas siguientes vamos a escribir a todos nuestros clientes de Drupal 6 (y de versiones mas antiguos) explicando la política en relación con el “fin de la vida” de D6 y la variedad de las opciones para el futuro.

Si te interesa el desarrollo de Drupal 8 y su cercanía a la publicación de un candidata para el lanzamiento (release candidate en ingles), mantén un ojo en esa pagina.

Páginas