Upgrading an existing installation

This section contains all the upgrade procedures required to use newest versions of Modoboa.

Note

Before running a migration, we recommend that you make a copy of your existing database.

Latest version

Fetch the latest version (see Get Modoboa) and install it. pip users, just run the following command:

$ pip install modoboa==<VERSION>

Replace <VERSION> by the appropriate value.

As for a fresh installation, modoboa-admin.py can be used to upgrade your local configuration. To do so, remove the directory where your instance was first deployed:

$ rm -rf <modoboa_instance_dir>

Warning

If you customized your configuration file (settings.py) with non-standard settings, you’ll have to re-apply them.

Finally, run the deploy comamand. Make sure to consult the Deployment section to know more about the available options.

If you prefer the manual way, check if Specific upgrade instructions are required according to the version you’re installing.

To finish, restart the web server process according to the environment you did choose. See Web servers for more details.

Specific upgrade instructions

1.5.0

The API has been greatly improved and a documentation is now available. To enable it, add 'rest_framework_swagger' to the INSTALLED_APPS variable in settings.py as follows:

INSTALLED_APPS = (
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.sites',
    'django.contrib.staticfiles',
    'reversion',
    'rest_framework.authtoken',
    'rest_framework_swagger',
)

Then, add the following content into settings.py, just after the REST_FRAMEWORK variable:

SWAGGER_SETTINGS = {
    "is_authenticated": False,
    "api_version": "1.0",
    "exclude_namespaces": [],
    "info": {
        "contact": "contact@modoboa.com",
        "description": ("Modoboa API, requires a valid token."),
        "title": "Modoboa API",
    }
}

You’re done. The documentation is now available at the following address:

http://<your instance address>/docs/api/

Finally, if you find a TEMPLATE_CONTEXT_PROCESSORS variable in your settings.py file, make sure it looks like this:

TEMPLATE_CONTEXT_PROCESSORS = global_settings.TEMPLATE_CONTEXT_PROCESSORS + [
    'modoboa.core.context_processors.top_notifications',
]

1.4.0

Warning

Please make sure to use Modoboa 1.3.5 with an up-to-date database before an upgrade to 1.4.0.

Warning

Do not follow the regular upgrade procedure for this version.

Some extension have been moved back into the main repository. The main reason for that is that using Modoboa without them doesn’t make sense.

First of all, you must rename the following applications listed inside the MODOBOA_APPS variable:

Old name	New name
modoboa_admin	modoboa.admin
modoboa_admin_limits	modoboa.limits
modoboa_admin_relaydomains	modoboa.relaydomains

Then, apply the following steps:

1. Uninstall old extensions:

   $ pip uninstall modoboa-admin modoboa-admin-limits modoboa-admin-relaydomains
   

2. Install all extension updates using pip (check the Modoboa > Information page)

3. Manually migrate database:

   $ cd <instance_dir>
   $ python manage.py migrate auth
   $ python manage.py migrate admin 0001 --fake
   $ python manage.py migrate admin
   $ python manage.py migrate limits 0001 --fake
   $ python manage.py migrate relaydomains 0001 --fake
   $ python manage.py migrate
   

4. Finally, update static files:

   $ python manage.py collectstatic
   

This version also introduces a REST API. To enable it:

1. Add 'rest_framework.authtoken' to the INSTALLED_APPS variable

2. Add the following configuration inside settings.py:

   # Rest framework settings
   
   REST_FRAMEWORK = {
       'DEFAULT_AUTHENTICATION_CLASSES': (
           'rest_framework.authentication.TokenAuthentication',
       ),
       'DEFAULT_PERMISSION_CLASSES': (
           'rest_framework.permissions.IsAuthenticated',
       )
   }
   

3. Run the following command:

   $ python manage.py migrate
   

1.3.5

To enhance security, Modoboa now checks the strength of user passwords <https://github.com/dstufft/django-passwords>_.

To use this feature, add the following configuration into the settings.py file:

# django-passwords

PASSWORD_MIN_LENGTH = 8

PASSWORD_COMPLEXITY = {
    "UPPER": 1,
    "LOWER": 1,
    "DIGITS": 1
}

1.3.2

Modoboa now uses the atomic requests mode to preserve database consistency (reference).

To enable it, update the DATABASES variable in settings.py as follows:

DATABASES = {
    "default": {
        # stuff before...
        "ATOMIC_REQUESTS": True
    },
    "amavis": {
        # stuff before...
        "ATOMIC_REQUESTS": True
    }
}

1.3.0

This release does not bring awesome new features but it is a necessary bridge to the future of Modoboa. All extensions now have their own git repository and the deploy process has been updated to reflect this change.

Another important update is the use of Django 1.7. Besides its new features, the migration system has been reworked and is now more robust than before.

Before we begin with the procedure, here is a table showing old extension names and their new name:

Old name	New package name	New module name
modoboa.extensions.admin	modoboa-admin	modoboa_admin
modoboa.extensions.limits	modoboa-admin-limits	modoboa_admin_limits
modoboa.extensions.postfix_autoreply	modoboa-postfix-autoreply	modoboa_postfix_autoreply
modoboa.extensions.postfix_relay_domains	modoboa-admin-relaydomains	modoboa_admin_relaydomains
modoboa.extensions.radicale	modoboa-radicale	modoboa_radicale
modoboa.extensions.sievefilters	modoboa-sievefilters	modoboa_sievefilters
modoboa.extensions.stats	modoboa-stats	modoboa_stats
modoboa.extensions.webmail	modoboa-webmail	modoboa_webmail

Here are the required steps:

1. Install the extensions using pip (look at the second column in the table above):

   $ pip install <the extensions you want>
   

2. Remove south from INSTALLED_APPS

3. Rename old extension names inside MODOBOA_APPS (look at the third column in the table above)

4. Remove modoboa.lib.middleware.ExtControlMiddleware from MIDDLEWARE_CLASSES

5. Change DATABASE_ROUTERS to:

   DATABASE_ROUTERS = ["modoboa_amavis.dbrouter.AmavisRouter"]
   

6. Run the following commands:

   $ cd <modoboa_instance_dir>
   $ python manage.py migrate
   

7. Reply yes to the question

8. Run the following commands:

   $ python manage.py load_initial_data
   $ python manage.py collectstatic
   

9. The cleanup job has been renamed in Django, so you have to modify your crontab entry:

   - 0 0 * * * <modoboa_site>/manage.py cleanup
   

   • 0 0 * * * <modoboa_site>/manage.py clearsessions

1.2.0

A new notification service let administrators know about new Modoboa versions. To activate it, you need to update the TEMPLATE_CONTEXT_PROCESSORS variable like this:

from django.conf import global_settings

TEMPLATE_CONTEXT_PROCESSORS = global_settings.TEMPLATE_CONTEXT_PROCESSORS + (
  'modoboa.core.context_processors.top_notifications',
)

and to define the new MODOBOA_API_URL variable:

MODOBOA_API_URL = 'http://api.modoboa.org/1/'

The location of external static files has changed. To use them, add a new path to the STATICFILES_DIRS:

# Additional locations of static files
STATICFILES_DIRS = (
  # Put strings here, like "/home/html/static" or "C:/www/django/static".
  # Always use forward slashes, even on Windows.
  # Don't forget to use absolute paths, not relative paths.
  "<path/to/modoboa/install/dir>/bower_components",
)

Run the following commands to define the hostname of your instance:

$ cd <modoboa_instance_dir>
$ python manage.py set_default_site <hostname>

If you plan to use the Radicale extension:

1. Add 'modoboa.extensions.radicale' to the MODOBOA_APPS variable

2. Run the following commands:

   $ cd <modoboa_instance_dir>
   $ python manage.py syncdb
   

Warning

You also have to note that the sitestatic directory has moved from <path to your site's dir> to <modoboa's root url> (it’s probably the parent directory). You have to adapt your web server configuration to reflect this change.

1.1.7: manual learning for SpamAssassin

A new feature allows administrators and users to manually train SpamAssassin in order to customize its behaviour.

Check Manual SpamAssassin learning to know more about this feature.

1.1.6: Few bugfixes

Catchall aliases were not really functional until this version as they were eating all domain traffic.

To fix them, a postfix map file (sql-mailboxes-self-aliases.cf) has been re-introduced and must be listed into the virtual_alias_maps setting. See Configuration for the order.

1.1.2: Audit trail issues

Update the settings.py file as follows:

1. Remove the 'reversion.middleware.RevisionMiddleware' middleware from the MIDDLEWARE_CLASSES variable
2. Add the new 'modoboa.lib.middleware.RequestCatcherMiddleware' middleware at the end of the MIDDLEWARE_CLASSES variable

1.1.1: Few bugfixes

For those who installed Dovecot in a non-standard location, it is now possible to tell Modoboa where to find it. Just define a variable named DOVECOT_LOOKUP_PATH in the settings.py file and include the appropriate lookup path inside:

DOVECOT_LOOKUP_PATH = ("/usr/sbin/dovecot", "/usr/local/sbin/dovecot")

1.1.0: relay domains and better passwords encryption

Due to code refactoring, some modifications need to be done into settings.py:

1. MODOBOA_APPS must contain the following applications:

   MODOBOA_APPS = (
     'modoboa',
     'modoboa.core',
     'modoboa.lib',
   
     'modoboa.extensions.admin',
     'modoboa.extensions.limits',
     'modoboa.extensions.postfix_autoreply',
     'modoboa.extensions.webmail',
     'modoboa.extensions.stats',
     'modoboa.extensions.amavis',
     'modoboa.extensions.sievefilters',
   )
   

2. Add 'modoboa.extensions.postfix_relay_domains' to MODOBOA_APPS, just before 'modoboa.extensions.limits'

3. AUTH_USER_MODEL must be set to core.User

4. Into LOGGING, replace modoboa.lib.logutils.SQLHandler by modoboa.core.loggers.SQLHandler

Then, run the following commands to migrate your installation:

$ python manage.py syncdb
$ python manage.py migrate core 0001 --fake
$ python manage.py migrate
$ python manage.py collectstatic

Finally, update both Dovecot and Postfix queries.

1.0.1: operations on mailboxes

The way Modoboa handles rename and delete operations on mailboxes has been improved. Make sure to consult Operations on the file system and Postfix configuration. Look at the smtpd_recipient_restrictions setting.

Run modoboa-admin.py postfix_maps --dbtype <mysql|postgres|sqlite> <tempdir> and compare the files with those that postfix currently use. Make necessary updates in light of the differences

1.0.0: production ready, at last

Configuration file update

Several modifications need to be done into settings.py.

1. Add the following import statement:

   from logging.handlers import SysLogHandler
   

2. Set the ALLOWER_HOSTS variable:

   ALLOWED_HOSTS = [
       '<your server fqdn>',
   ]
   

3. Activate the django.middleware.csrf.CsrfViewMiddleware middleware and add the reversion.middleware.RevisionMiddleware middleware to MIDDLEWARE_CLASSES like this:

   MIDDLEWARE_CLASSES = (
       'django.middleware.common.CommonMiddleware',
       'django.contrib.sessions.middleware.SessionMiddleware',
       'django.middleware.csrf.CsrfViewMiddleware',
       'django.contrib.auth.middleware.AuthenticationMiddleware',
       'django.contrib.messages.middleware.MessageMiddleware',
       'django.middleware.locale.LocaleMiddleware',
       # Uncomment the next line for simple clickjacking protection:
       # 'django.middleware.clickjacking.XFrameOptionsMiddleware',
       'reversion.middleware.RevisionMiddleware',
   
       'modoboa.lib.middleware.AjaxLoginRedirect',
       'modoboa.lib.middleware.CommonExceptionCatcher',
       'modoboa.lib.middleware.ExtControlMiddleware',
   )
   

4. Add the reversion application to INSTALLED_APPS

5. Remove all modoboa’s application from INSTALLED_APPS and put them into the new MODOBOA_APPS variable like this:

   INSTALLED_APPS = (
       'django.contrib.auth',
       'django.contrib.contenttypes',
       'django.contrib.sessions',
       'django.contrib.sites',
       'django.contrib.messages',
       'django.contrib.staticfiles',
       'south',
       'reversion',
   )
   
   # A dedicated place to register Modoboa applications
   # Do not delete it.
   # Do not change the order.
   MODOBOA_APPS = (
       'modoboa',
       'modoboa.auth',
       'modoboa.admin',
       'modoboa.lib',
       'modoboa.userprefs',
   
       'modoboa.extensions.limits',
       'modoboa.extensions.postfix_autoreply',
       'modoboa.extensions.webmail',
       'modoboa.extensions.stats',
       'modoboa.extensions.amavis',
       'modoboa.extensions.sievefilters',
   )
   
   INSTALLED_APPS += MODOBOA_APPS
   

6. Set the AUTH_USER_MODEL variable like this:

   AUTH_USER_MODEL = 'admin.User'
   

7. Modify the logging configuration as follows:

   LOGGING = {
       'version': 1,
       'disable_existing_loggers': False,
       'filters': {
           'require_debug_false': {
               '()': 'django.utils.log.RequireDebugFalse'
           }
       },
       'formatters': {
           'syslog': {
               'format': '%(name)s: %(levelname)s %(message)s'
           },
       },
       'handlers': {
           'mail_admins': {
               'level': 'ERROR',
               'filters': ['require_debug_false'],
               'class': 'django.utils.log.AdminEmailHandler'
           },
           'console': {
               # logging handler that outputs log messages to terminal
               'class': 'logging.StreamHandler',
               #'level': 'DEBUG', # message level to be written to console
           },
           'syslog-auth': {
               'class': 'logging.handlers.SysLogHandler',
               'facility': SysLogHandler.LOG_AUTH,
               'formatter': 'syslog'
           },
           'modoboa': {
               'class': 'modoboa.lib.logutils.SQLHandler',
           }
       },
       'loggers': {
           'django.request': {
               'handlers': ['mail_admins'],
               'level': 'ERROR',
               'propagate': True,
           },
           'modoboa.auth': {
               'handlers': ['syslog-auth', 'modoboa'],
               'level': 'INFO',
               'propagate': False
           },
           'modoboa.admin': {
               'handlers': ['modoboa'],
               'level': 'INFO',
               'propagate': False
           }
       }
   }
   

Postfix and Dovecot configuration update

It is necessary to update the queries used to retrieve users and mailboxes:

1. Run modoboa-admin.py postfix_maps --dbtype <mysql|postgres> <tempdir> and compare the files with those that postfix currently use. Make necessary updates in light of the differences
2. Into dovecot-sql.conf, update the user_query query, refer to MySQL users or PostgreSQL users
3. Update dovecot’s configuration to activate the new quota related features

Migration issues

When running the python manage.py syncdb --migrate command, you may encounter the following issues:

1. Remove useless content types

   If the script asks you this question, just reply no.

2. South fails to migrate reversion

   Due to the admin user model change, the script 0001_initial.py may fail. Just deactivate reversion from INSTALLED_APPS and run the command again. Once done, reactivate reversion and run the command one last time.