Installing Mautic in Ubuntu VM

Generally we recommend using AWS Fargate or Kubernetes for production environments. However, initially we’ll need to do things which are easier when using “live VM”:

• multi-tenancy (shared VM & CPU & RAM, shared PHP-FPM, shared nginx, multiple server blocks)
• dig deeper into software stack
• make ad-hoc changes
• diagnose troubles, etc.

At least until we have a way to have much better visibility operational procedures.

Hardware Requirements

• Mautic hardware requirements (TL;DR: 2 vCPU, 4 GB RAM)

Installation

• https://computingforgeeks.com/install-mautic-open-source-marketing-software-on-ubuntu/

Step 1: Launch EC2 instance

Use t4g.medium for installation. 8 GB storage is fine for a start. After running, Mautic uses only 200 MiB and 2.5 GiB storage. And in Kubernetes, it was proven that 384 MiB was enough for the container. Currently we’re using t4g.micro. Note: t4g.nano proved unstable with normal REST API usage even without composer upgrade.

Security groups: make sure to have “default” (so the load balancer can access it)

Step 2: Prepare Ubuntu System Configuration

Set hostname:

sudo hostnamectl set-hostname mautic-production-sg01.soluvas.com

Enable byobu and make sure LC_ALL is set by server even when connecting remotely:

# not just for multi-screens, but you'll get useful system load metrics
byobu-enable
sudo update-locale LC_ALL=en_US.UTF-8 LANG=en_US.UTF-8

Then re-login using ssh.

Update Ubuntu packages:

sudo apt update && sudo apt -y full-upgrade

To enable VSCode to watch files, please apply the following system tweak to the server.

Troubleshooting: UnhandledPromiseRejectionWarning: Error: ENOSPC: System limit for number of file watchers reached

Solution:

echo fs.inotify.max_user_watches=524288 | sudo tee -a /etc/sysctl.conf && sudo sysctl -p

Step 3: Prepare MariaDB User & Database

Use AWS RDS MariaDB.

CREATE USER 'mautic_production'@'%' IDENTIFIED BY 'password';
CREATE DATABASE mautic_production CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
GRANT SELECT, INSERT, UPDATE, DELETE, CREATE, DROP, INDEX, ALTER, CREATE TEMPORARY TABLES, CREATE VIEW, EVENT, TRIGGER, SHOW VIEW, CREATE ROUTINE, ALTER ROUTINE, EXECUTE ON mautic_production. * TO 'mautic_production'@'%';
FLUSH PRIVILEGES;

Step 4: Install dependencies, php-fpm 7.4

Mautic 3.2 supports PHP 7.4.

sudo apt update
# composer wants zip
sudo apt install -y zip
# no php7.4-pear?
sudo apt install -y php7.4-{opcache,cli,fpm,json,mysql,zip,gd,mbstring,curl,xml,bcmath,imap,intl}

Mautic 3.1.x test suite did not work on PHP 7.4 (#8447). Trying to install Mautic 3.1 on PHP 7.4 gave the following error:

ubuntu@mautic-production-sg01:/var/www/html/mautic.lovia.life$ sudo -H -u www-data composer install
Loading composer repositories with package information
Installing dependencies (including require-dev) from lock file
Your requirements could not be resolved to an installable set of packages.

  Problem 1
    - This package requires php >=7.2.21 <7.4 but your PHP version (7.4.3) does not satisfy that requirement.

Step 5: Edit php.ini

Edit /etc/php/7.4/fpm/php.ini:

memory_limit = 512M

[Date]
# Mautic recommends setting UTC explicitly
date.timezone = UTC

sudo systemctl restart php7.4-fpm

Check php-fpm status:

sudo systemctl status php7.4-fpm

Step 6: Edit nginx sites-available

sudo apt -y install nginx

Edit /etc/nginx/sites-available/mautic.lovia.life.conf:

server {
  listen       80;
  listen       [::]:80;
  server_name  mautic.lovia.life;
  root         /var/www/html/mautic.lovia.life;
  index index.html index.htm index.php;

  location / {
    try_files $uri /index.php$is_args$args;
  }

  location ~ \.php$ {
    if ($http_x_forwarded_proto = 'https') {
      set $my_https 'on';
    }
    fastcgi_param HTTPS $my_https; # set $_SERVER['HTTPS']
    fastcgi_param SERVER_NAME $host; # set $_SERVER['SERVER_NAME']
    fastcgi_param HTTP_HOST $host; # set $_SERVER['HTTP_HOST']
    fastcgi_pass unix:/run/php/php7.4-fpm.sock;
    fastcgi_index index.php;
    fastcgi_read_timeout 240;
    fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
    include fastcgi_params;
    fastcgi_split_path_info ^(.+.php)(/.+)$;
  }

  rewrite ^themes/.*/(layouts|pages|partials)/.*.htm /index.php break;
  rewrite ^bootstrap/.* /index.php break;
  rewrite ^config/.* /index.php break;
  rewrite ^vendor/.* /index.php break;
  rewrite ^storage/cms/.* /index.php break;
  rewrite ^storage/logs/.* /index.php break;
  rewrite ^storage/framework/.* /index.php break;
  rewrite ^storage/temp/protected/.* /index.php break;
  rewrite ^storage/app/uploads/protected/.* /index.php break;
}

Enable that site:

sudo ln -sv /etc/nginx/sites-available/mautic.lovia.life.conf /etc/nginx/sites-enabled/

Reload nginx:

sudo systemctl reload nginx

Step 7: Clone Mautic files from Git and install Composer 1.x

By using Git, if later we need it, it makes it possible to use Soluvas’s own fork.

cd /var/www/html/
sudo git clone --depth 1 --branch 3.1.1 https://github.com/mautic/mautic.git mautic.lovia.life
# Allow composer, etc. to write
sudo chown -R www-data:www-data /var/www

Note: Mautic 3.x is not compatible with Composer 2.x.

Install Composer as root, but later you’ll run composer as www-data:

curl -sS https://getcomposer.org/installer -o ~/composer-setup.php
sudo php ~/composer-setup.php --1 --install-dir=/usr/local/bin --filename=composer

Run composer install:

cd /var/www/html/mautic.lovia.life
sudo -H -u www-data composer install  ## This might take some time to complete!

Step 8: Set Up AWS Application Load Balancer & CNAME

Target group:

Listener:

Then create a CNAME record to the ALB.

Step 9: Finish Mautic’s Installation via Panel

Mautic may complain that HTTPS isn’t enabled, although it’s terminated by Load Balancer.

I’ve tried to set X-Forwarded-Proto header, but Mautic 3.1 seems to ignore it.

    proxy_set_header X-Forwarded-Proto $scheme;

Step 10: Cron Jobs

List of required and optional Mautic cron jobs.

Recommendation: Configure cron jobs using single shell command, not using script file.

Edit the crontab for www-data: (why use /usr/bin/logger?)

sudo crontab -e -u www-data

# Required cron jobs
*/5 * * * * (/var/www/html/mautic.lovia.life/bin/console mautic:segments:update | /usr/bin/logger -t mautic-segments-update) && (/var/www/html/mautic.lovia.life/bin/console mautic:campaigns:rebuild | /usr/bin/logger -t mautic-campaigns-rebuild) && (/var/www/html/mautic.lovia.life/bin/console mautic:campaigns:trigger | /usr/bin/logger -t mautic-campaigns-trigger)
# Required because we're using queue
*/3 * * * * /var/www/html/mautic.lovia.life/bin/console mautic:emails:send | /usr/bin/logger -t mautic-emails-send
# Optional cron jobs
*/3 * * * * (/var/www/html/mautic.lovia.life/bin/console mautic:social:monitoring | /usr/bin/logger -t mautic-social-monitoring) && (/var/www/html/mautic.lovia.life/bin/console mautic:import | /usr/bin/logger -t mautic-import) && (/var/www/html/mautic.lovia.life/bin/console mautic:webhooks:process | /usr/bin/logger -t mautic-webhooks-process)
# Clean up old data
6 19 * * * /var/www/html/mautic.lovia.life/bin/console mautic:maintenance:cleanup --days-old=365 --no-interaction | /usr/bin/logger -t mautic-maintenance-cleanup
# MaxMind updates their database the first Tuesday of the month
1 19 * * * /var/www/html/mautic.lovia.life/bin/console mautic:iplookup:download | /usr/bin/logger -t mautic-iplookup-download
# Send scheduled reports
1 0 * * * /var/www/html/mautic.lovia.life/bin/console mautic:reports:scheduler | /usr/bin/logger -t mautic-reports-scheduler
# Send scheduled broadcast messages. Needed for SMS, emails and other messaging
11 * * * * /var/www/html/mautic.lovia.life/bin/console mautic:broadcasts:send | /usr/bin/logger -t mautic-broadcasts-send

Step 11: Change The Site URL

Settings > Configuration > System Settings > General Settings > Site URL: change it using https:// and remove the index.php part.

Troubleshooting: Mautic images (including in email builder) with URL index.php/media/images/…jpg are not showing and gives 404 Not Found (broken image).

Cause & Solution: This because the Site URL contains “index.php”. To fix it, remove the “index.php”.

Step 12: Change Email Limits

Our basic SES limits is 50k / day or 14/second, about 840 / minute.

Settings > Configuration > Email Settings > Mail Send Settings > Message limit for queue processing: 600.

Upgrading Mautic using Git & Composer

Before running composer install, clear cache especially code cache, otherwise it might crash: Type error: Too few arguments to function Mautic\LeadBundle\Model\ListModel::__construct(), 3 passed in /var/www/html/mautic.lovia.life/var/cache/prod/Conta
iner9sm07l4/appProdProjectContainer.php on line 9584 and exactly 4 expected

cd /var/www/html/mautic.lovia.life
# Pull current branch (3.2) and also fetch other (newer) branches
sudo -H -u www-data git pull
# If upgrading minor version as well: sudo -H -u www-data git checkout 3.4
# to save RAM and prevent server crash, stop php-fpm
sudo systemctl stop php7.4-fpm
sudo -H -u www-data bin/console cache:clear
sudo -H -u www-data composer install  ## This might take some time to complete!
# Execute migrations
sudo -H -u www-data php bin/console doctrine:migration:status
sudo -H -u www-data php bin/console doctrine:migration:migrate
# Restart php-fpm
sudo systemctl start php7.4-fpm

Logging

Check /var/www/html/mautic.lovia.life/var/logs/*

Troubleshooting

[2020-10-13 13:24:02] console.ERROR: Error thrown while running command “mautic:emails:send”. Message: “Call to undefined method Swift_Message::getAttachments()” {“exception”:”[object] (Error(code: 0): Call to undefined method Swift_Message::getAttachments() at /var/www/html/mautic.lovia.life/app/bundles/EmailBundle/Swiftmailer/Transport/AmazonApiTransport.php:195)”,”command”:”mautic:emails:send”,”message”:”Call to undefined method Swift_Message::getAttachments()”} []

Issues: PR #9290, #9156, #9137, #9262, forum thread

Try to monkey patch:

www-data@mautic-production-sg01:~/html/mautic.lovia.life$ git fetch https://github.com/mabumusa1/mautic.git fix_ses_queue_mode
remote: Enumerating objects: 248005, done.
remote: Counting objects: 100% (248003/248003), done.
remote: Compressing objects: 100% (53165/53165), done.
remote: Total 245365 (delta 179369), reused 243001 (delta 177064), pack-reused 0
Receiving objects: 100% (245365/245365), 86.77 MiB | 12.68 MiB/s, done.
Resolving deltas: 100% (179369/179369), completed with 1616 local objects.
From https://github.com/mabumusa1/mautic
 * branch            fix_ses_queue_mode -> FETCH_HEAD
www-data@mautic-production-sg01:~/html/mautic.lovia.life$ git cherry-pick 5512d3e
[detached HEAD 5b37aefc2] Fix SES issues
 Author: Mohammad Abu Musa <[email protected]>
 Date: Mon Oct 12 20:24:15 2020 +0300
 Committer: www-data <[email protected]>
Your name and email address were configured automatically based
on your username and hostname. Please check that they are accurate.
You can suppress this message by setting them explicitly. Run the
following command and follow the instructions in your editor to edit
your configuration file:

    git config --global --edit

After doing this, you may fix the identity used for this commit with:

    git commit --amend --reset-author

 6 files changed, 490 insertions(+), 722 deletions(-)
 create mode 100644 app/bundles/EmailBundle/Swiftmailer/Amazon/AmazonFactory.php
 create mode 100644 app/bundles/EmailBundle/Swiftmailer/Amazon/AmazonFactoryInterface.php
 rewrite app/bundles/EmailBundle/Swiftmailer/Transport/AmazonApiTransport.php (85%)
 rename app/bundles/EmailBundle/Tests/{Swiftmailer/Amazon/AmazonCallbackTest.php => Transport/AmazonApiTransportTest.php} (76%)

This seemed to have caused another error:

[2020-10-13 13:55:11] mautic.CRITICAL: Uncaught PHP Exception Symfony\Component\Debug\Exception\FatalThrowableError: "Type error: Too few arguments to function Mautic\EmailBundle\Swiftmailer\Transport\AmazonApiTransport::__construct(), ...

Workaround: Use SMTP with STARTSSL (port 587, using port 465 will crash Mautic) instead:

Error: Cannot remove “Email Bounced” status.

[2020-10-13 14:04:55] mautic.CRITICAL: Uncaught PHP Exception InvalidArgumentException: "mautic.lead.model.dnc is not a valid model key." at /var/www/html/mautic.lovia.life/app/bundles/CoreBundle/Factory/ModelFactory.php line 41 {"exception":"[object] (InvalidArgumentException(code: 0): mautic.lead.model.dnc is not a valid model key. at /var/www/html/mautic.lovia.life/app/bundles/CoreBundle/Factory/ModelFactory.php:41)"} []

Issues: PR #9261.

Solution: Switch to branch 3.1

git checkout 3.1

www-data@mautic-production-sg01:~/html/mautic.lovia.life$ git fetch origin 3.1:3.1
From https://github.com/mautic/mautic
 * [new branch]            3.1        -> 3.1
www-data@mautic-production-sg01:~/html/mautic.lovia.life$ git checkout 3.1
Previous HEAD position was 271231a539 Merge pull request #9256 from mabumusa1/change_to_3.1_stable
Switched to branch '3.1'

Using GrapesJS Email Builder

Before 4.0, you need to enable it manually.

See: https://docs.mautic.org/en/builders