X
    Categories Tutorials

Set up Magento 2 with Redis, Varnish and Nginx as SSL termination

In this article, we will show you how to install Magento 2 on an Ubuntu 16.04 VPS with MariaDB, PHP-FPM 7.0, Varnish as a full page cache, Nginx as SSL termination and Redis for session storage and page caching.  This guide should work on other Linux VPS systems as well but was tested and written for an Ubuntu 16.04 VPS.

Login to your VPS via SSH

ssh my_sudo_user@my_server

Update the system and install necessary packages

sudo apt-get update && sudo apt-get -y upgrade
sudo apt-get -y install curl nano git

Install MariaDB 10.0

Install the latest MariaDB 10.0 server from the official Ubuntu repositories:

sudo apt-get install -y mariadb-server

When the installation is complete, run the following command to secure your installation:

mysql_secure_installation

Next, we need to create a database for our Magento installation.

mysql -uroot -p
MariaDB [(none)]> CREATE DATABASE magento;
MariaDB [(none)]> GRANT ALL PRIVILEGES ON magento.* TO 'magento'@'localhost' IDENTIFIED BY 'my_strong_password';
MariaDB [(none)]> FLUSH PRIVILEGES;
MariaDB [(none)]> \q

Install PHP 7.0, composer and all required PHP modules

To install the latest stable version of PHP 7.0 and all necessary modules, run:

sudo apt-get -y install php-fpm php-cli php-gd php-imagick php-mysql php-mcrypt php-pear php-curl php-intl php-xsl php-zip php-mbstring

Change few default PHP settings:

sudo sed -i "s/memory_limit = .*/memory_limit = 256M/" /etc/php/7.0/fpm/php.ini
sudo sed -i "s/upload_max_filesize = .*/upload_max_filesize = 128M/" /etc/php/7.0/fpm/php.ini
sudo sed -i "s/zlib.output_compression = .*/zlib.output_compression = on/" /etc/php/7.0/fpm/php.ini
sudo sed -i "s/max_execution_time = .*/max_execution_time = 18000/" /etc/php/7.0/fpm/php.ini

Composer is a dependency manager for PHP with which you can install packages. Composer will pull in all the required libraries and dependencies you need for your project.

curl -sS https://getcomposer.org/installer | php
sudo mv composer.phar /usr/local/bin/composer

Install Magento 2 from Github

Clone the Magento repository to the ~/myMagentoSite.com directory using the following command:

sudo git clone https://github.com/magento/magento2.git /var/www/myMagentoSite.com

Get the latest stable release, at the time of the writing it’s Magento 2.1.2:

cd /var/www/myMagentoSite.com
sudo git checkout $(git describe --tags $(git rev-list --tags --max-count=1))

Run composer to install all Magento dependencies:

sudo composer install

To continue with the installation you can either use the installation wizard or the command line, in this guide we will use the latter.

sudo bin/magento setup:install \
--base-url=http://myMagentoSite.com/ \
--db-host=localhost \
--db-name=magento \
--db-user=magento \
--db-password=my_strong_password \
--admin-firstname=First  \
--admin-lastname=Last \
--admin-email=user@myMagentoSite.com \
--admin-user=admin \
--admin-password=my_strong_password123 \
--language=en_US \
--currency=USD \
--timezone=America/Chicago \
--use-rewrites=1

If the installation is successful you will see something like below:

[SUCCESS]: Magento installation complete.
[SUCCESS]: Magento Admin URI: /admin_mejj1n

Run the crontab command to create a cronjob

crontab -u www-data -e

and add the following line:

* * * * * /usr/bin/php /var/www/myMagentoSite.com/bin/magento cron:run | grep -v "Ran jobs by schedule" >> /var/www/myMagentoSite.com/var/log/magento.cron.log

Finally, set the correct permissions:

sudo chown -R www-data: /var/www/myMagentoSite.com

Install and configure Nginx

Install Nginx from the official Ubuntu repositories::

sudo apt-get -y install nginx

Create a new Nginx server block with the following content:

sudo nano /etc/nginx/sites-available/myMagentoSite.com
upstream fastcgi_backend {
  server   unix:/run/php/php7.0-fpm.sock;
}

server {
    server_name myMagentoSite.com www.myMagentoSite.com;
    listen 80;
    set $MAGE_ROOT /var/www/myMagentoSite.com;
    set $MAGE_MODE developer; # or production

    access_log /var/log/nginx/myMagentoSite.com-access.log;
    error_log /var/log/nginx/myMagentoSite.com-error.log;

    include /var/www/myMagentoSite.com/nginx.conf.sample;        
}

Activate the server block by creating a symbolic link :

sudo ln -s /etc/nginx/sites-available/myMagentoSite.com /etc/nginx/sites-enabled/myMagentoSite.com

Delete the default configuration:

sudo rm -f /etc/nginx/sites-enabled/default

Test the Nginx configuration and restart nginx:

sudo nginx -t
sudo service nginx restart

You should be now able to login to your Magento back-end by going to http://myMagentoSite.com/admin_mejj1n using the information you set when running the bin/magento setup:install .

Install and configure Varnish

Installing Varnish is as simple as running the following command:

sudo apt-get install varnish

From you Magento Admin dashboard click on the STORES link (left sidebar) -> Configuration -> ADVANCED -> System -> Full Page Cache
Unselected Use system value and from the Caching Application list, select Varnish Cache (Recommended), save the configuration, click on the Varnish Configuration link and click on the Export VCL for Varnish 4 button. The varnish.vcl file which we will use will be exported in the /var/www/myMagentoSite.com/var/ directory.

Flush the Magento cache with:

sudo php bin/magento cache:flush

Delete the /etc/varnish/default.vcl and symlink it to the exported varnish configuration.

sudo rm -f /etc/varnish/default.vcl
sudo ln -sf /var/www/myMagentoSite.com/var/varnish.vcl /etc/varnish/default.vcl

To change varnish port from 6081 to 80, we need to edit the systemd service configuration.

Create a new customexec.conf file

sudo mkdir -p /etc/systemd/system/varnish.service.d
sudo nano /etc/systemd/system/varnish.service.d/customexec.conf

paste the following:

[Service]
ExecStart=
ExecStart=/usr/sbin/varnishd -j unix,user=vcache -F -a :80 -T localhost:6082 -f /etc/varnish/default.vcl -S /etc/varnish/secret -s malloc,256m

and reload systemd units

sudo systemctl daemon-reload

Now we need to change Nginx listening port from 80 to 8080 and enable Nginx SSL termination with HTTP2, to do that open the Nginx configuration file and change it as follows:

sudo nano /etc/nginx/sites-available/myMagentoSite.com
upstream fastcgi_backend {
  server   unix:/run/php/php7.0-fpm.sock;
}

server {
    server_name myMagentoSite.com www.myMagentoSite.com;
    listen 8080;
    set $MAGE_ROOT /var/www/myMagentoSite.com;
    set $MAGE_MODE production; # or developer

    access_log /var/log/nginx/myMagentoSite.com-access.log;
    error_log /var/log/nginx/myMagentoSite.com-error.log;

    include /var/www/myMagentoSite.com/nginx.conf.sample;        
}

server {

    listen 443 ssl http2;
    server_name myMagentoSite.com www.myMagentoSite.com;

    ssl_certificate /etc/ssl/certs/ssl-cert-snakeoil.pem; # change with your SSL cert
    ssl_certificate_key /etc/ssl/private/ssl-cert-snakeoil.key; # change with your SSL key
    ssl_protocols              TLSv1 TLSv1.1 TLSv1.2;
    ssl_ciphers               'AES128+EECDH:AES128+EDH:!aNULL';
    ssl_session_cache    shared:SSL:10m;
    ssl_session_timeout 24h;
    keepalive_timeout 300s;

    location / {
        proxy_pass http://127.0.0.1;
        proxy_set_header Host $http_host;
        proxy_set_header X-Forwarded-Host $http_host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header Ssl-Offloaded "1";
        proxy_set_header      X-Forwarded-Proto https;
        proxy_set_header      X-Forwarded-Port 443;
        #proxy_hide_header X-Varnish;
        #proxy_hide_header Via;
        proxy_set_header X-Forwarded-Proto $scheme;

    }

}

If you don’t already have an SSL certificate, you can purchase a trusted SSL certificate here.

Restart Varnish and Nginx:

sudo systemctl restart nginx
sudo systemctl restart varnish

Change the base url to https and flush the cache

sudo bin/magento setup:store-config:set --base-url="https://myMagentoSite.com"
sudo php bin/magento cache:flush

If everything is setup correctly now you should be able to login to your Magento back-end by going to https://myMagentoSite.com/admin_mejj1n.

Install and configure Redis caching

Redis is a key-value in memory data store and we will use it to replace the default Magento 2 Zend_Cache_Backend_File backend cache.  Install Redis by running the following command:

apt-get install php-redis redis-server

To configure your Magento installation to use Redis for session storage open the app/etc/env.php file and change/add the following:

sudo nano /var/www/myMagentoSite.com/app/etc/env.php

change:

  'session' =>
  array (
    'save' => 'files',
  ),

with:

'session' => 
   array (
   'save' => 'redis',
   'redis' => 
      array (
	'host' => '127.0.0.1',
	'port' => '6379',
	'password' => '',
	'timeout' => '2.5',
	'persistent_identifier' => '',
	'database' => '0',
	'compression_threshold' => '2048',
	'compression_library' => 'gzip',
	'log_level' => '1',
	'max_concurrency' => '6',
	'break_after_frontend' => '5',
	'break_after_adminhtml' => '30',
	'first_lifetime' => '600',
	'bot_first_lifetime' => '60',
	'bot_lifetime' => '7200',
	'disable_locking' => '0',
	'min_lifetime' => '60',
	'max_lifetime' => '2592000'
    )
),

and to use Redis for page caching add:

'cache' =>
array(
   'frontend' =>
   array(
      'default' =>
      array(
         'backend' => 'Cm_Cache_Backend_Redis',
         'backend_options' =>
         array(
            'server' => '127.0.0.1',
            'port' => '6379'
            ),
    ),
    'page_cache' =>
    array(
      'backend' => 'Cm_Cache_Backend_Redis',
      'backend_options' =>
       array(
         'server' => '127.0.0.1',
         'port' => '6379',
         'database' => '1',
         'compress_data' => '0'
       )
    )
  )
),

Finally flush the cache again:

sudo php bin/magento cache:flush

Further Optimizations

To further optimize your Magento installation from you Magento admin dashboard:

1. Go to STORES -> Configuration -> CATALOG -> Catalog -> Use Flat Catalog Category, select Yes and click Save Config.
2. Go to STORES -> Configuration -> ADVANCED -> Developer -> JavaScript Settings and set both Merge JavaScript Files and Minify JavaScript Files to Yes and click Save Config..
3. Go to STORES -> Configuration -> ADVANCED -> Developer -> CSS Settings and set both Merge CSS Files and Minify CSS Files to Yes and click Save Config.
4. Consider using a CDN – Content Delivery Network

Do not forget to flush the cache:

sudo php bin/magento cache:flush

That’s it. You have successfully installed Magento 2 with Redis as a session storage and page caching, Varnish as a full page caching and Nginx as SSL termination on your Ubuntu 16.04 VPS. For more information about how to manage your Magento installation, please refer to the official Magento documentation.


Of course, you don’t have to do any of this if you use one of our Magento VPS Hosting services, in which case you can simply ask our expert Linux admins to setup this for you. They are available 24×7 and will take care of your request immediately.

PS. If you liked this post please share it with your friends on the social networks using the buttons on the left or simply leave a reply below. Thanks.

View Comments

  • Found few typos; 1. sudo git https://github.com/magento/magento2.git /var/www/myMagentoSite.com Missing "clone". 2. latter -> later Like several howto's, I find myself looking at HTTP ERROR 500 with a bare new Ubuntu 16.04 VPS installation.

    • Thanks for your feedback, the post has been updated. As for the 500 error, please check the nginx error log.

      • The 500 error was due improper read permissions. Even tho entire root was chowned to www-data, some of the files needed manual chmodding.

  • Hello, I have a problem when I try to "sudo systemctl restart varnish". It shows "Job for varnish.service failed. See 'systemctl status varnish.service' and 'journalctl -xn' for details." Then I use 'systemctl status varnish.service' and it shows ● varnish.service - Varnish HTTP accelerator Loaded: loaded (/lib/systemd/system/varnish.service; enabled) Drop-In: /etc/systemd/system/varnish.service.d └─customexec.conf Active: failed (Result: exit-code) since Fri 2016-12-16 20:53:29 CET; 7s ago Process: 23563 ExecStart=/usr/sbin/varnishd -j unix,user=vcache -F -a :80 -T localhost:6082 -f /etc/varnish/default.vcl -S /etc/varnish/secret -s malloc,256m (code=exited, status=1/FAILURE) Process: 23553 ExecStartPre=/usr/sbin/varnishd -C -f /etc/varnish/default.vcl (code=exited, status=0/SUCCESS) Main PID: 18167 (code=exited, status=0/SUCCESS) Dec 16 20:53:29 mywebsite varnishd[23563]: # -s persist{experimenta} Dec 16 20:53:29 mywebsite varnishd[23563]: # -s file,,, Dec 16 20:53:29 mywebsite varnishd[23563]: -S secret-file # Secret file for CLI authentication Dec 16 20:53:29 mywebsite systemd[1]: varnish.service: control process exited, code=exited status=1 Dec 16 20:53:29 mywebsite systemd[1]: Failed to start Varnish HTTP accelerator. Dec 16 20:53:29 mywebsite systemd[1]: Unit varnish.service entered failed state. Dec 16 20:53:29 mywebsite varnishd[23563]: -T address:port # Telnet listen address and port Dec 16 20:53:29 mywebsite varnishd[23563]: -t # Default TTL Dec 16 20:53:29 mywebsite varnishd[23563]: -u user # Privilege separation user id Dec 16 20:53:29 mywebsite varnishd[23563]: -V # version I followed every step on this tutorial but I used Debian 8. I'm not sure is that cause the problem. Do you have any idea how to fix get varnish work?

      • Hello, Thank you for your response. the output is "tcp 0 0 0.0.0.0:8080 0.0.0.0:* LISTEN 23697/nginx -g daem" Plus, I didn't install apache. Thank you.

      • Hello, Thank you for your response. the output is tcp 0 0 0.0.0.0:8080 0.0.0.0:* LISTEN 23697/nginx -g daem

        • Delete the /etc/systemd/system/varnish.service.d/customexec.conf file and create a new customexec.conf file with the following content:

          [Service]
          ExecStart=
          ExecStart=/usr/sbin/varnishd -a :80 -T localhost:6082 -f /etc/varnish/default.vcl -S /etc/varnish/secret -s malloc,256m 
          
          reload the deamon sudo systemctl daemon-reload and restart the service sudo systemctl restart varnish

  • Hello, Can I call other file on /var/www/myMagentoSite.com example I upload phpinfo.php in /var/www/myMagentoSite.com but I can't call http://myMagentoSite.com/phpinfo.php

    • It is possible that phpino() is a forbidden function on your server. You can call any other file to check if it is working properly.

  • Hi, As soon as I finish varnish and restart nginx / varnish with sudo systemctl restart nginx sudo systemctl restart varnish I get error 503 / Backend Fetch Failed / Varnish Cache Server When I look at my varnish.vcl that I exported from my magento2 backend and uploaded to /var/www/mysite/var, I see that .host and .port are empty. I see no instructions in your guide about this, but Is it possible that that has something to do with it? If not, do you have any idea what might cause this? I have followed your guide exactly and everything worked great up to this point. Thank you! - Nick

    • Our tutorial is tested and working without a problem. Have you made sure that you’ve followed the instructions correctly? Please try doing all of the described steps again and be more careful.

  • You're right, I exported the VCL file before saving the configuration in Magento - Varnish now works except when I enable SSL. I've uploaded my certificate and key, and when I enter the SSL information in sudo nano /etc/nginx/sites-available/myMagentoSite.com and restart nginx / varnish and flush cache, I get a ERR_TOO_MANY_REDIRECTS error. Any idea what might cause this?

  • Hi, Removing the SSL part of the nginx conf file solves that problem for now, although for some reason there's still one issue that I can't seem to fix. I think it's the same problem Pong is having a few posts back. I went through your instructions carefully and have wiped the server and reinstalled everything about 5 times to make sure I didn't miss anything, but it keeps giving me the same problem: - After NGINX is installed, i've created my nginx block, created the symlink, deleted the default conf and restarted nginx and then the magento shop works great, however when I upload something through SFTP (for example 3.zip, or a php file to /var/www/mydomain.com/) or when I add an image while adding a product it always gives me a 404 error. I can't seem to find anything in the logs about it or elsewhere online, but hopefully you're able to provide a solution for this. Thanks and sorry for all the questions!