Set up Magento 2 with Redis, Varnish and Nginx as SSL termination


install-magento-2-with-redis-varnish-and-nginx-as-ssl-termination

In this article, we will show you how to install Magento 2 on an Ubuntu 16.04 VPS with MariaDB, PHP-FPM 7.0, Varnish as a full page cache, Nginx as SSL termination and Redis for session storage and page caching.  This guide should work on other Linux VPS systems as well but was tested and written for an Ubuntu 16.04 VPS.

Login to your VPS via SSH

ssh my_sudo_user@my_server

Update the system and install necessary packages

sudo apt-get update && sudo apt-get -y upgrade
sudo apt-get -y install curl nano git

Install MariaDB 10.0

Install the latest MariaDB 10.0 server from the official Ubuntu repositories:

sudo apt-get install -y mariadb-server

When the installation is complete, run the following command to secure your installation:

mysql_secure_installation

Next, we need to create a database for our Magento installation.

mysql -uroot -p
MariaDB [(none)]> CREATE DATABASE magento;
MariaDB [(none)]> GRANT ALL PRIVILEGES ON magento.* TO 'magento'@'localhost' IDENTIFIED BY 'my_strong_password';
MariaDB [(none)]> FLUSH PRIVILEGES;
MariaDB [(none)]> \q

Install PHP 7.0, composer and all required PHP modules

To install the latest stable version of PHP 7.0 and all necessary modules, run:

sudo apt-get -y install php-fpm php-cli php-gd php-imagick php-mysql php-mcrypt php-pear php-curl php-intl php-xsl php-zip php-mbstring

Change few default PHP settings:

sudo sed -i "s/memory_limit = .*/memory_limit = 256M/" /etc/php/7.0/fpm/php.ini
sudo sed -i "s/upload_max_filesize = .*/upload_max_filesize = 128M/" /etc/php/7.0/fpm/php.ini
sudo sed -i "s/zlib.output_compression = .*/zlib.output_compression = on/" /etc/php/7.0/fpm/php.ini
sudo sed -i "s/max_execution_time = .*/max_execution_time = 18000/" /etc/php/7.0/fpm/php.ini

Composer is a dependency manager for PHP with which you can install packages. Composer will pull in all the required libraries and dependencies you need for your project.

curl -sS https://getcomposer.org/installer | php
sudo mv composer.phar /usr/local/bin/composer

Install Magento 2 from Github

Clone the Magento repository to the ~/myMagentoSite.com directory using the following command:

sudo git clone https://github.com/magento/magento2.git /var/www/myMagentoSite.com

Get the latest stable release, at the time of the writing it’s Magento 2.1.2:

cd /var/www/myMagentoSite.com
sudo git checkout $(git describe --tags $(git rev-list --tags --max-count=1))

Run composer to install all Magento dependencies:

sudo composer install

To continue with the installation you can either use the installation wizard or the command line, in this guide we will use the latter.

sudo bin/magento setup:install \
--base-url=http://myMagentoSite.com/ \
--db-host=localhost \
--db-name=magento \
--db-user=magento \
--db-password=my_strong_password \
--admin-firstname=First  \
--admin-lastname=Last \
--admin-email=user@myMagentoSite.com \
--admin-user=admin \
--admin-password=my_strong_password123 \
--language=en_US \
--currency=USD \
--timezone=America/Chicago \
--use-rewrites=1

If the installation is successful you will see something like below:

[SUCCESS]: Magento installation complete.
[SUCCESS]: Magento Admin URI: /admin_mejj1n

Run the crontab command to create a cronjob

crontab -u www-data -e

and add the following line:

* * * * * /usr/bin/php /var/www/myMagentoSite.com/bin/magento cron:run | grep -v "Ran jobs by schedule" >> /var/www/myMagentoSite.com/var/log/magento.cron.log

Finally, set the correct permissions:

sudo chown -R www-data: /var/www/myMagentoSite.com

Install and configure Nginx

Install Nginx from the official Ubuntu repositories::

sudo apt-get -y install nginx

Create a new Nginx server block with the following content:

sudo nano /etc/nginx/sites-available/myMagentoSite.com
upstream fastcgi_backend {
  server   unix:/run/php/php7.0-fpm.sock;
}

server {
    server_name myMagentoSite.com www.myMagentoSite.com;
    listen 80;
    set $MAGE_ROOT /var/www/myMagentoSite.com;
    set $MAGE_MODE developer; # or production

    access_log /var/log/nginx/myMagentoSite.com-access.log;
    error_log /var/log/nginx/myMagentoSite.com-error.log;

    include /var/www/myMagentoSite.com/nginx.conf.sample;        
}

Activate the server block by creating a symbolic link :

sudo ln -s /etc/nginx/sites-available/myMagentoSite.com /etc/nginx/sites-enabled/myMagentoSite.com

Delete the default configuration:

sudo rm -f /etc/nginx/sites-enabled/default

Test the Nginx configuration and restart nginx:

sudo nginx -t
sudo service nginx restart

You should be now able to login to your Magento back-end by going to http://myMagentoSite.com/admin_mejj1n using the information you set when running the bin/magento setup:install .

Install and configure Varnish

Installing Varnish is as simple as running the following command:

sudo apt-get install varnish

From you Magento Admin dashboard click on the STORES link (left sidebar) -> Configuration -> ADVANCED -> System -> Full Page Cache
Unselected Use system value and from the Caching Application list, select Varnish Cache (Recommended), save the configuration, click on the Varnish Configuration link and click on the Export VCL for Varnish 4 button. The varnish.vcl file which we will use will be exported in the /var/www/myMagentoSite.com/var/ directory.

Flush the Magento cache with:

sudo php bin/magento cache:flush

Delete the /etc/varnish/default.vcl and symlink it to the exported varnish configuration.

sudo rm -f /etc/varnish/default.vcl
sudo ln -sf /var/www/myMagentoSite.com/var/varnish.vcl /etc/varnish/default.vcl

To change varnish port from 6081 to 80, we need to edit the systemd service configuration.

Create a new customexec.conf file

sudo mkdir -p /etc/systemd/system/varnish.service.d
sudo nano /etc/systemd/system/varnish.service.d/customexec.conf

paste the following:

[Service]
ExecStart=
ExecStart=/usr/sbin/varnishd -j unix,user=vcache -F -a :80 -T localhost:6082 -f /etc/varnish/default.vcl -S /etc/varnish/secret -s malloc,256m

and reload systemd units

sudo systemctl daemon-reload

Now we need to change Nginx listening port from 80 to 8080 and enable Nginx SSL termination with HTTP2, to do that open the Nginx configuration file and change it as follows:

sudo nano /etc/nginx/sites-available/myMagentoSite.com
upstream fastcgi_backend {
  server   unix:/run/php/php7.0-fpm.sock;
}

server {
    server_name myMagentoSite.com www.myMagentoSite.com;
    listen 8080;
    set $MAGE_ROOT /var/www/myMagentoSite.com;
    set $MAGE_MODE production; # or developer

    access_log /var/log/nginx/myMagentoSite.com-access.log;
    error_log /var/log/nginx/myMagentoSite.com-error.log;

    include /var/www/myMagentoSite.com/nginx.conf.sample;        
}

server {

    listen 443 ssl http2;
    server_name myMagentoSite.com www.myMagentoSite.com;

    ssl_certificate /etc/ssl/certs/ssl-cert-snakeoil.pem; # change with your SSL cert
    ssl_certificate_key /etc/ssl/private/ssl-cert-snakeoil.key; # change with your SSL key
    ssl_protocols              TLSv1 TLSv1.1 TLSv1.2;
    ssl_ciphers               'AES128+EECDH:AES128+EDH:!aNULL';
    ssl_session_cache    shared:SSL:10m;
    ssl_session_timeout 24h;
    keepalive_timeout 300s;

    location / {
        proxy_pass http://127.0.0.1;
        proxy_set_header Host $http_host;
        proxy_set_header X-Forwarded-Host $http_host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header Ssl-Offloaded "1";
        proxy_set_header      X-Forwarded-Proto https;
        proxy_set_header      X-Forwarded-Port 443;
        #proxy_hide_header X-Varnish;
        #proxy_hide_header Via;
        proxy_set_header X-Forwarded-Proto $scheme;

    }

}

If you don’t already have an SSL certificate, you can purchase a trusted SSL certificate here.

Restart Varnish and Nginx:

sudo systemctl restart nginx
sudo systemctl restart varnish

Change the base url to https and flush the cache

sudo bin/magento setup:store-config:set --base-url="https://myMagentoSite.com"
sudo php bin/magento cache:flush

If everything is setup correctly now you should be able to login to your Magento back-end by going to https://myMagentoSite.com/admin_mejj1n.

Stuck somewhere? Get a VPS from us and we’ll do all of this for you, free of charge!

Install and configure Redis caching

Redis is a key-value in memory data store and we will use it to replace the default Magento 2 Zend_Cache_Backend_File backend cache.  Install Redis by running the following command:

apt-get install php-redis redis-server

To configure your Magento installation to use Redis for session storage open the app/etc/env.php file and change/add the following:

sudo nano /var/www/myMagentoSite.com/app/etc/env.php

change:

  'session' =>
  array (
    'save' => 'files',
  ),

with:

'session' => 
   array (
   'save' => 'redis',
   'redis' => 
      array (
	'host' => '127.0.0.1',
	'port' => '6379',
	'password' => '',
	'timeout' => '2.5',
	'persistent_identifier' => '',
	'database' => '0',
	'compression_threshold' => '2048',
	'compression_library' => 'gzip',
	'log_level' => '1',
	'max_concurrency' => '6',
	'break_after_frontend' => '5',
	'break_after_adminhtml' => '30',
	'first_lifetime' => '600',
	'bot_first_lifetime' => '60',
	'bot_lifetime' => '7200',
	'disable_locking' => '0',
	'min_lifetime' => '60',
	'max_lifetime' => '2592000'
    )
),

and to use Redis for page caching add:

'cache' =>
array(
   'frontend' =>
   array(
      'default' =>
      array(
         'backend' => 'Cm_Cache_Backend_Redis',
         'backend_options' =>
         array(
            'server' => '127.0.0.1',
            'port' => '6379'
            ),
    ),
    'page_cache' =>
    array(
      'backend' => 'Cm_Cache_Backend_Redis',
      'backend_options' =>
       array(
         'server' => '127.0.0.1',
         'port' => '6379',
         'database' => '1',
         'compress_data' => '0'
       )
    )
  )
),

Finally flush the cache again:

sudo php bin/magento cache:flush

Further Optimizations

To further optimize your Magento installation from you Magento admin dashboard:

1. Go to STORES -> Configuration -> CATALOG -> Catalog -> Use Flat Catalog Category, select Yes and click Save Config.
2. Go to STORES -> Configuration -> ADVANCED -> Developer -> JavaScript Settings and set both Merge JavaScript Files and Minify JavaScript Files to Yes and click Save Config..
3. Go to STORES -> Configuration -> ADVANCED -> Developer -> CSS Settings and set both Merge CSS Files and Minify CSS Files to Yes and click Save Config.
4. Consider using a CDN – Content Delivery Network

Do not forget to flush the cache:

sudo php bin/magento cache:flush

That’s it. You have successfully installed Magento 2 with Redis as a session storage and page caching, Varnish as a full page caching and Nginx as SSL termination on your Ubuntu 16.04 VPS. For more information about how to manage your Magento installation, please refer to the official Magento documentation.


Of course, you don’t have to do any of this if you use one of our Magento VPS Hosting services, in which case you can simply ask our expert Linux admins to setup this for you. They are available 24×7 and will take care of your request immediately.

PS. If you liked this post please share it with your friends on the social networks using the buttons on the left or simply leave a reply below. Thanks.

Tutorials
How to install Open Source Social Network on a CentOS 7 VPS
Tutorials
Install ExpressionEngine on Debian 8
Tutorials
Install OSSEC agent on Ubuntu 14.04
  • Betro Hakala

    Found few typos;
    1. sudo git https://github.com/magento/magento2.git /var/www/myMagentoSite.com
    Missing “clone”.

    2. latter -> later

    Like several howto’s, I find myself looking at HTTP ERROR 500 with a bare new Ubuntu 16.04 VPS installation.


    • admin

      adminadmin

      Author Reply

      Thanks for your feedback, the post has been updated.

      As for the 500 error, please check the nginx error log.


      • Betro Hakala

        The 500 error was due improper read permissions. Even tho entire root was chowned to www-data, some of the files needed manual chmodding.


  • Lippil

    LippilLippil

    Author Reply

    Hello, I have a problem when I try to “sudo systemctl restart varnish”. It shows “Job for varnish.service failed. See ‘systemctl status varnish.service’ and ‘journalctl -xn’ for details.”
    Then I use ‘systemctl status varnish.service’ and it shows

    ● varnish.service – Varnish HTTP accelerator
    Loaded: loaded (/lib/systemd/system/varnish.service; enabled)
    Drop-In: /etc/systemd/system/varnish.service.d
    └─customexec.conf
    Active: failed (Result: exit-code) since Fri 2016-12-16 20:53:29 CET; 7s ago
    Process: 23563 ExecStart=/usr/sbin/varnishd -j unix,user=vcache -F -a :80 -T localhost:6082 -f /etc/varnish/default.vcl -S /etc/varnish/secret -s malloc,256m (code=exited, status=1/FAILURE)
    Process: 23553 ExecStartPre=/usr/sbin/varnishd -C -f /etc/varnish/default.vcl (code=exited, status=0/SUCCESS)
    Main PID: 18167 (code=exited, status=0/SUCCESS)

    Dec 16 20:53:29 mywebsite varnishd[23563]: # -s persist{experimenta}
    Dec 16 20:53:29 mywebsite varnishd[23563]: # -s file,,,
    Dec 16 20:53:29 mywebsite varnishd[23563]: -S secret-file # Secret file for CLI authentication
    Dec 16 20:53:29 mywebsite systemd[1]: varnish.service: control process exited, code=exited status=1
    Dec 16 20:53:29 mywebsite systemd[1]: Failed to start Varnish HTTP accelerator.
    Dec 16 20:53:29 mywebsite systemd[1]: Unit varnish.service entered failed state.
    Dec 16 20:53:29 mywebsite varnishd[23563]: -T address:port # Telnet listen address and port
    Dec 16 20:53:29 mywebsite varnishd[23563]: -t # Default TTL
    Dec 16 20:53:29 mywebsite varnishd[23563]: -u user # Privilege separation user id
    Dec 16 20:53:29 mywebsite varnishd[23563]: -V # version

    I followed every step on this tutorial but I used Debian 8. I’m not sure is that cause the problem.
    Do you have any idea how to fix get varnish work?


    • admin

      adminadmin

      Author Reply

      Could you please run:

      netstat -anlp | grep 80

      and provide us with the output?


      • Lippil

        LippilLippil

        Author Reply

        Hello,
        Thank you for your response.
        the output is
        tcp 0 0 0.0.0.0:8080 0.0.0.0:* LISTEN 23697/nginx -g daem


        • admin

          adminadmin

          Author Reply

          Delete the /etc/systemd/system/varnish.service.d/customexec.conf file and create a new customexec.conf file with the following content:

          [Service]
          ExecStart=
          ExecStart=/usr/sbin/varnishd -a :80 -T localhost:6082 -f /etc/varnish/default.vcl -S /etc/varnish/secret -s malloc,256m 
          

          reload the deamon sudo systemctl daemon-reload and restart the service sudo systemctl restart varnish


          • Lippil

            LippilLippil

            Author

            Thank you very much for your helping. it works now. :)


      • Lippil

        LippilLippil

        Author Reply

        Hello,

        Thank you for your response. the output is
        “tcp 0 0 0.0.0.0:8080 0.0.0.0:* LISTEN 23697/nginx -g daem”

        Plus, I didn’t install apache.

        Thank you.


  • Pong

    PongPong

    Author Reply

    Hello,
    Can I call other file on /var/www/myMagentoSite.com example I upload phpinfo.php in /var/www/myMagentoSite.com but I can’t call http://myMagentoSite.com/phpinfo.php


    • admin

      adminadmin

      Author Reply

      It is possible that phpino() is a forbidden function on your server. You can call any other file to check if it is working properly.


  • Nick

    NickNick

    Author Reply

    Hi,

    As soon as I finish varnish and restart nginx / varnish with

    sudo systemctl restart nginx
    sudo systemctl restart varnish

    I get error 503 / Backend Fetch Failed / Varnish Cache Server

    When I look at my varnish.vcl that I exported from my magento2 backend and uploaded to /var/www/mysite/var, I see that .host and .port are empty. I see no instructions in your guide about this, but Is it possible that that has something to do with it? If not, do you have any idea what might cause this? I have followed your guide exactly and everything worked great up to this point. Thank you!

    – Nick


    • admin

      adminadmin

      Author Reply

      Our tutorial is tested and working without a problem. Have you made sure that you’ve followed the instructions correctly?
      Please try doing all of the described steps again and be more careful.


  • Nick

    NickNick

    Author Reply

    You’re right, I exported the VCL file before saving the configuration in Magento – Varnish now works except when I enable SSL.
    I’ve uploaded my certificate and key, and when I enter the SSL information in sudo nano /etc/nginx/sites-available/myMagentoSite.com and restart nginx / varnish and flush cache, I get a ERR_TOO_MANY_REDIRECTS error.
    Any idea what might cause this?


    • admin

      adminadmin

      Author Reply

      Did you check the log files for warnings and errors?


  • Nick

    NickNick

    Author Reply

    Hi,

    Removing the SSL part of the nginx conf file solves that problem for now, although for some reason there’s still one issue that I can’t seem to fix. I think it’s the same problem Pong is having a few posts back. I went through your instructions carefully and have wiped the server and reinstalled everything about 5 times to make sure I didn’t miss anything, but it keeps giving me the same problem:

    – After NGINX is installed, i’ve created my nginx block, created the symlink, deleted the default conf and restarted nginx and then the magento shop works great, however when I upload something through SFTP (for example 3.zip, or a php file to /var/www/mydomain.com/) or when I add an image while adding a product it always gives me a 404 error. I can’t seem to find anything in the logs about it or elsewhere online, but hopefully you’re able to provide a solution for this. Thanks and sorry for all the questions!


  • Rajiv

    RajivRajiv

    Author Reply

    Hi after complete varnish installation error coming

    Error 503 Backend fetch failed

    Backend fetch failed

    Guru Meditation:

    XID: 32807

    Pls help


    • admin

      adminadmin

      Author Reply

      Please follow the tutorial instructions closely.


      • Rajiv

        RajivRajiv

        Author Reply

        double chk again but the issue yet not solve

        Error 503 Backend fetch failed

        Backend fetch failed

        Guru Meditation:

        XID: 32934


        • admin

          adminadmin

          Author Reply

          Please check the log files and see if there are some errors about this. Also, provide us with the output of ‘ps aux’ command.


          • Rajiv

            RajivRajiv

            Author

            same issue

            USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
            root 1 0.0 0.2 119780 5976 ? Ss Apr05 0:09 /sbin/init
            root 2 0.0 0.0 0 0 ? S Apr05 0:00 [kthreadd]
            root 3 0.0 0.0 0 0 ? S Apr05 0:03 [ksoftirqd/0]
            root 5 0.0 0.0 0 0 ? S< Apr05 0:00 [kworker/0:0H]
            root 7 0.0 0.0 0 0 ? S Apr05 0:03 [rcu_sched]
            root 8 0.0 0.0 0 0 ? S Apr05 0:00 [rcu_bh]
            root 9 0.0 0.0 0 0 ? S Apr05 0:00 [migration/0]
            root 10 0.0 0.0 0 0 ? S Apr05 0:00 [watchdog/0]
            root 11 0.0 0.0 0 0 ? S Apr05 0:00 [kdevtmpfs]
            root 12 0.0 0.0 0 0 ? S< Apr05 0:00 [netns]
            root 13 0.0 0.0 0 0 ? S< Apr05 0:00 [perf]
            root 14 0.0 0.0 0 0 ? S Apr05 0:00 [xenwatch]
            root 15 0.0 0.0 0 0 ? S Apr05 0:00 [xenbus]
            root 17 0.0 0.0 0 0 ? S Apr05 0:00 [khungtaskd]
            root 18 0.0 0.0 0 0 ? S< Apr05 0:00 [writeback]
            root 19 0.0 0.0 0 0 ? SN Apr05 0:00 [ksmd]
            root 20 0.0 0.0 0 0 ? SN Apr05 0:00 [khugepaged]
            root 21 0.0 0.0 0 0 ? S< Apr05 0:00 [crypto]
            root 22 0.0 0.0 0 0 ? S< Apr05 0:00 [kintegrityd]
            root 23 0.0 0.0 0 0 ? S< Apr05 0:00 [bioset]
            root 24 0.0 0.0 0 0 ? S< Apr05 0:00 [kblockd]
            root 25 0.0 0.0 0 0 ? S< Apr05 0:00 [ata_sff]
            root 26 0.0 0.0 0 0 ? S< Apr05 0:00 [md]
            root 27 0.0 0.0 0 0 ? S< Apr05 0:00 [devfreq_wq]
            root 30 0.0 0.0 0 0 ? S Apr05 0:00 [kswapd0]
            root 31 0.0 0.0 0 0 ? S< Apr05 0:00 [vmstat]
            root 32 0.0 0.0 0 0 ? S Apr05 0:00 [fsnotify_mark]
            root 33 0.0 0.0 0 0 ? S Apr05 0:00 [ecryptfs-kthrea]
            root 49 0.0 0.0 0 0 ? S< Apr05 0:00 [kthrotld]
            root 50 0.0 0.0 0 0 ? S< Apr05 0:00 [acpi_thermal_pm]
            root 51 0.0 0.0 0 0 ? S< Apr05 0:00 [bioset]
            root 52 0.0 0.0 0 0 ? S< Apr05 0:00 [bioset]
            root 53 0.0 0.0 0 0 ? S< Apr05 0:00 [bioset]
            root 54 0.0 0.0 0 0 ? S< Apr05 0:00 [bioset]
            root 55 0.0 0.0 0 0 ? S< Apr05 0:00 [bioset]
            root 56 0.0 0.0 0 0 ? S< Apr05 0:00 [bioset]
            root 57 0.0 0.0 0 0 ? S< Apr05 0:00 [bioset]
            root 58 0.0 0.0 0 0 ? S< Apr05 0:00 [bioset]
            root 59 0.0 0.0 0 0 ? S< Apr05 0:00 [bioset]
            root 60 0.0 0.0 0 0 ? S< Apr05 0:00 [bioset]
            root 61 0.0 0.0 0 0 ? S< Apr05 0:00 [bioset]
            root 62 0.0 0.0 0 0 ? S< Apr05 0:00 [bioset]
            root 63 0.0 0.0 0 0 ? S< Apr05 0:00 [bioset]
            root 64 0.0 0.0 0 0 ? S< Apr05 0:00 [bioset]
            root 65 0.0 0.0 0 0 ? S< Apr05 0:00 [bioset]
            root 66 0.0 0.0 0 0 ? S< Apr05 0:00 [bioset]
            root 67 0.0 0.0 0 0 ? S< Apr05 0:00 [bioset]
            root 68 0.0 0.0 0 0 ? S< Apr05 0:00 [bioset]
            root 69 0.0 0.0 0 0 ? S< Apr05 0:00 [bioset]
            root 70 0.0 0.0 0 0 ? S< Apr05 0:00 [bioset]
            root 71 0.0 0.0 0 0 ? S< Apr05 0:00 [bioset]
            root 72 0.0 0.0 0 0 ? S< Apr05 0:00 [bioset]
            root 73 0.0 0.0 0 0 ? S< Apr05 0:00 [bioset]
            root 74 0.0 0.0 0 0 ? S< Apr05 0:00 [bioset]
            root 75 0.0 0.0 0 0 ? S Apr05 0:00 [scsi_eh_0]
            root 76 0.0 0.0 0 0 ? S< Apr05 0:00 [scsi_tmf_0]
            root 77 0.0 0.0 0 0 ? S Apr05 0:00 [scsi_eh_1]
            root 78 0.0 0.0 0 0 ? S< Apr05 0:00 [scsi_tmf_1]
            root 84 0.0 0.0 0 0 ? S< Apr05 0:00 [ipv6_addrconf]
            root 85 0.0 0.0 0 0 ? S< Apr05 0:00 [bioset]
            root 98 0.0 0.0 0 0 ? S< Apr05 0:00 [deferwq]
            root 99 0.0 0.0 0 0 ? S< Apr05 0:00 [charger_manager]
            root 139 0.0 0.0 0 0 ? S< Apr05 0:00 [bioset]
            root 140 0.0 0.0 0 0 ? S< Apr05 0:00 [bioset]
            root 141 0.0 0.0 0 0 ? S< Apr05 0:00 [bioset]
            root 142 0.0 0.0 0 0 ? S< Apr05 0:00 [bioset]
            root 143 0.0 0.0 0 0 ? S< Apr05 0:00 [bioset]
            root 144 0.0 0.0 0 0 ? S< Apr05 0:00 [bioset]
            root 145 0.0 0.0 0 0 ? S< Apr05 0:00 [bioset]
            root 146 0.0 0.0 0 0 ? S< Apr05 0:00 [bioset]
            root 147 0.0 0.0 0 0 ? S< Apr05 0:00 [kpsmoused]
            root 264 0.0 0.0 0 0 ? S< Apr05 0:00 [raid5wq]
            root 299 0.0 0.0 0 0 ? S< Apr05 0:00 [bioset]
            root 323 0.0 0.0 0 0 ? D Apr05 0:02 [jbd2/xvda1-8]
            root 324 0.0 0.0 0 0 ? S< Apr05 0:00 [ext4-rsv-conver]
            root 371 0.0 0.0 0 0 ? S< Apr05 0:00 [kworker/0:1H]
            root 378 0.0 0.0 0 0 ? S< Apr05 0:00 [iscsi_eh]
            root 382 0.0 0.0 0 0 ? S< Apr05 0:00 [ib_addr]
            root 386 0.0 0.0 0 0 ? S< Apr05 0:00 [ib_mcast]
            root 388 0.0 0.0 0 0 ? S< Apr05 0:00 [ib_nl_sa_wq]
            root 393 0.0 0.0 0 0 ? S< Apr05 0:00 [ib_cm]
            root 396 0.0 0.0 0 0 ? S< Apr05 0:00 [iw_cm_wq]
            root 400 0.0 0.0 0 0 ? S< Apr05 0:00 [rdma_cm]
            root 412 0.0 0.3 38972 7548 ? Ss Apr05 0:02 /lib/systemd/systemd-journald
            root 416 0.0 0.0 0 0 ? S Apr05 0:00 [kauditd]
            root 441 0.0 0.0 102968 1492 ? Ss Apr05 0:00 /sbin/lvmetad -f
            root 476 0.0 0.1 42528 3692 ? Ss Apr05 0:00 /lib/systemd/systemd-udevd
            systemd+ 529 0.0 0.1 100324 2332 ? Ssl Apr05 0:00 /lib/systemd/systemd-timesyncd
            root 927 0.0 0.1 16120 2900 ? Ss Apr05 0:00 /sbin/dhclient -1 -v -pf /run/dhclient.eth0.pid -lf /var/lib/dhcp/dhclient.eth0.leases -I -df /var/lib/
            root 1083 0.0 0.0 5224 156 ? Ss Apr05 0:01 /sbin/iscsid
            root 1084 0.0 0.1 5724 3520 ? S<Ls Apr05 0:07 /sbin/iscsid
            root 1086 0.0 0.0 4400 1324 ? Ss Apr05 0:00 /usr/sbin/acpid
            root 1087 0.0 0.1 28620 2968 ? Ss Apr05 0:00 /lib/systemd/systemd-logind
            root 1090 0.0 0.3 272944 7800 ? Ssl Apr05 0:01 /usr/lib/accountsservice/accounts-daemon
            syslog 1096 0.0 0.1 260632 3608 ? Ssl Apr05 0:00 /usr/sbin/rsyslogd -n
            root 1109 0.0 0.1 26068 2452 ? Ss Apr05 0:00 /usr/sbin/cron -f
            message+ 1122 0.0 0.1 42904 3816 ? Ss Apr05 0:02 /usr/bin/dbus-daemon –system –address=systemd: –nofork –nopidfile –systemd-activation
            daemon 1135 0.0 0.1 26044 2092 ? Ss Apr05 0:00 /usr/sbin/atd -f
            root 1137 0.0 0.2 636488 6036 ? Ssl Apr05 0:00 /usr/bin/lxcfs /var/lib/lxcfs/
            root 1190 0.0 0.3 279324 7144 ? Ssl Apr05 0:00 /usr/lib/policykit-1/polkitd –no-debug
            root 1270 0.0 0.0 12844 1612 ttyS0 Ss+ Apr05 0:00 /sbin/agetty –keep-baud 115200 38400 9600 ttyS0 vt220
            root 1273 0.0 0.0 14660 1704 tty1 Ss+ Apr05 0:00 /sbin/agetty –noclear tty1 linux
            root 1326 0.0 0.2 65520 5824 ? Ss Apr05 0:00 /usr/sbin/sshd -D
            varnish+ 1709 0.1 0.1 99404 2244 ? Ss Apr05 1:43 /usr/bin/varnishlog -a -w /var/log/varnish/varnish.log
            varnish+ 1994 0.1 0.1 99416 2312 ? Ss Apr05 1:43 /usr/bin/varnishncsa -a -w /var/log/varnish/varnishncsa.log
            www-data 2059 0.0 3.3 490040 69440 ? S Apr05 0:03 php-fpm: pool www
            root 6972 0.0 0.0 0 0 ? S< Apr05 0:00 [xfsalloc]
            root 6973 0.0 0.0 0 0 ? S< Apr05 0:00 [xfs_mru_cache]
            root 7229 0.0 1.0 266532 20732 ? Ssl Apr05 0:00 /usr/lib/snapd/snapd
            root 9817 0.0 0.0 13376 164 ? Ss Apr05 0:00 /sbin/mdadm –monitor –pid-file /run/mdadm/monitor.pid –daemonise –scan –syslog
            root 15636 0.0 0.9 290448 19676 ? Sl Apr05 0:14 /usr/bin/python3 /usr/bin/fail2ban-server -s /var/run/fail2ban/fail2ban.sock -p /var/run/fail2ban/fail2
            root 17509 0.0 0.1 18232 2204 ? S Apr05 0:00 /bin/bash /usr/bin/mysqld_safe
            mysql 17653 0.0 7.9 629404 162276 ? Sl Apr05 0:46 /usr/sbin/mysqld –basedir=/usr –datadir=/var/lib/mysql –plugin-dir=/usr/lib/mysql/plugin –user=mysq
            root 17654 0.0 0.0 23180 1344 ? S Apr05 0:00 logger -t mysqld -p daemon error
            ubuntu 17921 0.0 0.2 45248 4676 ? Ss 20:27 0:00 /lib/systemd/systemd –user
            ubuntu 17924 0.0 0.0 61304 2032 ? S 20:27 0:00 (sd-pam)
            root 18549 0.0 0.3 95368 6668 ? Ss 21:33 0:00 sshd: ubuntu [priv]
            ubuntu 18603 0.0 0.1 95368 3312 ? S 21:33 0:00 sshd: ubuntu@notty
            ubuntu 18604 0.0 0.0 12884 1948 ? Ss 21:33 0:00 /usr/lib/openssh/sftp-server
            root 19076 0.0 0.0 135628 1752 ? Ss 22:16 0:00 nginx: master process /usr/sbin/nginx -g daemon on; master_process on;
            www-data 19077 0.0 0.3 135956 7440 ? S 22:16 0:00 nginx: worker process
            root 20345 0.0 0.0 0 0 ? S 22:30 0:00 [kworker/u30:0]
            root 20371 0.0 0.3 95368 6656 ? Ss 22:33 0:00 sshd: ubuntu [priv]
            ubuntu 20431 0.0 0.1 95368 3308 ? S 22:34 0:00 sshd: ubuntu@notty
            ubuntu 20432 0.0 0.0 12884 1936 ? Ss 22:34 0:00 /usr/lib/openssh/sftp-server
            root 20531 0.0 0.0 0 0 ? S 22:40 0:00 [kworker/0:0]
            root 20583 0.0 0.0 0 0 ? S 22:47 0:00 [kworker/0:1]
            root 20601 0.0 0.3 95372 6768 ? Ss 22:49 0:00 sshd: ubuntu [priv]
            ubuntu 20634 0.0 0.1 95372 4080 ? S 22:49 0:00 sshd: ubuntu@pts/1
            ubuntu 20642 0.0 0.2 21480 5220 pts/1 Ss 22:49 0:00 -bash
            root 20956 0.0 0.0 0 0 ? S 22:57 0:00 [kworker/u30:1]
            vcache 21084 0.0 0.3 125044 7808 ? Ss 23:08 0:00 /usr/sbin/varnishd -j unix,user=vcache -F -a :80 -T localhost:6082 -f /etc/varnish/default.vcl -S /etc/
            vcache 21098 0.0 4.7 274096 96952 ? Sl 23:08 0:00 /usr/sbin/varnishd -j unix,user=vcache -F -a :80 -T localhost:6082 -f /etc/varnish/default.vcl -S /etc/
            root 21538 0.0 0.2 66864 6052 ? Ss 23:16 0:00 sshd: [accepted]
            sshd 21539 0.0 0.1 66864 3292 ? S 23:16 0:00 sshd: [net]
            root 21540 0.0 0.1 55756 4036 pts/1 S+ 23:16 0:00 sudo ps aux
            root 21541 0.0 0.1 36084 3256 pts/1 R+ 23:16 0:00 ps aux
            root 29181 0.0 1.7 400108 35740 ? Ss Apr05 0:02 php-fpm: master process (/etc/php/7.0/fpm/php-fpm.conf)
            www-data 29185 0.0 2.9 483752 60236 ? S Apr05 0:06 php-fpm: pool www
            www-data 29891 0.0 3.2 488180 66716 ? S Apr05 0:04 php-fpm: pool www


          • admin

            adminadmin

            Author

            Please check your Varnish configuration and log files.


  • moe

    moemoe

    Author Reply

    hi,

    thank you for the great article really helpful,

    i have problem i configure everything like your post, but still varnish is not working , my website load perfect but when i check varnish through isvarnishworking.uk i got that varnish not working

    root@instance-1:~# netstat -anlp | grep 80
    tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN 10325/varnishd
    tcp 0 0 0.0.0.0:8080 0.0.0.0:* LISTEN 10310/nginx -g daem
    tcp 0 0 10.132.0.2:55638 169.254.169.254:80 ESTABLISHED 1793/python3
    tcp 0 0 10.132.0.2:55632 169.254.169.254:80 CLOSE_WAIT 1793/python3
    tcp 0 0 10.132.0.2:55628 169.254.169.254:80 CLOSE_WAIT 1781/python3
    tcp 0 0 10.132.0.2:55636 169.254.169.254:80 ESTABLISHED 1778/python3
    tcp 0 0 10.132.0.2:55634 169.254.169.254:80 ESTABLISHED 1781/python3
    tcp 0 0 10.132.0.2:55630 169.254.169.254:80 CLOSE_WAIT 1778/python3
    tcp6 0 0 :::80 :::* LISTEN 10325/varnishd
    udp6 0 0 fe80::4001:aff:fe84:123 :::* 1683/ntpd


    • admin

      adminadmin

      Author Reply

      To verify whether Varnish is working or not you can use the varnishlog command.


  • Moe

    MoeMoe

    Author Reply

    it is possible to install memcahce with the installation above along with varnish & Redis ?


    • admin

      adminadmin

      Author Reply

      All the software you mentioned can co-exist just fine on the same server.

      However, note that you don’t really need Memcached. If your intention is to use it for storing Magento PHP sessions – you can do the same just fine using Redis.


      • moe

        moemoe

        Author Reply

        thank you so much


  • Kevin

    KevinKevin

    Author Reply

    Do you have any idea how further to configure Nginx and Varnish without using any other third proxies (as hitch or HAproxy) for supporting the letsencrypt certbot to install SSL? – webroot doesn’t work with your tutorial, it shows (Failed authorization procedure. domain.com (http-01): urn:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from domain.com/.well-known/acme-challenge/ etc.)


    • admin

      adminadmin

      Author Reply

      You should be able to install letsencrypt without a problem.

      Things you want to check:

      Was the webroot path you provided correct? To test this, you can create a file manually under domain.com/.well-known/acme-challenge/test, put some random content in there, and verify that when you browse to domain.com/.well-known/acme-challenge/test, you get that content back.
      Is there a .htaccess rule (or something similar) that could be interfering with that request, and prevent the file from being served?


    • Cotiga

      CotigaCotiga

      Author Reply

      upstream fastcgi_backend {
      server unix:/run/php/php7.0-fpm.sock;
      }

      server {
      server_name myMagentoSite.com;
      listen 80;
      set $MAGE_ROOT /var/www/myMagentoSite.com;
      set $MAGE_MODE developer; # or production

      access_log /var/log/nginx/myMagentoSite.com-access.log;
      error_log /var/log/nginx/myMagentoSite.com-error.log;

      # ADD THIS
      location ^~ /.well-known {
      alias /var/www/myMagentoSite.com/.well-known/;
      auth_basic off;
      allow all;
      }

      include /var/www/myMagentoSite.com/nginx.conf.sample;
      }


      • Kevin

        KevinKevin

        Author Reply

        Cotiga with your suggestion it really works!! i have got the letsencrypt certificates !! I was trying to do the same with the 2 other directories (one for installing new magento2 theme) and one more but it doesn’t work for them!! Nginx shows 404 Not found.


  • Kevin

    KevinKevin

    Author Reply

    Admin & Cotiga thank you for your reply. To be honest i am new to Varnish & Nginx yet so maybe i am missing something from the configuration!!
    Well i have problem with any other directory under the webroot which i am creating such as /.well-known or /themesetup (for a magento theme installation) etc.. I am getting 404 error always!! Has nothing to do with credentials and ownership while 777 and magentouser:www-data even applied to those. (i.e https://myMagentoSite.com/.well-known/acme-challenge/test.txt or https://myMagentoSite.com/themesetup/ )

    ——————————————-The .htaccess file at the / has : ————————————————–

    ExpiresDefault “access plus 1 year”
    ExpiresByType text/html A0
    ExpiresByType text/plain A0

    RedirectMatch 403 /\.git

    order allow,deny
    deny from all

    order allow,deny
    deny from all

    order allow,deny
    deny from all

    order allow,deny
    deny from all

    order allow,deny
    deny from all

    order allow,deny
    deny from all

    order allow,deny
    deny from all

    order allow,deny
    deny from all

    order allow,deny
    deny from all

    order allow,deny
    deny from all

    order allow,deny
    deny from all

    order allow,deny
    deny from all

    order allow,deny
    deny from all

    order allow,deny
    deny from all

    order allow,deny
    deny from all

    order allow,deny
    deny from all

    order allow,deny
    deny from all

    order allow,deny
    deny from all

    ErrorDocument 404 /pub/errors/404.php
    ErrorDocument 403 /pub/errors/404.php

    Header set X-UA-Compatible “IE=edge”
    <FilesMatch "\.(appcache|atom|bbaw|bmp|crx|css|cur|eot|f4[abpv]|flv|geojson|gif|htc|ico|jpe?g|js|json(l$
    Header unset X-UA-Compatible

    ——————————-The .htaccess.sample at the / has : ——————————————–

    ############################################
    ## overrides deployment configuration mode value
    ## use command bin/magento deploy:mode:set to switch modes

    # SetEnv MAGE_MODE developer

    ############################################
    ## uncomment these lines for CGI mode
    ## make sure to specify the correct cgi php binary file name
    ## it might be /cgi-bin/php-cgi

    # Action php5-cgi /cgi-bin/php5-cgi
    # AddHandler php5-cgi .php

    ############################################
    ## GoDaddy specific options

    # Options -MultiViews

    ## you might also need to add this line to php.ini
    ## cgi.fix_pathinfo = 1
    ## if it still doesn’t work, rename php.ini to php5.ini

    ############################################
    ## this line is specific for 1and1 hosting

    #AddType x-mapp-php5 .php
    #AddHandler x-mapp-php5 .php

    ############################################
    ## default index file

    DirectoryIndex index.php

    ############################################
    ## adjust memory limit

    php_value memory_limit 768M
    php_value max_execution_time 18000

    ############################################
    ## disable automatic session start
    ## before autoload was initialized

    php_flag session.auto_start off

    ############################################
    ## enable resulting html compression

    #php_flag zlib.output_compression on

    ###########################################
    ## disable user agent verification to not break multiple image upload

    php_flag suhosin.session.cryptua off

    ############################################
    ## adjust memory limit

    php_value memory_limit 768M
    php_value max_execution_time 18000

    ############################################
    ## disable automatic session start
    ## before autoload was initialized

    php_flag session.auto_start off

    ############################################
    ## enable resulting html compression

    #php_flag zlib.output_compression on

    ###########################################
    ## disable user agent verification to not break multiple image upload

    php_flag suhosin.session.cryptua off

    ###########################################
    ## disable POST processing to not break multiple image upload
    php_flag suhosin.session.cryptua off

    ############################################
    ## adjust memory limit

    php_value memory_limit 768M
    php_value max_execution_time 18000

    ############################################
    ## disable automatic session start
    ## before autoload was initialized

    php_flag session.auto_start off

    ############################################
    ## enable resulting html compression

    #php_flag zlib.output_compression on

    ###########################################
    ## disable user agent verification to not break multiple image upload

    php_flag suhosin.session.cryptua off

    ###########################################
    ## disable POST processing to not break multiple image upload

    SecFilterEngine Off
    SecFilterScanPOST Off

    ############################################
    ## enable apache served files compression
    ## http://developer.yahoo.com/performance/rules.html#gzip

    # Insert filter on all content
    ###SetOutputFilter DEFLATE
    # Insert filter on selected content types only
    #AddOutputFilterByType DEFLATE text/html text/plain text/xml text/css text/javascript application/javas$

    # Netscape 4.x has some problems…
    #BrowserMatch ^Mozilla/4 gzip-only-text/html

    # Netscape 4.06-4.08 have some more problems
    #BrowserMatch ^Mozilla/4\.0[678] no-gzip

    # MSIE masquerades as Netscape, but it is fine
    #BrowserMatch \bMSIE !no-gzip !gzip-only-text/html

    # Don’t compress images
    #SetEnvIfNoCase Request_URI \.(?:gif|jpe?g|png)$ no-gzip dont-vary

    # Make sure proxies don’t deliver the wrong content
    #Header append Vary User-Agent env=!dont-vary

    ############################################
    ## make HTTPS env vars available for CGI mode

    SSLOptions StdEnvVars

    ############################################
    ## workaround for Apache 2.4.6 CentOS build when working via ProxyPassMatch with HHVM (or any other)
    ## Please, set it on virtual host configuration level

    ## SetEnvIf Authorization “(.*)” HTTP_AUTHORIZATION=$1
    ############################################

    ############################################
    ## enable rewrites

    Options +FollowSymLinks
    RewriteEngine on

    ############################################
    ## you can put here your magento root folder
    ## path relative to web root

    #RewriteBase /magento/

    ############################################
    ## workaround for HTTP authorization
    ## in CGI environment

    RewriteRule .* – [E=HTTP_AUTHORIZATION:%{HTTP:Authorization}]

    ############################################
    ## TRACE and TRACK HTTP methods disabled to prevent XSS attacks

    RewriteCond %{REQUEST_METHOD} ^TRAC[EK]
    RewriteRule .* – [L,R=405]

    ############################################
    ## redirect for mobile user agents

    #RewriteCond %{REQUEST_URI} !^/mobiledirectoryhere/.*$
    #RewriteCond %{HTTP_USER_AGENT} “android|blackberry|ipad|iphone|ipod|iemobile|opera mobile|palmos|webos$
    #RewriteRule ^(.*)$ /mobiledirectoryhere/ [L,R=302]

    ############################################
    ## never rewrite for existing files, directories and links

    RewriteCond %{REQUEST_FILENAME} !-f
    RewriteCond %{REQUEST_FILENAME} !-d
    RewriteCond %{REQUEST_FILENAME} !-l

    ############################################
    ## rewrite everything else to index.php

    RewriteRule .* index.php [L]

    ############################################
    ## Prevent character encoding issues from server overrides
    ## If you still have problems, use the second line instead

    AddDefaultCharset Off
    #AddDefaultCharset UTF-8
    AddType ‘text/html; charset=UTF-8’ html

    ############################################
    ## Add default Expires header
    ## http://developer.yahoo.com/performance/rules.html#expires

    ExpiresDefault “access plus 1 year”
    ExpiresByType text/html A0
    ExpiresByType text/plain A0

    ###########################################
    ## Deny access to root files to hide sensitive application information
    RedirectMatch 403 /\.git

    order allow,deny
    deny from all

    order allow,deny
    deny from all

    order allow,deny
    deny from all

    order allow,deny
    deny from all

    order allow,deny
    deny from all

    order allow,deny
    deny from all

    order allow,deny
    deny from all

    order allow,deny
    deny from all

    order allow,deny
    deny from all

    order allow,deny
    deny from all

    order allow,deny
    deny from all

    order allow,deny
    deny from all

    order allow,deny
    deny from all

    order allow,deny
    deny from all

    order allow,deny
    deny from all

    order allow,deny
    deny from all

    order allow,deny
    deny from all

    order allow,deny
    deny from all

    # For 404s and 403s that aren’t handled by the application, show plain 404 response
    ErrorDocument 404 /pub/errors/404.php
    ErrorDocument 403 /pub/errors/404.php

    ################################
    ## If running in cluster environment, uncomment this
    ## http://developer.yahoo.com/performance/rules.html#etags

    #FileETag none

    # ######################################################################
    # # INTERNET EXPLORER #
    # ######################################################################

    # ———————————————————————-
    # | Document modes |
    # ———————————————————————-

    # Force Internet Explorer 8/9/10 to render pages in the highest mode
    # available in the various cases when it may not.
    #
    # https://hsivonen.fi/doctype/#ie8
    #
    # (!) Starting with Internet Explorer 11, document modes are deprecated.
    # If your business still relies on older web apps and services that were
    # designed for older versions of Internet Explorer, you might want to
    # consider enabling `Enterprise Mode` throughout your company.
    #
    # https://msdn.microsoft.com/en-us/library/ie/bg182625.aspx#docmode
    # http://blogs.msdn.com/b/ie/archive/2014/04/02/stay-up-to-date-with-enterprise-mode-for-internet-explorer-$

    Header set X-UA-Compatible “IE=edge”

    # `mod_headers` cannot match based on the content-type, however,
    # the `X-UA-Compatible` response header should be send only for
    # HTML documents and not for the other resources.

    <FilesMatch "\.(appcache|atom|bbaw|bmp|crx|css|cur|eot|f4[abpv]|flv|geojson|gif|htc|ico|jpe?g|js|json(l$
    Header unset X-UA-Compatible

    —————————–The Varnish default.vcl has : ——————————————-

    vcl 4.0;

    import std;
    # The minimal Varnish version is 4.0
    # For SSL offloading, pass the following header in your proxy server or load balancer: ‘X-Forwarded-Proto: $

    backend default {
    .host = “localhost”;
    .port = “8080”;
    }

    acl purge {
    “localhost”;
    }

    sub vcl_recv {
    if (req.method == “PURGE”) {
    if (client.ip !~ purge) {
    return (synth(405, “Method not allowed”));
    }
    if (!req.http.X-Magento-Tags-Pattern) {
    return (synth(400, “X-Magento-Tags-Pattern header required”));
    }
    ban(“obj.http.X-Magento-Tags ~ ” + req.http.X-Magento-Tags-Pattern);
    return (synth(200, “Purged”));
    }

    if (req.method != “GET” &&
    req.method != “HEAD” &&
    req.method != “PUT” &&
    req.method != “POST” &&
    req.method != “TRACE” &&
    req.method != “OPTIONS” &&
    req.method != “DELETE”) {
    /* Non-RFC2616 or CONNECT which is weird. */
    return (pipe);
    }

    # We only deal with GET and HEAD by default
    if (req.method != “GET” && req.method != “HEAD”) {
    return (pass);
    }

    # Bypass shopping cart, checkout and search requests
    if (req.url ~ “/checkout” || req.url ~ “/catalogsearch”) {
    return (pass);
    }

    if (req.url ~ “^/ssl-seal”) {
    return (pass);
    }

    #Bypass the themesetup
    if (req.url ~ “^/themesetup”) {
    return (pass);
    }

    # Bypass .wel-known & acme-challenge
    if (req.url ~ “/.well-known” || req.url ~ “/.well-known/acme-challenge”) {
    return (pass);
    }

    # normalize url in case of leading HTTP scheme and domain
    set req.url = regsub(req.url, “^http[s]?://”, “”);

    # collect all cookies
    std.collect(req.http.Cookie);

    # Compression filter. See https://www.varnish-cache.org/trac/wiki/FAQ/Compression
    if (req.http.Accept-Encoding) {
    if (req.url ~ “\.(jpg|jpeg|png|gif|gz|tgz|bz2|tbz|mp3|ogg|swf|flv)$”) {
    # No point in compressing these
    unset req.http.Accept-Encoding;
    } elsif (req.http.Accept-Encoding ~ “gzip”) {
    set req.http.Accept-Encoding = “gzip”;
    } elsif (req.http.Accept-Encoding ~ “deflate” && req.http.user-agent !~ “MSIE”) {
    set req.http.Accept-Encoding = “deflate”;
    } else {
    # unkown algorithm
    unset req.http.Accept-Encoding;
    }
    }

    # Remove Google gclid parameters to minimize the cache objects
    set req.url = regsuball(req.url,”\?gclid=[^&]+$”,””); # strips when QS = “?gclid=AAA”
    set req.url = regsuball(req.url,”\?gclid=[^&]+&”,”?”); # strips when QS = “?gclid=AAA&foo=bar”
    set req.url = regsuball(req.url,”&gclid=[^&]+”,””); # strips when QS = “?foo=bar&gclid=AAA” or QS = “?f$

    # static files are always cacheable. remove SSL flag and cookie
    if (req.url ~ “^/(pub/)?(media|static)/.*\.(ico|css|js|jpg|jpeg|png|gif|tiff|bmp|mp3|ogg|svg|swf|wo$
    unset req.http.Https;
    unset req.http.X-Forwarded-Proto;
    unset req.http.Cookie;
    }

    return (hash);
    }

    sub vcl_hash {
    if (req.http.cookie ~ “X-Magento-Vary=”) {
    hash_data(regsub(req.http.cookie, “^.*?X-Magento-Vary=([^;]+);*.*$”, “\1”));
    }

    # For multi site configurations to not cache each other’s content
    if (req.http.host) {
    hash_data(req.http.host);
    } else {
    hash_data(server.ip);
    }

    # To make sure http users don’t see ssl warning
    if (req.http.X-Forwarded-Proto) {
    hash_data(req.http.X-Forwarded-Proto);
    }

    }

    sub vcl_backend_response {
    if (beresp.http.content-type ~ “text”) {
    set beresp.do_esi = true;
    }

    if (bereq.url ~ “\.js$” || beresp.http.content-type ~ “text”) {
    set beresp.do_gzip = true;
    }

    # cache only successfully responses and 404s
    if (beresp.status != 200 && beresp.status != 404) {
    set beresp.ttl = 0s;
    set beresp.uncacheable = true;
    return (deliver);
    } elsif (beresp.http.Cache-Control ~ “private”) {
    set beresp.uncacheable = true;
    set beresp.ttl = 86400s;
    return (deliver);
    }

    if (beresp.http.X-Magento-Debug) {
    set beresp.http.X-Magento-Cache-Control = beresp.http.Cache-Control;
    }

    # validate if we need to cache it and prevent from setting cookie
    # images, css and js are cacheable by default so we have to remove cookie also
    if (beresp.ttl > 0s && (bereq.method == “GET” || bereq.method == “HEAD”)) {
    unset beresp.http.set-cookie;
    if (bereq.url !~ “\.(ico|css|js|jpg|jpeg|png|gif|tiff|bmp|gz|tgz|bz2|tbz|mp3|ogg|svg|swf|woff|woff2$
    set beresp.http.Pragma = “no-cache”;
    set beresp.http.Expires = “-1”;
    set beresp.http.Cache-Control = “no-store, no-cache, must-revalidate, max-age=0”;
    set beresp.grace = 1m;
    }
    }

    # If page is not cacheable then bypass varnish for 2 minutes as Hit-For-Pass
    if (beresp.ttl <= 0s ||
    beresp.http.Surrogate-control ~ "no-store" ||
    (!beresp.http.Surrogate-Control && beresp.http.Vary == "*")) {
    # Mark as Hit-For-Pass for the next 2 minutes
    set beresp.ttl = 120s;
    set beresp.uncacheable = true;
    }
    return (deliver);
    }

    sub vcl_deliver {
    if (resp.http.X-Magento-Debug) {
    if (resp.http.x-varnish ~ " ") {
    set resp.http.X-Magento-Cache-Debug = "HIT";
    } else {
    set resp.http.X-Magento-Cache-Debug = "MISS";
    }
    } else {
    unset resp.http.Age;
    }

    unset resp.http.X-Magento-Debug;
    unset resp.http.X-Magento-Tags;
    unset resp.http.X-Powered-By;
    unset resp.http.Server;
    unset resp.http.X-Varnish;
    unset resp.http.Via;
    unset resp.http.Link;
    }

    ————————————-The nginx.conf.sample has : ————————————————–

    ## Example configuration:
    # upstream fastcgi_backend {
    # # use tcp connection
    # # server 127.0.0.1:9000;
    # # or socket
    # server unix:/var/run/php5-fpm.sock;
    # }
    # server {
    # listen 80;
    # server_name mage.dev;
    # set $MAGE_ROOT /var/www/magento2;
    # include /vagrant/magento2/nginx.conf.sample;
    # }
    #
    ## Optional override of deployment mode. We recommend you use the
    ## command 'bin/magento deploy:mode:set' to switch modes instead.
    ##
    ## set $MAGE_MODE default; # or production or developer
    ##
    ## If you set MAGE_MODE in server config, you must pass the variable into the
    ## PHP entry point blocks, which are indicated below. You can pass
    ## it in using:
    ##
    ## fastcgi_param MAGE_MODE $MAGE_MODE;
    ##
    ## In production mode, you should uncomment the 'expires' directive in the /static/ location block

    root $MAGE_ROOT/pub;

    index index.php;
    autoindex off;
    charset UTF-8;
    error_page 404 403 = /errors/404.php;
    #add_header "X-UA-Compatible" "IE=Edge";

    # PHP entry point for setup application
    location ~* ^/setup($|/) {
    root $MAGE_ROOT;
    location ~ ^/setup/index.php {
    fastcgi_pass fastcgi_backend;
    fastcgi_index index.php;
    fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
    include fastcgi_params;
    }

    location ~ ^/setup/(?!pub/). {
    deny all;
    }

    location ~ ^/setup/pub/ {
    add_header X-Frame-Options "SAMEORIGIN";
    }
    }

    # PHP entry point for update application
    location ~* ^/update($|/) {
    root $MAGE_ROOT;

    location ~ ^/update/index.php {
    fastcgi_split_path_info ^(/update/index.php)(/.+)$;
    fastcgi_pass fastcgi_backend;
    fastcgi_index index.php;
    fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
    fastcgi_param PATH_INFO $fastcgi_path_info;
    include fastcgi_params;
    }

    # Deny everything but index.php
    location ~ ^/update/(?!pub/). {
    deny all;
    }

    location ~ ^/update/pub/ {
    add_header X-Frame-Options "SAMEORIGIN";
    }
    }

    location / {
    try_files $uri $uri/ /index.php?$args;
    }

    location /pub/ {
    location ~ ^/pub/media/(downloadable|customer|import|theme_customization/.*\.xml) {
    deny all;
    }
    alias $MAGE_ROOT/pub/;
    add_header X-Frame-Options "SAMEORIGIN";
    }
    location /static/ {
    # Uncomment the following line in production mode
    # expires max;

    # Remove signature of the static files that is used to overcome the browser cache
    location ~ ^/static/version {
    rewrite ^/static/(version\d*/)?(.*)$ /static/$2 last;
    }

    location ~* \.(ico|jpg|jpeg|png|gif|svg|js|css|swf|eot|ttf|otf|woff|woff2)$ {
    add_header Cache-Control "public";
    add_header X-Frame-Options "SAMEORIGIN";
    expires +1y;

    if (!-f $request_filename) {
    rewrite ^/static/(version\d*/)?(.*)$ /static.php?resource=$2 last;
    }
    }
    location ~* \.(zip|gz|gzip|bz2|csv|xml)$ {
    add_header Cache-Control "no-store";
    add_header X-Frame-Options "SAMEORIGIN";
    expires off;

    if (!-f $request_filename) {
    rewrite ^/static/(version\d*/)?(.*)$ /static.php?resource=$2 last;
    }
    }
    if (!-f $request_filename) {
    rewrite ^/static/(version\d*/)?(.*)$ /static.php?resource=$2 last;
    }
    add_header X-Frame-Options "SAMEORIGIN";
    }

    location /media/ {
    try_files $uri $uri/ /get.php?$args;

    location ~ ^/media/theme_customization/.*\.xml {
    deny all;
    }

    location ~* \.(ico|jpg|jpeg|png|gif|svg|js|css|swf|eot|ttf|otf|woff|woff2)$ {
    add_header Cache-Control "public";
    add_header X-Frame-Options "SAMEORIGIN";
    expires +1y;
    try_files $uri $uri/ /get.php?$args;
    }
    location ~* \.(zip|gz|gzip|bz2|csv|xml)$ {
    add_header Cache-Control "no-store";
    add_header X-Frame-Options "SAMEORIGIN";
    expires off;
    try_files $uri $uri/ /get.php?$args;
    }
    add_header X-Frame-Options "SAMEORIGIN";
    }

    location /media/customer/ {
    deny all;
    }

    location /media/downloadable/ {
    deny all;
    }

    location /media/import/ {
    deny all;
    }

    # PHP entry point for main application
    location ~ (index|get|static|report|404|503)\.php$ {
    try_files $uri =404;
    fastcgi_pass fastcgi_backend;
    fastcgi_buffers 1024 4k;

    fastcgi_read_timeout 600s;
    fastcgi_connect_timeout 600s;

    fastcgi_index index.php;
    fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
    include fastcgi_params;
    }

    gzip on;
    gzip_disable "msie6";

    gzip_comp_level 6;
    gzip_min_length 1100;
    gzip_buffers 16 8k;
    gzip_proxied any;
    gzip_types
    text/plain
    text/css
    text/js
    text/xml
    text/javascript
    application/javascript
    application/x-javascript
    application/json
    application/xml
    application/xml+rss
    image/svg+xml;
    gzip_vary on;

    # Banned locations (only reached if the earlier PHP entry point regexes don't match)
    location ~* (\.php$|\.htaccess$|\.git) {
    deny all;
    }

    ——————————— the nginx myMagentoSite.com conf at sites-available has : ——————————————————-
    upstream fastcgi_backend {
    server unix:/run/php/php7.0-fpm.sock;
    }

    server {
    server_name myMagentoSite.com http://www.myMagentoSite.com;
    listen 8080;

    location ~ /.well-known {
    allow all;
    }

    set $MAGE_ROOT /var/www/myMagentoSite.com;
    set $MAGE_MODE production; # or developer

    access_log /var/log/nginx/myMagentoSIte.com-access.log;
    error_log /var/log/nginx/myMagentoSite.com-error.log;

    include /var/www/myMagentoSite.com/nginx.conf.sample;

    }

    server {

    listen 443 ssl http2;
    listen [::]:443 ssl http2;
    server_name myMagentoSite.com http://www.myMagentoSite.com;

    ssl_certificate /etc/nginx/ssl/ssl-bundle.crt; # change with your SSL cert
    ssl_certificate_key /etc/nginx/ssl/*.greekbox.ru.key; # change with your SSL key
    ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
    ssl_prefer_server_ciphers on;
    ssl_ciphers "EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH";
    ssl_ecdh_curve secp384r1;
    ssl_session_cache shared:SSL:10m;
    ssl_session_tickets off;
    ssl_stapling on;
    ssl_stapling_verify on;
    ssl_trusted_certificate /etc/nginx/ssl/trustchain.crt;
    resolver 8.8.8.8 8.8.4.4 valid=300s;
    resolver_timeout 5s;
    # Disable preloading HSTS for now. You can use the commented out header line that includes
    # the "preload" directive if you understand the implications.
    #add_header Strict-Transport-Security "max-age=63072000; includeSubdomains; preload";
    add_header Strict-Transport-Security "max-age=63072000; includeSubdomains";
    add_header X-Frame-Options DENY;
    add_header X-Content-Type-Options nosniff;

    ssl_dhparam /etc/ssl/certs/dhparam.pem;

    keepalive_timeout 300s;

    location ~ /ssl-seal {
    allow all;
    }
    location ~ /themesetup {
    allow all;
    }
    location / {
    proxy_pass http://127.0.0.1;
    proxy_set_header Host $http_host;
    proxy_set_header X-Forwarded-Host $http_host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header Ssl-Offloaded "1";
    proxy_set_header X-Forwarded-Proto https;
    proxy_set_header X-Forwarded-Port 443;
    #proxy_hide_header X-Varnish;
    #proxy_hide_header Via;
    #proxy_set_header X-Forwarded-Proto $scheme;

    }
    }

    ————————————————————————————————————————————–

    As far as i understand all the http requests forwarded to https , it means that the varnish must decide what too do with them, and as far as varnish can't find them at his cache send them back to nginx and Nginx asks php to execute them? Means that every time i need a resposne form a directry which is not belonging to Magento, must create an exception (PASS) to Varnish and an excception (ALLOW ALL) to NGINX ? true?
    I am not sure that i am not misising something esle….(


  • moe

    moemoe

    Author Reply

    i have a problem in the backend, web setup wizard missing from my backend i don’t know where is the problem do you think because we change nginx default port to 8080 or problem in github magento files sudo git clone https://github.com/magento/magento2.git

    i spend last 3 days trying to figure out what the problem but i never get a clear solutions ,

    please let me know how you solve it ?!


    • admin

      adminadmin

      Author Reply

      Nginx should be set on port 8080, so that shouldn’t be a problem if you have Varnish properly configured.

      Generally, if there is a problem with the application or the server configuration you should check the log files as they will help you to identify the problem.

      Thanks.


  • Bhumish

    BhumishBhumish

    Author Reply

    Hi,

    getting

    Error 503 Backend fetch failed

    Backend fetch failed

    Guru Meditation:

    XID: 32777

    Varnish cache server

    Please help


    • admin

      adminadmin

      Author Reply

      Our tutorial is tested and working without a problem. Have you made sure that you’ve followed the instructions correctly?
      Please try doing all of the described steps again and be more careful.


  • Eros

    ErosEros

    Author Reply

    Well done sir!!

    Works like a charm!
    No errors, no bugs! perfect instalation! my magento is flying righ now!

    Thanks a lot!


  • Steve

    SteveSteve

    Author Reply

    Hi!

    I’m getting “The requested PHP extension ext-soap * is missing from your system. Install or enable PHP’s soap extension” – with Ubuntu 16.0.4, after the ‘sudo composer install step’.

    I’ve worked through the guide three times now, even wiping our VPS instance and starting from scratch – same error each time.

    Any suggestions would be most welcome.

    Thanks,

    Steve


    • admin

      adminadmin

      Author Reply

      Hi Steve,

      You can install the extension using the apt-get install php-soap command.


  • Kevin

    KevinKevin

    Author Reply

    With this configuration it is not possible to get the real client-ip , so modules as geoip and fraudlabpro dotsn’t work properly! they always shows 127.0.0.1 as a client’s ip while this ip is using varnish and passing to nginx! Any idea how to solve it?
    Best Regards


    • admin

      adminadmin

      Author Reply

      Hi Kevin,

      You should be able to achieve this by modifying the ‘X-Forwarded-For’ header set by Varnish.


  • Kevin

    KevinKevin

    Author Reply

    Can you please be more specific? i was trying few times based on suggestions i ‘ve found on internet, bit everytime i was getting 502 bad getway after that! I have read that only the first time the headers passing the real-ip , if we need to get the real ip everytime , then it has some pipe() commands, which i am not sure i can understand what exactly they are doing, but in each try i was getting 502 error. Also i have read that except Varnish in order to get the right logs at nginx with real-ip it needs some extra configuration! So whil ei am not very familiar with it i would appreciate any further help.


    • admin

      adminadmin

      Author Reply

      To preserve the client IP address try adding the following lines in your Varnish configuration:

      sub vcl_recv {
        unset req.http.X-Forwarded-For;
        set req.http.X-Forwarded-For = client.ip;
      }

      Then restart Varnish and watch the log files.


  • Kevin

    KevinKevin

    Author Reply

    I have added it but it doesn’t work! i am still getting 127.0.0.1 at Magento backend for all clients! Probably something more must be added to to nginx??

    Best Regards


  • Clement

    ClementClement

    Author Reply

    Hi,

    Thank you for your post it’s very great.

    I have a problem with my configuration of nginx.
    Nginx doesn’t work

    `–# nginx -t
    nginx: [emerg] duplicate location “/” in /etc/nginx/sites-enabled/magento2:36
    nginx: configuration file /etc/nginx/nginx.conf test failed

    See my file of nginx (/etc/nginx/sites-enabled/magento2) :

    upstream fastcgi_backend {
    server unix:/run/php/php7.0-fpm.sock;
    }

    server {
    server_name xxxx;
    listen 8080;
    set $MAGE_ROOT /var/www/myMagentoSite.com;
    set $MAGE_MODE production; # or developer

    access_log /var/log/nginx/myMagentoSite.com-access.log;
    error_log /var/log/nginx/myMagentoSite.com-error.log;

    include /var/www/myMagentoSite.com/nginx.conf.sample;
    }

    server {

    listen 443 ssl http2;
    server_name xxxx;

    ssl_certificate /etc/letsencrypt/live/xxxx/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/xxxx/privkey.pem;
    ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
    ssl_ciphers ‘AES128+EECDH:AES128+EDH:!aNULL’;
    ssl_session_cache shared:SSL:10m;
    ssl_session_timeout 24h;
    keepalive_timeout 300s;

    location / {
    proxy_pass http://127.0.0.1;
    proxy_set_header Host $http_host;
    proxy_set_header X-Forwarded-Host $http_host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header Ssl-Offloaded “1”;
    proxy_set_header X-Forwarded-Proto https;
    proxy_set_header X-Forwarded-Port 443;
    #proxy_hide_header X-Varnish;
    #proxy_hide_header Via;
    proxy_set_header X-Forwarded-Proto $scheme;

    }

    }

    When I remove he part of “upstream fastcgi_backend” and “include /var/www/magento2/nginx.conf.sample” nginx work.
    “duplicate location /” What is it ?

    Thank you for your help

    Regards,

    Clément.


  • Kevin

    KevinKevin

    Author Reply

    i insert this snippet at sub vcl_recv :

    if (req.restarts == 0) {
    if (req.http.x-forwarded-for) {
    set req.http.X-Forwarded-For = req.http.X-Forwarded-For + “, ” + client.ip;
    } else {
    set req.http.X-Forwarded-For = client.ip;
    }
    }

    and i have these directives at nginx/sites-available/mydomain.com
    location / {
    proxy_pass http://127.0.0.1;
    proxy_set_header Host $http_host;
    proxy_set_header X-Forwarded-Host $http_host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header Ssl-Offloaded “1”;
    proxy_set_header X-Forwarded-Proto https;
    proxy_set_header X-Forwarded-Port 443;
    #proxy_hide_header X-Varnish;
    #proxy_hide_header Via;
    #proxy_set_header X-Forwarded-Proto $scheme;

    }

    and finally this is what i am getting with Varnishlog command :

    * <> 229756
    – Begin req 229755 rxreq
    – Timestamp Start: 1502220624.492312 0.000000 0.000000
    – Timestamp Req: 1502220624.492312 0.000000 0.000000
    – ReqStart 127.0.0.1 56092
    – ReqMethod GET
    – ReqURL /aromatizatori/aerozolnye-aromati.html?p=3&saturation_rating=52
    – ReqProtocol HTTP/1.0
    – ReqHeader Host: mydomain.com
    – ReqHeader X-Forwarded-Host: mydomain.com
    – ReqHeader X-Real-IP: 66.249.76.78
    – ReqHeader X-Forwarded-For: 66.249.76.78
    – ReqHeader Ssl-Offloaded: 1
    – ReqHeader X-Forwarded-Proto: https
    – ReqHeader X-Forwarded-Port: 443
    – ReqHeader Connection: close
    – ReqHeader Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
    – ReqHeader From: googlebot(at)googlebot.com
    – ReqHeader User-Agent: Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
    – ReqHeader Accept-Encoding: gzip,deflate,br
    – ReqUnset X-Forwarded-For: 66.249.76.78
    – ReqHeader X-Forwarded-For: 66.249.76.78, 127.0.0.1
    – VCL_call RECV
    – ReqUnset X-Forwarded-For: 66.249.76.78, 127.0.0.1
    – ReqHeader X-Forwarded-For: 66.249.76.78, 127.0.0.1, 127.0.0.1
    – ReqURL /aromatizatori/aerozolnye-aromati.html?p=3&saturation_rating=52
    – ReqUnset Accept-Encoding: gzip,deflate,br
    – ReqHeader Accept-Encoding: gzip
    – ReqURL /aromatizatori/aerozolnye-aromati.html?p=3&saturation_rating=52
    – ReqURL /aromatizatori/aerozolnye-aromati.html?p=3&saturation_rating=52
    – ReqURL /aromatizatori/aerozolnye-aromati.html?p=3&saturation_rating=52
    – VCL_return hash
    – VCL_call HASH
    – VCL_return lookup
    – VCL_call MISS
    – VCL_return fetch
    – Link bereq 229757 fetch
    – Timestamp Fetch: 1502220624.995656 0.503343 0.503343
    – RespProtocol HTTP/1.1
    – RespStatus 200
    – RespReason OK
    – RespHeader Server: nginx/1.10.3 (Ubuntu)
    – RespHeader Date: Tue, 08 Aug 2017 19:30:24 GMT
    – RespHeader Content-Type: text/html; charset=UTF-8
    – RespHeader X-Magento-Tags: store,cms_block,catalog_category_127,catalog_category,catalog_category_product_127,catalog_product_441,catalog_product,catalog_product_452,catalog_product_457,catalog_product_461,catalog_product_481,catalog_product_499,cms_block_fastest-g
    – RespHeader X-Content-Type-Options: nosniff
    – RespHeader X-XSS-Protection: 1; mode=block
    – RespHeader X-Frame-Options: SAMEORIGIN
    – RespHeader Vary: Accept-Encoding
    – RespHeader Pragma: no-cache
    – RespHeader Expires: -1
    – RespHeader Cache-Control: no-store, no-cache, must-revalidate, max-age=0
    – RespHeader Content-Encoding: gzip
    – RespHeader X-Varnish: 229756
    – RespHeader Age: 0
    – RespHeader Via: 1.1 varnish-v4
    – VCL_call DELIVER
    – RespUnset Age: 0
    – RespUnset X-Magento-Tags: store,cms_block,catalog_category_127,catalog_category,catalog_category_product_127,catalog_product_441,catalog_product,catalog_product_452,catalog_product_457,catalog_product_461,catalog_product_481,catalog_product_499,cms_block_fastest-g
    – RespUnset Server: nginx/1.10.3 (Ubuntu)
    – RespUnset X-Varnish: 229756
    – RespUnset Via: 1.1 varnish-v4
    – VCL_return deliver
    – Timestamp Process: 1502220624.995700 0.503387 0.000044
    – RespHeader Accept-Ranges: bytes
    – RespHeader Content-Length: 30547
    – Debug “RES_MODE 2”
    – RespHeader Connection: close
    – Timestamp Resp: 1502220624.995744 0.503432 0.000044
    – ReqAcct 499 0 499 396 30547 30943
    – End

    And this is what i am getting form Varnishncsa command :

    127.0.0.1 – – [08/Aug/2017:22:34:09 +0300] “GET http://mydomain.com/gretseskaja-konservatsija/tomatnaja-gruppa/tomaty-sushenye-v-masle-monastyrskie-delphi-340g.html HTTP/1.0″ 200 26286 “https://mydomain.com/gretseskaja-konservatsija/tomatnaja-gruppa.html” “Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:54.0) Gecko/20100101 Firefox/54.0”
    127.0.0.1 – – [08/Aug/2017:22:34:10 +0300] “GET http://mydomain.com/static/version1502219420/frontend/mage/calendar.css HTTP/1.0″ 304 0 “https://mydomain.com/gretseskaja-konservatsija/tomatnaja-gruppa/tomaty-sushenye-v-masle-monastyrskie-delphi-340g.html” “Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:54.0) Gecko/20100101 Firefox/54.0”

    In anyway Magento always shows 127.0.0.1 to client’s ip at order view!
    What i am missing yet?
    Best Regards


  • Kevin

    KevinKevin

    Author Reply

    And finally the nginx logs are :

    root@mydomain:/var/log/nginx# cat access.log
    81.19.78.134 – – [08/Aug/2017:22:48:57 +0300] “GET /media/email/logo/default/logo_2.png HTTP/1.0” 200 30463 “https://mail.rambler.ru/” “RamblerMail/6.0 (incompatible; ImageProxy/6.0)”
    66.249.76.80 – – [08/Aug/2017:22:49:01 +0300] “GET /gretsesk-olivkovoe-maslo.html?oil_packing_material=34&oliveoil_acidity=-&volume=8 HTTP/1.1” 200 29120 “-” “Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)”
    66.249.76.78 – – [08/Aug/2017:22:49:20 +0300] “GET /aromatizatori/aerozolnye-aromati.html?p=5&saturation_rating=53 HTTP/1.1” 200 30671 “-” “Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)”
    root@gmydoamin.com:/var/log/nginx# cat mydomain.com-access.log
    127.0.0.1 – – [08/Aug/2017:22:48:57 +0300] “GET /media/email/logo/default/ogo_2.png HTTP/1.1” 200 30463 “https://mail.rambler.ru/” “RamblerMail/6.0 (incompatible; ImageProxy/6.0)”
    127.0.0.1 – – [08/Aug/2017:22:49:01 +0300] “GET /gretsesk-olivkovoe-maslo.html?oil_packing_material=34&oliveoil_acidity=-&volume=8 HTTP/1.1” 200 29110 “-” “Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)”
    127.0.0.1 – – [08/Aug/2017:22:49:20 +0300] “GET /aromatizatori/aerozolnye-aromati.html?p=5&saturation_rating=53 HTTP/1.1” 200 30669 “-” “Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)”

    As it seems the general nginx access log shows the real ips but the domains nginx log shows 127.0.0.1 !!!
    Any help would be very valuable!!
    Best Regards


    • admin

      adminadmin

      Author Reply

      Try using the Nginx realIP module – http://nginx.org/en/docs/http/ngx_http_realip_module.html
      Add these lines in your virtual host file:

      set_real_ip_from   127.0.0.1;
      real_ip_header      X-Forwarded-For;
      

      • Kevin

        KevinKevin

        Author Reply

        this is the nginx build , which is already compiled –with-http_realip_module option!

        nginx/1.10.3 (Ubuntu)
        built with OpenSSL 1.0.2g 1 Mar 2016
        TLS SNI support enabled
        configure arguments: –with-cc-opt=’-g -O2 -fPIE -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2′ –with-ld-opt=’-Wl,-Bsymbolic-functions -fPIE -pie -Wl,-z,relro -Wl,-z,now’ –prefix=/usr/share/nginx –conf-path=/etc/nginx/nginx.conf –http-log-path=/var/log/nginx/access.log –error-log-path=/var/log/nginx/error.log –lock-path=/var/lock/nginx.lock –pid-path=/run/nginx.pid –http-client-body-temp-path=/var/lib/nginx/body –http-fastcgi-temp-path=/var/lib/nginx/fastcgi –http-proxy-temp-path=/var/lib/nginx/proxy –http-scgi-temp-path=/var/lib/nginx/scgi –http-uwsgi-temp-path=/var/lib/nginx/uwsgi –with-debug –with-pcre-jit –with-ipv6 –with-http_ssl_module –with-http_stub_status_module –with-http_realip_module –with-http_auth_request_module –with-http_addition_module –with-http_dav_module –with-http_geoip_module –with-http_gunzip_module –with-http_gzip_static_module –with-http_image_filter_module –with-http_v2_module –with-http_sub_module –with-http_xslt_module –with-stream –with-stream_ssl_module –with-mail –with-mail_ssl_module –with-threads

        and at nginx.conf exists rows :

        http {
        set_real_ip_from 127.0.0.1; #Put the Ip of your varnish/proxy here
        real_ip_header X-Forwarded-For; #Put the Header that your varnish/proxy set
        ……}


        • admin

          adminadmin

          Author Reply

          Remove the lines starting with error_log and access_log from the nginx 8080 server block and add them to the 443 server block.


  • Mike

    MikeMike

    Author Reply

    Tutorial is erroneous.

    “An error has happened during application run. See exception log for details.”


    • admin

      adminadmin

      Author Reply

      Please check the Magento exception log for more information about the problem. Thanks.


 
Free Migration to RoseHosting. Award Winning Linux VPS Hosting with Fully Managed Support. Coupon RHBLOG50 for 50% offOrder Now
+