Magento2 – Performance optimization, tips, improvements and hacks

Anže Harej07.05.2018eCommerce

Why speed matters

One of the main factors for a good conversion rate and low cart abandonment is page speed. A rule of thumb: if a page takes more than 3 seconds to load, it will encounter a loss in traffic, resulting in a high abandonment rate. Page speed is also a SEO ranking factor.

KISSmetrics made a cool infographic about loading time, which you can check here: https://blog.kissmetrics.com, while we’ll just leave you with the important numbers you need to keep in mind:

- More than 80% of users expect a website to load within 3 seconds.
- 40% of users leave the page if the load time is longer than 3 seconds.
- An average user is willing to wait max. 2.078 seconds for a page to load.
- Around 47% of users expect a page to load in 2 seconds or less.
- If the page loads between 1 and 2 seconds, visitor numbers rise by 2%.
- A page speed increase of 1 second will probably bring you about a 7% rise in conversion.
- 1 second of load lag would cost Amazon $1.6 billion in sales per year.
- Wal-Mart showed a 2% increase in conversions for every 1 second of improvement to their page speed.
- BBC has seen that they lose an additional 10% of users for every additional second it takes for their site to load.

Intro

Magento is known to have a lot of speed issues, e.g. if you just check the Magento2 Github repo you can find around 250 issues with the keyword “slow” https://github.com/. However, Magento is not slow by itself, it is we, the developers, who make it slow :)

This article will focus on Magento2 Community Edition and will try to point out all the important main features and tricks to speed up your website.

Everything you read below is based on our experience with M2 and what we did in order to debug and optimize the speed on our newest projects.

A good server and a knowledgeable server admin are your best friends

In our experience, Magento2 uses quite a lot of server resources, so a good hardware configuration is the first checkbox to tick. There are a lot of hosting providers, so choose the one that has good ratings and support. Don’t buy the cheapest server (choose a VPS or a dedicated server and definitely do not select a shared server) and allocate the server resources wisely. For a small to medium webshop (a couple of store views, 20k+ products, 100+ categories) we recommend 8CPUs and 16 GB RAM. If you can afford it, make sure you buy a server with SSD disks.

The software and the configuration are up to your server admin, so let them do their magic. :)

Our configuration for the best performance is:

MariaDB
PHP 7+
Nginx with HTTP2 for SSL/http2 “offloading”
Varnish full page cache
Apache for serving Magento
Redis session storage

For larger sites, we recommend that you put the database on a separate server.

10+ best tips and practices

(Production Mode, Varnish, Redis, Server configuration, Flat Catalog, PHP version, PHP opcache, CDN, Gzip, Image compression, ajax calls, minify js and css)

1. 0 – Enable Cache :)

This is pretty basic.

You can check the cache status in the administration or in the command line with the following command:

php bin/magento cache:status

Then you can enable all the cache with:

php bin/magento cache:enable

2. Enable GZIP compression

It may seem very basic, but we have seen too many shops without an enabled gzip compression. You can find a great article about GZIP here.

3. Update your Magento shop

Always have your Magento store updated to the latest (stable!) version. The new releases fix security issues, add new features and fix reported bugs and performance issues.

4. Put Magento in production mode

Magento2 has three different deploy modes (default, developer, production). Production mode speeds up the whole application, because it is serving pre-compiled DI and static files.
Check in which mode your Magento shop is currently running:

php bin/magento deploy:mode:show

To change to production run:

php bin/magento deploy:mode:set production

After compiling Magento you should create an optimized composer autoload file. This is very important because it will greatly reduce the IO load – composer will have a pre-generated map of classes instead of searching for them on a disk every time you load it. In order to do this, run:

composer dump-autoload -o

5. Enable and configure Varnish for full page cache

Magento 2 can use Varnish (one of the most popular caching solutions for serving dynamic content) for full page caching. Varnish is a reverse-HTTP proxy or web accelerator. The Varnish application stores HTML and other files or fragments so that they can be returned extremely quickly.

Varnish is such a performance booster that Magento even states in their documentation that it is not recommended to run your shop without it. It is extremely important to configure Varnish correctly so that you can take full advantage of this solution. We would therefore advise you to configure this together with your Server admin. Here, you can find more information on how to setup Varnish: http://devdocs.magento.com/guides.

To configure Varnish in Magento go to: Stores -> Configuration -> Advanced -> System -> Full page cache – Caching Application -> Varnish Caching

We were having problems with the cached login button because Varnish forwards cache headers to browser so it caches HTML. This resulted in showing the logged-in user the login button instead of their profile on pages that they’ve visited beforehand. We fixed this by adding the following lines to the varnish config. Locate the following code in vcl:

if (beresp.ttl > 0s && (bereq.method == "GET" || bereq.method == "HEAD")) {
unset beresp.http.set-cookie;
}

and change it to:

if (beresp.ttl > 0s && (bereq.method == "GET" || bereq.method == "HEAD")) {
   unset beresp.http.set-cookie;
   if (bereq.url !~ "\.(ico|css|js|jpg|jpeg|png|gif|tiff|bmp|gz|tgz|bz2|tbz|mp3|ogg|svg|swf|woff|woff2|eot|ttf|otf)(\?|$)")
   {
       set beresp.http.Pragma = "no-cache";
       set beresp.http.Expires = "-1";
       set beresp.http.Cache-Control = "no-store, no-cache, must-revalidate, max-age=0";
       set beresp.grace = 1m;
   }
}

6. Enable OpCache and configure it for Magento2

Another very important part is OpCache, but don’t just install it and then forget about it. Magento has a lot of files and this renders default OpCache config almost useless. Below, you can find the most important options that are required.

opcache.enable=1
opcache.enable_cli=1
opcache.memory_consumption=512
opcache.max_accelerated_files=100000
opcache.revalidate_freq=300
opcache.save_comments=1

This enables OpCache for the web server and CLI, increases memory and the maximum number of files to cope with Magento’s huge codebase. Revalidate frequency specifies how long (in seconds) it should cache php files. Save comments enables PHPDoc comments to be cached because they are used internally by Magento.
You should also check if you have 2 config files – one for the web server and another for CLI. If you do, you should keep them in sync in order to avoid any issues.

7. Redis for session storage and backend cache

Out of the box, Magento uses filesystem for backend cache, which can be slow in some cases. We suggest the switch to Redis, configuring it together with your Server admin.
More info: http://devdocs.magento.com/guides

The default configuration of Redis contains save directives, which dump cache to disk every x saves or after a certain period of time. In the process of dumping it could become unresponsive. Since losing cache is not critical, we suggest you remove the save directives from config.

8. Merge and minify JS and CSS files, minify HTML

Magento by default serves all CSS and JS files separately. In the case of a big number of files it takes more time to load every single file, which increases the final page load. However, with merging, Magento compiles the CSS and JS in one file.

Minification is a process of removing whitespace, comments, and other unnecessary characters in order to reduce the size of files served to the browser. This minification is carried out in advance of requests and is cached so the minified files are available for all website requests.

Magento uses industry-standard minification engines such as jShrink for Javascript and cssmin for CSS. However, be careful in the configuration and DON’T set “Enable Javascript Bundling” to yes. Even though bundling groups JS files together to lower the number of HTTP requests, it is useless if you use HTTP2. Moreover, in many cases the size of the JS file is well over 10MB.
Find the configuration in: Stores > Configuration > Advanced > Developer > CSS and Javascript settings:

Update:

As per July 2018 Google Page Speed update, Google has changed it’s algorithms for determining page speed score. We were working on one of our Magento 2 stores trying to improve our score. We’ve done a lot by implementing responsive images and compressing existing images (you can read more about it in our article on image compression), but the scores barely moved so then we looked deeper into the results and we found that a major score impact was due to one massive javascript file – a minified and merged javascript file. We then tried to disable JS merging and bundling (and left the minifying enabled) and re-ran the tests. The score jumped up by roughly 5-fold.

This is because the client must download the entire JS file before they can start rendering anything. By splitting the payload to more smaller chunks (by not merging the JS assets) the client can start rendering the page sooner ergo the user can start interacting with the page sooner. This is what brings the most value when testing the page with Page Insights tool.

As most of the core Magento 2 frontend is build on JS it comes as no surprise that there is a lot of code. Our minified and merged file resulted in about 1.2MB in size (of which less than 100kB was our custom code). A file this size can take several seconds to download on a 3G mobile network. The browser must wait for this download to complete before it starts rendering the page.

In conclusion: try disabling JS merging and bundling on your Magento 2 site and see the results on Page Speed Insights tool or in the Audits tab in Chrome developer tools.

You can also speed up the theme by minifying HTML and making the page size smaller. Find it in: Stores > Configuration > Advanced > Developer > Template Settings > Minify HTML

9. Use Flat Catalog Category and Product

Read operations to the database are one of the main reasons for slower Magento performance. Magento utilises an EAV structure for categories, product prices, attributes, etc. One of the things that you can do to speed up your store is to enable Flat Catalog. You can find the configuration under Store -> Configuration -> Catalog -> Catalog -> Storefront and set “Use Flat Catalog Category” and “Use Flat Catalog Product” to YES.

10. Optimize Ajax requests

When working on Magento 2 frontend you will often find yourself in a situation where you need to execute some Javascript code when customer data (minicart, wish list, compare items, etc.) is updated and/or displayed. Whenever this occurs you can hook your code to the ‘ajaxComplete’ event. While this is OK you should check how many times this event is triggered per page load in different scenarios (i.e. empty cart/wish list/compare, user logged in/out) using something along the lines of: $(document).on(‘ajaxComplete’, console.log(‘updated’)). Whatever you do, try not to call the same AJAX controller repeatedly.

Another piece of advice: when requesting certain data from the server, try returning as much of the data that the user might need. Let’s take the display SKU of a selected product as an example. Instead of calling the AJAX controller every time the user selects a product variation, finding and returning the SKU, you can call the controller once and have it return an array of option IDs and SKUs: {“38″:”SHOE_38″,”42″:”SHOE_42”}. This means you only have to call the controller once and then display the appropriate SKU from the array. You can see an example in this video.

11. Verified third-party extensions

As in any other development, you must follow some basic guidelines. One of the most important ones is that you use as little 3rd party extensions and dependencies as possible. When you do use custom extensions, please check the code for bugs and buy them only from well-known providers. Our best advice when buying extensions would be to check if the provider is an official Magento partner. After that also check the reviews, the support period, the money back policy and the changelog of the extension (this way you can see how often the extension is updated). We recommend Amasty, Mirasvit, AheadWorks, Magestore and Mageworx.

12. Custom theme from Luma

Do not buy a fancy theme with all the bells and whistles, because it will probably take you more time to disable all unnecessary features and debug all the problems. Our best advice would be to start with a custom theme that extends the Luma Theme. You can find more info on how to do this on Magento’s dev docs: http://devdocs.magento.com/guides

13. PHP 7

Magento2 is built to support PHP 7, so we strongly recommend the use of the PHP version 7.1.x. There were some small issues with Magento 2.1 and PHP 7.1, but they were fixed in the 2.2 version.

14. cacheable=”false”

is both a blessing and a curse. This flag will prevent your block from caching. This is great for development since you don’t have to flush your cache every time you make a change to your template. For example, cacheable=”false” will allow you to make changes to foobar.phtml and have them immediately show on frontend after reloading. The catch is that whenever a block appears that has cacheable=”false”, the entire page is not cached in the Full Page Cache. Always search your codebase for these and remove them before deploying your code as this will slow down your site significantly. However, make sure that you leave certain pages uncacheable (like compare products, cart, checkout, etc.). More details can be found here.

15. Custom cache type

When developing some custom functionalities for our clients we try to utilize Magento features as much as possible.
One of the key things in developing custom frontend functionalities is to use Magento custom cache types. You can read a more detailed tutorial from Urban here.

16. Product page appears slow because it takes forever to load the image (gallery)

We can improve the speed of the product page or rather make it seem much faster by displaying the base image to the user immediately. Normally, Magento will load the gallery library first and build it and only then display it.
My solution displays the base image as a background to the loader element and then hides it and displays the gallery. Please note that:

– there is a slight delay between the loader and the gallery (roughly .5s).
– this might only work on Magento/luma theme

Steps:

1. Copy vendor/magento/module-catalog/view/frontend/template/product/view/gallery.phtml to your theme
2. Add the following code somewhere at the top. This will get the base image url.

$images = $block->getGalleryImagesJson();
$imagesData = json_decode($images,true);
$mainImage = '';
if(count($imagesData) > 0) {
    foreach($imagesData as $key=>$value){
        if(isset($value['isMain']) && $value['isMain'] == 1){
            $mainImage = $value['img'];
            break;
        }
    }
}

3. Add this style to the .loading-mask element (inside div.gallery-placeholder)

style="background-image: url(/* @noEscape */ $mainImage)"

4. Add this styling to any of your scss files

.gallery-placeholder .loading-mask {
    background: 75% center /contain no-repeat;
    @media (max-width: 767px) {
        margin-bottom : 75px;
        background-position: center center;
    }
}

5. Adjust the positioning to your case using background-position and margins. This worked on my project but might not align so well on yours.

Cronjobs

One interesting issue we encountered with Magento 2 production environment is the cron_schedule table growth. Tilen has written an article about this topic and you can find it here.

CDN

Using a content delivery network (CDN) will cache static assets (such as images, CSS and Javascript) and serve the files to users from a node that is geographically closest to them. There are a number of CDN providers, some of the main players being Cloudflare, Amazon AWS CloudFront, BunnyCDN and Fastly.

Our friends at Inchoo wrote a great blog post about the implementation of Fastly with Magento2: http://inchoo.net/magento-2

PWA

Nowadays, one of the buzzwords that everybody is talking about is PWA (Progressive Web Apps). PWA is a web application that makes use of the latest web technologies with the intention of making a web application act and feel like an app. Optiweb has partnered up with Divante on the awesome project that brings PWA to Magento. This technology can enable a store to be rendered within milliseconds and it is immune to traffic overloads. You can read more about it here: http://www.vuestorefront.io/

Watch the Video Demo here.

Profiling (Blackfire)

Every developer sooner or later faces questions such as: “Why is this page taking so long to load?” or “Why is this operation taking such a long time?”. When you face a question like this, it is important to figure out the exact moment when it’s slowing down. Is it every time the page loads or when something specific occurs – like when a user is logged in, when you add 3 or more products to cart, etc. When you figure that out, you are only left with profiling.

That’s where Blackfire comes in. Install Blackfire on your dev environment. Next to that, install also the chrome plugin so you can trigger profiling of the page. Don’t forget to put Magento into production mode and optimise the composer autoload file or you can get significantly different results. Open the page that runs slow, click on Blackfire and voila – you get the performance profile of the page. Open the timeline view and check what is taking so long.

In the timeline view we can see 2 things that stand out:

The header takes ~33% of page load time – digging deeper we can see that loading the menu takes 5.6 seconds.
There are a lot of HTTP requests – according to the url it was a license check of a 3rd party module.

After pinpointing the problems comes the fun part – fixing it.

In the first case (the menu), a lot of categories were loaded to get information in the menu. There are multiple levels of categories in the menu. Since it’s a 3rd party module you don’t want to create a lot of changes in the code so that you can update it as effortlessly as possible. One way of fixing it would be to load categories on mouseover with ajax, but this would generate a lot of changes to the codebase – not good if you want to update it in the future. Instead, we found that we can cache output of getMegamenuHtml function, so we created a module that extended that module and cached the html for a few hours.

In the second case, we found that license check takes nearly 0.5s per module but it is cached if it succeeds. The problem was that our staging environment was not added on the license server of that 3rd party module. This means that the code can be slow because of misconfiguration, but without profiling it it would be hard to find. Usually you would think that a failed license check would not add load time and you would waste time searching for other bits that are slow. That’s why you should always choose the right tool for the task – in case of slow load time that is profiler.

Testing

There are a lot of free tools that let you easily test the speed of your website:

Conclusion

There are still a lot of ways to speed up your website that we did not mention in this blog post – we have basically just started scratching the surface. :) You can therefore expect a second blog post on this topic, which will focus more on the coding side, testing, server configurations and in general more technical aspects, so stay tuned.

Sources:
https://wpostats.com/
http://devdocs.magento.com
https://aionhill.com/magento-performance
https://www.atwix.com/magento-2/ways-to-make-theme-faster/

Anže Harej

You won’t find Anže in our offices anymore, but we know that no matter where he is now, he is doing a phenomenal job. We’ve kept a piece of them with us as a memory. :)

Magento2 – Performance optimization, tips, improvements and hacks

Why speed matters

Intro

10+ best tips and practices

Cronjobs

CDN

PWA

Profiling (Blackfire)

Testing

Conclusion

Share this entry

Anže Harej

Don’t miss out on similar articles

Kafka: Unleashing the power of real-time data streaming

Connecting the Dots: Ideas for Data Synchronization

All roads lead to Škofja Loka