Magento 2 cron issues

Tilen Habjan14.03.2018eCommerce

One of the more interesting issues we encountered with Magento 2 production environment is the cron_schedule table growth.

A few weeks ago, one of our Magento 2 production environments was getting slow. There was constant CPU load on the server, mostly used by Mysql queries. At times, more than six duplicated cron jobs were running. If we disabled the default cron job, the server load normalised.

We started to debug cron jobs with flock. Flock creates a lock file for the lifetime of cron execution, which means that only one cron job could run at a time.

Flock creates a lock file for the lifetime of cron execution, which means that only one cron job could run at a time.

Worrying results

This gave us a worrying result, since one cron job execution lasted for almost 10 minutes. With a normal execution this cron job should be executed within a 1-minute time frame and then the next cron job should start.

Because one cron ran for more than 1 minute, another cron job started after this minute. Since cron jobs write completed tasks to the database only after they are executed, the second one was created with the same tasks. After another minute the third one started and then the fourth and so on until there was one cron per CPU core.

When we checked the cron_schedule table, there was more than 300.000 rows, with 99% of them with a pending status.

By default, the cron_schedule table is cleaned for each cron group separately. How often each cron group is cleaned is set in cron config definition:

<history_cleanup_every>10</history_cleanup_every> <history_success_lifetime>60</history_success_lifetime> <history_failure_lifetime>600</history_failure_lifetime>

However, this only cleans up cron entries with the status »success« or »missed« and does not clean old entries with the status »pending«.

After we cleaned the old records from the cron_schedule table, only a few thousand rows remained. Cron jobs are now executed within a few seconds.

Here are some graphs of Mysql queries:

* The red line shows when the table was cleaned.

After a few days, we checked our cron_schedule table and we again had over 10k rows in it. Therefore, we created a simple script for clearing one-day-old cron jobs from the cron_schedule table:

use Magento\Framework\App\Bootstrap;

$rootDirectory = *Magento root directory*;
require $rootDirectory . '/app/bootstrap.php';

$params = $_SERVER;
$bootstrap = Bootstrap::create(BP, $params);
$objectManager = $bootstrap->getObjectManager();
$resource = $objectManager->get('Magento\Framework\App\ResourceConnection');
$connection = $resource->getConnection();
$tableName = $resource->getTableName('cron_schedule');


$sql = "DELETE FROM " . $tableName . " WHERE created_at < DATE_SUB(NOW(), INTERVAL 1 DAY)";

try{
    $result = $connection->query($sql);
}catch(Exception $ex){
    print_r($ex);
}

You could create a Mysql event schedule for query:

DELETE FROM `cron_schedule` WHERE `created_at` < DATE_SUB(NOW(), INTERVAL 1 DAY)

Conclusion

We were all surprised that the cron_schedule table size has such an impact on the cron execution performance and the overall server load.

For now, this fix is sufficient. However, we will surely look into the cron_schedule table growth again in the future and try to fix the issue within the cron execution itself.

For more performance tests, tips, improvements and hacks for Magento2 you can checkout the blog post from my coworker here.

Tilen Habjan

You won’t find Tilen in our offices anymore, but we know that no matter where he is now, he is doing a phenomenal job. We’ve kept a piece of them with us as a memory. :)

Magento 2 cron issues

Worrying results

Conclusion

Share this entry

Tilen Habjan

Don’t miss out on similar articles

Kafka: Unleashing the power of real-time data streaming

Connecting the Dots: Ideas for Data Synchronization

All roads lead to Škofja Loka