Recommendations on scheduler setup

Selecting the Quartz policies for the processing of overdue tasks

All Quartz policies for the processing of overdue tasks can be divided into three groups: Ignore misfire policy, Run immediately and continue and Discard and wait for next. Recommendations about using each specific policy are given below.

Ignore misfire policy

This policy is represented by the MisfireInstruction.IgnoreMisfirePolicy = -1 constant. It is recommended to use it when it is necessary to ensure that all trigger firings will be executed even with overdue tasks. For example, the task A with 2 minutes execution periodicity. Due to lack of Quartz threads or the scheduler shutdown, the next fire time of task A (NEXT_FIRE_TIME) lags 10 minutes from the current time. If execution of all 5 overdue tasks is required, use the IgnoreMisfirePolicy utility.

This policy is recommended to use for triggers that operate with a unique data at each fire and it is important to perform all trigger fires. For example, the task B that is executed once per 1 hour and generates the report for the time period from PREV_FIRE_TIME till PREV_FIRE_TIME + 1 hour. The scheduler was turned off for 8 hours. In this case, all 8 fires of the task B should be performed after launch of the scheduler and all reports should be generated.

Applying this policy to triggers that do not operate with unique data may cause to unnecessary clogging of the scheduler queue and application performance decrease. For example, the Exchange email synchronization is configured with the 1 minute interval for each Creatio user. An update was performed for 1.5 hours. After the update, the Quartz will synchronize user mailboxes 90 times before proceeding with tasks scheduled for the current time. Although it is enough to perform the delayed synchronization of mailboxes once, and then proceed the task according to the schedule.

Run immediately and continue

This group includes:

• SimpleTrigger.FireNow;
• SimpleTrigger.RescheduleNowWithExistingRepeatCount;
• SimpleTrigger.RescheduleNowWithRemainingRepeatCount;
• CronTrigger.FireOnceNow;
• CalendarIntervalTrigger.FireOnceNow.

More information about these policies can be found in the “Quartz policies for the processing of overdue tasks” article.

This policies should be used if the overdue task should be executed one time and as a priority and then execute scheduled tasks. For example, the email synchronization (<user>@<server>_LoadExchangeEmailsProcess_<userId>, SyncImap_<user>@<server>_<userId>) or the RemindingCountersJob and SyncWithLDAPProcess tasks.

For example, email synchronization is configured with the 5 minutes interval for all users and is fired by the <user>@<server>_LoadExchengeEmailProcess_<userId>Trigger triggers. The update was performed since 1:30 AM till 2:43 AM. After update, the next fire time for the <user>@<server>_LoadExchengeEmailsProcess_<userId>Trigger triggers will be changed to 2:43 AM. All overdue tasks will be fired once at 2:43 AM and further will be fired according to the schedule (2,48 AM, 2:53 AM, 2:58 Am, etc.).

Discard and wait for next

This group includes:

• SimpleTrigger.RescheduleNextWithRemainingCount;
• SimpleTrigger.RescheduleNextWithExistingCount;
• CronTrigger.DoNothing;
• CalendarIntervalTrigger.DoNothing.

More information about these policies can be found in the “Quartz policies for the processing of overdue tasks” article.

These policies should be applied to the tasks that should be fired strictly at a specific time. For example the statistics collection launched daily at 3:00 AM when there is no active users on the website (the CronTrigger is used). This task is resource intensive and time consuming, and it can not be run during working hours, because this can slow down the work of users. In this case, use the CronTrigger.DoNothing policy. As a result, if the task was not fired, the next fire will be at 3:00 AM of the next day.

Quartz configuration

thread count

If the scheduler delays the tasks or if some tasks have not been executed, increase the number of Quartz threads. For this, set necessary number of threads in the Web.config of the application loader:

<add key="quartz.threadPool.threadCount" value="5" />

misfireThreshold

If the increase in the number of Quartz threads is undesirable (for example, due to limited resources), the change in the misfireThreshold setting in the Web.config file of the application loader can optimize the task execution.

For example, an application with a number of tasks much bigger than a number of threads. The most of tasks are executed with a small interval (1 minute). The value of the misfireThreshold setting is 1 minute and the number of threads is 3:

<add key="quartz.jobStore.misfireThreshold" value="60000" />
<add key="quartz.threadPool.threadCount" value="3" />

For the most of the tasks used the policies from the Run immediately and continue group. This means following. For the most tasks that have the MISFIRE_INSTR not equal “-1” and the NEXT_FIRE_TIME less than the current time for 1 minute (60000 ms) the Quartz will periodically set the time of the next fire for the current time. This means that the initial order of the scheduled tasks will be lost because all tasks will have the current time as the fire time. The probability that Quartz will often process the same tasks and ignore the other tasks will increase.

The scheduler queue after 15 minutes of working is displayed on the Fig. 1. Tasks that have the PREV_FIRE_TIME = NULL never have been executed. There is a big number of these tasks.

Fig. 1. Scheduler queue with the misfireThreshold, = 1 minute

Increase the misfireThreshold value to 10 minutes (and clear the PREV_FIRE_TIME in QRTZ_TRIGGERS):

<add key="quartz.jobStore.misfireThreshold" value="600000" />

The scheduler queue after 15 minutes of working is displayed on the Fig. 2. 

Fig. 2. Scheduler queue with the misfireThreshold, = 10 minutes

The number of non-executed tasks is decreased.

Increasing of the value of the misfireThreshold will lead to more equal tasks execution. Almost all jobs from the queue will be executed. Due to lack of threads, the scheduler does not have time to fire each of the tasks in a minute. This is displayed in the [Last repeat interval] column that has the value NEXT_FIRE_TIME - PREV_FIRE_TIME, min. However, the scheduler fires each of the tasks.

batchTriggerAcquisitionMaxCount

Increase the batchTriggerAcquisitionMaxCount to optimize the scheduler performance if you do not use the clustered Quartz configuration (one scheduler node is used).