Jobs and Job Data

Job has one method, execute(JobExecutionContext). What matters is who creates the instance and when. Quartz does not keep a job object between runs: the JobRunShell of every execution asks the scheduler’s JobFactory for a new job instance, calls execute on it, and drops it. Instance fields therefore start empty on every firing and nothing stored in them survives to the next one — state that must survive belongs in the JobDataMap (next section) or in your own database.

The one exception is a refire: when a job asks to be re-executed immediately (see JobExecutionException and its flags), the shell calls execute again on the same instance. The following job counts calls in a field and asks for exactly one refire:

Source: Quartz documentation: Lesson 3, More About Jobs and Job Details

Scheduled three times 400 ms apart, the output was:

Because every firing gets a new instance, a job class needs no thread-safety for its own fields. It does not follow that the class may be stateless in the world: two firings of the same JobDetail can run on two threads at once unless @DisallowConcurrentExecution forbids it, so shared resources the job touches still need to be safe.

A JobDataMap is a Map<String, Object> with typed getters, and a job can find one in two places that Quartz merges before the job runs:

The merged map is built by the execution context: a new JobDataMap receives everything from the JobDetail map and then everything from the Trigger map, so on an equal key the trigger value wins. getMergedJobDataMap() returns that copy.

The snippets assume static imports of JobBuilder.newJob, TriggerBuilder.newTrigger, SimpleScheduleBuilder.simpleSchedule and CronScheduleBuilder.cronSchedule. The Bookshelf overdue-loan job keeps a default batch size of 200 on the JobDetail and overrides it to 50 on its trigger. It also counts its runs:

Source: Quartz documentation: Cookbook, Define a job with data

Source: Quartz Javadoc: JobBuilder

Run with three firings, the job saw the same merged values every time, except for the counter that comes back from the stored map:

The figure shows the same flow: two maps merge into a copy, and only a write to the JobDetail map can come back.

@DisallowConcurrentExecution forbids two executions of the same JobDetail at the same time. It is not a lock on the class: the unit is the job’s key (name and group). Two JobDetail`s of the same class run side by side, and one `JobDetail with two triggers that fire together is serialized, with the second trigger held back until the first run completes.

Source: Quartz Javadoc: DisallowConcurrentExecution

The highest number of simultaneous executions was measured as 1 for job A, 1 for job B and 2 across both, so the annotation serialized A’s two triggers but did not stop A and B from overlapping. A subclass of an annotated class is annotated too: JobDetail.isConcurrentExecutionDisallowed() returned true for a class that only extends SerialJob.

Two further points follow from the stores' source. While a non-concurrent job runs, its other triggers are moved to a BLOCKED state and released when it completes; with the JDBC store that state lives in the database, so the rule holds across the nodes of a cluster, not just within one JVM (see Clustering). And what a held-back trigger does when the wait is long — a misfire — is the subject of Misfires. The Spring-only alternative to this annotation, MethodInvokingJobDetailFactoryBean with concurrent=false, is covered in Getting Started with Spring Boot.

A JobDetail is durable when it is built with storeDurably(). A non-durable job exists only as long as it has triggers: when the last trigger completes or is removed, the store deletes the job. A durable job stays, with or without triggers, and can be triggered or given new triggers later.

Source: Quartz documentation: Lesson 3, More About Jobs and Job Details

The rule of thumb is simple. The four fixed Bookshelf jobs, declared as JobDetail beans, are durable, so unscheduling or pausing their trigger never deletes them. The per-member reminder is created together with its single trigger and is not durable: once it has fired, it disappears and nothing needs cleaning up. How these beans are registered is covered in Getting Started with Spring Boot; creating and removing jobs at run time is covered in Dynamic Scheduling and Management.

If the JVM dies in the middle of a job, a persistent store still has a row saying "this trigger was firing". What happens when the scheduler (or, in a cluster, another node) discovers that depends on one JobDetail flag:

Recovery needs a persistent store: the in-memory store loses the record with the process, so requestRecovery() has an effect only with the JDBC store (or a store that persists its fired-trigger records).

Source: Quartz Javadoc: JobExecutionContext

The failover test of the verification run killed (kill -9) a node that was running a 25-second job with requestRecovery=true, while a second node sat in standby. QRTZ_FIRED_TRIGGERS showed the job as EXECUTING with requests_recovery=true; after the second node was resumed it re-executed the job with isRecovering() true 11.2 seconds after the kill, which is the configured check-in interval (2 s) plus the fixed 7.5 s grace plus one check-in cycle. How a node is declared dead is described in Clustering.

Recovery means "run it again from the start", never "continue where it stopped". The job must be written for that, which is the section on idempotent design below.

Job.execute may throw only JobExecutionException; it is Quartz’s channel from the job to the scheduler. Quartz logs it at INFO, tells the job listeners (jobWasExecuted(context, exception)) and the trigger, and carries on: a failing run does not stop the schedule. Three booleans decide what else happens:

The word "unschedule" is slightly misleading: the triggers are marked complete, not deleted, so they remain visible in the store and in /actuator/quartz until removed.

Refiring is the flag to use with care. It re-runs the job immediately, in the same worker thread, and Quartz applies no limit to the number of refires: a job that always asks for one loops forever. It is safe only when something changes between attempts. The pattern of the official Example 6 (a job that fixes its data and asks to run again) reads like this for the Bookshelf export:

Source: Quartz examples: Example 6, job exceptions

Started with batchSize=500, the job ran four times in a row (500, 250, 125 and 62, getRefireCount() 0 to 3) and the stored value afterwards was 62. The job reads the stored map on purpose: during a refire the merged map is not rebuilt, so a change made to it would not be seen, whereas the JobDetail map is the same object on every attempt. The other official example, BadJob2, sets setUnscheduleAllTriggers(true) so that a job that can never succeed stops being scheduled.

The practice that follows:

Quartz cannot stop a running job; it can only ask. A job opts in by implementing InterruptableJob, whose interrupt() method Quartz calls from another thread. What the method does is up to the job: set a flag that the loop checks, or interrupt the thread that runs the job.

Source: Quartz Javadoc: InterruptableJob

When the work blocks (a sleep, a socket, a queue), the flag is not enough, and the job can keep a reference to its own thread and interrupt it. The weekly statistics report of the Bookshelf pages through the catalogue, 100 ms a page:

Source: Quartz documentation: FAQ

A Quartz job can run more than once for the same logical occasion, and a correct service assumes it will:

The remedy is the same everywhere: make "doing it again" a no-op. The techniques, in the order to prefer them:

The overdue-loan reminders as a small transactional service with a claim table:

Source: Spring Framework: Using @Transactional

Two firings on the same day produced, in order, newReminders=5 (batch size 5 from the trigger) and then newReminders=0. The table held 5 rows and the counter of sent reminders stood at 5.

A Quartz job runs on a Quartz worker thread, in no Spring transaction. Quartz’s own transaction (the one that updates trigger state in the JDBC store) is separate from anything the job does. The job therefore calls a @Transactional service method, which opens and commits its own transaction, and the job class itself stays free of @Transactional so that the transaction boundary is explicit and lives in one place.

Source: Spring Framework: QuartzJobBean

A transaction protects database writes only. In the measured failure case the service claimed two new loans, handed their reminders to the (simulated) mail gateway, and then failed: the database rolled the claims back to the previous 3 rows, but the gateway counter had already advanced by 2 (from 5 to 7), so the next run sends those two mails again. External side effects are at-least-once; the claim table only prevents duplicate runs that complete. Where duplicate mails matter, write an outbox row in the same transaction and let a separate step dispatch it, or pass an idempotency key to the provider. The wider treatment of transactions is in Transaction Isolation and Locking.

Two consequences for the job store:

Source: Spring Framework: LocalDataSourceJobStore

Spring Batch does not schedule itself; a Quartz job is a good launcher for a weekly or nightly batch run. The job should be a thin adapter. Three rules make it robust: build the JobParameters from the scheduled fire time (so a refire, misfire or recovery maps to the same JobInstance), treat "already completed" as success, and mark the job @DisallowConcurrentExecution.

Source: Spring Batch: Running a Job

With the JDBC job repository (spring-boot-starter-batch-jdbc, here on H2) the first launch for 5 October completed (status=COMPLETED), and launching again for the same day completed without running the step: the exception was caught and reported as already completed. The launch is synchronous by default, so the Quartz worker thread is held for the whole batch run, which is one more reason for @DisallowConcurrentExecution.

The existing page Running a job covers JobOperator in full and has a shorter Quartz example, BatchQuartzJob, to which the rule about collaborators above applies: pass JobOperator and the Job through the constructor, or through the scheduler context, because plain setters are not autowired.

On Spring Boot 3.5 the same adapter uses Spring Batch 5.2.x: org.springframework.batch.core.Job, JobParameters, JobParametersBuilder and JobLauncher.run(job, parameters), with org.springframework.batch.core.repository.JobInstanceAlreadyCompleteException.

Quartz ships ready-made jobs in a second artifact, org.quartz-scheduler:quartz-jobs. Spring Boot manages its version together with quartz, so the dependency carries no version:

Source: Quartz source: quartz-jobs

The jobs in 2.5.2, read from the module’s sources:

NativeJob, which ran an operating-system command described in the job data, is not in the module any more: Quartz removed it in 2.4.0 for security reasons. Comparing the artifacts confirms it: the quartz-jobs 2.3.2 jar contains org.quartz.jobs.NativeJob, and the 2.4.1 and 2.5.2 jars do not. Run an external command from your own job instead, with an explicit, fixed command line.

Three practical notes on the rest. The POM of quartz-jobs 2.5.2 declares only quartz and slf4j-api at runtime, so the mail, JMS, EJB and JMX libraries come from your application, and the 2.4.1 and 2.5.2 jars also contain copies of the core org.quartz classes (2.5.2: about 650 KB), harmless as long as both artifacts are the same version. SendMailJob reads the SMTP password from the job data, which puts a credential into the job store and, when values are shown, into the Actuator; in a Spring application use a job that calls JavaMailSender and reads its credentials from configuration. The invoker jobs run whatever the job data names, so write access to job data is write access to those calls; treat the store as a trusted component.

The scan jobs are the ones worth using. Both are annotated @DisallowConcurrentExecution and @PersistJobDataAfterExecution (the last-seen modification time is kept in the stored map), and both find their listener in the SchedulerContext by name:

Source: Quartz documentation: Lesson 3, More About Jobs and Job Details

A file written into the folder was reported to the listener once it was older than MINIMUM_UPDATE_AGE, and its rewrite was reported again. FileScanJob accepts the same MINIMUM_UPDATE_AGE key, but it had no effect in the test: rewriting the watched file just before a scan, with the age set to 60000 ms, notified the listener immediately. Its source adds the minimum age to the current time instead of subtracting it, so only use FileScanJob for files that are not written in place, or prefer DirectoryScanJob. The listener runs on a Quartz worker thread, in no transaction, and the scan job re-runs after a crash: apply the idempotency rules above to importCatalogue.

The configuration below ties the page together for Spring Boot: a durable job that asks for recovery, trigger-level data, a cron trigger with a fire-and-proceed misfire policy and an injectable cron expression. Run with the cron overridden to fire every two seconds, it fired twice, left the persisted counter at runs=2 and sent each reminder once; a manual triggerJob with batchSize=3 reused the claims and created nothing new.

Source: Spring Boot: Quartz Scheduler

The JobDetail read back from the store reported durable=true, requestsRecovery=true, isPersistJobDataAfterExecution=true and isConcurrentExecutionDisallowed=true; the job class is the OverdueLoanRemindersJob of the previous sections.