Test Plan Elements, Execution Order & Scoping

The variables of the Test Plan are the right place for values that change between runs but stay constant within a run: the host name, the port, the number of threads. They are written ${SERVER} wherever they are used. Combined with the __P function they can be given a default and overridden on the command line:

Run it against another host without editing the file: jmeter -n -t bookshelf-load.jmx -Jhost=staging.internal -Jport=9090. Properties and the functions are covered in Properties, Variables & Functions.

Source: Elements of a Test Plan (User Manual)

By default JMeter starts every Thread Group at the same moment and they run in parallel. Selecting Run Thread Groups consecutively changes that to one group at a time, in tree order, each finishing before the next starts. Use it when the groups are phases of one scenario (a smoke check, then a warm-up, then the measured load) and not concurrent populations (browsing users plus a few administrators). For a one-off preparation step or a clean-up step use a setUp or tearDown Thread Group instead; they have fixed positions in the lifecycle whatever the tree order and whatever this option says.

Source: setUp Thread Group (Component Reference)

A tearDown group does not run after a graceful shutdown of the test unless Run tearDown Thread Groups after shutdown of main threads is checked. Even with the box checked it never runs when the test is stopped (as opposed to shut down). A plan whose clean-up matters (it leaves orders or loans in a shared database) therefore needs the option on and operators who use Shutdown, not Stop.

Source: tearDown Thread Group (Component Reference)

Functional Test Mode makes every sample carry its response body and its sampler data into the saved result files, so that a listener can show exactly what was sent and received later. It is meant for small verification runs (a functional test of the plan, or a debugging session) and not for load. The cost is on the load generator:

When the extra data is needed for one sampler only, the manual’s advice is to leave Functional Test Mode off and attach a listener to that sampler, configuring the fields it saves. Treat the checkbox as a switch that must be off in every load profile of the section.

Source: Elements of a Test Plan (User Manual)

Add directory or jar to classpath provides an easy way to add a classpath entry specific to one test plan. The feature is additive: entries can be added while JMeter runs, but removing an entry requires a restart. It cannot be used for JMeter GUI plugins, because those are processed earlier, but it is handy for utility jars such as JDBC drivers. The jars are added to the search path of the JMeter loader, not to the system class loader.

The same job can be done for every plan at once in jmeter.properties with user.classpath or plugin_dependency_paths, or by dropping the jar into lib/ (libraries) or lib/ext (plugins). The trade-off is scope: a Test Plan entry travels with the .jmx (but the file path inside it is local to the machine, which is a problem on remote servers in a distributed test), whereas a lib/ jar needs to be installed on every load generator.

Source: Test Plan (Component Reference)

The sample result of a sampler can be null (the Flow Control Action sampler, for example, generates no sample result). Post-processors, assertions and listeners are skipped for a null result, and the timers and pre-processors before it still ran. A listener therefore never sees a null result.

The plan below (Bookshelf, one Thread Group, one Transaction Controller with two samplers) mixes the elements so that the order of the types is visible. Each element writes a line to the JMeter log:

Source: JSR223 PreProcessor (Component Reference)

The pre-processor, the timer and the assertion sit as siblings or children in an order that is deliberately "wrong" (timer after pre-processor, assertion after post-processor). The expected sequence for one iteration follows from the rules above: the HTTP Request Defaults (config element) are applied first, then the pre-processor log line, then the 300 ms timer delay, then the request, then the post-processor log line, then the assertion, and finally any listener. If the Browse controller had two samplers, the whole sequence would repeat for the second sampler, with the timer delaying it too, because the Browse-level timer is in scope of both.

The element file of a .jmx is a flat list of hashTree nodes; the nesting of those nodes is the tree. The example above leaves out some attributes the GUI would write (for example the HTTP sampler’s "follow redirects"); JMeter fills those in with defaults when it loads the file, and the GUI saves them again when you save.

Timers are processed before each sampler in the scope in which they are found, and when several timers are in the scope of one sampler, all of them are processed before it and their delays add up. A thread group with a 1000 ms Constant Timer and a Uniform Random Timer (constant delay offset 500 ms, random delay maximum 200 ms) delays each sampler by 1500 ms to 1700 ms: the sum of the two, not the larger. The same holds when one timer sits at Thread Group level and another under a controller; a sampler in that controller gets both.

Two practical consequences. First, a plan that "has a 1 second think time" can have a much higher one because a second timer was added earlier in the tree and forgotten; open the plan and check every level. Second, the number of requests a thread can issue is bounded by the sum of the delays plus the response time: a thread with 1500 ms of timers and 200 ms of response time can do at most 1 / 1.7 = 0.59 requests per second, so 50 such threads generate at most about 29 requests per second at full load. The think-time page of this section, timers-and-think-time, goes deeper into the timers themselves.

Source: Timers (Component Reference)

Configuration defaults (such as HTTP Request Defaults) are merged into one set of values that the sampler can use. A sampler leaves a field empty to inherit it. Several defaults in scope are combined field by field, and the one closest to the sampler wins where they disagree.

The user manual test-plan chapter states a different rule for three elements: the Header Manager, the Cookie Manager and the Authorization Manager are not merged; if more than one is in scope, only one is used, and there is no way to say which. Two details from the component reference refine this and are worth knowing before relying on it:

A planned change for JMeter 6.0 (from changes.xml on the master branch, not released) is that the Cookie Manager stops sending expired cookies. It does not change the scoping rule.

Source: Elements of a Test Plan (User Manual), HTTP Header Manager, HTTP Cookie Manager and HTTP Authorization Manager (Component Reference)

Consider this Bookshelf plan, where the numbers are the order of the tree:

For each sampler the rules give:

Moving the Constant Timer one level up, to the Thread Group, would add 500 ms to POST /api/loans too, on top of its own random delay. Moving the JSON Assertion up to the Browse controller would assert both browse requests (and make GET /api/books/1 fail if the assertion looks for the search result shape). Moving the Summary Report under the Browse controller would report only the two browse requests and silently ignore the loan request.

Source: Elements of a Test Plan (User Manual)

The Module Controller substitutes a test plan fragment (a controller and all the elements it contains) into the current plan at run time. The fragment can be located in any Thread Group; disable its controller so it only runs through the Module Controller, or store fragments in a dummy Thread Group and disable the whole group. Several fragments with different samplers can coexist, and the Module Controller switches between them by choosing a different controller in its drop-down list, which is a convenient way to run alternative scenarios.

A fragment name is made up of the controller’s own name and the names of all its parents, for example Test Plan / Protocol: JDBC / Control / Interleave Controller (Module1). Fragments used by a Module Controller must have unique names, because the name is how the target is found when the plan is reloaded; change the default controller name, or a duplicate may appear when elements are added.

Source: Module Controller (Component Reference)

The Include Controller uses an external .jmx file. To prepare one, create a Test Fragment under the Test Plan, add the samplers and controllers below it, and save the plan. The file can then be included by other plans. For convenience a Thread Group may be added to the external file for debugging; it is ignored when the file is included.

The Bookshelf plan can use it this way. login-fragment.jmx holds a Test Fragment with the login sampler and a JSON Extractor for the token; bookshelf-load.jmx contains the Include Controller:

Run with a prefix so the same plan works from any directory:

Source: Include Controller (Component Reference)

On a distributed test, an included file must exist on every remote server at the resolved path, which is another reason to use the prefix property. The remote-testing page of this section describes how plans reach the servers.