Custom Thread Groups & Load Shaping

The panel has the shared Action to be taken after a Sampler error section at the top (the same five choices as the core Thread Group, stored in ThreadGroup.on_sample_error), then the load fields, the preview chart and the additional fields. Labels and property names below are those of the 3.1.1 source; the time labels show (sec) or (min) according to the Time Unit.

Source: Concurrency Thread Group (JMeter Plugins)

The 3.1.1 thread starter computes the planned concurrency \(C(t)\) at elapsed time \(t\), with target \(N\), ramp-up \(R\), hold \(H\) and \(s\) steps:

Three consequences are worth knowing:

The starter checks every second (or as soon as a thread finishes) whether more threads are needed, and starts them. Threads that end because of an error action, an iteration limit or a failed login are therefore replaced, which is what "maintain the level of concurrency" means.

The wiki lists three JMeter properties that tune all the bzm dynamic groups. Set them in user.properties or with -J.

Source: Concurrency Thread Group (JMeter Plugins)

The Bookshelf stress profile climbs from 20 to 200 threads in ten steps of two minutes and then holds for five minutes, to find the concurrency at which the API saturates. The same journey as the load profile runs under the group (log in, search, view a book, create a loan, view the loan, with 3 s of think time). The fields read properties so that the same plan runs smaller steps in a test environment. Time Unit is set to seconds so that rampup keeps the meaning it has in the rest of the section.

The element and property names are those written by the 3.1.1 GUI (the shape matches the demo.jmx shipped with the Throughput Shaping Timer sources). Save the plan from the GUI to get the exact form your installation writes.

Worked numbers. One Bookshelf thread sends a request every \(R_t + Z = 0.2 + 3 = 3.2\) s, that is 0.3125 requests per second, as long as the response time stays at 0.2 s. If the server kept up, each step would add \(20 \times 0.3125 = 6.25\) requests per second: 6.25 at 20 threads, 31.25 at 100, 62.5 at 200. The test runs \(1200 + 300 = 1500\) s (25 minutes). Saturation shows as the step at which throughput stops rising while response times climb. Suppose the summariser reports about 30 requests per second at 120 threads and still about 30 at 140. Little’s Law, \(R_t = N/X - Z\), then gives \(120 / 30 - 3 = 1.0\) s and \(140 / 30 - 3 \approx 1.67\) s: the extra threads only queue. The capacity is about 30 requests per second, reached between 100 and 120 threads.

A representative summariser excerpt (the numbers depend on the system under test; each 30 s line falls inside one two-minute step):

In jmeter.log the group’s thread starter logs Start supplying threads when it begins and Done supplying threads when the schedule is over.

Source: Concurrency Thread Group (JMeter Plugins)

The Threads Schedule table has five columns. All times are in seconds (whole numbers).

The panel also has the shared Action to be taken after a Sampler error section. The table is stored as a collectionProp named ultimatethreadgroupdata, one inner collectionProp per row holding five stringProp values in column order. New rows are pre-filled with 100, 0, 30, 60 and 10 by the 3.1.1 GUI.

In 3.1.1 each thread of a row gets a start time of group start + initial delay + its share of the startup time, and an end time of start of hold + hold + its share of the shutdown time. The row lasts initial delay + startup + hold
shutdown in total. A thread at its end time stops between two samples, so with a multi-request journey some iterations are cut part way; leave a few seconds of shutdown time so that the cut is spread out rather than simultaneous.

Source: Ultimate Thread Group (JMeter Plugins)

The wiki documents a way to replace the table from a JMeter property, in user.properties or with -J. The schedule is a list of spawn directives:

The four durations accept case-insensitive suffixes s, m, h and d, combinable as in 1d11h23m6s. The wiki’s example is threads_schedule=spawn(15,1s,1s,1s,1s) spawn(40,1s,3s,1s,2s).

When the property is set and not empty, the 3.1.1 source logs GUI threads profile will be ignored and uses the property instead of the table. The property name is fixed, so it overrides every Ultimate Thread Group in the plan; a plan with two Ultimate groups cannot give them different schedules this way. A directive that cannot be parsed is skipped with a warning in the log, not reported as an error, so check the log line Setting threads profile from property threads_schedule: ... and the thread count of the start-up line (Starting thread group number 1 threads 350) after changing it.

Source: Ultimate Thread Group (JMeter Plugins)

The Bookshelf spike profile keeps the 50-thread baseline for about 20 minutes and fires two spikes of 150 extra threads, each reaching 200 threads in 10 s and holding for a minute. Two spikes, eight minutes apart, show whether the API recovers between them (response times back to the baseline, connection pools drained, no growing error rate) and whether the second spike is worse than the first.

The numeric name attributes of the inner properties are generated by the GUI (for the values, they are the Java string hash codes of the cell text) and carry no meaning. The same schedule as a property, for a CLI run that varies the spike size without editing the plan:

Worked numbers. Peak concurrency is \(50 + 150 = 200\) threads, four times the baseline, reached in 10 s, which is a start rate of 15 threads per second, against 0.83 per second during the baseline ramp-up. Each spike holds 200 threads for 60 s; at the light-load rate of 0.3125 requests per second per thread that would be 62.5 requests per second, but a spike exists to show that the server cannot keep that rate, so expect response times to jump and the measured throughput to stay well below it. Note that the group creates 350 threads at the start of the test (the sum of all rows), even though at most 200 are active at once: threads are created up front and sleep until their start time.

Source: Ultimate Thread Group (JMeter Plugins)

Both are called "spike" tools, but they spike different things. The Ultimate Thread Group adds users: the extra threads run full journeys for as long as the row holds them. The core Synchronizing Timer, described on Timers & Think Time, holds existing threads until a number of them have gathered and then releases them at the same instant, a burst of simultaneous requests to one sampler with no extra users. Use the Ultimate Thread Group for "the audience triples during the lunchtime newsletter"; use the Synchronizing Timer for "500 users press Borrow the second a rare book becomes available". They combine: a Synchronizing Timer on the loan request under the spike rows produces both.

The load fields are the same as the Concurrency Thread Group’s, with a rate instead of a thread count; the additional fields add the concurrency limit.

Source: Arrivals Thread Group (JMeter Plugins)

The wiki only gives the definition; the 3.1.1 source (ArrivalsThreadStarter) shows four behaviours that shape the results:

The section’s arrival-rate target, 20 requests per second from the five-request Bookshelf journey, is 4 arrivals per second, as on Open Model & Throughput Control. One journey takes about 5 s (five requests of 0.2 s and four 1 s think times), so on average \(4 \times 5 = 20\) threads are busy. With 5 s response timeouts the worst journey is 29 s, so a stall could need up to \(4 \times 29 = 116\) threads: a Concurrency Limit of 120 bounds the generator without dropping arrivals in the worst case the timeouts allow.

Over the run: \(4 \times 60 / 2 = 120\) arrivals during the linear ramp and \(4 \times 600 = 2400\) during the hold, about 2520 journeys and 12,600 requests. The plugin’s preview chart shows the same total in its legend (Arrival Rate (~2520 total arrivals)). At the end of the test jmeter.log holds a line of the form Done 2520 arrivals, 2518 completions, 2 abandonments. A noticeably smaller arrival count means arrivals were dropped by the concurrency limit; abandonments are iterations ended early by Start Next Thread Loop.

Source: Arrivals Thread Group (JMeter Plugins)

The panel has one table, Requests Per Second (RPS) Schedule, with Add Row, Copy Row and Delete Row buttons and a preview chart (Expected RPS).

The table is stored as a collectionProp named load_profile. Like every timer, the Throughput Shaping Timer applies to every sampler in its scope and counts all their samples: under a thread group with a five-request journey, 20 RPS means 20 HTTP samples per second in total (4 journeys), and the delay falls before each of the five requests. Put it as the child of a single sampler to shape only that request. In the 2.6 source the timer is one shared instance for all threads (it implements NoThreadClone), and other timers in scope still add their delays, so either remove think time from a shaped scope or size the threads for both.

Source: Throughput Shaping Timer (JMeter Plugins)

The wiki lists these rules:

When the timer runs short of threads, the 2.6 source logs a warning every second of the form No free threads available in current Thread Group Search, made 31 samples/s for expected rps 40 samples/s, increase your number of threads. When the schedule ends it logs No further RPS schedule, asking threads to stop... and escalates over the next seconds from stopping the thread group gracefully to stopping the engine if threads do not end.

The wiki gives the sizing rule for a target rate:

Its example: a target of 1230 RPS with a response time of up to 2.5 s needs \(1230 \times 2500 / 1000 = 3075\) threads. It recommends some overhead for unexpected slow responses, but not too much, since excess threads lead to a "final spike". The rule is Little’s Law with the maximum response time instead of the average: the threads in flight are rate × time in flight, and sizing for the slowest expected response keeps the schedule fed when the server slows down.

For Bookshelf search at 40 RPS (the top of the stress steps below) with response times of up to 2 s: \(40 \times 2000 / 1000 = 80\) threads, so a maximum of about 100 leaves a 25 % margin. At the 20 RPS hold with the usual 0.2 s response time, only \(20 \times 0.2 = 4\) threads are busy on average; the rest wait in the timer.

The schedule can come from a JMeter property instead of the table. The wiki defines three "functions":

Durations take the suffixes s, m, h and d, combinable as 1d11h23m6s. Which property is used, in order of precedence:

The wiki’s examples are my_timer.load_profile=const(10,10s) line(10,100,1m) step(5,25,5,1h) and load_profile=step(5, 50, 5, 2m) const(50, 10m). In the 2.6 source the RPS values in these directives are parsed as whole numbers; fractional rates (for example 0.5 RPS) can only be entered in the GUI table. step expands into one constant row per level, including both ends: step(25,40,5,2m) is four rows of 25, 30, 35 and 40 RPS, two minutes each. A directive that cannot be parsed is skipped with a warning in the log.

Source: Throughput Shaping Timer (JMeter Plugins)

The wiki documents four properties that the timer updates every second, readable with __P or props.get(...) in a JSR223 element. elementName is the timer’s name.

For example, a JSR223 Sampler in a monitoring thread group can log props.get('search_tst_cntDelayed') every few seconds to show whether the schedule is starved.

Sizing for the maximum response time wastes threads for most of a run, and sizing for the average starves the schedule when the server slows down. The wiki’s answer is a feedback loop: write ${__tstFeedback(tst-name,1,100,10)} in the Target Concurrency field of a Concurrency Thread Group, and the group adds threads when the timer reports that the requested RPS is not met, and removes them when too many are idle.

The wiki’s instructions for the group: leave Ramp Up Time and Ramp-Up Steps Count empty, and set Hold Target Rate Time to at least the total duration of the timer’s schedule. In the 2.6 source the function keeps its estimate in the property <timer name>_concurrency, raises it in proportion to the shortfall when samples were missed in the last second with no thread waiting, lowers it when threads are waiting, never goes below 1 or above the maximum, and uses a spare ratio of 0.1 when the fourth argument is omitted.

The class names are those of the 2.6 timer (kg.apc.jmeter.timers.VariableThroughputTimer), as in the demo.jmx shipped with its sources, which uses the same pattern with ${__tstFeedback(ramp_up_tst,1,1000,2)}. The search-terms.csv CSV Data Set Config that provides term is omitted.

Source: Throughput Shaping Timer (JMeter Plugins)

The table above is a warm-up line from 1 to 20 RPS over 2 minutes and a 10-minute hold at 20 RPS: \(21 \times 60 = 1260\) requests in the ramp (the average of 1 and 20 is 10.5 RPS over 120 s) plus 12,000 in the hold. To turn the same plan into an open-ish stress test, add RPS steps from the command line without touching the table:

The named property wins over the table, so the timer runs 2 + 10 + 4 × 2 + 1 = 21 minutes (1260 s); the Concurrency Thread Group’s Hold of 1320 s covers it, as the wiki requires, and the timer stops the threads when its schedule ends. The steps hold 25, 30, 35 and 40 RPS for two minutes each, and the final line takes the rate down to 5 RPS so that the end is not a sudden drop. The figure below shows this schedule.

A representative summariser excerpt from the 30 RPS step, showing Active adjusted by the feedback function and the rate on target (numbers depend on the system):

Started keeps growing while Active follows the need, because the Concurrency Thread Group ends surplus threads at the end of their iteration and starts new ones when the function raises the target.

Source: Throughput Shaping Timer (JMeter Plugins)