Listeners & Results Files

A listener follows the normal scoping rules, with one particularity stated in the component reference: listeners are processed at the end of the scope in which they are found. In practice:

Listeners always run after assertions, so the success flag they record is final. The full execution order is on Test Plan Elements, Execution Order & Scoping.

Several listeners may write to the same file; the user manual allows it as long as they all use the same save configuration. In 5.6.3 a result is written only once per file name even if two listeners in scope point to the same file.

Source: Introduction to listeners (User Manual)

In GUI mode, every enabled listener receives the samples and updates its view while the test runs. That is useful while building a plan, and expensive during a load test, because the Swing views refresh continuously (the property jmeter.gui.refresh_period, 500 ms by default, controls how often).

In CLI mode (jmeter -n), the rule in the component reference is short: listeners do not process sample data in CLI mode, but the raw data will be saved if an output file has been configured. So:

The official advice follows from this: run load tests in CLI mode, record results with -l to a CSV file, and use as few listeners as possible; if -l is used, the listeners in the plan can all be disabled or deleted. Analyse the file afterwards, either by loading it into a listener (see Loading a JTL file into a listener after the run) or by generating the HTML dashboard. The CLI options themselves are covered on Running Tests in CLI Mode and the dashboard on HTML Dashboard Report.

Source: Listeners (Component Reference), CLI mode (batch) test runs (User Manual) and Best Practices: Reducing resource requirements

Listeners can use a lot of memory when there are many samples, because most of them keep a copy of every sample they display. The component reference lists the exceptions. The table summarises the 5.6.3 core listeners.

To minimise memory, the documentation recommends the Simple Data Writer with the CSV format.

Source: Resource usage (User Manual) and Listeners (Component Reference)

For load tests, use CSV. Use XML only for short functional or debugging runs where request and response details are needed.

Source: Default Configuration (User Manual) and Listeners (Component Reference)

With the shipped defaults, a 5.6.3 CSV results file starts with this header (the order of columns is fixed; only the selected ones are written):

Three rows written by the Bookshelf test (one thread, a search, a book view and the Transaction Controller that groups them, plus a failing login of another thread) look like this:

Reading them:

The row of the Transaction Controller is written after the rows of its children, because a result is written when it is complete. Values containing the delimiter, like that response message, are quoted.

Source: CSV Log format (User Manual) and CSVSaveService.java (5.6.3 source)

The user manual lists the columns in this order: timeStamp, elapsed, label, responseCode, responseMessage, threadName, dataType, success, failureMessage, bytes, sentBytes, grpThreads, allThreads, URL, Filename, latency, connect, encoding, SampleCount, ErrorCount, Hostname, IdleTime, then any variables. The 5.6.3 writer uses the header names below and puts Connect last, after IdleTime; when reading a file, JMeter uses the header, so the order does not matter to it.

Source: CSV Log format, Sample Attributes (User Manual) and Glossary (User Manual)

These three numbers are the core of every JMeter report, and the official glossary defines them precisely.

All three start from the same instant, so for one sample Connect ≤ Latency ≤ elapsed. The figure shows them on a request timeline, together with the idle time explained in the next section.

A worked reading for the Bookshelf search in the figure: the connection took 40 ms, the first byte arrived 160 ms after the start and the last byte 230 ms after it. So:

Patterns that follow from these definitions:

Source: Glossary (User Manual)

IdleTime is the time inside a sample that JMeter deliberately excludes from it. In 5.6.3 the elapsed time is computed as end time − start time − idle time, and latency and connect time are computed relative to the same corrected start.

For an ordinary sampler IdleTime is 0 (the user manual says "normally 0"). The case that matters in practice is the Transaction Controller when its option Include duration of timer and pre-post processors in generated sample is not selected: the controller then accumulates the gaps between its children (timer delays, pre- and post-processor time) and records them as the idle time of the transaction sample. In the figure, the transaction spans 650 ms on the wall clock, 300 ms of which are a timer, so the row shows elapsed 350 and IdleTime 300. With the option selected, IdleTime is 0 and elapsed is 650.

This is why the convention in this section is to set that option explicitly in every plan: the reported transaction time changes by the whole think time depending on it. The option is described on Logic Controllers & Transactions.

The idle_time property has defaulted to true since 3.0, so the column is present unless removed.

Source: CSV Log format (User Manual), Transaction Controller (Component Reference) and SampleResult.java (5.6.3 source)

The properties reference documents sampleresult.timestamp.start as "save the start time stamp instead of the end", with a default of false. The bin/jmeter.properties shipped with 5.6.3, however, sets it explicitly:

So, unless that line is changed, timeStamp is the start of each sample. To get the end time, add elapsed (and IdleTime). The format is milliseconds since the epoch by default; timestamp_format accepts none, ms or a SimpleDateFormat pattern, applies to CSV only and is used both to write and to read files. When the column does not parse as a number, JMeter also tries yyyy/MM/dd HH:mm:ss.SSS, yyyy/MM/dd HH:mm:ss, yyyy-MM-dd HH:mm:ss.SSS, yyyy-MM-dd HH:mm:ss and MM/dd/yy HH:mm:ss. Keep the default ms when the file will feed the dashboard or a script: it is unambiguous and time-zone free.

Two related properties: sampleresult.useNanoTime (default true) uses System.nanoTime() for timing, and sampleresult.nanoThreadSleep (default 5000 ms) runs a background thread to compute the nano-time offset; a value of zero or less disables that thread.

Source: Settings that affect SampleResults (Properties Reference) and Default Configuration (User Manual)

Some samples contain sub-samples: an HTTP Request that followed redirects or downloaded embedded resources, and a Transaction Controller in Generate parent sample mode. Since 5.0, with jmeter.save.saveservice.subresults=true (the default), sub-results are also written to the CSV file, one line each. Since 5.0, sub-results are also renamed with a numeric suffix (for example POST /api/auth/login-0, POST /api/auth/login-1 for a request and its redirect); subresults.disable_renaming=true restores the old naming. To keep only parent samples in a CSV file, set:

Count carefully when sub-results are present: a request that follows one redirect produces three lines (the parent and two children), so the number of lines is not the number of requests the virtual users made. The HTML dashboard and the listeners understand this; a spreadsheet does not.

Source: Changes history (5.0 incompatible changes) and Settings that affect SampleResults (Properties Reference)

The XML results file (format 2.1, unchanged in 2.2) has a testResults root with httpSample elements for HTTP samples and sample elements for other samplers. Sub-samples are nested elements. The attributes are those in the XML column of the table above (t, lt, ct, ts, s, lb, rc, rm, tn, dt, de, by, sby, ng, na, sc, ec, hn, it, plus one attribute per sample variable). Optional child elements hold the request and response headers, the sampler data (including method, queryString and cookies), the response data and the responseFile name. Only text responses can be saved as response data; images cannot.

A Bookshelf sample with response headers and data saved looks like this (shortened):

Source: XML Log format 2.1 and Sample Attributes (User Manual)

Every file-writing listener has a Configure button that opens the Sample Result Save Configuration dialog. Its check boxes start from the jmeter.save.saveservice.* properties and are then stored in the .jmx with the listener, so a listener configured in the GUI keeps its choices even if the properties change later. Items marked (CSV) apply only to CSV, items marked (XML) only to XML. Cookies, the method and the query string are part of Sampler Data.

The check boxes appear in this order in 5.6.3:

Two items have no check box but are stored in saveConfig: responseDataOnError (property response_data.on_error) and assertionsResultsToSave (property assertion_results).

Source: Sample Result Save Configuration (User Manual) and Listeners (Component Reference)

These properties define the defaults of the dialog and the entire configuration of the -l file. Set them in user.properties (never edit jmeter.properties) or with -J on the command line.

Where the documentation disagrees. The properties reference gives false as the default of connect_time and url. The 5.6.3 bin/jmeter.properties comments and the 5.6.3 source both use true, the changes history records the switch of connect_time to true in 3.1, and the default header shown above contains both columns. The defaults in this table are those of 5.6.3. The properties reference also lists SESSION_ID,REFERENCE as the default of sample_variables; that value is the commented-out example in jmeter.properties, and the property is not set unless you set it.

Source: Results file configuration (Properties Reference), Default Configuration (User Manual) and bin/jmeter.properties (5.6.3)

The defaults are already close to what a load test needs. The Bookshelf user.properties adds the search term and the book id as columns (useful to find which inputs were slow), the host name (the same file layout works for distributed runs), and makes sure nothing heavy is saved:

Never put token, password or other secrets in sample_variables: they would be written to every line of the file. The variable names are also used as XML attribute names, so they must be valid XML names, and JMeter sends the list to remote servers so that the values are available on the controller.

The CSV header then ends with ...,Hostname,IdleTime,Connect,"term","bookId". A sampler that ran before term or bookId was set writes an empty value.

Source: Sample Variables (User Manual) and Results file configuration (Properties Reference)

Source: Listeners (Component Reference), Getting Started: command-line options and Properties Reference

When a load test reports errors, the first question is "what did the server answer?", and a CSV file cannot say: it has no response bodies. Saving every body is out of the question at load (it multiplies file size and I/O), so the goal is to save bodies only for failed samples. There are three ways, from the most to the least recommended.

1. A second file-writing listener for errors only, in XML. Keep the main results in CSV via -l, and add a Simple Data Writer with Errors selected, configured to save as XML with response data and response headers. Because it only receives failed samples, everything it saves is about errors. In the GUI:

This listener writes in CLI mode too, because it has a file name.

2. Save Responses to a file with "Save Failed Responses only". It writes one file per failed sample (body only) and can add the file name to the results (see Save Responses to a file). Useful for binary bodies, which XML cannot hold.

3. The response_data.on_error property for the -l file. With

the -l file saves the response data and the sampler data of every failed sample. It works only in XML, so the whole main file becomes XML, which is the opposite of the recommendation for load tests. Keep it for short smoke runs.

Separately, the Functional Test Mode check box of the Test Plan forces all response data to be saved by every listener; it must stay off in load tests.

Source: Saving response data (User Manual), Simple Data Writer (Component Reference) and Results file configuration (Properties Reference)

Every listener built on the result collector (View Results Tree, View Results in Table, Summary Report, Aggregate Report, Aggregate Graph, Response Time Graph, Graph Results, Assertion Results, Simple Data Writer) has the same file panel.

The class in the .jmx is always ResultCollector; the guiclass attribute tells which view it is: ViewResultsFullVisualizer (View Results Tree), TableVisualizer, SummaryReport, StatVisualizer (Aggregate Report), StatGraphVisualizer (Aggregate Graph), RespTimeGraphVisualizer, GraphVisualizer, AssertionVisualizer or SimpleDataWriter.

Source: Listeners (Component Reference)

Add → Listener → View Results Tree. The View Results Tree shows a tree of all sample results; selecting a sample shows how long it took, its response code, the request JMeter sent and the response. It is the main debugging tool while building a plan, and the official documentation states it must not be used during a load test, because of its memory and CPU cost.

The right-hand side has these tabs:

Renderers, selected in the drop-down at the bottom of the tree, change how the response is displayed or let you test an extractor expression against it:

The order of the renderers in the drop-down comes from view.results.tree.renderers_order. Custom renderers can be added by implementing org.apache.jmeter.visualizers.ResultRenderer and putting the jar in lib/ext.

Search: the Search check box opens a search field, with Case sensitive and Regular exp. options; matches are highlighted in the displayed text. It searches the visible text, so results differ between the Text and HTML views, and its regular expressions use the Java engine. Scroll automatically? keeps the newest sample selected.

The properties that keep the tree usable:

A typical debugging loop for Bookshelf: run one thread for one iteration (threads=1), select POST /api/auth/login, check in Sampler result that the response code is 200, open Response data with the JSON Path Tester and try $.token to confirm the extractor expression, then disable the listener before any real run. The View Results Tree shows the token and the login body in full, which is another reason never to save its content to a shared file.

Source: View Results Tree (Component Reference) and Properties Reference

View Results in Table (Add → Listener → View Results in Table) shows one row per sample with Sample #, Start Time, Thread Name, Label, Sample Time(ms), Status, Bytes, Sent Bytes, Latency and Connect Time(ms). It shows parent samples only unless Child samples? is selected. It also keeps every row and is for debugging only.

Source: View Results in Table (Component Reference)

Both listeners create one row per distinct label, plus a TOTAL row, and compute throughput the same way. The difference is what they keep: the Summary Report keeps running totals only (constant memory), the Aggregate Report keeps the distribution needed for the median and percentiles.

Both panels also have Save Table Data (writes the table to a CSV file) with Save Table Header, and Include group name in label?. In the .jmx these are stored as useGroupName (default false) and saveHeaders (default true). The three percentiles are set with properties, since 2.12:

How throughput is computed. Throughput = number of samples ÷ (end of the last sample − start of the first sample), which includes any pauses between samples, because it is meant to represent the load on the server. Two consequences from the component reference:

Worked example. In the Bookshelf load run, GET /api/books has 5,616 samples between the first start at 10:00:00.000 and the last end at 10:11:00.000, 660 s later: throughput = 5,616 ÷ 660 ≈ 8.5/s. With an average of 5,320 bytes received per sample, the received rate is about 45,000 bytes per second, displayed as roughly 44 KB/sec. If the same endpoint had been split into two samplers named GET /api/books (search) and GET /api/books (browse) with 2,808 samples each, each row would show about 4.25/s.

Which one to use. Use the Summary Report when a quick table in the GUI is enough or in a debugging run. Use the Aggregate Report for percentiles, preferably by loading the JTL after the run, which the component reference names as the recommended way to avoid performance impacts. Aggregate Report memory grows with the number of distinct elapsed times, so long responses (several seconds, rarely identical) need more memory. For a report to share, the HTML dashboard is the better tool.

Source: Aggregate Report, Summary Report (Component Reference), Glossary and Aggregate Report and Aggregate Graph configuration (Properties Reference)

Add → Listener → Aggregate Graph. The same statistics as the Aggregate Report, with a bar chart that can be saved as a PNG. It has a Settings tab and a Graph tab; after changing a filter, click Display Graph (and Apply filter) to refresh. None of these settings is saved in the .jmx.

Buttons: Save Graph (PNG) and Save Table Data. Like the Aggregate Report, it aggregates samples with identical elapsed times to save memory.

Source: Aggregate Graph (Component Reference)

Add → Listener → Response Time Graph. A line chart of response time over the test, one line per label; samples are grouped into intervals and when several samples fall into the same timestamp the mean is drawn. Unlike the Aggregate Graph, its settings are saved in the .jmx file (since 2.12).

For the Bookshelf run, load the JTL afterwards with an interval of 10000 ms and the filter GET /api/books.* to see whether search times drift during the 10-minute hold. The HTML dashboard’s "Response Times Over Time" graph gives the same view without the GUI.

Source: Response Time Graph (Component Reference)

Add → Listener → Simple Data Writer. It records results to a file and displays nothing, which removes the GUI overhead. Its only fields are the common file panel; the fields written come from its Configure dialog, initialised from the properties. In CLI mode, -l does the same job for the whole plan, so a Simple Data Writer is useful for a second file with a different scope or configuration, such as the errors-only XML file above or a file for one Thread Group only.

Placed as a child of the Borrowers Thread Group, it writes only that group’s samples.

Source: Simple Data Writer (Component Reference)

Add → Listener → Generate Summary Results. It periodically writes a summary of the run so far to the log and to standard output: a delta line (+) for the last interval and a running total line (=). Output is aligned to the interval boundary, so the first and last deltas are usually shorter; a line is written only if samples arrived in the interval. Its only field is Name, which is the label of the output: all summarisers with the same name add their samples together.

In CLI mode a summariser is created automatically, named by summariser.name (default summary). That is where the progress lines of a CLI run come from. The component reference warns that if you add a Generate Summary Results named summary to the plan as well, its samples are accumulated with the default summariser’s under that label and the totals become wrong; give your own a different name.

A Bookshelf load run (50 threads, 60 s ramp-up, 660 s duration, 3 s think time, about 0.2 s per request, so about 50 ÷ 3.2 ≈ 15.6 requests per second at full load) prints:

How to read them: each thread does about 1 ÷ 3.2 ≈ 0.3125 requests per second. In the first 30 s the ramp-up grows from 0 to 25 active threads, 12.5 on average, so 12.5 × 0.3125 ≈ 3.9/s; in the next 30 s the average is 37.5 threads, about 11.7/s; after the ramp-up, 50 threads give the expected 15.6/s; the final total over 660 s is lower than 15.6/s because it includes the ramp-up. The Active, Started and Finished columns appear on delta lines only. These lines are the first place to watch during a run: a rising Avg at constant Active or a non-zero Err is a reason to look closer.

Source: Generate Summary Results (Component Reference) and Summariser configuration (Properties Reference)

Add → Listener → Save Responses to a file. For each sample in its scope it writes the response data to a file. Its main use is functional testing, or keeping responses too large for the View Results Tree; in a load test, use it only with Save Failed Responses only. It can be placed anywhere in the plan. (The user manual’s "Saving response data" section still calls it a post-processor; in 5.6.3 it is in the Listener menu.)

The file name is prefix + timestamp (if selected) + sequence number + . + the subtype of the content type (json for application/json, unknown if there is none). The sequence number is shared by all threads and restarts at each test start. The parent sample is saved first, then its sub-samples. The generated name is stored in the sample result, so Save Response Filename in a results file records which file holds each body, and a listener that loads that results file can show the saved body again.

As the child of POST /api/auth/login, the first failed login of a run that started at 10:00 produces results/failed/login-20261014-1000_0001.json next to the .jmx.

Source: Save Responses to a file (Component Reference) and ResultSaver.java (5.6.3 source)

Add → Listener → JSR223 Listener. Runs a script for every sample result in its scope. It replaces the BeanShell Listener, whose documentation itself recommends migrating to JSR223 with Groovy.

Bindings available to the script: log, Label, FileName, Parameters, args, ctx, vars, props, sampleResult and prev (the result being processed), sampleEvent, sampler and OUT (System.out).

The script runs in the thread of the virtual user that produced the sample, for every sample, after the sample is complete. Its time is therefore not part of elapsed, but it delays that user’s next request and so lowers throughput: keep it short, avoid I/O, and do not write to a shared file from many threads. The example logs slow Bookshelf searches with their timing breakdown, reading the threshold from the search_sla_ms run property:

With -Jsearch_sla_ms=600, jmeter.log gets lines such as WARN ... SLOW GET /api/books elapsed=742 latency=731 connect=0 idle=0 code=200 thread=Bookshelf users 1-33, which show at a glance that the time was spent waiting for the server (latency close to elapsed) and not on the connection. Scripting in general is on Scripting with JSR223 & Groovy.

Source: JSR223 Listener (Component Reference) and Best Practices: JSR223 Elements

Source: Listeners (Component Reference)