Streaming expressions & Parallel SQL

This section documents the current Solr line (10.0; 9.10.x the maintained 9.x branch), written and verified against the Apache Solr Reference Guide. No specific patch version is pinned. Some capabilities (the Solr Operator on Kubernetes, the package-manager ecosystem, Learning To Rank model training, and expert plugin development) are linked, not documented in depth.

This content was generated with the assistance of AI and should be verified against the official documentation before being relied on in production.

This section’s bibliography lists the reference material consulted while preparing these pages.

Every page so far reaches Solr through a single request against one collection. Streaming expressions are the opposite: composable functions that read and combine entire result sets across a SolrCloud collection (or several), running as a distributed dataflow instead of a top-N query — Solr’s answer to "join two collections", "aggregate more values than a single request can rank", or "walk a graph of relationships between documents". Parallel SQL is a SELECT front end that compiles down to the same streaming-expression engine, so ordinary SQL clients and JDBC drivers can drive it without learning the expression syntax directly. This page covers both. For the request-parameter and JSON-body query surfaces these sit alongside, see Query basics & parameters and The JSON Request API; for the cluster topology streaming expressions fan out across, see SolrCloud architecture.

The /stream and /sql handlers

Both handlers are implicitly defined — no solrconfig.xml entry is required to use them. /stream takes an expr parameter holding a streaming expression and returns a JSON stream of tuples (flat key/value maps, one per result row). /sql takes a stmt parameter holding a SQL statement, compiles it to a streaming expression internally, and returns the same tuple stream.

curl "http://localhost:8983/solr/books/stream" \
  --data-urlencode 'expr=search(books,
    q="genre:fiction",
    fl="title,author,price",
    sort="price desc",
    rows=10)'

curl "http://localhost:8983/solr/books/sql" \
  --data-urlencode 'stmt=SELECT author, COUNT(*) AS book_count
    FROM books
    WHERE in_stock = true
    GROUP BY author
    ORDER BY book_count DESC
    LIMIT 10' \
  --data-urlencode 'aggregationMode=facet'

See Streaming Expressions and SQL Query Language for the full reference to each handler.

Streaming expression basics

A streaming expression is a function call whose arguments can themselves be streaming expressions, nesting arbitrarily deep — the same shape as a Unix pipeline written as nested calls instead of |. The outermost function is always a source (it produces the initial tuple stream); decorators wrap a source (or another decorator) to transform, join, or aggregate that stream before it reaches the client. Every field referenced anywhere in the expression must be stored or a docValues field — streaming expressions read column data, not the inverted index directly.

sort(
  select(
    search(books, q="*:*", fl="title,genre,price", sort="id asc"),
    title, genre, round(price) as rounded_price
  ),
  by="rounded_price desc"
)

Stream sources

A source is where a streaming expression’s data comes from. The three most common:

Source Produces

Source	Produces
`search`	A tuple per document matching a query against one collection — `q`, `fq`, `fl`, `sort`, `rows`, `qt`, exactly like a `/select` request but streamed as tuples rather than a `response` block. `sort` must include a unique tiebreaker field for a deterministic export.
`facet`	One tuple per bucket of a `json.facet`-style aggregation (see Faceting), pushed down into Solr’s own faceting engine — cheaper than `rollup` over a raw `search` when the grouping fields have low-to-moderate cardinality.
`random`	A tuple per document from a random sample of a query’s matches (`rows` caps the sample size) — for statistical sampling and quick data exploration without reading the whole result set.

search

A tuple per document matching a query against one collection — q, fq, fl, sort, rows, qt, exactly like a /select request but streamed as tuples rather than a response block. sort must include a unique tiebreaker field for a deterministic export.

facet

One tuple per bucket of a json.facet-style aggregation (see Faceting), pushed down into Solr’s own faceting engine — cheaper than rollup over a raw search when the grouping fields have low-to-moderate cardinality.

random

A tuple per document from a random sample of a query’s matches (rows caps the sample size) — for statistical sampling and quick data exploration without reading the whole result set.

curl "http://localhost:8983/solr/books/stream" \
  --data-urlencode 'expr=facet(books,
    q="*:*",
    buckets="genre",
    bucketSorts="count(*) desc",
    bucketSizeLimit=20,
    sum(price), count(*))'

Other sources worth knowing by name: export and topic for full, cursor-driven or change-log reads of a collection; jdbc to pull tuples from an external relational database into the same pipeline; knnSearch for a vector-similarity tuple stream (see Dense vector search). The complete list, with every parameter, is in Streaming Expressions under Stream Source Reference.

Stream decorators

Decorators wrap a source (or another decorator) and transform the tuple stream flowing through them:

Decorator Effect

Decorator	Effect
`select`	Project, rename and compute fields on each tuple — `select(stream, field, expr as alias, …)` — the streaming-expression equivalent of a SQL `SELECT` list.
`hashJoin`	An in-memory equi-join between a larger stream and a smaller one held entirely in memory (`hashJoin(search(…), hashed=search(…), on="field")`) — the streaming-expression path to combining two collections, since Solr has no cross-collection query-time join otherwise.
`rollup`	Group tuples by one or more fields and compute aggregate metrics per group (`sum`, `avg`, `min`, `max`, `count`) over an already-sorted stream — the streaming-expression equivalent of `GROUP BY`, used when the source is `search`/`merge` rather than `facet`.
`having`	Filter tuples after a `rollup` (or other aggregation) by a predicate over the computed fields — the equivalent of SQL’s `HAVING` versus `WHERE`.

select

Project, rename and compute fields on each tuple — select(stream, field, expr as alias, …) — the streaming-expression equivalent of a SQL SELECT list.

hashJoin

An in-memory equi-join between a larger stream and a smaller one held entirely in memory (hashJoin(search(…), hashed=search(…), on="field")) — the streaming-expression path to combining two collections, since Solr has no cross-collection query-time join otherwise.

rollup

Group tuples by one or more fields and compute aggregate metrics per group (sum, avg, min, max, count) over an already-sorted stream — the streaming-expression equivalent of GROUP BY, used when the source is search/merge rather than facet.

having

Filter tuples after a rollup (or other aggregation) by a predicate over the computed fields — the equivalent of SQL’s HAVING versus WHERE.

curl "http://localhost:8983/solr/books/stream" \
  --data-urlencode 'expr=having(
    rollup(
      search(books, q="*:*", fl="genre,price", sort="genre asc"),
      over="genre",
      sum(price), count(*)
    ),
    gt(count(*), 5)
  )'

Further decorators worth naming: sort and unique to order and dedupe a stream, merge to combine several already-sorted sources, parallel to fan an expression out across a SolrCloud architecture worker collection (see below), and top / reduce / group for other grouping shapes. The full set is in Streaming Expressions under Stream Decorator Reference.

Evaluators — math expressions

Evaluators are the functions usable inside select, having and other decorator arguments — arithmetic (add, mult, div), comparisons (gt, lt, eq), conditionals (if), string and date functions, and a much larger math expressions library shared with Solr’s statistics and machine-learning streaming functions (regression, correlation, distributions, matrix and vector operations, and time-series functions such as timeseries(), moving averages, and ARIMA-style forecasting — commonly wired into Apache Zeppelin notebooks for interactive analysis and visualization). A having predicate and a select computed field both use this same evaluator grammar. This page only establishes the streaming/SQL foundation those build on; see Streaming Expressions under Stream Evaluator Reference, Math Expressions, and Visualization for that surface in full.

Graph traversal — nodes()

nodes() performs breadth-first graph traversal over a collection treated as a set of edges, walking from a starting set of node values across a gather field and (optionally) a walk edge definition, with built-in cycle detection so a traversal never revisits a node. Earlier Solr versions exposed this as a separate gatherNodes() function; nodes() is the current, merged form and is the one to reach for. Traversals nest to walk multiple hops, and the result can feed into any other streaming expression — scoreNodes for recommendation-style ranking, or export of the gathered node set as GraphML for visualization.

curl "http://localhost:8983/solr/emails/stream" \
  --data-urlencode 'expr=nodes(emails,
    walk="johndoe@example.com->from",
    gather="to",
    count(*))'

See Graph Traversal for the full parameter set, filtering, and multi-collection traversal.

Parallel SQL over JDBC

SELECT statements against /sql compile through Apache Calcite into a streaming expression — each collection queried behaves as though it were a SQL table, with indexed/docValues fields as its columns. The supported surface is SELECT (with DISTINCT), WHERE, GROUP BY, HAVING, ORDER BY, LIMIT, and the same statistical/aggregate functions the streaming-expression layer exposes — there is no INSERT/UPDATE/DELETE, and no arbitrary multi-way JOIN beyond what a hashJoin-backed plan can express. Contrast the full language on SQL Queries.

SELECT genre, COUNT(*) AS book_count, AVG(price) AS avg_price
FROM books
WHERE in_stock = true
GROUP BY genre
HAVING COUNT(*) > 5
ORDER BY book_count DESC
LIMIT 20

Connect with the bundled JDBC driver using a ZooKeeper-based connection string:

jdbc:solr://<zkHost>?collection=books&aggregationMode=facet&numWorkers=2

Aggregation modes — facet vs. map_reduce

aggregationMode picks how GROUP BY is executed: facet (the default) pushes the aggregation down into Solr’s own faceting engine per shard — fast, but best suited to low-to-moderate cardinality grouping fields, the same trade-off facet.method makes for classic faceting (see Faceting). map_reduce instead shuffles tuples across a worker collection so they land on the same worker by grouping-key hash, then aggregates locally — higher network cost, but no cardinality ceiling.

Worker collections and numWorkers

map_reduce aggregation, explicit parallel() streaming expressions, and large hashJoin`s all need a worker collection: an ordinary SolrCloud collection (it holds no data of its own for this purpose) whose shards act as the compute tier tuples get partitioned across. `numWorkers on the JDBC connection string (or the workers parameter on parallel()) sets how many of that collection’s shards participate. Undersizing it caps parallelism; oversizing it beyond the available shards wastes nothing but is not necessary either. See Collections API, configsets & replica placement for creating a collection, and SolrCloud architecture for how shards and replicas map to this compute role.