Relevance & scoring
|
This section documents the current Solr line (10.0; 9.10.x the maintained 9.x branch), written and verified against the Apache Solr Reference Guide. No specific patch version is pinned. Some capabilities (the Solr Operator on Kubernetes, the package-manager ecosystem, Learning To Rank model training, and expert plugin development) are linked, not documented in depth. This content was generated with the assistance of AI and should be verified against the official documentation before being relied on in production. This section’s bibliography lists the reference material consulted while preparing these pages. |
Solr ranks documents by handing off scoring to Lucene’s practical scoring model: the same term
that appears more often in a document counts more (term frequency), and a term that appears in
fewer documents across the collection counts more per occurrence (inverse document frequency).
This page covers how that model is implemented today — BM25Similarity, the current default — how
to see the calculation with debugQuery, the several places a document or a term can be boosted,
the Query Elevation Component for pinning results editorially, and the precision/recall trade-off
that relevance tuning is ultimately in service of. Query-time re-ranking (rq, ReRank) and
Learning To Rank live with the query-suggestion machinery on
Spell checking, suggestions & similar documents rather than here; scoring by arbitrary field math
is covered in depth on Function queries.
From classic TF/IDF to BM25Similarity
Older Solr versions (luceneMatchVersion below 6.0) scored with ClassicSimilarity — a vector-
space TF/IDF model where term frequency contributes roughly as sqrt(freq) and never saturates: the
hundredth occurrence of a term in a field still keeps adding to the score. BM25Similarity is
Solr’s current default similarity, in effect since Solr 6 (any collection with
luceneMatchVersion 6.0 or higher uses it unless a Similarity is configured otherwise) — ClassicSimilarity/TF-IDF was the old default before Solr 6, and is no longer what a modern
collection uses out of the box. BM25 keeps the same two intuitions — term frequency and inverse
document frequency — but saturates term frequency so extra occurrences matter less and less, and
normalises for field length so a match in a short field is not diluted the way it is under classic
TF/IDF. See
Major Changes in Solr 6
for the version cutover and
Schema Elements
(the Similarity section) for how a collection’s active similarity is determined.
BM25 exposes two tunables on the field type’s similarity: k1 (default 1.2) controls how
quickly term-frequency saturates, and b (default 0.75) controls how strongly field length is
normalised — b=0 disables length normalisation entirely.
<!-- in schema.xml / managed-schema, on a fieldType -->
<fieldType name="text_general" class="solr.TextField">
<similarity class="solr.BM25SimilarityFactory">
<float name="k1">1.3</float>
<float name="b">0.7</float>
</similarity>
...
</fieldType>
SchemaSimilarityFactory: per-field similarity
Left unconfigured, Solr uses SchemaSimilarityFactory implicitly: it applies BM25Similarity to
every field type that does not declare its own <similarity>, and honors a per-field-type
<similarity> element where one is present — so most schemas mix a BM25 default with the odd field
type pinned to something else (DFRSimilarityFactory, IBSimilarityFactory, or back to
ClassicSimilarityFactory for a field that specifically wants TF/IDF behaviour). Declare
SchemaSimilarityFactory explicitly only if you also want to set a defaultSimFromFieldType naming
which field type’s similarity the whole schema should fall back to instead of BM25Similarity.
See Schema & fields for where <similarity> sits among the other schema
elements, and Field types for fieldType declarations in general.
<similarity class="solr.SchemaSimilarityFactory">
<str name="defaultSimFromFieldType">text_dfr</str>
</similarity>
Seeing the calculation: debugQuery and explainOther
debug=results (or the legacy debugQuery=true, equivalent to debug=all) attaches an
explain block to every hit, breaking its score down term by term — the same idf, tf and
length-normalisation factors BM25 multiplied together. explainOther runs that same explanation
against a second Lucene query, so you can compare why a document you expected to rank highly
scored lower than the documents that actually came back. See
Common Query Parameters
(the debug and explainOther sections).
# Full debug output (parsed query, timing and per-hit explain) for a query.
curl -G "http://localhost:8983/solr/books/select" \
--data-urlencode 'q=title:solr AND author:smith' \
--data-urlencode 'debug=all'
# Compare against a specific document's own explanation.
curl -G "http://localhost:8983/solr/books/select" \
--data-urlencode 'q=title:solr' \
--data-urlencode 'debug=results' \
--data-urlencode 'explainOther=id:book-42'
debug=query and debug=timing narrow the output to just the parsed query or just the phase
timings when the full per-document explanation is not needed.
Boosting the score
Index-time vs. query-time boosting
Lucene 7 removed index-time (per-document or per-field) boosting outright — a boost attribute on
a field at index time is now silently ignored. The supported replacement is to index the boosting
factor (a popularity count, a recency timestamp, an editorial weight) as an ordinary numeric field
and fold it into the score at query time with a function query, typically via the bf or boost
parameters of the DisMax/eDisMax parsers. See
Major Changes in Solr 7
and Function queries for the bf/boost parameters and the function
library available to them. Query-time boosting — the ^ operator, bq/bf, and the techniques
below — remains the normal way to shape ranking.
Per-term boosting
The standard and DisMax/eDisMax parsers both accept a ^ suffix on any clause to multiply its
contribution to the score; values above 1 promote, values between 0 and 1 demote.
curl -G "http://localhost:8983/solr/books/select" \
--data-urlencode 'q=title:solr^3 OR description:solr^0.5' \
--data-urlencode 'defType=edismax'
Payload boosting
A field analyzed with DelimitedPayloadTokenFilterFactory (see
Text analysis: analyzers, tokenizers & token filters) carries a per-token numeric payload alongside the term — useful for a weight that varies token-by-token within the same field, such as a confidence score
attached to each extracted keyword. The payload() function query reads it back into the score, and
local params such as \{!payload_score f=keywords func=max} select which payload-aware query parser
combines multiple matching tokens. See Function queries for payload()
and the other scoring functions.
Function-query boosting
Beyond a flat ^ multiplier, the boost query parser (\{!boost b=…}) and the bf/boost
eDisMax parameters multiply or add an arbitrary function-query result — a recency decay, a
log-scaled popularity count, a geo-distance falloff — into the base relevance score. This is the
mechanism that replaces removed index-time boosting, above. Full coverage, including the function
library and worked bf/boost examples, is on Function queries.
Term-proximity boosting
edismax’s `pf, pf2 and pf3 parameters run the user’s query as an implicit phrase query
(whole query, or every bigram/trigram of it, respectively) against the named fields and add that
phrase match’s score on top of the normal per-term match — so documents where the query terms
appear near each other, not just anywhere in the field, rank higher. ps (and ps2/ps3) sets how
much slop that implicit phrase tolerates, the same way ~N sets slop on an explicit "…"~N
phrase query.
curl -G "http://localhost:8983/solr/books/select" \
--data-urlencode 'q=distributed consensus' \
--data-urlencode 'defType=edismax' \
--data-urlencode 'qf=title description' \
--data-urlencode 'pf=title^2' \
--data-urlencode 'ps=2'
The Query Elevation Component
The Query Elevation Component lets an editor force specific documents to the top of the results
for a specific query string, independent of what BM25 would otherwise compute — the standard way to
implement "sponsored" or "best bets" results, or to hand-correct a query that relevance tuning alone
does not fix. Elevated (and excluded) document IDs are declared per query string in an elevate.xml
file referenced from the component’s solrconfig.xml configuration; enableElevation,
forceElevation and exclusive control its behaviour at request time, and elevateIds/excludeIds
can override the file for a single request. See
Query Elevation Component.
<!-- elevate.xml -->
<elevate>
<query text="ipod">
<doc id="MA147LL/A" /> <!-- pinned to the top -->
<doc id="IW-02" exclude="true" /> <!-- never shown for this query -->
</query>
</elevate>
curl -G "http://localhost:8983/solr/products/select" \
--data-urlencode 'q=ipod' \
--data-urlencode 'enableElevation=true' \
--data-urlencode 'forceElevation=true'
For re-ranking the top N results with a second, costlier query — rather than pinning specific
documents by ID — see the rq/ReRank coverage on
Spell checking, suggestions & similar documents, which also covers Learning To Rank.
Precision vs. recall
Every relevance decision above — which similarity, which boosts, how aggressively to elevate or
re-rank — ultimately trades off precision (what fraction of the returned results are relevant)
against recall (what fraction of all relevant documents in the collection were returned). A legal
e-discovery search wants recall near 100% even at the cost of precision; a product search’s first
page wants high precision and can afford to miss a few borderline matches. Query structure choices
covered elsewhere on this site — minimum_should_match-style clause requirements, fuzzy and
wildcard matching, synonym expansion — shift that balance as much as scoring does. See
Relevance for the
conceptual overview this page’s scoring mechanics implement.