Hibernate Search Analyzers

This section documents Hibernate ORM 7.4.x (User Guide, Introduction, Query Language Guide, Data Repositories Guide), Jakarta Persistence 3.2, Hibernate Search 8.4.x, and the Hibernate Validator / Hibernate Reactive references — which are the references these pages are written and verified against.

This content was generated with the assistance of AI and should be verified against those official docs before being relied on in production.

Three older reference books were consulted as bibliography only while preparing these pages and are not the primary or main source for any page. All three predate Jakarta Persistence 3.2 and Hibernate ORM 6/7 (the javax.persistence → jakarta.persistence namespace change, the ORM 6 query-engine rewrite, the Hibernate Search 6+ Elasticsearch backend), so the official documentation above wins on any discrepancy.

This section’s bibliography lists the reference material consulted while preparing these pages.

Hibernate Search Fundamentals introduces analyzers/normalizers conceptually and shows referencing one by name (analyzer = "english"); this page is where that name is actually defined, plus what to do when a single entity’s text content is not all in the same language.

Built-in analyzers

Both backends ship ready-made analyzers that need no configurer at all — reach for one of these before writing a custom chain; the "Defining a custom analyzer" section below is for the cases these don’t cover (n-grams, a token-filter combination no built-in offers).

Lucene backend — register a built-in org.apache.lucene.analysis.* class under a name with .instance(…​) instead of .custom():

context.analyzer("simple").instance(new org.apache.lucene.analysis.standard.StandardAnalyzer());
Analyzer class Purpose and typical use

StandardAnalyzer (the default when no analyzer is configured for a field)

General-purpose Unicode text segmentation, lowercasing, and stop-word removal — the right default for ordinary prose in a single, unspecified language.

SimpleAnalyzer

Splits on any non-letter character and lowercases; no stop-word removal, no stemming. Cheap and predictable, for content where "words" are simple and language-specific processing is not wanted.

WhitespaceAnalyzer

Splits only on whitespace; does not lowercase or strip punctuation. For pre-normalized tokens (codes, identifiers) where any further processing would be wrong.

StopAnalyzer

SimpleAnalyzer plus English stop-word removal (an explicit stop-word set can be supplied).

KeywordAnalyzer

Emits the entire input as a single, unmodified token — the Analyzer-level equivalent of a @KeywordField; rarely needed directly since @KeywordField already applies a normalizer for this case.

EnglishAnalyzer, FrenchAnalyzer, GermanAnalyzer, and the other org.apache.lucene.analysis.<lang>.* classes

A ready-made equivalent of the hand-assembled SnowballPorterFilterFactory-based chain shown below —  lowercasing, language-specific stop words, and stemming for that language, in one class. Prefer these over hand-rolling the same chain unless a token filter they don’t include is also needed.

Elasticsearch/OpenSearch backend — reference a built-in analyzer by name, directly in the mapping annotation, with no ElasticsearchAnalysisConfigurer needed:

Analyzer name Purpose and typical use

standard (the default)

The same general-purpose behavior as Lucene’s StandardAnalyzer — segmentation, lowercasing, stop words.

simple

Splits on non-letters and lowercases; no stop words, no stemming.

whitespace

Splits only on whitespace; no lowercasing.

stop

simple plus stop-word removal.

keyword

A no-op analyzer — the whole input becomes one token, unmodified.

pattern

Splits on a supplied regular expression instead of Unicode word boundaries — for delimiter-separated content (a custom code format) that doesn’t fit any of the above.

fingerprint

Sorts, deduplicates, and concatenates a field’s tokens into one normalized token — built for near-duplicate detection (e.g. flagging two records whose text differs only in word order/casing/repetition), not general search.

english, french, and the other language analyzer names

Ready-made per-language stemming and stop words, equivalent to Lucene’s EnglishAnalyzer/FrenchAnalyzer above — reference by name directly (analyzer = "french"), no configurer needed at all on this backend.

Defining a custom analyzer

A LuceneAnalysisConfigurer bean, registered via the hibernate.search.backend.analysis.configurer property, builds each custom analyzer as a tokenizer plus a chain of token filters. The language-specific analyzers below are shown hand-assembled from individual filters for illustration — in practice, prefer registering the built-in EnglishAnalyzer/FrenchAnalyzer classes from the table above via .instance(…​) unless a token filter outside what they already include is also needed (this hand-assembled form is what to reach for in that case, or to combine language-specific stemming with something the built-in class doesn’t do):

public class MyAnalysisConfigurer implements LuceneAnalysisConfigurer {

    @Override
    public void configure(LuceneAnalysisConfigurationContext context) {
        // language-specific stemming -- "running"/"runs" both index toward "run" (English rules here;
        // the "language" param selects the Snowball stemming algorithm for that language)
        context.analyzer("english").custom()
                .tokenizer(StandardTokenizerFactory.class)
                .tokenFilter(LowerCaseFilterFactory.class)
                .tokenFilter(SnowballPorterFilterFactory.class).param("language", "English");

        context.analyzer("french").custom()
                .tokenizer(StandardTokenizerFactory.class)
                .tokenFilter(LowerCaseFilterFactory.class)
                .tokenFilter(ASCIIFoldingFilterFactory.class)          // strips accents (é -> e)
                .tokenFilter(SnowballPorterFilterFactory.class).param("language", "French");

        // edge n-grams -- indexes every leading substring of each token ("hiber", "hibern", "hibernate", ...)
        // so a partial, as-you-type query prefix matches before the user finishes typing the whole word
        context.analyzer("autocomplete").custom()
                .tokenizer(StandardTokenizerFactory.class)
                .tokenFilter(LowerCaseFilterFactory.class)
                .tokenFilter(EdgeNGramFilterFactory.class)
                .param("minGramSize", "3")
                .param("maxGramSize", "15");
    }
}
hibernate:
  search:
    backend:
      analysis:
        configurer: class:com.example.search.MyAnalysisConfigurer

Reference any of these by name from a mapping annotation, exactly as Hibernate Search Fundamentals' @FullTextField(analyzer = "english") already does — analyzer = "autocomplete" on a title field indexes it for prefix/as-you-type matching, analyzer = "french" applies French-specific stemming and accent-folding instead of English’s. Regular (non-edge) n-grams (NGramFilterFactory, no Edge prefix) index every substring rather than only leading ones — useful for "contains" style matching (a partial ISBN, a product code fragment) at the cost of a noticeably larger index, since far more substrings get indexed per token.

The Elasticsearch/OpenSearch backend configures the equivalent analysis chain through its own ElasticsearchAnalysisConfigurer (same wiring property, an Elasticsearch-shaped analyzer/filter JSON structure instead of Lucene factory classes) — and, for the language-stemming case specifically, Elasticsearch already ships ready-made built-in language analyzers ("english", "french", and many more) that can often be referenced directly with no custom configurer needed at all. See the Hibernate Search reference’s Analysis chapter for the full tokenizer/token-filter catalog on both backends. Choosing which backend to run against at all is covered on Hibernate Search Backends — Elasticsearch’s own analyzer/analysis-chain concepts, which this section’s Elasticsearch built-ins and ElasticsearchAnalysisConfigurer map onto directly, are covered in depth on Elasticsearch.

Selecting an analyzer dynamically, per document (multi-language content)

An older version of Hibernate Search (5.x and earlier, before the ground-up rewrite that shipped as 6.0) had a dedicated @AnalyzerDiscriminator annotation for exactly this: pick which pre-defined analyzer to apply to a given entity instance at index time, based on the value of one of its own fields (a language column, say). @AnalyzerDiscriminator was not carried forward into the current (6.0+/8.4) architecture — there is no direct, single-annotation equivalent in the modern API. Two current, composable pieces of the API cover the same ground instead, at index time and query time respectively:

Index time — one field per language, populated dynamically by a custom bridge. Declare one indexed field per supported language, each with its own static, language-specific analyzer defined as above (title_en, title_fr, …​), then use a PropertyBinder/PropertyBridge — the same bridge mechanism Hibernate Search Fundamentals introduces for converting an unmapped value type — to read the entity’s own language field and write the text into only the one field that matches it, leaving the other language fields empty for that document:

public class LocalizedTitleBinder implements PropertyBinder {

    @Override
    public void bind(PropertyBindingContext context) {
        context.dependencies()
                .use("language")
                .use("title");

        IndexFieldReference<String> titleEn = context.indexSchemaElement()
                .field("title_en", f -> f.asString().analyzer("english")).toReference();
        IndexFieldReference<String> titleFr = context.indexSchemaElement()
                .field("title_fr", f -> f.asString().analyzer("french")).toReference();

        context.bridge(Book.class, (target, book, bridgeContext) -> {
            IndexFieldReference<String> field = switch (book.getLanguage()) {
                case "fr" -> titleFr;
                default -> titleEn;
            };
            target.addValue(field, book.getTitle());
        });
    }
}

@Entity
@Indexed
public class Book {
    // ...
    @PropertyBinding(binder = @PropertyBinderRef(type = LocalizedTitleBinder.class))
    private String title;

    private String language;   // "en", "fr", ...
}

This is the direct, current-architecture equivalent of what @AnalyzerDiscriminator used to do: the decision of which analyzer effectively applies to this document is still made per-document, at index time, from another field’s value — it is just expressed as "which field gets written", via a bridge, rather than as a single discriminator switch built into the framework.

Query time — override which analyzer parses the search string, per predicate. Independent of how a field was indexed, MatchPredicateOptionsStep.analyzer(name) lets one match() predicate specify which analyzer parses the query value itself, instead of using the field’s own configured analyzer:

List<Book> hits = searchSession.search(Book.class)
        .where(f -> f.match().field("title_fr").matching(userQuery).analyzer("french"))
        .fetchHits(20);

This is useful whenever the caller already knows which language the query is in (a UI language switch, a detected browser locale) and wants it parsed accordingly — it does not retroactively change how content already in the index was analyzed, so it complements the per-language-field indexing pattern above rather than replacing it.

Links