Parser API & AutoDetectParser

This content was generated with the assistance of AI and should be verified against the official documentation before being relied on in production.

The reference material consulted while preparing these pages is listed in the section bibliography.

Baseline date: 2026-10-12. This page was written against Apache Tika 4.1.0 (the latest release on Maven Central on that date) with the 3.3.x differences noted, Java 21 and later for the examples (Tika 4.x needs Java 17), Spring Boot 4.x on Spring Framework 7 and the Docker image apache/tika:4.1.0. The Parser interface is the heart of Tika: one method that takes a document stream and turns it into XHTML events for a ContentHandler and into key/value pairs in a Metadata object. Everything else on the extraction side (the Tika facade, RecursiveParserWrapper, PipesForkParser, tika-server) is a way of calling a Parser and taking care of its failures. This page explains the contract and its 4.x shape, AutoDetectParser step by step, the parsers that compose other parsers, ParseContext, TikaInputStream and spooling, passwords, and how to share a parser between threads, and it ends with a Bookshelf intake parser that classifies every outcome instead of throwing.

The one-line calls through the facade are in Getting Started with the Tika Facade, the module map is in Introduction & Architecture, the detection that AutoDetectParser performs first is in Media Types & Detection and the dependencies are in Setup & Installation. What to do with the XHTML events is the subject of Content Handlers & Output Formats, the metadata keys are in Metadata and embedded documents are in Embedded Documents & Recursive Parsing.

All examples on this page were compiled and run against Tika 4.1.0 (tika-core, tika-parsers-standard-package as a pom dependency and tika-serialization for TikaLoader). The sample files are small stand-ins: a one-heading manuscript.docx titled "The Lighthouse Keeper" by Ana Torres, a one-page catalogue.pdf, a publisher-email.eml that carries samples.zip, a legacy-catalogue.txt in windows-1252, an empty file, a block of random bytes, and password protected copies of the manuscript and the catalogue (password reader-pw, created only for these tests). The class and method names were also checked against the tika-core 4.1.0 sources and signatures.

<dependencies>
    <dependency>
        <groupId>org.apache.tika</groupId>
        <artifactId>tika-core</artifactId>
        <version>4.1.0</version>
    </dependency>
    <dependency>
        <groupId>org.apache.tika</groupId>
        <artifactId>tika-parsers-standard-package</artifactId>
        <version>4.1.0</version>
        <type>pom</type>
    </dependency>
    <dependency>
        <groupId>org.apache.tika</groupId>
        <artifactId>tika-serialization</artifactId>
        <version>4.1.0</version>
    </dependency>
</dependencies>
dependencies {
    implementation(platform("org.apache.tika:tika-bom:4.1.0"))
    implementation("org.apache.tika:tika-core")
    implementation("org.apache.tika:tika-parsers-standard-package")
    implementation("org.apache.tika:tika-serialization")
}

Source: Using Tika as a Library (Java API) and the resolved POMs on Maven Central; the Gradle form follows the tika-bom platform and was not run here, as on the setup page.

The Parser contract

In 4.1.0 the interface has two methods and two marker supertypes.

public interface Parser extends Serializable, SelfConfiguring {

    Set<MediaType> getSupportedTypes(ParseContext context);

    void parse(TikaInputStream tis, ContentHandler handler, Metadata metadata, ParseContext context)
            throws IOException, SAXException, TikaException;
}

(Signature taken from the 4.1.0 tika-core class file; the real source carries Javadoc.)

Source: Parser, Javadoc 4.1.0.

Argument Direction What it carries

TikaInputStream tis

Input

The document bytes. The parser consumes it and does not close it: the caller owns the stream. In Tika 3.x and earlier the type was a plain InputStream.

ContentHandler handler

Output

Receives the content as a sequence of XHTML SAX events: html, head, title, body, paragraphs, tables and so on. The XHTML only conveys structure; it is not meant for browsers. It can be null when the caller wants only metadata.

Metadata metadata

Input and output

Hints in (the resource name, a declared Content-Type), facts out (Content-Type, title, author, tk:parsed-by and many more).

ParseContext context

Input

A typed bag of everything else: the parser and detector to use for embedded documents, a PasswordProvider, limits, per-component configuration. See the section below.

getSupportedTypes(ParseContext) returns the media types the parser claims, as an immutable set. It takes a context because some parsers answer differently depending on what is configured (an OCR parser claims image types only when an engine is available).

SelfConfiguring is a 4.x marker interface: a parser that implements it reads its own JSON configuration from the ParseContext at parse time (the Javadoc example is a PDF parser reading pdf-parser), instead of having it resolved for it. Third-party parsers normally do not need it. The ParseContext section shows how the JSON configs arrive.

3.x versus 4.x

Aspect Tika 3.x Tika 4.x

Stream argument

parse(InputStream stream, ContentHandler handler, Metadata metadata, ParseContext context)

parse(TikaInputStream tis, ContentHandler handler, Metadata metadata, ParseContext context). There is no InputStream overload, so a third-party Parser fails to compile until it is updated.

Detector

detect(InputStream, Metadata)

detect(TikaInputStream, Metadata, ParseContext)

Caller code

parser.parse(inputStream, handler, metadata, context)

try (TikaInputStream tis = TikaInputStream.get(inputStream)) { parser.parse(tis, handler, metadata, context); }

Configuration

TikaConfig (tika-config.xml) built the parser

TikaLoader (tika-config.json) builds it; see the Configuration page of this section.

Limits

AutoDetectParserConfig carried zip-bomb and XML-depth thresholds

OutputLimits in the ParseContext; AutoDetectParserConfig keeps only throwOnZeroBytes and the handler decorator factory.

Direct use of a concrete parser with a bare ParseContext

Embedded documents were parsed by an SPI-discovered AutoDetectParser

Embedded documents are skipped silently, unless the context holds a Parser and a Detector.

Source: Migrating to Tika 4.x (core SPI signatures, behaviour changes) and Apache Tika 3.3.2, The Parser interface.

Who closes the stream

The Javadoc says the stream "is consumed but not closed" by the parser. The caller opens it, passes it and closes it, so every example here uses try-with-resources on the TikaInputStream. When you build the stream from a Path with TikaInputStream.get(path, metadata), the factory also records the file name (tk:resource-name) and the length (Content-Length) in the metadata, which gives the detector its name hint for free.

A first parse

The shortest correct parse in Tika 4.x: build the parser once, then per document create a stream, a handler, a metadata object and a context.

import java.nio.file.Path;
import org.apache.tika.io.TikaInputStream;
import org.apache.tika.metadata.Metadata;
import org.apache.tika.metadata.TikaCoreProperties;
import org.apache.tika.parser.AutoDetectParser;
import org.apache.tika.parser.ParseContext;
import org.apache.tika.sax.BodyContentHandler;

public class FirstParse {

    public static void main(String[] args) throws Exception {
        AutoDetectParser parser = new AutoDetectParser();

        Metadata metadata = new Metadata();
        BodyContentHandler handler = new BodyContentHandler(-1);
        ParseContext context = new ParseContext();

        try (TikaInputStream tis = TikaInputStream.get(Path.of("manuscript.docx"), metadata)) {
            parser.parse(tis, handler, metadata, context);
        }

        System.out.println("Content-Type : " + metadata.get("Content-Type"));
        System.out.println("Parsed-By    : "
                + String.join(", ", metadata.getValues(TikaCoreProperties.TIKA_PARSED_BY)));
        System.out.println("Title        : " + metadata.get(TikaCoreProperties.TITLE));
        System.out.println("Text         : " + handler.toString().strip());
    }
}

Output:

Content-Type : application/vnd.openxmlformats-officedocument.wordprocessingml.document
Parsed-By    : org.apache.tika.parser.DefaultParser, org.apache.tika.parser.microsoft.ooxml.OOXMLParser, org.apache.tika.parser.microsoft.ooxml.xwpf.XWPFEventBasedWordExtractor
Title        : The Lighthouse Keeper
Text         : The Lighthouse Keeper
Chapter one begins at dusk.

Three things in this output deserve attention.

  • Content-Type was filled in by AutoDetectParser, not by you: the detector’s answer is written to the metadata before the parser runs.

  • tk:parsed-by lists every parser that touched the document, outermost first. It is the quickest way to see which parser really handled a file, and a way to detect "nothing handled it" (see EmptyParser below).

  • new BodyContentHandler(-1) disables the default 100,000-character limit of the no-argument constructor, which otherwise stops long documents with a SAXException. The other handlers and the Markdown default of tika-app, tika-server and Tika Pipes are described on the Content Handlers & Output Formats page.

The AutoDetectParser class also has a three-argument convenience method, parse(tis, handler, metadata), which creates an empty ParseContext and puts itself into it as the Parser for embedded documents.

Source: AutoDetectParser, Javadoc 4.1.0 and Migrating to Tika 4.x, which shows the try (TikaInputStream tis = TikaInputStream.get(...)) pattern.

What can go wrong: the exceptions

A parse has three checked exception types by contract, and a handful of subclasses that carry information.

Exception Super type Meaning and what to do

IOException

Exception

The stream could not be read. Nothing is wrong with the document; retry or fail the request. CompositeParser rethrows an IOException that came from the stream and wraps any other one (TIKA-198, "Illegal IOException from …​") in a TikaException.

SAXException

Exception

The handler failed, or a limit stopped the parse. Wrapped in a TikaException by CompositeParser (TIKA-237) unless it came from your own handler or is a write limit.

TikaException

Exception

The document could not be parsed (corrupt, unsupported variant, a parser bug). A parser RuntimeException is also wrapped in a TikaException ("Unexpected RuntimeException from …​"). One document, not one batch: record it and continue.

EncryptedDocumentException

TikaException

The document is encrypted and no (correct) password was available. Message: "Unable to process: document is encrypted".

ZeroByteFileException

TikaException

Thrown by AutoDetectParser for a stream with no bytes (InputStream must have  0 bytes). Configurable, see below.

WriteLimitReachedException

SAXException

The write limit of the handler was reached. The text up to the limit is still in the handler. WriteLimitReachedException.isWriteLimitReached(e) finds it inside a cause chain.

UnsupportedFormatException, CorruptedFileException, TikaMemoryLimitException, AccessPermissionException

TikaException

More specific reasons a parser may throw instead of a plain TikaException.

TikaTimeoutException

TikaException

A single operation exceeded its timeout. Checked in 4.x (it was not in 3.x).

EmbeddedLimitReachedException

RuntimeException

Thrown only when throwOnMaxDepth or throwOnMaxCount is set in the embedded limits; CompositeParser rethrows it unwrapped on purpose, so it reaches the top-level caller.

Source: the 4.1.0 tika-core sources, org.apache.tika.exception and CompositeParser.parse, and the exception package, Javadoc 4.1.0.

The example below parses the five situations that the Bookshelf intake meets every day and shows what each one throws.

import java.io.IOException;
import java.nio.file.Path;
import org.apache.tika.exception.EncryptedDocumentException;
import org.apache.tika.exception.TikaException;
import org.apache.tika.exception.WriteLimitReachedException;
import org.apache.tika.exception.ZeroByteFileException;
import org.apache.tika.io.TikaInputStream;
import org.apache.tika.metadata.Metadata;
import org.apache.tika.parser.AutoDetectParser;
import org.apache.tika.parser.ParseContext;
import org.apache.tika.sax.BodyContentHandler;
import org.xml.sax.SAXException;

public class Failures {

    static final AutoDetectParser PARSER = new AutoDetectParser();

    static void attempt(String file, int writeLimit) throws IOException {
        Metadata metadata = new Metadata();
        BodyContentHandler handler = new BodyContentHandler(writeLimit);
        try (TikaInputStream tis = TikaInputStream.get(Path.of(file), metadata)) {
            PARSER.parse(tis, handler, metadata, new ParseContext());
            System.out.println(file + ": OK, " + handler.toString().strip().length() + " characters");
        } catch (ZeroByteFileException e) {
            System.out.println(file + ": empty file (" + e.getMessage() + ")");
        } catch (EncryptedDocumentException e) {
            System.out.println(file + ": encrypted (" + e.getMessage() + ")");
        } catch (SAXException e) {
            if (WriteLimitReachedException.isWriteLimitReached(e)) {
                System.out.println(file + ": write limit reached, partial text: '"
                        + handler.toString().strip() + "'");
            } else {
                System.out.println(file + ": XML/handler problem " + e);
            }
        } catch (TikaException e) {
            System.out.println(file + ": cannot parse " + e);
        }
    }

    public static void main(String[] args) throws IOException {
        attempt("manuscript.docx", -1);
        attempt("empty.bin", -1);
        attempt("members-only.pdf", -1);
        attempt("members-only.docx", -1);
        attempt("manuscript.docx", 10);
    }
}

Output:

manuscript.docx: OK, 49 characters
empty.bin: empty file (InputStream must have > 0 bytes)
members-only.pdf: encrypted (Unable to process: document is encrypted)
members-only.docx: encrypted (Unable to process: document is encrypted)
manuscript.docx: write limit reached, partial text: 'The Lighth'

Order the catch blocks from specific to general. SAXException and TikaException are unrelated types, so the compiler does not care about their order, but the meaning does: a write limit is a normal, expected outcome and not a broken document, so test for it before treating a SAXException as a failure.

Source: Apache Tika 4.1.x, Robustness and Apache Tika 3.3.2, Parser guide (the exception contract is unchanged since 1.x).

Zero-byte files

AutoDetectParser throws ZeroByteFileException for an empty stream, so that an upload that lost its body is noticed. To accept empty files, turn the check off with AutoDetectParserConfig:

import org.apache.tika.parser.AutoDetectParser;
import org.apache.tika.parser.AutoDetectParserConfig;

AutoDetectParser lenient = new AutoDetectParser();
AutoDetectParserConfig config = new AutoDetectParserConfig();
config.setThrowOnZeroBytes(false);
lenient.setAutoDetectParserConfig(config);
// an empty file now parses to no text and Content-Type application/octet-stream

The run on empty.bin returned Content-Type application/octet-stream and empty text. For embedded documents, a ZeroByteFileException.IGNORE_ZERO_BYTE_FILE_EXCEPTION object in the ParseContext makes AutoDetectParser and RecursiveParserWrapper ignore zero-byte embedded files (per its Javadoc comment; not exercised here).

Source: AutoDetectParserConfig, Javadoc 4.1.0 and ZeroByteFileException, Javadoc 4.1.0.

AutoDetectParser step by step

AutoDetectParser is a CompositeParser that detects the type itself. It is the parser you should reach for first: it needs no media type from the caller, it falls back gracefully, and it is what the Tika facade uses. Reading its 4.1.0 source gives the sequence below.

sequenceDiagram autonumber participant C as Caller participant A as AutoDetectParser participant D as Detector participant S as SecureContentHandler participant P as CompositeParser lookup participant X as Concrete parser participant H as Your ContentHandler C->>A: parse(tis, handler, metadata, context) A->>A: compute digests if a DigesterFactory is configured A->>D: detect(tis, metadata, context) D-->>A: media type A->>A: metadata Content-Type = detected type A->>A: zero-byte check (throwOnZeroBytes) A->>S: wrap the handler using OutputLimits A->>A: put itself and its detector into the context if absent A->>P: super.parse(tis, secure handler, metadata, context) P->>P: pick parser by type, then supertype, then fallback P->>X: parse(tis, tagged handler, metadata, context) X->>H: XHTML events through the wrappers X->>A: metadata filled (title, author, parsed-by) X-->>P: return or throw P-->>A: wrap unexpected exceptions in TikaException A-->>C: return, the caller closes the stream

  1. Digests. If a DigesterFactory is in the ParseContext, DigestHelper.maybeDigest computes the configured checksums first (the source comment says "before type detection").

  2. Detect. The detector (a DefaultDetector unless you passed another) looks at the magic bytes, the name and the container, and AutoDetectParser stores the answer in Content-Type. A ParsingIntent.WILL_PARSE entry in the context tells detectors that a parse follows, so a detector can prepare (for example salvage a damaged ZIP).

  3. Guards. A zero-byte stream throws ZeroByteFileException (unless disabled). The handler is decorated by the ContentHandlerDecoratorFactory of the AutoDetectParserConfig at the top level and wrapped in a SecureContentHandler, which enforces the zip-bomb ratio and threshold, the maximum XML depth and the maximum nesting of package entries from OutputLimits.

  4. Embedded defaults. If the context has no Parser, the AutoDetectParser itself is stored under Parser.class; if it has no Detector, its own detector is stored under Detector.class. This is why embedded documents work when you go through AutoDetectParser and silently disappear when you call a concrete parser directly (see ParseContext).

  5. Hooks. The 4.1.0 class also carries ParseHooks (setParseHooks/getParseHooks), which run around every document, top-level or embedded. They are internal plumbing for Tika Pipes and are not covered here.

  6. Delegate. CompositeParser.parse chooses the parser for the detected type and calls it; the next section explains the choice. It also records tk:parsed-by, wraps unexpected exceptions and sets embedded-document warnings and limit flags on the top-level metadata.

Two practical consequences follow. A caller-supplied Content-Type is a hint at most: detection overwrites it. And the ContentHandler the concrete parser receives is not yours but a chain of wrappers around yours. An exception thrown by your own handler is recognised through the tagging wrapper that CompositeParser adds and is rethrown to you as it was, instead of being wrapped in a TikaException.

Source: the 4.1.0 AutoDetectParser and CompositeParser sources in the tika-core sources jar, AutoDetectParser, Javadoc 4.1.0 and Apache Tika 3.3.2, Parser guide.

Constructors

Constructor Use it for

new AutoDetectParser()

Everything on the classpath: a DefaultDetector and a DefaultParser. The normal choice.

AutoDetectParser(Detector)

Your detector with all default parsers, for example one that reads custom-mimetypes.xml or adds the PKCS7 detector.

AutoDetectParser(Parser...)

A hand-picked set of parsers with the default detector. Useful to avoid ClassNotFoundException when only some parser jars are present, or to allow only some formats. See the pitfall below.

AutoDetectParser(Detector, Parser...)

Both.

AutoDetectParser(MediaTypeRegistry, Parser, Detector, AutoDetectParserConfig)

Full control, used by the loader. AutoDetectParser.build(compositeParser, detector, config) is the static shortcut.

TikaLoader.loadAutoDetectParser()

Builds the parser, detector and configuration from tika-config.json; shown with the ParseContext example below.

setDetector, setAutoDetectParserConfig and setFallback exist but are plain setters. Call them while building the parser, before it is shared between threads.

Source: AutoDetectParser, Javadoc 4.1.0.

Parsers that compose parsers

Class Role

CompositeParser

Holds a list of parsers and a MediaTypeRegistry. It builds a map from media type to parser and delegates to one of them, or to a fallback.

DefaultParser

A CompositeParser of every Parser found through the Java service provider mechanism, with the Tika parsers first so that user-supplied parsers can override them. Offers constructors that exclude parser classes.

AutoDetectParser

A CompositeParser plus a detector, as shown above.

ParserDecorator

Wraps one parser and changes what it claims: withTypes, withoutTypes, withMimeFilters. withFallbacks is deprecated in favour of FallbackParser.

EmptyParser

Accepts anything and does nothing: no events, no metadata, no exception. It is the default fallback.

How the parser is chosen

CompositeParser.getParser implements a short algorithm.

  1. If the metadata holds tk:content-type-parser-override (the constant TikaCoreProperties.CONTENT_TYPE_PARSER_OVERRIDE), use that type; otherwise use Content-Type.

  2. Normalise the type through the registry (aliases map to the canonical name).

  3. Look it up in the map. If there is no parser, move to the supertype (MediaTypeRegistry.getSupertype, the sub-class-of relation in tika-mimetypes.xml) and try again, up to the root.

  4. If nothing matches, use the fallback parser, EmptyParser by default.

When two parsers claim the same type, the later one in the list wins, with one exception: a content enricher (a parser that only adds to what another one extracts, such as an OCR enricher) fills only types that no other parser in the list claims. DefaultParser sorts the discovered parsers and reverses the list; per the comment in its source this is done so that user-supplied parsers come last and therefore win over the Tika ones. findDuplicateParsers(context) lists every type claimed twice, which is the tool for tracking down a conflict.

The next program asks the default composite parser what it knows (4.1.0, tika-parsers-standard-package).

import java.util.List;
import java.util.Map;
import java.util.Set;
import org.apache.tika.config.ServiceLoader;
import org.apache.tika.mime.MediaType;
import org.apache.tika.mime.MediaTypeRegistry;
import org.apache.tika.parser.DefaultParser;
import org.apache.tika.parser.ParseContext;
import org.apache.tika.parser.Parser;
import org.apache.tika.parser.ParserDecorator;
import org.apache.tika.parser.pdf.PDFParser;

public class Composition {

    public static void main(String[] args) {
        MediaTypeRegistry registry = MediaTypeRegistry.getDefaultRegistry();
        DefaultParser defaults = new DefaultParser(registry);
        ParseContext context = new ParseContext();

        Map<MediaType, Parser> byType = defaults.getParsers(context);
        System.out.println("media types with a parser: " + byType.size());
        for (String type : List.of("application/pdf", "message/rfc822", "text/html", "application/epub+zip")) {
            System.out.println("  " + type + " -> " + byType.get(MediaType.parse(type)).getClass().getName());
        }
        System.out.println("component parsers: " + defaults.getAllComponentParsers().size());
        System.out.println("fallback: " + defaults.getFallback().getClass().getSimpleName());
        System.out.println("types claimed twice: " + defaults.findDuplicateParsers(context).size());

        // Exclude a parser class from the default set
        DefaultParser noPdf = new DefaultParser(registry, new ServiceLoader(), Set.of(PDFParser.class));
        System.out.println("PDF supported without PDFParser: "
                + noPdf.getSupportedTypes(context).contains(MediaType.application("pdf")));

        // Decorate a parser so that it no longer claims PDF
        Parser filtered = ParserDecorator.withoutTypes(new PDFParser(), Set.of(MediaType.application("pdf")));
        System.out.println("decorated PDFParser supports: " + filtered.getSupportedTypes(context).size() + " types");
    }
}

Output:

media types with a parser: 258
  application/pdf -> org.apache.tika.parser.pdf.PDFParser
  message/rfc822 -> org.apache.tika.parser.mail.RFC822Parser
  text/html -> org.apache.tika.parser.html.JSoupParser
  application/epub+zip -> org.apache.tika.parser.epub.EpubParser
component parsers: 89
fallback: EmptyParser
types claimed twice: 21
PDF supported without PDFParser: false
decorated PDFParser supports: 0 types

The counts (258 types, 89 component parsers, 21 duplicates) belong to the standard package on the classpath of this test project; yours will differ with the modules you add. The OOXML parser (OOXMLParser) answered for the Word type in the same run, and the supported formats are discussed on the Supported Formats & Parser Families page of this section.

Source: DefaultParser, Javadoc 4.1.0, ParserDecorator, Javadoc 4.1.0 and CompositeParser, Javadoc 4.1.0.

EmptyParser and silent gaps

An unknown type does not fail: it reaches EmptyParser, which succeeds with no text. A hand-picked parser set makes the same thing happen to every other format.

AutoDetectParser pdfOnly = new AutoDetectParser(new PDFParser());
// catalogue.pdf       -> 68 characters, parsed
// manuscript.docx     -> 0 characters, Content-Type still reported as the Word type, no exception

The first line parsed the catalogue; the second returned no text and no error. This is by design ("whenever possible, unsupported formats are handled by the EmptyParser", in the UnsupportedFormatException Javadoc), and it is the main trap of AutoDetectParser: a missing parser jar looks like an empty document. Detect it by looking at tk:parsed-by:

boolean parsedByNobody = Arrays.asList(metadata.getValues(TikaCoreProperties.TIKA_PARSED_BY))
        .contains(EmptyParser.class.getName());

On a 3,000-byte file of random bytes the run reported application/octet-stream and tk:parsed-by equal to org.apache.tika.parser.EmptyParser. The intake parser at the end of this page uses this check to produce a NO_PARSER status.

Source: EmptyParser, Javadoc 4.1.0 and UnsupportedFormatException, Javadoc 4.1.0.

Forcing a parser

Setting tk:content-type-parser-override before the parse makes CompositeParser pick the parser for that type while the detector still sets Content-Type. The 4.1.0 Javadoc describes it as the way parsers override detection of embedded resources.

Metadata metadata = new Metadata();
metadata.set(TikaCoreProperties.CONTENT_TYPE_PARSER_OVERRIDE, "text/plain");
// manuscript.docx parsed with the text parser: Parsed-By ends with TextAndCSVParser,
// and the "text" is the raw bytes of the ZIP file.

Use the override only for a file that really is the declared type and that detection gets wrong; it is a way to produce garbage on purpose otherwise. Prefer fixing detection with a custom-mimetypes.xml or a custom Detector, as described on the Media Types & Detection page.

Source: TikaCoreProperties, Javadoc 4.1.0, constant CONTENT_TYPE_PARSER_OVERRIDE.

Decorating and excluding

  • ParserDecorator.withTypes(parser, types) makes a parser claim exactly those types (also a way to bind a parser to a type it does not know).

  • ParserDecorator.withoutTypes(parser, types) removes types from what a parser claims, so another parser can take them.

  • ParserDecorator.withMimeFilters(parser, include, exclude) does both and is what the JSON configuration serialises to.

  • new DefaultParser(registry, new ServiceLoader(), excludedClasses) removes whole parser classes (a class and its subclasses) from the service-loaded set, which is how a deployment drops a parser it does not trust. In tika-config.json the same idea is the default-parser entry with an exclusion list; the exact keys are covered on the Configuration page of this section, and the decorator behaviour above was run, the JSON form was not run for this page.

  • A custom Parser added through META-INF/services/org.apache.tika.parser.Parser is found by DefaultParser and beats the Tika parser for the same type; see the Extending Tika page of this section.

  • A parser can also wrap another one by hand. The EncryptedPrescriptionParser in tika-example decrypts the stream with a Cipher and passes it to another parser, with the 4.x TikaInputStream signature. It is a decorating parser in the plain sense, not a ParserDecorator subclass, and an illustration, not production code.

ParseContext

ParseContext is a map from a class to an instance (plus a second map of JSON configuration by component name). It travels with every parse, including into embedded documents, and it is how a caller hands anything to a parser without widening the four-argument signature.

ParseContext context = new ParseContext();
context.set(PasswordProvider.class, metadata -> "reader-pw");  // set by interface or config class
PasswordProvider passwords = context.get(PasswordProvider.class);        // null if absent
PasswordProvider orNone = context.get(PasswordProvider.class, md -> null); // with a default

The common entries, each verified in the 4.1.0 sources or the 4.1.x documentation:

Key class (in the ParseContext) What it controls

Parser

The parser used for embedded documents. AutoDetectParser fills it with itself if absent. Without it a concrete parser skips embedded documents.

Detector

The detector used for embedded documents. Without it, embedded files are reported as application/octet-stream.

PasswordProvider

The password for an encrypted document; see the passwords section.

EmbeddedDocumentExtractor

What to do with each embedded document: parse it, skip it, save its bytes. See Embedded Documents & Recursive Parsing.

OutputLimits

writeLimit, throwOnWriteLimit, maxXmlDepth, maxPackageEntryDepth, zipBombThreshold and zipBombRatio.

EmbeddedLimits

maxDepth, maxCount and the two flags throwOnMaxDepth and throwOnMaxCount.

TimeoutLimits

totalTaskTimeoutMillis and progressTimeoutMillis, the 4.x unified timeout model.

MetadataWriteLimiterFactory

Limits on the size of Metadata (total bytes, field size, values per field); used through ParseContext.newMetadata().

SpoolingStrategy

Which types may be spooled to disk; see the spooling section.

DigesterFactory

Checksums computed by AutoDetectParser before detection.

ZeroByteFileException.IgnoreZeroByteFileException

Ignore zero-byte embedded files.

Source: the 4.1.0 tika-core sources (ParseContext, AutoDetectParser, SecureContentHandler), Apache Tika 4.1.x, Setting limits and ParseContext, Javadoc 4.1.0.

Typed objects and JSON configs

4.x adds a second map: setJsonConfig(name, json) stores the JSON for a component by name, such as pdf-parser, and the component resolves it when it runs. The resolved object is cached per component name and config class (getResolvedConfig(name, class)). The 4.x migration guide records two consequences:

  1. parseContext.get(SomeConfig.class) no longer returns a config that came from JSON. Configs are cached by (component name, config class), so that one VLM provider’s base URL and key can no longer leak into the other two parsers that bind the same config class.

  2. Precedence is inverted: when a key has a JSON config, the JSON wins over a programmatic context.set(XConfig.class, ...); a programmatic value still counts for a key with no JSON config.

TikaLoader is the usual way to fill a context. The next program sets the same output limit in code and from a file, and parses the manuscript with a handler that honours it.

{
  "parsers": [
    { "default-parser": {} }
  ],
  "parse-context": {
    "output-limits": {
      "writeLimit": 20,
      "throwOnWriteLimit": false
    },
    "embedded-limits": {
      "maxDepth": 5,
      "maxCount": 100
    }
  }
}
import java.nio.file.Path;
import org.apache.tika.config.EmbeddedLimits;
import org.apache.tika.config.OutputLimits;
import org.apache.tika.config.loader.TikaLoader;
import org.apache.tika.io.TikaInputStream;
import org.apache.tika.metadata.Metadata;
import org.apache.tika.parser.AutoDetectParser;
import org.apache.tika.parser.ParseContext;
import org.apache.tika.parser.Parser;
import org.apache.tika.sax.BasicContentHandlerFactory;
import org.xml.sax.ContentHandler;

public class ContextConfig {

    static void parse(Parser parser, ParseContext context, String label) throws Exception {
        Metadata metadata = new Metadata();
        // a handler built from the context honours OutputLimits.writeLimit
        ContentHandler handler = BasicContentHandlerFactory
                .newInstance(BasicContentHandlerFactory.HANDLER_TYPE.TEXT, context).createHandler();
        try (TikaInputStream tis = TikaInputStream.get(Path.of("manuscript.docx"), metadata)) {
            parser.parse(tis, handler, metadata, context);
        }
        System.out.println(label + ": '" + handler.toString().strip().replace('\n', '|') + "'");
        for (String name : metadata.names()) {
            if (name.contains("limit")) {
                System.out.println("   " + name + " = " + metadata.get(name));
            }
        }
    }

    public static void main(String[] args) throws Exception {
        // 1. programmatic: one typed object per class
        ParseContext programmatic = new ParseContext();
        programmatic.set(OutputLimits.class, new OutputLimits(20, false, 100, 10, 1_000_000, 100));
        parse(new AutoDetectParser(), programmatic, "programmatic");

        // 2. from tika-config.json
        TikaLoader loader = TikaLoader.load(Path.of("tika-config.json"));
        Parser configured = loader.loadAutoDetectParser();
        ParseContext fromJson = loader.loadParseContext();
        System.out.println("OutputLimits from JSON: " + OutputLimits.get(fromJson));
        System.out.println("EmbeddedLimits from JSON: " + EmbeddedLimits.get(fromJson));
        parse(configured, fromJson, "json");

        // 3. defaults when nothing is configured
        System.out.println("defaults: " + OutputLimits.get(new ParseContext()));
    }
}

Output:

programmatic: 'The Ligh'
   tk:exception:write-limit-reached = true
OutputLimits from JSON: OutputLimits{writeLimit=20, throwOnWriteLimit=false, maxXmlDepth=100, maxPackageEntryDepth=10, zipBombThreshold=1000000, zipBombRatio=100}
EmbeddedLimits from JSON: EmbeddedLimits{maxDepth=5, throwOnMaxDepth=false, maxCount=100, throwOnMaxCount=false}
json: 'The Ligh'
   tk:exception:write-limit-reached = true
defaults: OutputLimits{writeLimit=-1, throwOnWriteLimit=false, maxXmlDepth=100, maxPackageEntryDepth=10, zipBombThreshold=1000000, zipBombRatio=100}

Two points about this output. First, the text was cut with a limit of 20 and stopped after eight visible characters; the counting is done by the handler on all the characters it receives, which are not all in the visible text, and this page did not analyse the exact accounting. Second, and more important, a first attempt that passed new BodyContentHandler(-1) together with the same OutputLimits was not limited at all: writeLimit is read by the handler factories (BasicContentHandlerFactory.newInstance(type, context) and WriteOutContentHandler.newInstance(handler, context)), which Tika Pipes, tika-server and RecursiveParserWrapper use. A handler you build yourself keeps the limit you gave it. The other OutputLimits fields (zip-bomb ratio and threshold, XML depth, package nesting) are read by the SecureContentHandler that AutoDetectParser wraps around your handler, so they apply to any direct parse.

Source: Apache Tika 4.1.x, Setting limits, Migrating to Tika 4.x, ParseContext config resolution and the 4.1.0 BasicContentHandlerFactory, WriteOutContentHandler and SecureContentHandler sources. The configuration file format is described on the Configuration page of this section.

A bare context and concrete parsers

Calling a concrete parser instead of AutoDetectParser bypasses the step that fills in Parser and Detector. The 4.x migration guide documents the consequence and the run below confirms it with RFC822Parser on publisher-email.eml:

RFC822Parser mail = new RFC822Parser();

run(mail, new ParseContext());                       // bare context

ParseContext full = new ParseContext();
full.set(Parser.class, new AutoDetectParser());
full.set(Detector.class, new DefaultDetector());
run(mail, full);                                     // parser and detector supplied

run(new AutoDetectParser(), new ParseContext());     // the usual way

Output (whitespace collapsed):

bare context     :
Parser + Detector: Hello, Please find the spring samples in the attached archive. Marta chapter-one.docx Chapter one The lamp was lit at dusk. figure1.png catalogue.pdf Bookshelf spring catalogue Forty titles from independent publishers. notes/cover-note.txt Cover note for the Bookshelf intake team. Two samples attached: a chapter with one figure and the spring catalogue.
AutoDetectParser : Hello, Please find the spring samples in the attached archive. Marta chapter-one.docx Chapter one The lamp was lit at dusk. figure1.png catalogue.pdf Bookshelf spring catalogue Forty titles from independent publishers. notes/cover-note.txt Cover note for the Bookshelf intake team. Two samples attached: a chapter with one figure and the spring catalogue.

With the bare context the output was empty, without an exception: not even the plain-text body of the message came out, which suggests that for this parser every MIME part goes through the embedded-document path (this was observed, not traced in the parser source). With a Parser and a Detector in the context, the body, the ZIP attachment and the documents inside it are all extracted. The rule is simple: go through AutoDetectParser, or fill the context yourself.

Source: Migrating to Tika 4.x, behaviour changes for a bare ParseContext.

A context is per parse

ParseContext is backed by plain HashMap instances and the parsers write into it during a parse (the ParseRecord, the timeout state, the cache of the parser map). Create one per document and do not share it between concurrent parses. Reusing one for several sequential parses is possible but carries state, such as the cached parser map, from the first parse to the next.

TikaInputStream and spooling

TikaInputStream is a TaggedInputStream with extras that parsers and detectors need: the file name and length, mark and reset, random access, and an "open container" slot where a parser can pass an already-open object to the next parser. How it is built decides what it can do.

Factory Backing Behaviour

TikaInputStream.get(Path) and get(File)

The file itself

Rewind, getPath() and getFile() always work and cost nothing. Sets tk:resource-name and Content-Length. Prefer it whenever you have a file.

TikaInputStream.get(byte[])

Memory

Rewind works; getPath() writes a temp file on demand. Sets Content-Length.

TikaInputStream.get(InputStream)

Passthrough

Reads straight through, nothing is retained, mark/reset work only inside a small buffer. Spools only when asked.

TikaInputStream.get(Blob), get(URL), get(URI)

Blob: memory up to 1 MB, otherwise a caching stream; URL: a caching stream; file: URIs become file-backed

get(URL) also copies the HTTP Content-Type, Content-Encoding, length and resource name into the metadata.

TikaInputStream.get(IOSupplier<InputStream>, TemporaryResources, Metadata)

Re-openable source

Rewinding re-invokes the supplier instead of caching, so digesting never spills to disk.

TikaInputStream.getPlaceholder()

Empty

A stand-in for content that is never extracted, used with MetadataOnlyParse.

Source: the 4.1.0 TikaInputStream sources and Apache Tika 4.1.x, TikaInputStream and Spooling.

No spooling unless asked

Spooling is writing a stream to a temporary file so that code that wants random access can have it: OLE2 files for POI, the central directory at the end of a ZIP, binary plists, and the PDF cross-reference table. Since 4.0 a TikaInputStream over a plain InputStream never writes to disk by itself. Only two things spool: getPath() or getFile() (they must create a file by definition) and enableRewind() (it starts caching in memory and spills to a file past 1 MB). The rules that follow are strict, and the example shows each one.

import java.io.FileInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.nio.file.Files;
import java.nio.file.Path;
import org.apache.tika.io.TikaInputStream;

public class Spooling {

    public static void main(String[] args) throws IOException {
        Path pdf = Path.of("catalogue.pdf");

        try (TikaInputStream tis = TikaInputStream.get(pdf)) {
            System.out.println("path-backed   hasFile=" + tis.hasFile() + " length=" + tis.getLength());
        }

        try (TikaInputStream tis = TikaInputStream.get(Files.readAllBytes(pdf))) {
            System.out.println("byte[]-backed hasFile=" + tis.hasFile() + " hasLength=" + tis.hasLength());
        }

        // A plain stream starts in passthrough mode: nothing is retained
        try (InputStream in = new FileInputStream(pdf.toFile());
             TikaInputStream tis = TikaInputStream.get(in)) {
            System.out.println("stream        hasFile=" + tis.hasFile() + " hasLength=" + tis.hasLength());
            tis.read(new byte[5]);
            System.out.println("read 5 bytes, position=" + tis.getPosition());
            try {
                tis.getPath();
            } catch (IOException e) {
                System.out.println("getPath() after reading: " + e.getMessage());
            }
        }

        // enableRewind() at position 0 caches, so rewinding works afterwards
        try (InputStream in = new FileInputStream(pdf.toFile());
             TikaInputStream tis = TikaInputStream.get(in)) {
            tis.enableRewind();
            tis.read(new byte[5]);
            tis.rewind();
            System.out.println("after rewind  position=" + tis.getPosition());
        }

        // getPath() at position 0 spools once; the temp file disappears on close
        Path temp;
        try (InputStream in = new FileInputStream(pdf.toFile());
             TikaInputStream tis = TikaInputStream.get(in)) {
            temp = tis.getPath();
            System.out.println("spooled       exists=" + Files.exists(temp)
                    + " same on second call=" + temp.equals(tis.getPath()));
        }
        System.out.println("after close   exists=" + Files.exists(temp));

        // rewind() without enableRewind() fails
        try (InputStream in = new FileInputStream(pdf.toFile());
             TikaInputStream tis = TikaInputStream.get(in)) {
            tis.read(new byte[5]);
            try {
                tis.rewind();
            } catch (IOException e) {
                System.out.println("rewind(): " + e.getMessage());
            }
        }
    }
}

Output:

path-backed   hasFile=true length=785
byte[]-backed hasFile=false hasLength=true
stream        hasFile=false hasLength=false
read 5 bytes, position=5
getPath() after reading: Cannot spill to file: position is 5, must be 0. Call enableRewind() before reading if you need file access.
after rewind  position=0
spooled       exists=true same on second call=true
after close   exists=false
rewind(): Cannot seek in passthrough mode. Call enableRewind() first. Current position: 5, requested: 0
Call on a stream-backed TikaInputStream At position 0 After reading

enableRewind()

Switches to caching: bytes kept in memory, spilled to a temp file past 1 MB; rewind and random access then work.

IOException: "Cannot enable rewind: position is N, must be 0".

getPath() / getFile()

Implicitly enables rewind, spills to a temp file once and returns it; the same file is returned on later calls.

IOException: "Cannot spill to file: position is N, must be 0".

rewind()

No-op.

IOException: "Cannot seek in passthrough mode. Call enableRewind() first."

mark(int) / reset()

Work inside an in-memory buffer, no disk. A large readlimit grows a heap buffer: a memory cost, not a disk cost.

Same.

The pitfall this creates is a 3.x habit that worked and now fails: reading part of the stream and then asking for the file. Decide up front. If a code path might need the file, take it first, or call enableRewind() before the first read. For streaming-only work (plain text, HTML) skip it, to avoid the caching cost. The Tika.detect(InputStream) change described on the Getting Started with the Tika Facade page is the same idea from the other side: the internal wrapper that reads ahead is discarded, so the caller’s stream comes back advanced.

Source: Apache Tika 4.1.x, TikaInputStream and Spooling and Migrating to Tika 4.x, TikaInputStream no longer spools unless asked; the output above was produced by running the program against 4.1.0.

SpoolingStrategy

SpoolingStrategy is a class you can put in the ParseContext to say which types may be spooled; its shouldSpool method returns true when the stream has no file yet and the media type matches one of spoolTypes (exact type, base type or a specialisation through the registry). The default types are application/zip, application/x-tika-msoffice, application/x-bplist and application/pdf. The honest summary from the 4.1.x documentation is that no detector or parser shipped with Tika consults it: it is an extension point for your own components. Configuring it therefore does not change what the standard parsers do.

{
  "parse-context": {
    "spooling-strategy": {
      "spoolTypes": ["application/zip", "application/x-tika-msoffice", "application/pdf"]
    }
  }
}

The configuration form comes from the 4.1.x documentation and was not run for this page.

Source: Apache Tika 4.1.x, TikaInputStream and Spooling and SpoolingStrategy, Javadoc 4.1.0.

Cost and best practice

  • Pass the Path or the byte[] to the factory instead of wrapping a FileInputStream or a ByteArrayInputStream: the stream then knows it has a file or bytes and avoids copying and spooling.

  • Always close the stream (try-with-resources): it deletes the temp files it created.

  • Temp files go to the JVM temp directory (java.io.tmpdir). On a container with a read-only root file system or a small /tmp, mount a writable volume for it; large uploads of ZIP, OLE2 and PDF types will spool.

  • Memory: a mark with a large read limit and the in-memory part of a caching stream live on the heap; a file-backed stream does not.

Source: Apache Tika 4.1.x, TikaInputStream and Spooling, best practices. The advice on temp directories is general JVM practice, not a Tika statement.

Passwords and encrypted documents

An encrypted document that Tika cannot open throws EncryptedDocumentException. To open it, put a PasswordProvider into the ParseContext. The interface has one method, getPassword(Metadata), and returns the password as a string, or null for "no password"; because it receives the metadata, it can choose a password per document (by name, say).

import java.nio.file.Path;
import org.apache.tika.io.TikaInputStream;
import org.apache.tika.metadata.Metadata;
import org.apache.tika.parser.AutoDetectParser;
import org.apache.tika.parser.ParseContext;
import org.apache.tika.parser.PasswordProvider;
import org.apache.tika.sax.BodyContentHandler;

public class Passwords {

    public static void main(String[] args) throws Exception {
        AutoDetectParser parser = new AutoDetectParser();

        // In a real service this comes from a secret store or from the request, never from source code.
        String password = System.getenv("MEMBERS_PDF_PASSWORD");
        if (password == null) {
            throw new IllegalStateException("Set MEMBERS_PDF_PASSWORD to the password of the test files");
        }

        ParseContext context = new ParseContext();
        context.set(PasswordProvider.class, metadata -> {
            System.out.println("password requested for " + metadata.get("tk:resource-name"));
            return password;
        });

        for (String file : new String[] {"members-only.pdf", "members-only.docx"}) {
            Metadata metadata = new Metadata();
            BodyContentHandler handler = new BodyContentHandler(-1);
            try (TikaInputStream tis = TikaInputStream.get(Path.of(file), metadata)) {
                parser.parse(tis, handler, metadata, context);
            }
            System.out.println(file + " -> " + handler.toString().strip().replace('\n', '|'));
        }
    }
}

Output:

password requested for members-only.pdf
members-only.pdf -> Bookshelf spring catalogue|Forty titles from independent publishers.
password requested for members-only.docx
members-only.docx -> The Lighthouse Keeper||Chapter one begins at dusk.

With a wrong password the same parse threw EncryptedDocumentException ("Unable to process: document is encrypted"): there is no separate "wrong password" exception in this run. The provider was consulted by the PDF parser and by the Office parser, which covers the two formats of the example; this page did not test other encrypted formats (ZIP, 7z, ODF), and a parser decides for itself whether to ask the provider, so check the parser you rely on.

  • Secrets. Do not hard-code passwords, do not log them, and do not write them to metadata. In the example the password comes from an environment variable, and the program stops if it is missing. The test files of this page use the password reader-pw, which exists only for these samples.

  • Brute force. The provider is called once per document; trying a dictionary of passwords is your decision and your CPU cost, and Tika offers no help.

  • Docx details. How the Office encryption itself works (agile encryption, the password-protected docx) is in Encryption, Signing & Protection.

Source: PasswordProvider, Javadoc 4.1.0 and the run above. Official prose about passwords is sparse in the 4.x documentation, so the Javadoc and the behaviour observed in the run are the sources.

Thread safety and parser reuse

Building an AutoDetectParser is expensive: it discovers every parser and detector on the classpath, loads the media type registry and, for some parsers, reads configuration. Create one per application and reuse it for every document.

The Tika 4.1.x documentation states thread safety explicitly for PipesForkParser ("thread-safe, so one instance serves many threads"). No equivalent sentence for AutoDetectParser was found in the 4.1.x or 3.3.2 guides or in the Javadoc that were read for this page. What the sources and a test show is this: AutoDetectParser keeps no per-parse state in its fields (everything per parse lives in the ParseContext, the metadata and the handler), and a run of 40 parses on a pool of four threads through one shared parser returned identical results for each of the four sample files. Treat that as the common and tested pattern, not as a guarantee, and do not call the setters (setDetector, setAutoDetectParserConfig, setFallback) once the parser is shared.

import java.nio.file.Path;
import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.Future;
import org.apache.tika.io.TikaInputStream;
import org.apache.tika.metadata.Metadata;
import org.apache.tika.parser.AutoDetectParser;
import org.apache.tika.parser.ParseContext;
import org.apache.tika.sax.BodyContentHandler;

public class SharedParser {

    // One parser for the whole application: it is not modified after construction.
    private static final AutoDetectParser PARSER = new AutoDetectParser();

    record Extract(String type, int characters) { }

    static Extract extract(Path file) throws Exception {
        // Everything below is per call and never shared between threads.
        Metadata metadata = new Metadata();
        BodyContentHandler handler = new BodyContentHandler(-1);
        ParseContext context = new ParseContext();
        try (TikaInputStream tis = TikaInputStream.get(file, metadata)) {
            PARSER.parse(tis, handler, metadata, context);
        }
        return new Extract(metadata.get("Content-Type"), handler.toString().length());
    }

    public static void main(String[] args) throws Exception {
        List<Path> files = List.of(Path.of("manuscript.docx"), Path.of("catalogue.pdf"),
                Path.of("publisher-email.eml"), Path.of("legacy-catalogue.txt"));
        ExecutorService pool = Executors.newFixedThreadPool(4);
        try {
            List<Future<Extract>> results = new ArrayList<>();
            for (int i = 0; i < 40; i++) {
                Path file = files.get(i % files.size());
                results.add(pool.submit(() -> extract(file)));
            }
            for (int i = 0; i < 4; i++) {
                System.out.println(files.get(i).getFileName() + " -> " + results.get(i).get());
            }
            boolean consistent = true;
            for (int i = 4; i < 40; i++) {
                consistent &= results.get(i).get().equals(results.get(i % 4).get());
            }
            System.out.println("all 40 results consistent: " + consistent);
        } finally {
            pool.shutdown();
        }
    }
}

Output:

manuscript.docx -> Extract[type=application/vnd.openxmlformats-officedocument.wordprocessingml.document, characters=50]
catalogue.pdf -> Extract[type=application/pdf, characters=72]
publisher-email.eml -> Extract[type=message/rfc822, characters=405]
legacy-catalogue.txt -> Extract[type=text/plain; charset=windows-1252, characters=152]
all 40 results consistent: true
Share across threads Create per document

The AutoDetectParser, the DefaultDetector, a loaded TikaLoader

The TikaInputStream, the ContentHandler, the Metadata, the ParseContext

Sharing the parser does not make parsing safe for the JVM. A pathological file can still exhaust the heap or spin forever in any thread of your application; the 4.1.x documentation tells embedders to use PipesForkParser for untrusted input (see Introduction & Architecture for the choice between in-process, forked and service use), and Tika’s own security model counts such failures of in-process parsing as the caller’s responsibility. For the Java concurrency tools used above see High-Level Concurrency.

Source: Getting Started with the Java API, Apache Tika Security Model and the 4.1.0 AutoDetectParser source; the four-thread run was executed against Tika 4.1.0 for this page.

The Bookshelf intake parser

The class below is the building block that the later pages and the Spring Boot Integration page reuse in other forms: it parses one file and always returns an Outcome, whatever the document does. It builds the parser once, creates the stream, handler, metadata and context per call, enforces a write limit, optionally supplies a password and classifies the result.

import java.io.IOException;
import java.nio.file.Path;
import java.util.Arrays;
import java.util.List;
import org.apache.tika.exception.EncryptedDocumentException;
import org.apache.tika.exception.TikaException;
import org.apache.tika.exception.WriteLimitReachedException;
import org.apache.tika.exception.ZeroByteFileException;
import org.apache.tika.io.TikaInputStream;
import org.apache.tika.metadata.Metadata;
import org.apache.tika.metadata.TikaCoreProperties;
import org.apache.tika.parser.AutoDetectParser;
import org.apache.tika.parser.EmptyParser;
import org.apache.tika.parser.ParseContext;
import org.apache.tika.parser.Parser;
import org.apache.tika.parser.PasswordProvider;
import org.apache.tika.sax.BodyContentHandler;
import org.xml.sax.SAXException;

/** Parses one file and reports what happened, without throwing for a bad document. */
public class IntakeParser {

    enum Status { EXTRACTED, TRUNCATED, EMPTY_FILE, ENCRYPTED, NO_PARSER, FAILED }

    record Outcome(String file, Status status, String mediaType, String parsedBy,
                   int characters, String detail) { }

    private static final int MAX_CHARACTERS = 200_000;

    // Built once, shared by every thread.
    private final Parser parser = new AutoDetectParser();

    Outcome parse(Path file, PasswordProvider passwords) throws IOException {
        Metadata metadata = new Metadata();
        BodyContentHandler handler = new BodyContentHandler(MAX_CHARACTERS);
        ParseContext context = new ParseContext();
        if (passwords != null) {
            context.set(PasswordProvider.class, passwords);
        }
        try (TikaInputStream tis = TikaInputStream.get(file, metadata)) {
            parser.parse(tis, handler, metadata, context);
            return outcome(file, metadata, handler, Status.EXTRACTED, null);
        } catch (ZeroByteFileException e) {
            return outcome(file, metadata, handler, Status.EMPTY_FILE, e.getMessage());
        } catch (EncryptedDocumentException e) {
            return outcome(file, metadata, handler, Status.ENCRYPTED, e.getMessage());
        } catch (SAXException e) {
            if (WriteLimitReachedException.isWriteLimitReached(e)) {
                return outcome(file, metadata, handler, Status.TRUNCATED, "limit " + MAX_CHARACTERS);
            }
            return outcome(file, metadata, handler, Status.FAILED, e.toString());
        } catch (TikaException e) {
            return outcome(file, metadata, handler, Status.FAILED, e.toString());
        }
    }

    private static Outcome outcome(Path file, Metadata metadata, BodyContentHandler handler,
                                   Status status, String detail) {
        List<String> parsers = Arrays.asList(metadata.getValues(TikaCoreProperties.TIKA_PARSED_BY));
        if (status == Status.EXTRACTED && parsers.contains(EmptyParser.class.getName())) {
            status = Status.NO_PARSER;
        }
        String last = parsers.isEmpty() ? "-" : parsers.get(parsers.size() - 1);
        return new Outcome(file.getFileName().toString(), status, metadata.get("Content-Type"),
                last.substring(last.lastIndexOf('.') + 1), handler.toString().length(), detail);
    }

    public static void main(String[] args) throws IOException {
        IntakeParser intake = new IntakeParser();
        for (String name : new String[] {"manuscript.docx", "catalogue.pdf", "publisher-email.eml",
                "legacy-catalogue.txt", "empty.bin", "members-only.pdf", "blob.bin"}) {
            System.out.println(intake.parse(Path.of(name), null));
        }
        System.out.println(intake.parse(Path.of("members-only.pdf"), md -> "reader-pw"));
    }
}

Output:

Outcome[file=manuscript.docx, status=EXTRACTED, mediaType=application/vnd.openxmlformats-officedocument.wordprocessingml.document, parsedBy=XWPFEventBasedWordExtractor, characters=50, detail=null]
Outcome[file=catalogue.pdf, status=EXTRACTED, mediaType=application/pdf, parsedBy=PDFParser, characters=72, detail=null]
Outcome[file=publisher-email.eml, status=EXTRACTED, mediaType=message/rfc822, parsedBy=RFC822Parser, characters=405, detail=null]
Outcome[file=legacy-catalogue.txt, status=EXTRACTED, mediaType=text/plain; charset=windows-1252, parsedBy=TextAndCSVParser, characters=152, detail=null]
Outcome[file=empty.bin, status=EMPTY_FILE, mediaType=application/octet-stream, parsedBy=-, characters=0, detail=InputStream must have > 0 bytes]
Outcome[file=members-only.pdf, status=ENCRYPTED, mediaType=application/pdf, parsedBy=PDFParser, characters=0, detail=Unable to process: document is encrypted]
Outcome[file=blob.bin, status=NO_PARSER, mediaType=application/octet-stream, parsedBy=EmptyParser, characters=0, detail=null]
Outcome[file=members-only.pdf, status=EXTRACTED, mediaType=application/pdf, parsedBy=PDFParser, characters=72, detail=null]

What the class deliberately does not do: it does not isolate the JVM from a hostile document (use PipesForkParser or a service for that), it does not parse embedded documents into separate records (the Embedded Documents & Recursive Parsing page does), and it keeps all text in memory up to 200,000 characters. For files you receive from outside, the safety side is covered on the Security page of this section and on Security and Untrusted Files.

Source: composed for this page from the examples derived from Migrating to Tika 4.x and the tika-example module; the output was produced by running the class against Tika 4.1.0.

Memory, performance and robustness

Concern What to know

Parser construction

Expensive (classpath scan, registry). Build once. A Tika facade or AutoDetectParser per request is a common performance bug.

Memory per parse

The handler’s buffer (a BodyContentHandler(-1) keeps all text), the Metadata, the in-memory part of a caching stream, and whatever the parser loads (POI and PDFBox can hold a whole document; the PDF parser has a maxMainMemoryBytes setting that spills to a temp file past 512 MB by default, per the 4.1.x PDF parser page).

Disk

Temp files from getPath(), getFile() and enableRewind() past 1 MB, deleted on close.

Time

There is no timeout in Parser.parse. In-process parsing cannot be cancelled safely; use TimeoutLimits with Tika Pipes, PipesForkParser or tika-server for a hard limit, as covered on the Robustness, Limits & Timeouts page of this section.

Hostile input

SecureContentHandler guards against zip bombs and deep nesting through OutputLimits, but it cannot stop an infinite loop or an out-of-memory error. Process isolation is the sandbox.

Reuse

Share the parser, never the stream, handler, metadata or context.

Source: Apache Tika 4.1.x, Robustness, Apache Tika 4.1.x, Setting limits and Apache Tika Security Model.

Pitfalls

  • Forgetting to close the TikaInputStream. Temp files and caches then live until garbage collection. Always use try-with-resources.

  • Treating "no exception" as "extracted". EmptyParser and a hand-picked parser set both succeed with no text. Check tk:parsed-by and the text length.

  • Sharing a ContentHandler or Metadata between parses. The text or keys of two documents then mix.

  • Sharing a ParseContext between concurrent parses. It is a plain map and parsers write state into it.

  • Calling a concrete parser with a bare context. Embedded content (and, with RFC822Parser, even the message body in the run above) disappears without an error.

  • Reading before spooling. getPath() and enableRewind() fail after the first read in 4.x.

  • Expecting BodyContentHandler(int) to follow OutputLimits. It does not; build handlers from the context (BasicContentHandlerFactory.newInstance(type, context)) or give the limit to the constructor.

  • Catching TikaException and swallowing it. Record the document, the media type and the message; an unparseable file is data about your intake, not noise.

  • Running untrusted files in-process. An out-of-memory error is not a TikaException. Use a forked parser or a service.

1.x to 4.x at a glance

Topic 1.x (the book, Tika in Action) 2.x 3.x 4.x

Stream argument

InputStream

InputStream

InputStream

TikaInputStream

Parse entry

parser.parse(stream, handler, metadata, context); AutoDetectParser (ch. 5)

Same

Same

Same four arguments; the caller wraps the stream in a TikaInputStream

Parser configuration

TikaConfig and tika-config.xml

Same

Same

TikaLoader and tika-config.json, which replace TikaConfig

Limits

BodyContentHandler(int)

Adds AutoDetectParserConfig (since 2.1.1)

AutoDetectParserConfig

OutputLimits, EmbeddedLimits, TimeoutLimits in the ParseContext

Spooling

Implicit for OLE2 and ZIP

Implicit

Implicit

Only on getPath(), getFile() or enableRewind()

Embedded documents

context.set(Parser.class, parser) (ch. 5 and 8)

Same; RecursiveParserWrapper

Same

Stateless EmbeddedDocumentExtractor; a bare context skips them

Crash isolation

ForkParser (ch. 9)

ForkParser

ForkParser

ForkParser is no longer in tika-core (checked in the 4.1.0 jar); use PipesForkParser

The 2.x and 3.x cells that say "same" were not checked release by release: they rest on the 4.1.x migration guide, which describes the 3.x to 4.x delta, and on the absence of any contrary note in the 3.3.2 guides. AutoDetectParserConfig carries a Javadoc note "since 2.1.1". There is no official 2.x to 3.x migration page. The book covers the parser model in chapter 5; its ideas are current, its signatures are not.

Source: Migrating to Tika 4.x, Apache Tika 3.3.2, Parser guide and Tika in Action, chapter 5.

Checklist

  • Build one AutoDetectParser per application (or per TikaLoader configuration) and share it.

  • Per document: a new TikaInputStream (from a Path or byte[] when you can), a new handler, a new Metadata, a new ParseContext.

  • Close the stream with try-with-resources; never hand the parser a stream you still need open and positioned.

  • Catch ZeroByteFileException, EncryptedDocumentException, the write limit and TikaException separately and record each outcome.

  • Check tk:parsed-by for EmptyParser to find formats that nothing parsed.

  • Supply a Parser and a Detector in the context when you call a concrete parser directly, or go through AutoDetectParser.

  • Give a PasswordProvider from a secret store, never from source code; expect EncryptedDocumentException for a wrong password.

  • Use limits from the ParseContext (OutputLimits, EmbeddedLimits, TimeoutLimits) and handlers built from it.

  • For untrusted files use PipesForkParser or a service; in-process parsing is not a sandbox.

References