`ItemReader`s: databases
|
This section documents the current Spring Batch line — 6.0.x, on Spring Framework 7 and Spring Boot 4.1.x, with a Java 17+ baseline — as published at the Spring Batch reference documentation. No specific patch version is pinned. Some surfaces (Spring Cloud Task and Spring Cloud Data Flow orchestration, the deployer-based partition handler, and JSR-352) are linked, not documented in depth. This content was generated with the assistance of AI and should be verified against the official documentation before being relied on in production. |
Database input comes in two shapes: hold a cursor open and stream rows from it, or issue a page query repeatedly. The choice determines restartability, thread-safety and how long a transaction stays open.
Cursor vs. paging
| Cursor | Paging | |
|---|---|---|
How it works |
One |
Repeated `SELECT`s with a sort key and an offset or a key predicate; each page is a separate round-trip. |
Transaction / connection |
One long-lived connection and result set for the whole step. |
Short queries; nothing is held between pages. |
Restart |
Re-executes the query and skips forward to the saved row number. |
Re-executes from the saved page — naturally stateless. |
Threads |
Bound to one thread: a |
Safe to use from several threads if the sort key is unique and deterministic. |
Cost |
Cheapest per row; no repeated query planning. |
More round-trips; deep offsets degrade unless the sort key is indexed. |
Use it when |
A single-threaded step reads a large, stable table. |
The step is multi-threaded or partitioned, or the source changes under a long transaction. |
The sort key is what makes paging correct: it must be unique and stable, otherwise pages overlap or skip rows.
JdbcCursorItemReader
@Bean
public JdbcCursorItemReader<Trade> cursorTradeReader(DataSource dataSource) {
return new JdbcCursorItemReaderBuilder<Trade>()
.name("cursorTradeReader")
.dataSource(dataSource)
.sql("SELECT id, isin, quantity, price FROM trade WHERE trade_date = ?")
.preparedStatementSetter(new ArgumentPreparedStatementSetter(
new Object[] { Date.valueOf(LocalDate.now().minusDays(1)) }))
.rowMapper(new TradeRowMapper())
.fetchSize(1000) // JDBC driver hint: rows per network round-trip
.verifyCursorPosition(false)
.build();
}
fetchSize is the single most valuable setting here — the default of many drivers is to fetch a handful of
rows at a time, or (on some) to buffer the entire result set in memory.
JdbcPagingItemReader
The reader needs a PagingQueryProvider, which builds the per-page SQL for the specific database.
SqlPagingQueryProviderFactoryBean detects the dialect from the DataSource:
@Bean
public PagingQueryProvider tradeQueryProvider(DataSource dataSource) throws Exception {
SqlPagingQueryProviderFactoryBean factory = new SqlPagingQueryProviderFactoryBean();
factory.setDataSource(dataSource);
factory.setSelectClause("SELECT id, isin, quantity, price");
factory.setFromClause("FROM trade");
factory.setWhereClause("WHERE trade_date = :tradeDate");
factory.setSortKey("id");
return factory.getObject();
}
@Bean
@StepScope
public JdbcPagingItemReader<Trade> pagingTradeReader(
DataSource dataSource, PagingQueryProvider tradeQueryProvider,
@Value("#{jobParameters['run.date']}") LocalDate runDate) {
return new JdbcPagingItemReaderBuilder<Trade>()
.name("pagingTradeReader")
.dataSource(dataSource)
.queryProvider(tradeQueryProvider)
.parameterValues(Map.of("tradeDate", Date.valueOf(runDate)))
.pageSize(1000)
.rowMapper(new TradeRowMapper())
.build();
}
A good default is pageSize equal to the chunk size, so one page fills one chunk.
JPA and Hibernate readers
JpaCursorItemReader streams a JPQL query; JpaPagingItemReader pages it. Both need an
EntityManagerFactory:
@Bean
public JpaPagingItemReader<Trade> jpaTradeReader(EntityManagerFactory entityManagerFactory) {
return new JpaPagingItemReaderBuilder<Trade>()
.name("jpaTradeReader")
.entityManagerFactory(entityManagerFactory)
.queryString("SELECT t FROM Trade t WHERE t.tradeDate = :tradeDate ORDER BY t.id")
.parameterValues(Map.of("tradeDate", LocalDate.now().minusDays(1)))
.pageSize(500)
.build();
}
@Bean
public HibernateCursorItemReader<Trade> hibernateTradeReader(SessionFactory sessionFactory) {
return new HibernateCursorItemReaderBuilder<Trade>()
.name("hibernateTradeReader")
.sessionFactory(sessionFactory)
.queryString("FROM Trade t WHERE t.tradeDate = :tradeDate ORDER BY t.id")
.parameterValues(Map.of("tradeDate", LocalDate.now().minusDays(1)))
.useStatelessSession(true) // no first-level cache: the right default for batch
.build();
}
Two persistence-specific hazards, both amplified by batch volumes:
-
the first-level cache grows with every entity read, so a long-running JPA step leaks memory unless the context is cleared per chunk —
useStatelessSession(true)for Hibernate, or a chunk listener callingentityManager.clear(); -
lazy associations turn one query into N. Fetch what the processor needs in the query itself.
Both are covered in depth in Hibernate Reference; the Spring Data JPA side is Spring Data JPA — neither is restated here.
StoredProcedureItemReader
When the query is a procedure returning a cursor:
@Bean
public StoredProcedureItemReader<Trade> procedureTradeReader(DataSource dataSource) {
return new StoredProcedureItemReaderBuilder<Trade>()
.name("procedureTradeReader")
.dataSource(dataSource)
.procedureName("read_trades_for_date")
.parameters(new SqlParameter[] { new SqlParameter("p_trade_date", Types.DATE) })
.preparedStatementSetter(new ArgumentPreparedStatementSetter(
new Object[] { Date.valueOf(LocalDate.now().minusDays(1)) }))
.refCursorPosition(2) // for databases that return a REF CURSOR out parameter
.rowMapper(new TradeRowMapper())
.build();
}
Spring Data readers
RepositoryItemReader calls a paging repository method, which keeps the query definition next to the domain
model:
@Bean
public RepositoryItemReader<Trade> repositoryTradeReader(TradeRepository tradeRepository) {
return new RepositoryItemReaderBuilder<Trade>()
.name("repositoryTradeReader")
.repository(tradeRepository)
.methodName("findByTradeDate")
.arguments(List.of(LocalDate.now().minusDays(1)))
.sorts(Map.of("id", Sort.Direction.ASC))
.pageSize(500)
.build();
}
MongoPagingItemReader (and its cursor sibling) does the same over a MongoTemplate:
@Bean
public MongoPagingItemReader<Trade> mongoTradeReader(MongoTemplate mongoTemplate) {
return new MongoPagingItemReaderBuilder<Trade>()
.name("mongoTradeReader")
.template(mongoTemplate)
.collection("trade")
.jsonQuery("{ 'tradeDate': ?0 }")
.parameterValues(List.of(LocalDate.now().minusDays(1)))
.sorts(Map.of("_id", Sort.Direction.ASC))
.pageSize(500)
.targetType(Trade.class)
.build();
}
The driving-query pattern
Instead of selecting whole rows with all their joins, select only the keys, and let the processor hydrate each one. The reader’s query stays small and index-only, the transaction stays short, and the work of assembling an object graph moves to a place where it can be parallelised:
@Bean
public JdbcPagingItemReader<Long> tradeIdReader(DataSource dataSource,
PagingQueryProvider tradeIdQueryProvider) {
return new JdbcPagingItemReaderBuilder<Long>()
.name("tradeIdReader")
.dataSource(dataSource)
.queryProvider(tradeIdQueryProvider) // SELECT id FROM trade WHERE ... ORDER BY id
.pageSize(1000)
.rowMapper((resultSet, rowNumber) -> resultSet.getLong("id"))
.build();
}
@Bean
public ItemProcessor<Long, Trade> hydratingProcessor(TradeRepository tradeRepository) {
return tradeRepository::loadWithPositions; // one keyed lookup per item
}
The trade-off is one extra query per item, so it pays off when the hydration is selective or the driving query would otherwise be an expensive multi-way join. The pattern is described in Common batch patterns; the query technique itself is in SQL Queries.
Thread safety
No stateful reader is thread-safe. In a multi-threaded step, either use a paging reader with a unique sort key
or wrap the reader in a SynchronizedItemStreamReader — see
Scaling & parallel processing, which also
covers giving each partition its own key range instead of sharing one reader.