Streaming Large Result Sets in Firebird with Spring Boot for Real-Time Processing

February 05, 2026

When you work with large tables in Firebird, loading millions of records at once is inefficient and can exhaust your system’s memory. Instead of retrieving everything, streaming allows you to process data in chunks - directly as it’s being read from the database.
This article explains how to stream data efficiently in Spring Boot using Firebird JDBC, ensuring your application remains responsive and scalable.

1. Why Use Streaming

Traditional queries load all rows into memory before processing begins. For large datasets, this approach causes:

Out-of-memory errors.
Slow response times.
High CPU load.

Streaming solves these problems by fetching data row-by-row from the database and processing it immediately — ideal for analytics, reporting, or ETL tasks.

2. Enable Streaming with Spring Data JPA

Spring Data JPA supports streaming results via Stream<T>. To enable it, define a repository method that returns a stream:

public interface EmployeeRepository extends JpaRepository<Employee, Long> {
    @Query("SELECT e FROM Employee e")
    Stream<Employee> streamAll();
}

Then process data like this:

try (Stream<Employee> stream = employeeRepository.streamAll()) {
    stream.forEach(employee -> processEmployee(employee));
}

This approach reads records one by one, keeping memory usage minimal.

3. Use Transactional Streaming

To maintain an open connection while streaming, wrap the operation in a read-only transaction:

@Transactional(readOnly = true)
public void processLargeDataset() {
    try (Stream<Employee> employees = employeeRepository.streamAll()) {
        employees.forEach(this::processEmployee);
    }
}

This ensures the stream remains active for the entire transaction without closing prematurely.

4. Set Fetch Size for Efficient Data Flow

Firebird’s JDBC driver allows configuring fetch size to optimize how many rows are retrieved per batch from the database:

@PersistenceContext
private EntityManager entityManager;

public void fetchWithCustomSize() {
    Session session = entityManager.unwrap(Session.class);
    session.doWork(connection -> {
        try (PreparedStatement ps = connection.prepareStatement("SELECT * FROM EMPLOYEE")) {
            ps.setFetchSize(200); // fetch 200 rows per network call
            ResultSet rs = ps.executeQuery();
            while (rs.next()) {
                // process each row immediately
            }
        }
    });
}

The optimal fetch size depends on your system’s memory and network latency.

5. Combine Streaming with Real-Time Processing

Streaming becomes powerful when combined with real-time transformations or event pipelines.
For example, as you stream data, you can send updates to a dashboard or another service:

public void streamAndSendToQueue() {
    try (Stream<Employee> stream = employeeRepository.streamAll()) {
        stream.forEach(employee -> kafkaProducer.send("employee-updates", employee));
    }
}

This makes it possible to process large Firebird datasets continuously without downtime.

6. Monitor Streaming Performance

Use Spring Boot Actuator to monitor connection health and query performance:

management.endpoints.web.exposure.include=health,metrics
management.endpoint.metrics.enabled=true

Then access metrics like /actuator/metrics/hikaricp.connections.active to verify streaming stability.
If streaming slows down, tune fetch size, indexes, or database cache settings.

Ads go here

#ads