8,570 questions
Score of 0
0 answers
159 views
Best approach for full-file aggregation before processing 4M records — memory vs performance vs lock contention
Business Requirement
I need to process large CSV files (4M records) which contains ID and other contact info as batch job. The batch updates three SQL Server tables.Core constraint — cannot use line-...
- reputation score 1
Advice
0
votes
0
replies
72
views
Investigating message duplication in Kafka topics + Spring Batch (KafkaItemReader)
Context
I have a Spring Batch project acquiring messages from Kafka topics using KafkaItemReader, every 4-hours scheduling, for a total of hundred-thousands messages per day. Unfortunately, I don't ...
- reputation score 27165
Best practices
0
votes
0
replies
113
views
What is the Best Practice for Counting Processed Items to Print out a Summary
Hello I would like to know how to best Count Processed Items to Print out a Summary.
I find it quite Complicated because Rolledbacked and Skipped Items should not be Counted. Only successful ones. I ...
- reputation score 431
Score of 0
0 answers
50 views
noRollback equivalent in Spring Batch 6.X
I am currently migrating a project from Spring Batch 5 to 6 and was wondering whether there is an equivalent to the noRollback method that used to be available in Spring Batch 5. Is the functionality ...
- reputation score 3
Score of 2
1 answer
95 views
Order of listeners with Spring Batch 6
My project is using Spring Batch version 6.0.3.
My step has 2 StepExecutionListener like this:
@Bean(name = "myStep")
public Step myStep(JobRepository jobRepository, ...
- reputation score 461
Score of 2
1 answer
200 views
Spring Boot 4.0.6 and Spring Batch 6 - the Spring Batch tables are not being created
I am using Spring Boot 4.0.6 with Spring Batch and MySQL with Docker; however, the Spring Batch tables (e.g. BATCH_JOB_EXECUTION, etc.) in the spring_batch schema are not being created automatically.
...
- reputation score 21
Score of 2
0 answers
228 views
spring.batch.jdbc.initialize-schema=always not working anymore
Although, I don't use anotation @EnableBatchProcessing, but my application not use config of spring batch in file application.yml to create schema of metadata's spring batch. How can I fix it?
The ...
- reputation score 77
Best practices
0
votes
0
replies
66
views
Scheduling tasks inside a spring batch job using ThreadPoolTaskScheduler , how much can a jvm handle?
i have a spring batch job that runs every 10 minutes, gets the rows inserted in a table within the last 10 minutes and based on the insertion time of each row, it schedules an sms for them
once they ...
- reputation score 21
Score of 3
1 answer
144 views
Parallel Spring Batch jobs mixing up data
I am trying to run a separate instance of a Spring Batch job for every file, up to 20 at a time. For some reason, the job is being launched with the same parameters multiple times (same filename + ...
- reputation score 2310
Score of 0
1 answer
249 views
Getting issues writing test to run End-to-End job flow using JobOperatorTestUtils after Spring Batch 6.0.2 migration
I have recently upgraded my test app to Spring Boot 4.0.2 (transitively upgraded my batch version to 6.0.2). Since the JobLauncherTestUtils is deprecated since 6.0 version, I'm trying to use the ...
- reputation score 1
Best practices
0
votes
0
replies
130
views
How to automatically detect and map multiple CSV formats into a single entity without unnecessary Java code?
I am building a Spring Boot 4.0.1 application using Spring Batch 6.0.1 to ingest bank transactions from multiple CSV sources (specifically N26 and Revolut).
The Problem
The files are all dropped into ...
- reputation score 53
Score of -3
1 answer
234 views
Spring Batch metadata tables are not automatically created in Spring Boot 4
I am trying to set up a Spring Batch project using Spring Boot 4.0.0
The application starts successfully, but the standard Spring Batch meta-data tables (e.g., BATCH_JOB_INSTANCE, BATCH_JOB_EXECUTION, ...
- reputation score 3
Best practices
0
votes
2
replies
87
views
Should we use GCP Cloud Run (not Cloud Run Jobs) for Java Spring Batch?
Currently, I have a Java Spring Batch source code that contains multiple batch jobs, with their execution schedules configured directly in the code. The application is currently running on a Windows ...
- reputation score 1
Score of 1
2 answers
469 views
Spring Boot Batch 6.x: Job Parameters are null in Reader despite using @StepScope
I'm trying to pass job parameters from a REST controller to a batch job reader in Spring Boot Batch 6.x, but the parameters are always null in the reader. I'm using @StepScope and SpEL expressions as ...
- reputation score 23
Best practices
0
votes
2
replies
62
views
What is the most efficient way to validate file data against Couchbase documents using Spring Batch— N1QL timestamp range, KV lookups, or USE KEYS?
I have a validation use-case where an input file contains a list of records that were updated at a certain timestamp. I need to verify that the data in the file matches the data stored in Couchbase.
...
- reputation score 71
