MultiResourceItemWriter creates files with varying line count #1722
Milestone
Comments
|
Raja Rajan R S commented Note : <beans:property name="delimitter" value=""|"" /> |
|
Brice Dutheil commented Actually this does not help with different writers. I think the real issue is that the check is done after writing, while it should do that before, and eventually create a new resource if needed. if (!opened) {
File file = setResourceToDelegate();
// create only if write is called
file.createNewFile();
Assert.state(file.canWrite(), "Output resource " + file.getAbsolutePath() + " must be writable");
delegate.open(new ExecutionContext());
opened = true;
}
delegate.write(items);
currentResourceItemCount += items.size();
if (currentResourceItemCount >= itemCountLimitPerResource) {
delegate.close();
resourceIndex++;
currentResourceItemCount = 0;
setResourceToDelegate();
opened = false;
}EDIT : unfortunately there's the transaction issue here, that might cause problems : Middling with the code a litle bit I came up with these changes, still the transaction fails because the stream is closed, but I don't know yet how to get manage that part. public void write(List<? extends T> items) throws Exception {
openNewResourceIfNecessary();
currentResourceItemCount += items.size();
if (currentResourceItemCount >= itemCountLimitPerResource) {
closeCurrentResource();
openNewResourceIfNecessary();
}
delegate.write(items);
}
private void closeCurrentResource() throws IOException {
delegate.close();
resourceIndex++;
currentResourceItemCount = 0;
setResourceToDelegate();
opened = false;
}
private void openNewResourceIfNecessary() throws IOException {
if (!opened) {
File file = setResourceToDelegate();
// create only if write is called
file.createNewFile();
Assert.state(file.canWrite(), "Output resource " + file.getAbsolutePath() + " must be writable");
delegate.open(new ExecutionContext());
opened = true;
}
} |
spring-projects-issues
added
status: waiting-for-triage
|
Duplicated by #4660 MCVE: #4660 (comment) |
fmbenhassine
added
type: bug
and removed
status: waiting-for-triage
fmbenhassine
changed the title
fmbenhassine
changed the title
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Raja Rajan R S opened BATCH-1868 and commented
When it is required to create output files with MultiResourceItemWriter with all files having the exact line count as mentioned in the itemCountLimitPerResource attribute, Spring batch creates files with varying line counts (varying between the itemCountLimitPerResource value and commit-interval value). The attached class FixedLinesMultiResourceItemWriter.java creates the multiple files with every file having exactly same no of lines as defined by itemCountLimitPerResource attribute.
The usage of the class in the configuration should be as given below.
where the value for the property fields should be the comma seperate list of field names to be printed in the output file. Field names should always match the name of the attributesas declared in the POJO class definition. Say for example for a Customer Name bean class the value of "fields" property can be "firstName,LastName,MiddleName,title" if these attributes are declared in the bean class.
This has been tested in our spring batch application and confirmed working fine.
Affects: 2.1.8
Attachments:
The text was updated successfully, but these errors were encountered: