Skip to content

Conversation

@southernriver
Copy link
Contributor

Param scanAllFiles Used to check whether all the data files should be processed, or only added files.Here we should replace scanAllFiles to !scanAllFiles.

Param scanAllFiles Used to check whether all the data files should be processed, or only added files.Here we should replace  scanAllFiles to  !scanAllFiles.
@github-actions github-actions bot added the core label Jun 3, 2021
@southernriver
Copy link
Contributor Author

cc @rdblue

@rdblue
Copy link
Contributor

rdblue commented Jun 12, 2021

This looks correct to me, but I don't know why tests would be passing with a logic error like this. I approved running tests and we'll see if they still pass. If they do, then that indicates that this path isn't tested.

@rdblue
Copy link
Contributor

rdblue commented Jun 18, 2021

@SreeramGarlapati, FYI. Looks like there may be some issues with MicroBatches. I'm committing this fix but we should probably have some good testing for your Spark 3 work.

@rdblue rdblue merged commit 7798094 into apache:master Jun 18, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants