Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

FS Crawler freezes when crawling a 0 byte TXT file #834

Closed
dansfelix opened this issue Oct 30, 2019 · 2 comments · Fixed by #1137
Closed

FS Crawler freezes when crawling a 0 byte TXT file #834

dansfelix opened this issue Oct 30, 2019 · 2 comments · Fixed by #1137
Assignees
@dadoonet
Labels
bug For confirmed bugs
Milestone
2.7

Comments

@dansfelix
Copy link

Hello all,

I'm trying to use FS Crawler 2.6 in a Windows Server machine to index a huge number of files in my company. It's a very large Windows folders tree in a network drive: 14,92 Tb size, 7,2M files in 2,3M folders. Data are in a remote filer, in the same data center.

I realized that FS Crawler stops responding at some point of the process. Even if running in TRACE mode, there is no response, it only freezes.

After using CONTROL + C command, I navigate in the error stack to find the original problem: a 0 byte TXT file caused the exception.

Think it is easy to fix in the next versions.

Thank you
@dadoonet dadoonet self-assigned this Oct 30, 2019
@dadoonet dadoonet added the bug For confirmed bugs label Oct 30, 2019
@dadoonet dadoonet added this to the 2.7 milestone Oct 30, 2019
@den-kip
Copy link

den-kip commented Dec 12, 2019

Has this issue been fixed?

@den-kip den-kip mentioned this issue Dec 12, 2019
Open
@dadoonet
Copy link
Owner

No. Wanna contribute a patch?

@dadoonet dadoonet mentioned this issue Mar 30, 2021
Closed
dadoonet added a commit that referenced this issue Apr 23, 2021
@dadoonet
Ignore errors when parsing a 0 byte file
7d3a5f4 
FSCrawler is freezing when crawling a 0 byte TXT file or it complains with a `InputStream must have > 0 bytes` message.
This fix will ignore such a case so the crawling can continue.

Closes #834
Closes #1084
@dadoonet dadoonet mentioned this issue Apr 23, 2021
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@dadoonet dadoonet

Labels
bug For confirmed bugs
Projects
None yet
Milestone
2.7
Development

Successfully merging a pull request may close this issue.

Ignore errors when parsing a 0 byte file dadoonet/fscrawler
3 participants
@dadoonet @den-kip @dansfelix