We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
源为mongodb数据库时,使用多并发读取,Job在进行split切分算法时,并没有针对query条件进行数据过滤 而是读取全量数据通过 _id 进行切分,这样在collection中数据量较大时,切分会十分缓慢 真实场景:在源表数据为11亿条(存储空间约为4.5T)时,2channel切分耗时40min,3channel切分耗时90min
不针对query条件做过滤是有什么其它的考量吗
The text was updated successfully, but these errors were encountered:
No branches or pull requests
源为mongodb数据库时,使用多并发读取,Job在进行split切分算法时,并没有针对query条件进行数据过滤
而是读取全量数据通过 _id 进行切分,这样在collection中数据量较大时,切分会十分缓慢
真实场景:在源表数据为11亿条(存储空间约为4.5T)时,2channel切分耗时40min,3channel切分耗时90min
不针对query条件做过滤是有什么其它的考量吗
The text was updated successfully, but these errors were encountered: