Whylogs vs Recap? Whats the differences and similarities?

[nehiljain]

I was looking at Whylogs. Is there a big chunk of work that is overlap and reuseable by either codebase?

[criccomini]

Wow, this is really interesting. I am not familiar with it. From a cursory look, it appears that whylogs is more focused on ML and the Pandas ecosystem. Recap is much more about the infra level (DBs, filesystems, streaming systems). That said, it has some similar features.

[nehiljain]

Yeah, I agree that the semantics are different. The two libraries want to exist in different parts of the big puzzle, but based on the underlying abstractions to implement these there may be reusable ideas and code.

[criccomini]

Yea, totally. I have a strong opinion on schema, but for lineage and profiling, I think it's worth digging into what they're doing to see if we can use their ideas and/or code.