🐛Bug: Problem with special Characters in Stream Names

[tinomerl]

Preface: This isn't directly a PyAirbyte Bug and more of an OS Bug. Im using Ubuntu 22.04 on WSL.

Description

In some systems custom Tables or the like are possible. If one of these custom tables has a special character in it's name the jsonl.gz file, used to import everything into the DuckDB cache, cannot be created. Therefore the whole sync fails. We have a case where a custom table in a customer's system which contains a backward slash \. Since we are using a custom connector for this customer and it's the system of said customer, i'm not able to share logs. maybe at a later stage we can release the connector to the main airbyte repo.

Possible Fix

In file_writers.py the method _get_new_cache_file_path generates a unique name by adding a uuid / batch_id. By my understanding it should be possible to add a line which would remove special characters in the jsonl.gz file.

Willing to create PR

Yes. Eager to try something out.