Skip to content

[LowLow Priority Feature Request] BSON writing  #116

@jamesb93

Description

@jamesb93

For big datasets (either made in a supported CCE or elsewhere), being able to read/write BSON would help to speed up parsing and the general workflow. With some cursory tests in Python:

A smallish dataset in JSON format on disk at around 80mb and in BSON format at around 25mb parses around 7 times faster.

A large dataset in JSON format on disk at around 3.84GB and 1.74GB in BSON format parses around 4 times faster.

Obviously, there is the benefit of disk space being less consumptive too in BSON format.

In either case I am working with around 1000 short audio files, so the scope of working musically with such a collection is not out of the question. Anyway, this is more of a "could be nice" than something that is essential whatsoever.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions