csvsort seems to hang on large files/sort jobs

[eads]

I have a rather large (42m) CSV. I'm sitting here looking crazy waiting while csvsort has been running for over 10 minutes trying to sort the file. To perform the same sort in Excel or LibreOffice is slow, but still less than a minute.

[onyxfish]

Yep. It's the type inference--it has to buffer the whole file into memory, iterate over it to figure out what the type is, iterate over it again to coerce to the correct type and then finally sort all that data. It's crazy slow. Not sure what to do about it.

[eads]

Bummer. I wonder how CSVFix handles it, and how the performance is.

[gpoulter]

How about an option to skip type coercion? This would be similar to specifying that all columns are "text" when opening in LibreOffice.

[onyxfish]

You can now pass --no-inference to disable type inference on csvsql. This option still needs to be extended to csvsort, but we've got a ticket for it in #222, so I'm closing this one as a dupe.