Optimize the CSV/TSV parsing

[sharmrj]

The current bottleneck in our benchmarks is the parsing and file reading, which eclipses the rest of the operations. This can be seen in the flame graph for the chipotle example (generated using https://www.speedscope.app/)

The proposed solution is to use a state-machine. We'll avoid using Attoparsec to minimize the memory footprint (since we don't need backtracking and Attoparsec stores the entire input in memory in case it needs to backtrack).
The general outline of the approach can be found here: https://gist.github.com/sharmrj/8f745afd04fcd066851627e3e59fe5eb

[mchav]

I'll start working on a test suite for this and we can make this the main path for CSV.

[sharmrj]

I can start working on a way to pass arbitrary separator character and specialize that into a tsv and csv function like the current implementation does.

Regarding type inference: when I raised the PR, I commented out two tests that compared the old implementation to the new one because the type inference worked differently for some reason. But upon checking it now it seems those tests are passing. I think it may have to do with the type inference changes in #107. Afaict the type inference in the old and new implementations are at parity now.

Also, @mchav lmk if there's something I can pick up related to writing the test suite.

[mchav]

This implementation behaves strangely for large CSVs. We should go over it.

[sharmrj]

Sure. How large do the CSVs have to be until you start seeing the behavior? Does it happen for the compiled version or just the interpreted version? I'd like to look at it before we go over the implementation.

Preliminarily, we mmap the csv file, and we also allocate memory the size of the file, fill it with the indices of the delimiters, and cast it into a Storable Vector. Then we slice it into a vector of length equal to the actual number of delimiters which is usually way lower than the size of the file.
We generate the final vector of columns by iterating through the vector we created above and taking slices of the mmapped file.

It doesn't seem like we're doing a bunch of intermediate allocations from looking at the memory profile, but we canmaybe use some other tools to make sure of that.