I wanted so much to use csvkit and all the features it had, but its horrendous performance made it unscalable and therefore the more I used it, the more technical debt I accumulated.
This was one of the reasons I wrote zsv (https://github.com/liquidaty/zsv). Maybe csvkit could incorporate the zsv engine and we could get the best of both worlds?
Examples (using majestic million csv):
---
csvcut -c 1,3 = 5.3 seconds
zsv select -n -- 1 3 = 0.19 seconds
28x faster
---
csvsql --query "select count(*) from file" file.csv = 148 seconds
zsv sql "select count(*) from data" file.csv = 0.68 seconds
216x faster
---
