undefined | Better HN

0 pointsSdtEE14d ago0 comments

This starts from my frustration when opening large CSV files, but later evolved to a log/data analyzer that loads arbitrary format in constant time (O(1)).

https://github.com/Verticalysis/Hitomi

The secret: I engineered an incremental combinatorial parser capable of processing customized format from a steam. Any inputs, including file or the stdout from a command, are first chunked and then fed to the pipeline. The UI is ready when the first small chunk is processed.

Other highlights: 2-mode filter, one with a convenient UI and the other is based on an extensible DSL for complex cases;

Timeline mode scrollbar, a secret weapon for log or time series analysis;

Column widths fit to content automatically;

Native code, no web bloat;

Cross-platform (currently Windows and Linux, MacOS WIP).

If you are tired of all the quirks Excel have when working with CSV files, you'll gonna love it!

0 comments

klntsky14d ago

Does it update the parser incrementally?

SdtEEOP13d ago

I'm not sure what do you mean by "update". If you mean the state of the parser - of course, and I did some trick to reduce the supposedly infinite states for arbitrary format to a fixed set of 4. If you mean by the parser program it self -No. The parser was developed with Dart, a statically compiled language, with no ability to modify the program itself.

j / k navigate · click thread line to collapse

0 comments

klntsky14d ago

Does it update the parser incrementally?

SdtEEOP13d ago

j / k navigate · click thread line to collapse