Show HN: Using SIMD to decode and encode gigabytes of varints per second in Rust (opens in new tab)

(github.com)

6 pointsascom5y ago2 comments

2 comments

Can somebody explain what this is actually doing? I am quite confused

This is a Rust library for reading and writing the LEB128 integer compression format. LEB128 is a representation of arbitrary-size integers: https://en.wikipedia.org/wiki/LEB128

As I understand it, this library is optimized to avoid branching (which incur an overhead) and take advantage of SIMD instructions (which process data in parallel).

j / k navigate · click thread line to collapse

2 comments

herohamp5y ago

Can somebody explain what this is actually doing? I am quite confused

TheAsprngHacker5y ago

This is a Rust library for reading and writing the LEB128 integer compression format. LEB128 is a representation of arbitrary-size integers: https://en.wikipedia.org/wiki/LEB128

As I understand it, this library is optimized to avoid branching (which incur an overhead) and take advantage of SIMD instructions (which process data in parallel).

j / k navigate · click thread line to collapse