undefined | Better HN

0 pointsdiscardorama11y ago0 comments

I read that. But how does that compare with the nodes they're comparing against ("10x fewer nodes")?

0 comments

rxin11y ago

The old entry had 10Gb/s <full-duplex> (40 nodes/rack 160Gbps rack to spine. 2.5:1 subscription), 64GB of RAM, and 12 x 3TB SATA.

The network part is probably the most important one here, and both have comparable network.

discardoramaOP11y ago

Since each node was handling 500GB of data (roughly), I think the disk speed may have been a more critical factor since each node had 244GB of memory. Their nodes used SSDs; the older nodes used spinning rust. The seek times alone will be a killer.

rxin11y ago

Not sure why you mentioned seek time. In large scale, distributed sorting, I/O is mostly sequential.

1 more reply

Twirrim11y ago

3TB SATA would indicate spinning rust too, so slower storage. It's far from an apples to apples comparison.

j / k navigate · click thread line to collapse

0 comments

rxin11y ago

The old entry had 10Gb/s <full-duplex> (40 nodes/rack 160Gbps rack to spine. 2.5:1 subscription), 64GB of RAM, and 12 x 3TB SATA.

The network part is probably the most important one here, and both have comparable network.

discardoramaOP11y ago

rxin11y ago

Not sure why you mentioned seek time. In large scale, distributed sorting, I/O is mostly sequential.

1 more reply

Twirrim11y ago

3TB SATA would indicate spinning rust too, so slower storage. It's far from an apples to apples comparison.

j / k navigate · click thread line to collapse