undefined | Better HN

Skip to content

Top New Best Ask Show Jobs

undefined | Better HN

0 pointsAznHisoka9y ago0 comments

Ruby and Sidekiq as the messaging queue

Postgres to store the data

Elasticsearch as a search index.

My ES cluster has around 10 nodes, 64 GB RAM, quad-core.

Postgres database cluster is 4 nodes, 1 TB, 64 GB RAM, quad-core.

800 crawler threads distributed across 10 dedicated servers.

0 comments

outpan9y ago

Thanks a lot! This sounds reasonable. Did you guys look into professional services for this?

AznHisokaOP9y ago

Nope. We have lots of custom needs.

j / k navigate · click thread line to collapse