Show HN: Insights.hn – Real-time Hacker News posts and comments analytics (opens in new tab)

(insights.hn)

4 pointsburmecia1y ago9 comments

Hi HN, I'm excited to share a project I've been working on for the past few months: https://insights.hn/, a real-time analytics platform for Hacker News posts and comments.

Why did I build this project?

I built this project to help me understand the trends and patterns in the HN community. I've always been fascinated by the discussions and insights shared on HN, and I wanted to create a tool that could help me explore and analyze this data in real-time.

How it works?

The platform collects data from the Hacker News API and processes it in ClickHouse database to provide insights on the top posts, comments, and users on HN. The data is updated in nearly real time, so you can see the latest trends and discussions as they happen.

I hope you find this project useful and interesting. I would love to hear your feedback and suggestions for how I can improve it further.

Previous similar HN threads:

- https://news.ycombinator.com/item?id=3745965

- https://news.ycombinator.com/item?id=34110321

9 comments

jimmySixDOF1y ago

Love that you have been improving this idea since 2012 !! I think enriching HN is a super interesting problem I have also thought about quite a lot. The realtime angle looks really good fun to see likes/page position as it happens. I notice you show "Top Comments" in the details and wonder how you determine this since afaik there is no concept of likes for individual comments only stories from the API. I need go looking at the API again I found one method somewhere that would return the whole story with all comments fully sorted in order together in a single json so I guess if the "best" comments float to the top of their branch you could track that over time. Just curious and wanted to say this looks amazing !!

burmeciaOP1y ago

Thanks, the "Top Comments" is sourced from the "kids" field of the "items" endpoint. As the API docs (https://github.com/HackerNews/API?tab=readme-ov-file#items) says, it is "in ranked display order" so I just used it to sort the top comments.

jimmySixDOF1y ago

Yah that makes sense and after looking I could not find the API call that pre sorted each thread (it must not have been native algolia or firebase) - and without that the way you are doing it sure beats recursively reconstructing the thread. All the best and I look forward watching any features you add in the future!

zX41ZdbW1y ago

This is really great!

I can suggest more ideas that will be easy to add:

- a spark line or heat map of upvotes for every thread: https://github.com/ClickHouse/ClickHouse/issues/59020

- a built-in SQL editor for custom queries;

If you need help in supporting or hosting it, write to milovidov at clickhouse.com

burmeciaOP1y ago

Thanks for the ideas, heat map looks great! Will also consider add custom queries, it can be quite useful.

PeterZaitsev1y ago

One more thing, I would suggest to make comparisons linkable - I was looking to share with folks the graph of how MySQL vs PostgreSQL is doing, sadly the URL still links to generic keyword comparison tool

burmeciaOP1y ago

That's a good idea, will put it on to-do list.

PeterZaitsev1y ago

Very cool. I was surprised though to see keywords seems to be hardcoded, would be great to be able to graph and keywords and phrases. Any decent search engine should be able to support that

burmeciaOP1y ago

yes, the keywords search is using full-text searching and it is not efficient so I hardcoded them. Will consider switch to use embedding, that will be more efficient and accurate, also can support custom keywords search. Thanks for your advice.

j / k navigate · click thread line to collapse

Show HN: Insights.hn – Real-time Hacker News posts and comments analytics (opens in new tab)

(insights.hn)

4 pointsburmecia1y ago9 comments

Hi HN, I'm excited to share a project I've been working on for the past few months: https://insights.hn/, a real-time analytics platform for Hacker News posts and comments.

Why did I build this project?

How it works?

I hope you find this project useful and interesting. I would love to hear your feedback and suggestions for how I can improve it further.

Previous similar HN threads:

- https://news.ycombinator.com/item?id=3745965

- https://news.ycombinator.com/item?id=34110321

9 comments

jimmySixDOF1y ago

burmeciaOP1y ago

jimmySixDOF1y ago

zX41ZdbW1y ago

This is really great!

I can suggest more ideas that will be easy to add:

- a spark line or heat map of upvotes for every thread: https://github.com/ClickHouse/ClickHouse/issues/59020

- a built-in SQL editor for custom queries;

If you need help in supporting or hosting it, write to milovidov at clickhouse.com

burmeciaOP1y ago

Thanks for the ideas, heat map looks great! Will also consider add custom queries, it can be quite useful.

PeterZaitsev1y ago

burmeciaOP1y ago

That's a good idea, will put it on to-do list.

PeterZaitsev1y ago

Very cool. I was surprised though to see keywords seems to be hardcoded, would be great to be able to graph and keywords and phrases. Any decent search engine should be able to support that

burmeciaOP1y ago

j / k navigate · click thread line to collapse