Hacker_news_scraper

1 pointsvstuart5y ago2 comments

hacker_news_scraper | https://github.com/victoriastuart/hacker_news_scraper

A Python 3 script for scraping the Hacker News feed, filtering that content by

* number of points, and/or

* number of comments, and/or

* excluding posts {dead | flagged | youtube | wikipedia | ...} according to a keywords list

Run via ~/.bashrc alias or crontab.

Experimental code; I will monitor it locally (scheduled to run {06:00 | 18:00} daily, starting 2020-04-14) for bugs, etc.

2 comments

Etheryte5y ago

Not to discount your effort, but is there a benefit for scraping the data when HN offers an official API[1]? Does the API not expose all the data you need?

[1] https://github.com/HackerNews/API

vstuartOP5y ago

Good to know; thank you. :-) I suppose the advantages (mine) may include keyword exclusions (I'm not sure if that functionality is available in the API), and local hosted (hence, more "private" -- whatever that means these days ;-) code.

j / k navigate · click thread line to collapse