There are ways to read HN other than this website. However, I have not found one that meets all of these requirements:
- Self-hosted.
- Offline access of data.
- Query data via SQL.
- Full text search of stories and comments.
- Notification of replies to comments.
1) A tool that maintains a copy of the HN API[1] in an SQLite database, with indexing and full-text search[2]. This supports the development of the other tools.
2) A command line tool that demonstrates how to use the database and supports the development of scripts. For example:
#!/bin/sh
# Run notify whenever someone replies to a comment.
$ hn replies username | notify
What features interest you?
[1]: https://github.com/HackerNews/API
[2]: https://www.sqlite.org/fts5.htmlI have never explored the API, and I'm not a professional programmer, but allow me to describe the ideas from top of head when I read your posts:
Requirements:
1 - Self-hosted: So essentially, it's too heavy to keep the whole sites offline. However, because I always read by topic (e.g. I'd type "SICP ycombinator" in Google and read the top pages), I think one approach is to let the user enter a topic, say "SICP", as well as number of top-level stories to return, say 25, and invoke the API to return stories. The app will then dump the stories and their comments into a database with data modelling that suits for a forum.
2 - Offline access of data: Essentially what I mentioned above. The app should also allow user to remove a story (and its children), modify topics, favorite a story and create new topics. I think those are the barebone requirements. The backend would interact with the database and do updates.
3 - Query data via SQL: I think it might be too much work to parse queries, and the easiest thing to do is to just access a string as query and pass that to the database engine. Or maybe only allow user certain actions and let the backend assemble the queries.
4 - Full text search of stories and comments: Not sure what to do as my programming knowledge is very limited. I saw the second link you provided and it is very interesting.
5 - Notification of replies to comments: Maybe give user a button to update all his favorite stories.
TBH I really want to see what other people's implementations will be.
1) It is possible to keep the whole site offline. A database from 2017 is 9 GB: https://archive.org/details/hackernews-2017-05-18.db I think a 2020 DB could be less than 20 GB.
2) My focus is on reading, not writing. Local favorites make sense. Maybe with importing public favorites. A user can set their name without logging in.
3/4) SQLite does the heavy lifting here.
Option to download a copy of the article the hn discsussion is about and create a thumbnail..
There are times when I add HN articles to mobile chrome's "read later" list, and when I have time go through them. Being able to read them later when not having internet access would be amazing (London's tube is annoying when it comes to internet, internet only exists in the stations).
Source: Ran hub for a BBS network. All the offline readers, like SLMR, had twit filters.
It has never been shared and I only use it personally so expect lots of bugs.
I may open source the code after I clean it up.
I wouldn't need offline mode, but a CLI tool to read HN would be very much welcome.
Sometimes the scroll gets real and I want to get past a discussion to see other angles of whatever subject - scrolling trying to find where thread 2 ends and a different subject comes up is challenging at times - so I often go to a new tab on a different story and don't come back.
Even click/tap to draw a line from comment 2 to all sub comments so I can find where comment 3 starts would be helpful.
In particular I like the fact that I can bookmark posts and comments with it.
I very often jump to the discussion to read opinions - there's often a treasure trove of stuff there of someone who knows something more than what the article states.
A really annoying, missing feature for the discussion/comments parts, is that the threads are extremely difficult to tell apart and traverse. On a small screen, the threads are very close together and only distinguishable through color differences. Scrolling through them is a nightmare, whereas I've really come to love the Reddit app's feature with a button that allows you to scroll to the next sibling comment, e.g. if you want TO quickly see the next comment to the OP, instead of scrolling endlessly and not knowing how far along you are.
Which is why I usually read it on https://hackerweb.app/ instead.
Forking https://hackerweb.app and adding offline support would be a good starting point, IMHO.
The biggest qualms for me would be; notifications of comments and searching my previous comments.
Other than that Hacker News is actually highly reliable and available and I've never personally needed an off-line version (except maybe on a plane)
IMHO the default HN ui has many quite obvious UX issues on mobile with e.g. tiny click targets that are positioned close together, long threads that are hard to navigate, low contrast colors (grey on grey), etc. It's clearly not been designed with mobile in mind and even on desktop it's not all that good. Brutalist/retro is the best you can say about it and that looks like it's intentional to me.
If you read HN on the way to work in an area that doesn't have great mobile coverage in e.g. subway tunnels, etc. offline support would be pretty nice.
There are various Android clients that do some interesting things with the UX (e.g. thread navigation, swipe to upvote, etc.) and offline usage. However, most of them fall a bit short when it comes to things like commenting or other features. Most of the clients I tried did not cover all the features.
- save individual posts as "favorites", and allow users to tag them somehow (e.g. 'finance', or 'front-end') for easy searchability. The saved quote should be available for offline reading, and should link back to the original topic. The original topic doesn't need to be available offline.
- mark links and comments as seen
- hide seen
- fold comments to only show top level
- favorite links (preferably to a list that is easily dumped in to e.g. pocket with a few buttons for that)
This would provide some of the listed features when using a suitable NNTP client; my own NNTP client called "bystand" supports SQL (it stores all data in a SQLite database; if you use the SQLite FTS extension then full text search is available), but other clients may have different features, so may be more suitable for your use possibly.
(I think that all web forums and mailing lists should be NNTP instead. It is OK to have a web interface too as long as the NNTP server and message ID can be read from the web interface (even if JavaScript and CSS is disabled), so that if you are given links to the web interface then you can access the NNTP if you want to do. It would also be possible to provide a email interface too if wanted.)
- an Android app embodying the same features,
- The ability to comment while offline.
(This isn’t criticism, but I definitely don’t understand why it’s a criteria in your case yet.)
