Ask HN: CLI tool to download webpages and convert to Markdown?

12 pointsraytracer5y ago9 comments

Is there a CLI tool that downloads webpages and converts them to Markdown?

I've just started using Obsidian https://obsidian.md/ and would like a way to save interesting blog posts and articles.

9 comments

  curl --silent https://example.com/foo.html | pandoc --from html --to markdown_strict -o foo.md

From Converting HTML to Markdown using Pandoc http://www.cantoni.org/2019/01/27/converting-html-markdown-u...

crazypython5y ago

I'd like to add that Calibre (ebook-convert on the CLI) has a mode for outputting a .txt file with markdown formatting. It can take HTML as input.

I_complete_me5y ago

I tried that on a HackerNews page and the content of the output file foo.md was in html.

miles5y ago

Sorry, had only tested it on my own site, which worked as expected:

  curl --silent https://tinyapps.org/ | pandoc --from html --to markdown_strict -o index.md

longimanus5y ago

I find Joplin https://joplinapp.org does a good job of producing markdown from web pages and already has sync capability built in. Looks like it would be ideal for working with Obsidian

longimanus5y ago

Exported my Joplin markdown and opened it up in Obsidian. Works like a dream and the best part is Jopli already has it's own web clipper. Seems like a superb match

dougskinner5y ago

I made this a couple weeks ago: https://www.npmjs.com/package/@dougskinner/markdowner

anandology5y ago

https://github.com/aaronsw/html2text

miles5y ago

Thank you, Anand. Despite being 8 years old, Aaron's html2text.py worked perfectly to convert the HN homepage to Markdown. His memory (and code) continues to be a blessing!

j / k navigate · click thread line to collapse

9 comments

miles5y ago

  curl --silent https://example.com/foo.html | pandoc --from html --to markdown_strict -o foo.md

From Converting HTML to Markdown using Pandoc http://www.cantoni.org/2019/01/27/converting-html-markdown-u...

crazypython5y ago

I'd like to add that Calibre (ebook-convert on the CLI) has a mode for outputting a .txt file with markdown formatting. It can take HTML as input.

I_complete_me5y ago

I tried that on a HackerNews page and the content of the output file foo.md was in html.

miles5y ago

Sorry, had only tested it on my own site, which worked as expected:

  curl --silent https://tinyapps.org/ | pandoc --from html --to markdown_strict -o index.md

longimanus5y ago

I find Joplin https://joplinapp.org does a good job of producing markdown from web pages and already has sync capability built in. Looks like it would be ideal for working with Obsidian

longimanus5y ago

Exported my Joplin markdown and opened it up in Obsidian. Works like a dream and the best part is Jopli already has it's own web clipper. Seems like a superb match

dougskinner5y ago

I made this a couple weeks ago: https://www.npmjs.com/package/@dougskinner/markdowner

anandology5y ago

https://github.com/aaronsw/html2text

miles5y ago

Thank you, Anand. Despite being 8 years old, Aaron's html2text.py worked perfectly to convert the HN homepage to Markdown. His memory (and code) continues to be a blessing!

j / k navigate · click thread line to collapse