Ask HN: What are you using headless Chrome for?

17 pointscptnntsoobv5y ago21 comments

The headless version of chrome has seen great adoption since it was launched 4-5 years ago. I am wondering what types of use cases it is being used for.

Running automated tests is one the most common use cases I am familiar with. Are there any other purposes folks are using it for?

I am also curious about large scale cloud deployments use cases for headless chrome that the HN crowd might be aware of.

21 comments

enlightens5y ago

Generating diagrams from Markdown files using the Mermaid JS library. They have a CLI but it's NPM based so you basically have to pick your poison as to what you'll be configuring. In my case Chrome made more sense.

eberkund5y ago

I used it for creating PDFs at my last job.

digianarchist5y ago

We've done this. You can run it in AWS lambda.

kypro5y ago

Could someone elaborate? How does this work?

Do you create a HTML document then print the output as a PDF with headless Chrome?

eberkund5y ago

Yes exactly, HTML document using whatever templating language you want (or even client-side JavaScript framework since Chrome can handle that too) and then output a PDF.

cjfont5y ago

I use currently use puppeteer to do the same thing

jjeaff5y ago

I use it to interface and download reports and other data on behalf of customers from sites that don't have any type of 3rd party api.

jmnicolas5y ago

I use it to get rendered web pages I want to scrap.

jamil75y ago

I was doing the same for an article reader / archiver I built but it was slow. Did you ever figure out how to speed it up? Cancelling image and media requests helped but it increased the likelihood of being flagged as a bot.

jmnicolas5y ago

Sorry I never looked at the perf aspect of it since my needs were modest (less than 50 web pages per day). The first thing that would come to my mind is to parallelize it but then it's going to become pretty complex.

matt_f5y ago

*scrape

mothinx5y ago

I use it for a trading bot on Fifa 2020 Fut mode

joefarish5y ago

I imagine you don't want to share too much info about this, but I'd be interested in any more detail you are willing to share.

jamieweb5y ago

I use it to test my Content-Security-Policy by crawling my whole site and capturing any violation reports that are sent.

doopy15y ago

Mind sharing your script?

jamieweb5y ago

This has been absorbed into an internal repo now, but the original version that I wrote a couple of years ago is here: https://gitlab.com/jamieweb/travis-ci_csp-tester/

1 more reply

peteypao5y ago

What's wrong with just plain old curl?

jamieweb5y ago

cURL doesn't load subresources or honour Content-Security-Policy headers as this is beyond the scope of what it's supposed to do.

pkz5y ago

I use it to archive complex web pages.

r0s5y ago

Automated testing.

j / k navigate · click thread line to collapse

21 comments

enlightens5y ago

eberkund5y ago

I used it for creating PDFs at my last job.

digianarchist5y ago

We've done this. You can run it in AWS lambda.

kypro5y ago

Could someone elaborate? How does this work?

Do you create a HTML document then print the output as a PDF with headless Chrome?

eberkund5y ago

Yes exactly, HTML document using whatever templating language you want (or even client-side JavaScript framework since Chrome can handle that too) and then output a PDF.

cjfont5y ago

I use currently use puppeteer to do the same thing

jjeaff5y ago

I use it to interface and download reports and other data on behalf of customers from sites that don't have any type of 3rd party api.

jmnicolas5y ago

I use it to get rendered web pages I want to scrap.

jamil75y ago

jmnicolas5y ago

matt_f5y ago

*scrape

mothinx5y ago

I use it for a trading bot on Fifa 2020 Fut mode

joefarish5y ago

I imagine you don't want to share too much info about this, but I'd be interested in any more detail you are willing to share.

jamieweb5y ago

I use it to test my Content-Security-Policy by crawling my whole site and capturing any violation reports that are sent.

doopy15y ago

Mind sharing your script?

jamieweb5y ago

This has been absorbed into an internal repo now, but the original version that I wrote a couple of years ago is here: https://gitlab.com/jamieweb/travis-ci_csp-tester/

1 more reply

peteypao5y ago

What's wrong with just plain old curl?

jamieweb5y ago

cURL doesn't load subresources or honour Content-Security-Policy headers as this is beyond the scope of what it's supposed to do.

pkz5y ago

I use it to archive complex web pages.

r0s5y ago

Automated testing.

j / k navigate · click thread line to collapse