As mentioned, Puppeteer was used in this project, which is Chrome based.

I've also used Selenium (via Python -- I used BeautifulSoup to parse the resulting HTML) in the past for precisely for the reasons you stated. Selenium uses "web drivers", which lets you start other headless browsers as well (Firefox, Opera, IE, etc.).

http://www.seleniumhq.org/

http://selenium-python.readthedocs.io/

All it took was a couple of lines of Python code...