Polymer was released and worked on as a library while the spec was being worked on. Polymer 0.x and 1.x targeted a version that was only implemented in Chrome, which was used to build the eventual standard, and Polymer 2.x targets the final standard.
The API for web components isn't that bad. You have to stamp DOM templates yourself using whatever process, you have to handle attributes and properties yourself, and that's about it. Lots of boilerplate, but a library like Polymer can take a simple, opinionated approach to handle it for you.
From my experience Web Components have been pretty pleasant to use with or without Polymer, it's just a matter of wiring everything up if you go vanilla.