If you start using Google's password manager with their auto-suggested long complex passwords, there is no way you can remember the passwords yourself. You'll need to be logged into Google Chrome to have access to the password. Therefore, you'll need to be logged into Google Chrome to log into any website. Currently, there is not way to export passwords from one browser to another, so if all your long complex random passwords are in Chrome, as a practical matter, you won't be able to use Firefox, Safari, or Internet Explorer, you'll be required to use Chrome.
The apprehension over the feature is not about security at all (in fact, this system is likely more secure). It's about control. If you allow Google to manage all of your passwords, then you'll need Google to do anything.