It's a pain in the ass to use. The plugins are often not backwards compatible. It's amateur quality. The database is mySQL which is not ACID compliant, among other problems. Joomla also is terrible for SEO as the URL handling is badly implemented.. (the whole thing runs off of Index.php? for example.)
Joomla is for people that don't know anything about web development and want something they can throw together. If your client "must" use something like Joomla, you'd be better off with WordPress -- there are far more plugins, the documentation is better and it's not a pain in the ass to deploy. Or better yet, just build a Rails CMS. It isn't that hard.
Joomla also has a bad reputation for security. While it can be secure, it often isn't because of the way 'developers' build the sites.
Joomla is based on Mambo (from the PHP 4 days) and is badly engineered. It isn't even object oriented. It's also not MVC, so you can't separate business logic from presentation, among other things. It's just a mess.
The documentation is terrible compared to WordPress -- a pretty big deal if you want to create custom extensions.
I'll go out on a limb here and add that the types of clients that love Joomla are the types of clients I can do without. I did some Joomla work for a client and it was the worst experience I've ever had working with anyone, because the types of people that value Joomla, rarely value craftsmanship, good design (both front-end and back-end) and are generally ignorant (or hardheaded) about good software practices. Never again.
Oh, and it's in PHP.
