That doesn't match with my experience. I've got a carbon fibre road bike and some parts of the frame are remarkably stiff whereas other areas such as the handlebars have noticeable flex.
It can be surprising to people just how tough/strong carbon fibre parts can be - here's Danny MacAskill's destructive testing of some CF wheels: https://www.youtube.com/watch?v=VfjjiHGuHoc