Now maybe we're not quite at that point yet (after all, the Daily Stormer did eventually find a CDN that would take them), but we may very well be getting close; there are only so many CDNs big enough to effectively shrug off large-scale DDOS attacks after all.
You also have to consider how difficult it is to match the quality of service provided by Cloudflare, and the hassle involved in switching to a new CDN. Cloudflare's refusal to service some organizations on the basis of ideology might have a chilling effect on Free Speech, even if it's not an insurmountable barrier.
I agree there is a balance between Freedom of Speech and Freedom of Association. The question is: where do we draw the line?