Many times they do taste better. The spoilage rate is higher so only the healthiest foods survive, and also farmers who are expecting a higher price point recognize that they need to also taste good (I.e. there is a market for bad tasting $1/pound conventional vs good tasting $2/pound conventional, not so much of a market for bad tasting $4/pound organic vs $5/pound good tasting organic)
A lot of organic foods are different strains than typically sold at grocery stores, and they do tend to taste better simply by virtue of being a strain likely chosen more for its taste than transportability.
Of course though it all depends on where you shop. If you're just comparing two identical apples organic vs non-organic, then you'd almost certainly not taste the difference.
One interesting point. When watching the Netflix series Chef's Table, one of the chefs worked closely with genetic engineers to make better tasting GMOs. The researcher interviewed said it was the first time in his long career that anyone had asked for him to improve flavor rather than yield or size (sometimes at the detriment of flavor).
While this doesn't pertain to organic food directly it does suggest that the industrial food system may be optimizing for different things than a consumer may want.
