In my experience, no not at all. Really the hardest part is eliminating any/all of your ANRs (Application Not Responding), which is pretty easy if you just build it from the ground-up with a liberal use of AsyncTasks, Background Services, Content Providers, and BroadcastReceivers. If you can imagine a possible way to take a portion of your code off the main UI thread: do it.
The only advantage that I've seen with iOS development is default UI and animation behavior. With Android you don't get much by default, and there's not really any Open Source scene that I've been able to find. Some would also say that debugging is easier with iOS's emulators, but I just debug straight onto my Android phone hooked up to my workstation.
Fragmentation isn't the problem that people make it out to be. If you write flexible UIs and performance-driven code to begin with, you're not going to run into any real problems outside of game development. Only then do things get tricky with all the possible hardware configurations.
