You're saying that someone who talks about the percentage growth in their customers would be just as concerned about the percentage shrinkage of their non-customers? Unless you're Facebook or Coca-Cola, you probably don't assume that the majority of people in the world will at some point use your product. So it doesn't make sense to measure your shrinkage of non-marketshare.
The percentage change in users is the interesting number. If someone said "My conversions went up by 100 per week!" I would say "What percentage increase is that?" If someone said "My conversions went up five percentage points!" I would say "What percentage increase was that?" If they said "My conversions went up 20%," I would know what I wanted to know.