Your customers are only entitled to the source if you accepted the terms of the GPL and technically you can claim that you didn't accept them and that you simply distributed the derived work illegally.
If your customers wanted the source, they would have to ask the original authors to sue you and have the court force you to accept the GPL post-factum. It's not clear how this would turn out because afaik such line of "defense" has never been tried by violators so far. A few such lawsuits settled out of court, usually with source code being part of the deal.
A ruling in this kind of case, where the violator stubbornly refused to comply and maybe tried to escape with just paying damages or some other trickery, that would be interesting.