I think the question is flawed... Or at least irrelevant.
Since we are talking about the stock market (and stock brokers don’t make anything of value) a better question would be:
You hire a worker for $2 to make something you sell for $5. Someone buys it for $5 and sells it to a forth person for $10. Who lost money?
Now to make this simplification more accurate, there is a system in place for the $5 to $10 transaction called The stock market where there are bazillions of these transaction. Another question arises. Who is selling something for $5 when they could sell it straight to the forth person for %10? Who has is $10 and why are they paying $10 for it? How did they get that $10? Did they make something worth $10 and sold it for $10?
I think the real answer for who is loosing money is: Workers across the globe are.