If the shirt costs $10 in labor, and the pre-tariff wholesale price is also $10, how does the manufacturer make a profit? Surely the wholesale price would at least include some markup for profit? So like $10 labor + $3 markup (per employee-day) + $10 tariff ($23) contrasted with $15 labor + $3 markup + $4 tariff ($22), in your scenario.
But now you see that the low wage manufacturer has a third option, $10 labor + $1 markup + 10 tariff ($21), which would maintain their competitive advantage and in this scenario only cut their daily per employee profit by $2, as opposed to the $5 hit they would suffer by raising their employee wages to $15 day from $10.