Legal or not, if you tell your manager that you are also working for another company half the time (unless you work odd hours), will they be OK? If so then good. If not then it's a matter of transparency and trust.
Also doesn't the contract has something about the number of hours? It may not be enforceable, but it's on paper.