Actually, your karma still goes down, there's just no display-level lower than -4.