A 4 digit pin isn't good enough if you care at all about thwarting serious attackers. Do a quick calculation of how long it would take an attacker with a script and 10 phone lines to get access.
Maybe you could translate a longer password into the corresponding touchtones?