You'd need to keep enough of the chain live for all "live" transactions; that would likely have a time bound (days, perhaps). People who run a "full" client that verifies the complete blockchain then just need to have enough storage for the complete transaction volume over that time period. (And people who only need to worry about their own transactions and don't care about verifying other people's transactions can hold onto even less data.)