It really depends on the type of contract. (I also did defense contracting, both bootstrapping my own and working for a larger (25-50 person) company)
I'd say overall it is beyond what an early stage startup can do (non-dod-focused), without a great partner. Selling products is a lot easier than selling ongoing services, but the profit is all in services. A lot of the difficulty was due to it being a classified contract (the work itself wasn't, but the work locations and interoperability were). It's probably feasible for a successful small business with a full time person, particularly if the business is set up to go after government from the beginning.
Big companies aren't cost-free as clients, but there's less "if you do X wrong, you could go to jail" (which essentially never happens absent willful fraud, but still). The citizenship requirements also make it really hard for tech companies.