Yeah, but once we get good enough to annihilate them at will we will also have a similar power to improve them through tinkering.
Just like computer viruses continue to be developed by people, biological agents will be developed as well.
Some human tinkering will be for the heck of it. Some will be government level projects that get out of control. Eventually, organized crime will be in on the action.
Imagine Cryptolocker (http://en.wikipedia.org/wiki/CryptoLocker) but with a biological agent.