the bacteria will be looking for food. either it finds it or it doesn't. if it finds it it will grow and reproduce. that in itself is not the end yet, as there are already many bacteria in our body that live in a symbiotic relationship. only if it reproduces to much and overwhelms our body somehow it becomes a problem.
but also, if it doesn't find food we are not safe yet. the bacteria itself could be toxic in some way and it could reproduce elsewhere and enter our body through food or other ways.
it doesn't need to target our bodies. it just needs to cause an averse reaction of some form.
compare to chemicals for example. they don't target anything. they are inert. yet some chemicals are poisonous.