You're correct that container root file systems should be considered ephemeral, and writing anything that needs to be persisted to them is a smell. However you can mount persistent volumes to a container specifically for the purpose of deploying stateful applications and referencing persistent data. How safe you can consider that data to be depends on the underlying tech that is provisioning the storage. For example a GCE persistent disk with the retain flag set is probably pretty damn durable. I would still back it up, however.