Install each of the pip/bundler requirements with a separate dockerfile RUN command. Each gets cached into its own container filesystem layer that way and only new requirements are pulled. Use your favourite templating tool to generate the dockerfile with multiple run commands.

I wrote djtempl ( https://github.com/emailgregn/djtempl ) for my purposes.