I've been working with this for a couple of weeks ... there are real challenges in traditional setups going from prompt in -> prediction out to building the actual backends that blend business logic and inference for multiple models, large inputs, etc.
It's fun to work on solving these challenges with new tooling. Happy to answer any questions.
