Show HN: Beating OpenAI's structured outputs on cost, accuracy and speed (opens in new tab)

(boundaryml.com)

8 pointshellovai1y ago2 comments

2 comments

Interesting performance with GPT 3.5, what does performance on Llama look like? What about smaller models like Llama 3.1?

hellovaiOP1y ago

We have some preliminary data with llama3.1 and we find that the smaller model gets to around 70% with BAML (+20% from base), but we'll update this dashboard with llama3.1 by end of week!

j / k navigate · click thread line to collapse

2 comments

ddematheu1y ago

Interesting performance with GPT 3.5, what does performance on Llama look like? What about smaller models like Llama 3.1?

hellovaiOP1y ago

We have some preliminary data with llama3.1 and we find that the smaller model gets to around 70% with BAML (+20% from base), but we'll update this dashboard with llama3.1 by end of week!

j / k navigate · click thread line to collapse