undefined | Better HN

0 pointsnl3d ago0 comments

Use function calling/tool use, not XML output. The models are all trained for that now.

Ie, instead of telling it to generate

  <name>Name</name>
  <age>19</name>
  <address>whatever</name>

give it a function

  details(name: string, age: int, address: string)

That is actually a JSON schema, and the models do great at it. Here's the claude docs, but they are all similar: https://platform.claude.com/docs/en/agents-and-tools/tool-us...

0 comments

carpo3d ago

Hey, just want to thank you for this suggestion. Spent this morning swapping to open router and changing all my prompts to use tools instead of XML. Not only is Gemma and Gemini much cheaper, the output tokens from the tool call are much less too. Cost to analyse one 20 minute video with 10 snapshots went from $0.21 to $0.009, and I'm even sending full HD snapshots instead of the 960x540 ones I was sending before (to save costs). The results so far are pretty good. It looks like the larger images are giving the model more context, so in some cases making the cheaper models results better than the expensive models. I'm going to run this over a few hundred videos today and see how it goes in bulk!

nlOP22h ago

Ha! So glad it helped you!

Very interested in the full run details.

carpo3d ago

Very interesting. Thank you!

j / k navigate · click thread line to collapse

0 comments

carpo3d ago

nlOP22h ago

Ha! So glad it helped you!

Very interested in the full run details.

carpo3d ago

Very interesting. Thank you!

j / k navigate · click thread line to collapse