undefined | Better HN

0 pointskillthebuddha2y ago0 comments

I agree, it looks great. A couple similar projects you might find interesting:

- https://github.com/newhouseb/clownfish

The first one is JSON only and the second one uses regular expressions, but they both take the same "logit masking" approach as the project GP linked to.

0 comments

Der_Einzige2y ago

I love the love from you two - I am trying right now to significantly improve CTGS. I'm not actually using the "Logitsprocessor" from Huggingface, and I really ought to as it will massively speed up inference performance. Unfortunately, fixing up my current code to work with that will take quite awhile. I've started working on it but I am extremely busy these days and would really love for other smart people to help me on this project.

If not here, I really want proper access to the constraints APIs (LogitsProcessor and the Constraints classes in Huggingface) in the big webUIs for LLMs like oogabooga. I'd love to make that an extension.

I'm also upset at the "undertooling" in the world of LLM prompting. I wrote a snarky blog post about this: https://gist.github.com/Hellisotherpeople/45c619ee22aac6865c...

j / k navigate · click thread line to collapse