Show HN: Easily generate text and compute probabilities for any Hugging Face LLM (opens in new tab)

A python library and command line scripts to simplify working with language models.

In the course of studying EleutherAI's Language Model Evaluation Harness, we found ourselves wanting to be able to use their clean LM interface for our own projects, particularly with models from the Hugging Face model hub. Starting from the lm-evaluation-harness code, we built a standalone HFLM class that you can easily import into your Python projects to quickly generate text and compute log probabilities of arbitrary strings.

We also wrote two scripts to let you play with LMs from the command line: lmprob takes a model and a string and returns the log probability of the string according to the model, and lmgen takes a model and a string and generates some completion text. If you've ever wanted to write shell scripts that make decisions and take actions using an LLM, these scripts will help you do that.