Ask HN: Why does ChatGPT love the word "eager" so much?

ChatGPT will write in essentially any style you ask it to. The default behaviour is an artifact of RLHF training and the hidden pre-prompt, not a reflection of the capabilities of the underlying model. A pre-prompt along the following lines will radically change the nature of the output: "You are a subject matter expert in x. You communicate in a terse, information-dense manner. You make clear, strong arguments without vacillation."

>> like a highschool student trying to inflate their word count on an essay.

So the chameleon is a bad chameleon because it looks too much like a leaf, too much like the thing it is attempting to imitate?

A few years ago I was tasked with editing the performance reviews for my unit (military). Every supervisor had a finite number of characters in which to describe each soldier's performance. I went through and removed all the extra/useless words. Oh the anger! While supervisors agreed I had improved their writing, they now felt obligated to fill up all the blank space I had created.

I have had good luck telling it to use simplified technical English.

It's called thinking out loud. Gets you better results. Try it out yourself.

I don't want to look for the source of analysis right now, but I recall reading a study demonstrating that a large part, if not most of the word frequency shift was caused by RLHF training done on data predominantly generated by people hired from lower income English-speaking countries which simply have a different dialect of English with a noticeably different frequency of certain phrases and expressions, so e.g. at least some versions of ChatGPT got RLHF-trained to speak more in a Nigerian English dialect.

Since there isn't a single English (English learners generally get informed about the choice of UK vs US English only, but most English is spoken outside of UK and USA in other places and other dialects), but multiple different Englishes, any English speaker will probably find something to be surprised by, and there is an economic incentive to get data from people other than the relatively expensive native speakers of UK or USA English.

The window of time where word frequency of chatgpt's favourites and usage of chatgpt is closely related is rather small I think. Academic language has a number of 'marker' words that are basically just style and will be more or less copied once you read many papers. 'Rigorous' is a general example, but most fields have their own. If many papers you read while writing your own paper use words like 'delve', you will be much more likely to use it yourself.

On another note, while the paper itself is pretty cool, in discussions on it I thought people where kind of looking down on using LLM's to help you write. There's a philistine moat in many fields around writing style. While writing well is in my experience correlated with paper quality, it is not predicated by it. And introducing tools that help people write more readable papers is probably a net benefit overall.

I wonder why some words are overrepresented. Isn't the whole idea of language models to model word distribution as close as possible? Does it have something to do with RLHF? Or it's the training data?

To what extent can this style be overcome by prompting?

If it can be overcome in existing models, it’s probably going to involve different aspects including vocabulary, style, and organization.

This, and ChatGPT was trained on a lot of web content produced by businesses, so in my view this may lead it to overuse "buzzword" words such as "delve," "deep dive," "leverage," "optimize," "synergy," "impact," and others.

My new frequency illusion is the phrase "ride or die". I'm 32 and I had never heard that in my life until I watched that show Cruel Summer and now it seems like I hear it everywhere, mostly on tv/ads/podcasts and almost never when talking to people I know. I refuse to say it because I don't fully understand what it means and when I do hear it, it always sounds a little cringey.

So I just heard about Frequency Illusion earlier today, and now I see it here again. Such meta.

(Not kidding, from today’s NYT crossword column: https://www.nytimes.com/2024/08/11/crosswords/daily-puzzle-2...)

> Eager isn't an especially uncommon word

I'd say it's very common, at least in my part of the US. It's one of the words I hear on a daily basis, anyway.

"Delve" used to be a very commonly used term before "deep dive" largely replaced it. I'm sure there are a whole lot of writings online that use "delve" because of the time period they were produced in.

As a graybeard, I'm personally still much more likely to say "delve" than "deep dive".

It is generally the antonym, in technical contexts, to "lazy", as in "lazy evaluation" etc.

An extreme overuse of everything. Like "give me a formula to color every second Excel row blue". "Sure! First make sure you have installed Excel, here are the steps..."

I often find that it's easy for me to explain things in enumeration. I'm wondering if a) there is a latent middle manager living inside me, always shuffling his little powerpoint decks, or b) there is a little LM living inside me, making me compulsively enumerate things.

Sorry, this is a British company, we only hire unseasoned developerd here.