What I missed: How does the superficial alignment hypothesis related to model size (they only investigate disjoint aspects on 7B vs 65B llama models). Since the paper focuses on data quality, I would have expected an annotation guideline.
Still, I think the paper is an excellent read.
> "B Anticorrelation between Perplexity and Generation Quality"
> "When fine-tuning LIMA, we observe that perplexity on held-out Stack Exchange data (2,000 examples) negatively correlates with the model’s ability to produce quality responses. To quantify this manual observation, we evaluate model generations using ChatGPT, following the methodology described in Section 5. Figure 9 shows that as perplexity rises with more training steps – which is typically a negative sign that the model is overfitting – so does the quality of generations increase"
I think where they say "anticorrelation" it should say "correlation" and where they say "negatively correlates" it should say "positively correlates" if they are basing their statement on what they observed in their experiments.
EDIT: I see they say "Preprint. Under review" so maybe they will fix it if it's wrong. This is the kind of thing that peer review is really good at fixing. Also not every submission on arxiv is a preprint or under review but I guess this one is.
If all it takes is ~1k high-quality examples (of course, quality can be tricky to define) to tune an LLM successfully, then we should expect to see these tuned LLMs for many different narrow use cases.
Of course, devil is likely in the details. Even in this paper, the prompts on which the model is evaluated were written by the authors and "inspired by their own interests or those of their friends." Can be tricky to make a jump from these prompts and answers to real world LLM use cases, but super super promising.
Using the term "alignment" for what they're trying to do is misleading.
Not that the absolute results are extremely strong, most likely I'd suspect as the base model is just not competitive to GPT4 atm, but the relative results seem very impactful. Maybe fine-tuning a large LM for specific tasks is more practical than thought before?
