undefined | Better HN

0 pointsjacquesm15d ago0 comments

The main difference is humans are learning all the time and models learn batch wise and forget whatever happened in a previous session unless someone makes it part of the training data so there is a massive lag.

Whoever cracks the continuous customized (per user, for instance) learning problem without just extending the context window is going to be making a big splash. And I don't mean cheats and shortcuts, I mean actually tuning the model based on received feedback.

0 comments

aurareturn15d ago

Why not just provide more compute for say, 1 billion token context for each user to mimic continuous learning. Then retrain the model in the background to include learnings.

The user wouldn’t know if the continuous learning came from the context or the model retrained. It wouldn’t matter.

Continuous learning seems to be a compute and engineering problem.

jacquesmOP15d ago

Because that re-training is not strong enough to hold, or so it seems. The same dumb factual errors keep coming up on different generations of the same models. I've yet to see proof that something 'stuck' from model to model. They get better in a general sense but not in the specific sense that what was corrected stays put, not from session to session and not from one generation to the next.

My solution is to have this massive 'boot up' prompt but it becomes extremely tedious to maintain.

eloisant15d ago

They can write to files then refer to them in a next session.

A bit like the main character played by Guy Pierce in the movie Memento (which doesn't work great for him to be honest).

j / k navigate · click thread line to collapse