undefined | Better HN

0 pointsverdverm1mo ago0 comments

That later "learning" part is updating weights in your brain

What Ai's get is a cheat sheet for the session

0 comments

That's what I mean by continual learning, skills, memory are a crutch until real learning can happen, which could be weights changing in the local instance.

verdvermOP1mo ago

And my point is that weight changes are not likely to have the economic ROI for their justification on a person-by-person basis

What you are suggesting is a very expensive late-training phase activity. It's also not clear anymore when fine-tuning helps or hurts. Progress is rapid

small_model1mo ago

I see, I misunderstood your original message. Given how much progress has been made without it, It's perhaps not necessary especially if the economics make it prohibitive.

j / k navigate · click thread line to collapse

0 comments

small_model1mo ago

That's what I mean by continual learning, skills, memory are a crutch until real learning can happen, which could be weights changing in the local instance.

verdvermOP1mo ago

And my point is that weight changes are not likely to have the economic ROI for their justification on a person-by-person basis

What you are suggesting is a very expensive late-training phase activity. It's also not clear anymore when fine-tuning helps or hurts. Progress is rapid

small_model1mo ago

I see, I misunderstood your original message. Given how much progress has been made without it, It's perhaps not necessary especially if the economics make it prohibitive.

j / k navigate · click thread line to collapse