undefined | Better HN

0 pointslostmsu1y ago0 comments

What's the context length?

0 comments

The model has a context of 131,072, but I only have 48G of VRAM so I run it with a context of 32768.

j / k navigate · click thread line to collapse

The model has a context of 131,072, but I only have 48G of VRAM so I run it with a context of 32768.

j / k navigate · click thread line to collapse