Is there an hour by hour model quality tracker?

1 pointssshadmand3mo ago1 comments

I'd love to put some actual science behind a feeling I think a lot of us share: certain hosted models seem to dip in quality at certain times of day. In my group, it’s almost treated as a “yeah, of course” kind of thing. Having something near-real-time that shows which models are performing best at any moment would be… honestly, pretty awesome.

I asked Perplexity and got nothing: https://www.perplexity.ai/search/is-there-an-hour-by-hour-model-8.03Z_BlSomAY7o9F5HwmQ#0

1 comments

istvan_intrnd3mo ago

I have noticed this 100%. My theory is that they are serving quantized versions of the models randomly to save costs.

j / k navigate · click thread line to collapse

Is there an hour by hour model quality tracker?

1 pointssshadmand3mo ago1 comments

I asked Perplexity and got nothing: https://www.perplexity.ai/search/is-there-an-hour-by-hour-model-8.03Z_BlSomAY7o9F5HwmQ#0

1 comments

istvan_intrnd3mo ago

I have noticed this 100%. My theory is that they are serving quantized versions of the models randomly to save costs.

j / k navigate · click thread line to collapse