undefined | Better HN

0 pointsallyourhorses4y ago0 comments

It knew which queries to run before I clicked, because those queries are baked in. Why did I have to wait for the app to do something it knew it had to do if someone clicked? Of course we can't make e.g. protein folding an instantaneous task, that doesn't mean common workflows and request patterns can't be. I guess it's the difference between "did I ask the computer a hard question?" and "did I simply ask the computer to do its job?"

In the example dashboards, I'm guessing something like 100% of requests make exactly the same queries. Maybe in a typical corporate dashboard, 70% of users will pull up the default view before leaving. These cases easy to optimize for and definitely worth optimizing for

0 comments

jedberg4y ago

If you own the frontend and the database, sure. But this product hits other people's databases. The only way they can optimize this is by making queries it thinks the user might want. They can't cache the response because they have no way of knowing if it changed, since again, they don't control the data source.

If I owned the database they were getting data from, I'd be mighty upset at the insane amount of useless queries they'd have to make guessing what the user wants.

allyourhorsesOP4y ago

Easily solved by a user configurable staleness value with some reasonable default. Google don't crawl the entire web in response to every query because for the vast majority of queries it's unnecessary. For those where it might be necessary (like news), they instead crawl at a higher frequency or use some special flow (like they do for tweets), either way the result is seamless, involves no progress spinners and is well suited for the vast majority of users.

jedberg4y ago

Funny you should bring up Google. Google is so hard on infrastructure that most big sites have special handling for Google scraper requests. At reddit we put google on their own slower server cluster just so they didn't break the website.

We only did this because of the extreme value Google brings via traffic. But most crawlers and other things that made speculative queries like that were just banned.

j / k navigate · click thread line to collapse

0 pointsallyourhorses4y ago0 comments

0 comments

jedberg4y ago

If I owned the database they were getting data from, I'd be mighty upset at the insane amount of useless queries they'd have to make guessing what the user wants.

allyourhorsesOP4y ago

jedberg4y ago

We only did this because of the extreme value Google brings via traffic. But most crawlers and other things that made speculative queries like that were just banned.

j / k navigate · click thread line to collapse