undefined | Better HN

0 pointsWarOnPrivacy1y ago0 comments

> Realistically, there are only 2 search engines now.

From the article:

     Many alternatives to GBY [Google, Bing, and Yandex] exist, but almost none of them have their own results;

This seems to assert that ~0 other search providers do any crawling at all. Ever. Are we sure that's the case?

   (they could crawl but never ever return those results == more odd).

0 comments

ColinHayhurst1y ago

It's a very long article so understandable that you did not read on and learn about other search engines crawling beyond GBY. Still there are indeed very few that are crawling at web scale, and internationally. We are at 8 billion pages and totally independent [0], hence expressing our concerns to 404 media after being blanked by Reddit.

[0] https://www.mojeek.com/about/why-mojeek

WarOnPrivacyOP1y ago

> did not read on and learn about other search engines crawling beyond GBY. Still there are indeed very few that are crawling at web scale, and internationally

That's helpful clarification.

In criticism of the article, you might agree that

none of them have their own results

is a fairly absolute statement. It signals: Final word on the matter; no nuance to follow.

topaz01y ago

Omitting the "almost" from "almost none" makes it sound disingenuously more absolute than it actually is.

1 more reply

shadowgovt1y ago

I mean, I didn't read on because it's paid.

I'm not taking their reporting without compensation, but that also means I didn't have the whole story. Such is life in this era of the internet.

1 more reply

culi1y ago

I believe Brave Search is also starting their own index. There are some tiny independent indexes too:

https://www.crawlson.com/ https://search.marginalia.nu/ https://wiby.me/ https://searchmysite.net/

MichaelZuo1y ago

Bing provides far fewer verbatim results for pretty much all search queries that I've tested.

And Yandex isn't much better for non cyrillic search, Baidu is only for the Chinese web effectively.

And all other search engines either don't even attempt to do full web crawls anymore and/or buy from one of the four above.

So realistically there's just one search engine for the full web that actually does the work.

dev1ycan1y ago

Brave has their own search engine, yandex I only use for reverse image search, baidu's interface is really clean and feels like old school google... but I don't speak chinese so I can't use it.

I hope that one day they get a western version

MichaelZuo1y ago

Brave doesn't have its own index of the full web, and it's even less useful than Yandex. And very likely buys some of it, according to what I've heard. So it falls into the last category.

2 more replies

WarOnPrivacyOP1y ago

> And Yandex isn't much better for non cyrillic search,

I like Yandex when I'm rabbit-holing after obscure musicians/music. I routinely have a better experience than I do with DDG or Kagi or Goog.

freediver1y ago

Kagi also uses Yandex index so it would be unusual that something exists in Yandex and not in Kagi results.

1 more reply

MichaelZuo1y ago

It's also vastly better for finding livejournal blogs.

darreninthenet1y ago

I believe Kagi has its own crawler as well and it merges all the results and does whatever Kagi does behind the scenes to show the mix

tdeck1y ago

Aside: Does anyone know how the GBY term became a thing and why it includes Yandex but not Baidu?

j / k navigate · click thread line to collapse

0 pointsWarOnPrivacy1y ago0 comments

> Realistically, there are only 2 search engines now.

From the article:

     Many alternatives to GBY [Google, Bing, and Yandex] exist, but almost none of them have their own results;

This seems to assert that ~0 other search providers do any crawling at all. Ever. Are we sure that's the case?

   (they could crawl but never ever return those results == more odd).

0 comments

ColinHayhurst1y ago

[0] https://www.mojeek.com/about/why-mojeek

WarOnPrivacyOP1y ago

> did not read on and learn about other search engines crawling beyond GBY. Still there are indeed very few that are crawling at web scale, and internationally

That's helpful clarification.

In criticism of the article, you might agree that

none of them have their own results

is a fairly absolute statement. It signals: Final word on the matter; no nuance to follow.

topaz01y ago

Omitting the "almost" from "almost none" makes it sound disingenuously more absolute than it actually is.

1 more reply

shadowgovt1y ago

I mean, I didn't read on because it's paid.

I'm not taking their reporting without compensation, but that also means I didn't have the whole story. Such is life in this era of the internet.

1 more reply

culi1y ago

I believe Brave Search is also starting their own index. There are some tiny independent indexes too:

https://www.crawlson.com/ https://search.marginalia.nu/ https://wiby.me/ https://searchmysite.net/

MichaelZuo1y ago

Bing provides far fewer verbatim results for pretty much all search queries that I've tested.

And Yandex isn't much better for non cyrillic search, Baidu is only for the Chinese web effectively.

And all other search engines either don't even attempt to do full web crawls anymore and/or buy from one of the four above.

So realistically there's just one search engine for the full web that actually does the work.

dev1ycan1y ago

Brave has their own search engine, yandex I only use for reverse image search, baidu's interface is really clean and feels like old school google... but I don't speak chinese so I can't use it.

I hope that one day they get a western version

MichaelZuo1y ago

Brave doesn't have its own index of the full web, and it's even less useful than Yandex. And very likely buys some of it, according to what I've heard. So it falls into the last category.

2 more replies

WarOnPrivacyOP1y ago

> And Yandex isn't much better for non cyrillic search,

I like Yandex when I'm rabbit-holing after obscure musicians/music. I routinely have a better experience than I do with DDG or Kagi or Goog.

freediver1y ago

Kagi also uses Yandex index so it would be unusual that something exists in Yandex and not in Kagi results.

1 more reply

MichaelZuo1y ago

It's also vastly better for finding livejournal blogs.

darreninthenet1y ago

I believe Kagi has its own crawler as well and it merges all the results and does whatever Kagi does behind the scenes to show the mix

tdeck1y ago

Aside: Does anyone know how the GBY term became a thing and why it includes Yandex but not Baidu?

j / k navigate · click thread line to collapse