If you actually need whole-web search, Google now points you to an “interest form” for enterprise solutions (Vertex AI Search etc.), with no public pricing and no guarantee they’ll even reply.
This seems like it effectively ends the era of indie / niche search engines being able to build on Google’s index. Anything that looks like general web search is getting pushed behind enterprise gates.
I haven’t seen much discussion about this yet, but for anyone who built a small search product on Programmable Search, this feels like a pretty big shift.
Curious if others here are affected or already planning alternatives.
UPDATE: I logged into Programmable Search and the message is even more explicit: Full web search via the "Search the entire web" feature will be discontinued within the next year. Please update your search engine to specify specific sites to search. With this link: https://support.google.com/programmable-search/answer/123971...
Dear Programmable Search Engine user,
Thank you for contacting us via the Web Search Products Interest Form. We have received your feedback and are actively reviewing the specific use cases you shared.
We are writing to share important details regarding the transition plan and the available solutions for your search needs.
1. For Unrestricted Web Search: Future Web Search Service
For partners requiring unrestricted "Search the entire web" capabilities, we are developing a new enterprise-grade Web Search Service. As you evaluate your future needs, please be aware of the commercial terms planned for this new service:
Pricing: USD $15 CPM (Cost Per Mille / 1,000 requests).
Minimum Commitment: A minimum monthly fee of USD $30,000 will apply.
We’ll release more information on this service later in 2026. Existing 'Search the entire web' engines remain functional until January 1, 2027.
2. For AI & Advanced Search: Google Vertex AI
We strongly encourage you to explore Google Vertex AI as another option for partners who need enterprise search and AI capabilities across 50 or fewer domains. Vertex AI offers powerful capabilities for:
Grounded Generation: Connecting your AI agents to your own data and/or to Google Search to provide accurate, up-to-date responses.
Custom Data Search: Building enterprise-grade search engines over your own data and specific websites.
This solution is available now and is designed to scale with your specific application needs.
Clarification on Current Service Status:
While you evaluate which path fits your business needs, please remember the timeline for your current implementations:
Existing Projects: If you have an existing Programmable Search Engine configured to "Search the entire web," your service will continue to function until January 1, 2027. You have the full year to plan your migration.
New Projects: As of January 20, 2026, new engines created in the Programmable Search Engine admin console are restricted to "Site Search Only" (specific domains only).
Later in 2026 we’ll provide you with more updates regarding the new Web Search Service and the means to express your desire to use it to power your web search experiences.
Sincerely,
Programmable Search Engine Team
People rely too much on other people's infra and services, which can be decommissioned anytime. The Google Graveyard is real.
I just searched for “stackoverflow” and the first result was this: https://www.perl.com/tags/stackoverflow/
The actual Stackoverflow site was ranked way down, below some weird twitter accounts.
Can you talk a bit about your stack? The about page mentions grep but I'd assume it's a bit more complex than having a large volume and running grep over it ;)
Is it some sort of custom database or did you keep it simple? Do you also run a crawler?
There's a reason Google became so popular as quickly as it did. It's even harder to compete in this space nowadays, as the volume of junk and SEO spam is many orders of magnitude worse as a percentage of the corpus than it was back then.
https://github.com/rumca-js/Internet-Places-Database
Demo for most important ones https://rumca-js.github.io/search
I can only weep at this point, as the heroes that were the Silent and Greatest generations (in the U.S.), who fought hard to pass on as much institutional knowledge as possible through hardcore organization and distribution via public and University library, have had that legacy shit on by these ad obsessed cretins. The entirety of human published understanding; and we make it nigh impossible for all but the most determined to actually avail themselves of it.
TIL they allowed that before. It sounds a bit crazy. Like Google is inviting people to repackage google search itself and sell it / serve with their own ads.
Others have to replace Google. We need access to public information. States can not allow corporations to hold us here hostage.
https://dn710204.ca.archive.org/0/items/gov.uscourts.dcd.223...
> Google must provide Web Search Index data (URLs, crawl metadata, spam scores) at marginal cost.
Maybe they're shutting down the good integration and then Kagi, Ecosia and others can buy index data in an inconvenient way going forward?
Kagi makes deals with many search engines so they can have raw search results in exchange for money.
Google says: no, you can't have raw search results because only whales can get those. Only thing we can offer you is search results riddled with ads and we won't allow you to reorder or filter them.
Kagi thinks Google's offer is unacceptable, so Kagi goes to a third party SERP API, which scrapes Google at scale and sells the raw search results to Kagi and others.
August 2024: Court says Google is breaking the law by selling raw search results only to whales.
December 2025: Court orders that for the next six years, 1. Google must no longer exclude non-whales from buying raw search results, 2. Google must offer the raw search results for a reasonable price, and 3. Google can no longer force partners to bundle the results with ads.
December 2025: Google sues the third-party scraping companies.
January 2026: Google says "hey, the old search offering is going to go away, there's going to be a new API by 2027, stay tuned."
Granted, that is scoped to 50 domains. But we don't know if the enterprise package, which allows full web search, isn't roughly market rate.
They were:
> aiming to serve 30% of French search queries [by end of 2025]
https://blog.ecosia.org/launching-our-european-search-index/
> The French index is at an advanced stage of completion, we have started creating the German language index, and the English one should start shortly. All progress is quickly integrated into the Qwant STAAN API.
For anyone affected: consider this a forcing function to either: 1. Build your own lightweight search infrastructure (tools like Meilisearch, Typesense make this more accessible now) 2. Use adversarial interop via services like SerpAPI (though Google is already taking legal action there) 3. Pivot to specialized vertical search where you control the data sources
The real lesson here is the importance of owning your core value proposition. If your product's moat depends entirely on a third-party API that can be yanked away with 12 months notice, you don't really have a sustainable business.
Google is essentially saying: indie search is dead, pay enterprise prices or leave. This will probably accelerate the trend toward specialized, domain-specific search engines that don't rely on Google's index at all.
What alternatives are there besides Bing? Is it really so hard that it’s not considered worth doing? Some of the AI companies (Perplexity, Anthropic) seem to have managed to get their own indexing up and running.
Imagine a decentralized network where volunteers run crawler nodes that each fetch and extract a tiny slice of the web. Those partial results get merged into open, versioned indexes that can be distributed via P2P (or mirrored anywhere). Then anyone can build ranking, vertical search, or specialized tools on top of that shared index layer.
I get that reproducing Google’s “Coca-Cola formula” (ranking, spam fighting, infra, freshness, etc.) is probably unrealistic. But I’d happily use the coconut-water version: an open baseline index that’s good enough, extensible, and not owned by a single gatekeeper.
I know we have common crawl, but small processing nodes can be more efficient and fresh
YaCy, a distributed Web Search Engine, based on a peer-to-peer network https://news.ycombinator.com/item?id=39612950
I suppose I'm asking whether this is actually a _good thing_ in that it will stimulate competition in the space, or if it's just a case that Google's index is now too good for anyone to reasonably catch up at this point.
If you want programmatic access to search results there aren't really many options left.
Bing charge per query for the average user. Ecosia and Qwant use Bing to power their results, probably under some type of license, which results in them paying much less per query than a normal user.
But for searching in more niche languages google is usually the only decent option and I have little hope that others will ever reach the scale where they could compete.
I keep seeing posts about how ~"the volume of AI scrapers is making hosting untenable."
There must a ton of new full-web datasets out there, right?
What are the major hurdles that prevent the owners of these datasets from providing them to third parties via API? Is it the quality of SERP, or staleness? Otherwise, this seems like a potentially lucrative pivot/side hustle?
Sadly, no. There's CommonCrawl (https://commoncrawl.org/) which still, sadly, far removed from "full-web dataset."
So everyone runs their own search instead, hammering the sites, going into gray areas (you either ignore robots.txt or your results suck), etc. It's a tragedy of the commons that keeps Google entrenched: https://senkorasic.com/articles/ai-scraper-tragedy-commons
Aside from that potential, it's also not true.
A Pentium Pro or PIII SSE with circa 1998-99 Apache happily delivers a billion hits a month w/o breaking a sweat unless you think generating pages for every visit is better than generating pages when they change.
RIP, another one to the Google Graveyard.
Hard part is doing it at any sort of scale and producing useful results. It's easy to build something that indexes a few million documents. Pushing into billions is a bigger challenge, as you start needing a lot of increasingly intricate bespoke solutions.
Devlog here:
https://www.marginalia.nu/tags/search-engine/
And search engine itself:
https://marginalia-search.com/
(... though it operates a bit sub-optimally now as I'm using a ton of CPU cores to migrate the index to use postings lists compression, will take about 4-5 days I think).
I found it didn’t really work as a real search engine but it was interesting.
Although, it needs some more work and peers to be usable as a general-purpose search engine.
Not long ago they ruined ublock origin (for chrome; ublock origin lite is nowhere near as good and effective, from my own experience here).
Now Google is also committing towards more evil and trying to ruin things for more - people, competitors, you name it. We can not allow Google to continue on its wiched path here. It'll just further erode the quality. There is a reason why "killed by google" is more than a mere meme - a graveyard of things killed by google.
We need alternatives, viable ones, for ALL Google services. Let's all work to make this world better - a place without Google.
We're in the second era. The era of the MBAs are shutting down the last remnants of openness the company ever had.
Just dissolve them in acid.
If you actually enforce them.
Unfortunately, during the Reagan administration, political sentiment toward monopolies shifted and since then antitrust law has been a paper tiger at best.
The correct parsing is: "Google is ending (full-web search for niche search engines)"
Given that the title supplied is effectively editorialised, and the original article's title is effectively content-free ("Updates to our Web Search Products & Programmable Search Engine Capabilities"), my rewording would be at least as fair.
HN's policy is to try to use text from the article itself where the article title is clickbait, sensational, vague, etc., however. I suspect Google's blog authors are aware of this, and they've carefully avoided any readily-extracted clear statements, though I'll take a stab...
Here's the most direct 'graph from TFA:
Custom Search JSON API: Vertex AI Search is a favorable alternative for up to 50 domains. Alternatively, if your use case necessitates full web search, contact us to express your interest in and get more information about our full web search solution. Your transition to an alternative solution needs to be completed by January 1, 2027.
We can get a clearer, 80-character head that's somewhat faithful to that with:
"Google Search API alternative Vertex AI Search limited to 50 domains" (70 chars).
That's still pretty loosely adherent, though it (mostly) uses words from the original article. I'm suggesting it to mods via email at hn@ycominator.com; others may wish to suggest their own formulations.
One thing touched upon in comments here: I never understood how it was proper for 3rd parties to scrape Google search results and reuse/resell them.
Really off topic, sorry, but I am surprised that more companies don’t build local search indices for just the few hundred web domains that are important to their businesses. I have tried this in combination with local (small and fast) LLMs and I think this is unappreciated tech: fast, cheap, and local.
What they are ending is their support for websites to search across the entire web. The websites that search across the entire web are usually niche search engine websites.
