Maybe we shouldn't be too critical of Mozilla for providing a privacy-first LLM service - for free. The other big privacy-first LLM provider is Apple, which requires users to have their devices/subscriptions to use, and definitely uses advanced telemetry by default.
Not to speak of all the other providers who are either paid or free-but-mine-your-data.
It is just Mozilla have a tendency of chasing hype rather than focusing on what they are doing. During early smartphone era they spend most of the resources trying to write an OS with Javascript ( Firefox OS ) that works on a $35 Smartphone.
Now they are doing it again with AI. Although this time around Firefox is in fairly good shape I guess this isn't too bad. But they need to figure out a way to generate revenue rather than relying on Google. And LLM service isn't it.
It was a tiny percentage of the overall staff; I traveled to the Toronto office to work with some of the graphics devs (the area I primarily worked in) and the floor we were on easily had 200 people in that one -- relatively small -- location.
B2G/FFOS gets a lot of well-deserved hate -- I quit after a year -- but its impact has been wildly blown out of proportion.
Like by trying other things? Such as a mobile OS, "read later" app, LLM service which they could embed in Firefox? People criticise Mozilla a lot for even trying, but in the same breath say they should figure out alternative income sources. What do you think they're doing? LLMs are part of the hype cycle, yes, but maybe there's still a market for them even after it's blown over?
Most of Mozilla’s resources have always been spent on Firefox. There was never a cycle where most of Mozilla’s resources were spent on FirefoxOS.
But they need to figure out a way to generate revenue rather than relying on Google.
The percentage of revenue from Google has fallen every year since 2016. Mozilla Corporation had a 33% profit margin in 2022 (The latest data on Wikipedia).
It’s a fantastic business model.
It's bad by incompetence, 7B models of a year ago were terribly bad. It's not privacy-first enough, as it's possible to run the AI directly in the browser, but for some reason they didnt do it.
Also consider accessibility for low-end devices and development countries.
Perhaps this is a proof of concept and they will have optional Firefox integration at a later time. Firefox uses local AI for webpage translation already.
Running a year old model, if it's power efficient and economically constrained makes complete sense to me.
In Apple's case, they are putting some amount of work into making their privacy claims verifiable. Good will is no longer good enough. Verifiability should be the bar for trust in 3P privacy claims.
Also consider that Apple has the big pockets to build their own server hardware, to claim multiple layers of privacy - but also remember that when they first introduced "differential privacy" and claimed it would be totally anonymous, privacy researchers soon found out that Apple set the epsilon so low that even after a few requests to their service, the user could be de-anonymized.
source: "Apple has boasted of its use of a cutting-edge data science known as "differential privacy." Researchers say they're doing it wrong." https://www.wired.com/story/apple-differential-privacy-short...
There is a very fundamental critique here: a service being offered for free like this is being subsidized, messing with the general market dynamics that really should be making all of these tools cost way more money to begin with.
Of course Apple is also doing similar things, but for Mozilla to be doing it is quite frustrating.
Also consider that some free use, for example 5 summaries per hour or whatever, is a pretty basic offer for any kind of software, not just LLMs.
Prices have been dropping very rapidly due to technological progress. I feel like a lot of threads recently have had people confused about costs because they either are stuck on prices they learned about 6-12 months ago, or because they're failing to project further rapid reductions of price (I can't predict the future myself, but I'm not gonna bet on inference prices stabilizing all of a sudden.)
I mean, it definitely costs more than FREE. But I think it's closer to free than many commenters realize, and continuing to go that direction faster than many commenters realize.
By definition then, Apple's is most certainly not privacy first by any stretch of mental gymnastics.
Mozilla users care more about privacy than e.g. Microsoft or Google users do, so when Mozilla adds tracking to one of their products, they get more criticism from their users than their competitors would. This isn't unexpected or hypocritical of those users.
I have nothing to base it off other than the results, but someone with a tinfoil hat would wonder if Google chose or influenced the choice of Mozillas leadership to allow Chrome to grow by destroying it from the inside. They were already basically bankrolling the whole company so it's not a big stretch.
It's far from that: https://untested.sonnet.io/notes/defaults-matter-dont-assume...
(I still think FF is one of the best choices privacy wise, it's just that we don't have that many choices left)
> HN is fine with tracking, smartphones, and every surveillance capitalist trick in the book
(meta comment, thinking aloud here, feel free to skip) You might be conflating two different groups of users, each vocal about different subjects. That said, there's a big group of people on HN, who just enjoy being annoyingly contrarian. Then some people derive pleasure from pointing out some "moral fallacy" on whatever they perceive as the opposite part of their political/ideological spectrum ("you think my flashlight app collecting fingerprinting data from your phone is evil? well, your browser doesn't block all cookies by default you hypocrite!"). Life's too short for psychoanalysing the orange site, so I'll stop here.
You must be reading a very different HN to mine. Every single submission that reach front page has comment against all the thing you stated. With little to no support for it.
Ads are all evil and there shouldn't be any has been the theme in just a very recent thread.
First, they have forced telemetry. Okay, it's an early release, it's very basic information, they want to understand how they're doing - I don't like it, but I can understand it. Sets a wrong vibe, though - I had to check if it was from Microsoft and not Mozilla. ;-)
Then, I figured there is no option to use locally-hosted LLMs, which can be something as minimal as simply allowing to configure custom API URL. Somehow, less and less things about Firefox are tinkerer-friendly than they used to be.
That made me wonder if Mozilla used OpenAI-like API, or if they invented their own unique thing for some reason. So I went to look and according to the extension page, it's proprietary ("All Rights Reserved") and I'm too lazy to bother deminifying code from the xpi or remembering how to debug extensions.
Finally, '00s have called and said they wanted their weird floating round thingy UI back, and so I had to return it to the store. (I'm kidding, I don't really mind the visual style. The real issue with that thingy was how it floats on the page. Like, why on Earth it isn't normal right-click menu option that doesn't obstruct the view until it's needed? Or a menu on that toolbar button? It's not even a paperclip to be worth it.)
And then I realized I somehow missed the big "AI you can trust" header, which should've already been a huge red flag.
Based on this I would assume they are using GCP Vertex AI as that's going to be WAY cheaper than rolling it all themselves and hosting the model on a GCP server instance. I would also assume they'd be using the gcloud SDK for Vertex AI/Model Garden, which I believe means they can't just provide for a different endpoint and payload shape if you had a service elsewhere.
Eitherway, at the (presumed) scale they'll probably also be using GCPs API management service, so I would expect further abstraction between what the extension is sending and what the model/Vertex AI expects as a payload. This means providing that kind of "bring your own endpoint" experience would require more bespoke build-out time.
BUT who knows? Maybe this is just straight up hitting the out-of-the-box GCP Vertex AI REST API directly from the extension like some hobby project.
You metnion GCP Vertex AI, is that something like MS Copilot Studio?
Finally someone admits that BonziBuddy was 25 years ahead of its time.
- Currently using Mistral 7B, but ability (by Mozilla) to swap the model used to another open source at any point.
- Hosted "by Mozilla" on their GCP instance.
- No obvious info about what it will cost them to run this since it is free to use.
- No training on user data.
Like others here, I'm very curious about the cost for Mozilla to run this service. It may be less than it initially appears given the 7B model they chose. I do wish they would focus their efforts on creating a very long-term endowment to pay devs for continued Firefox development in lieu of projects like this given the tenuous situation with their Google funding.
I'm not against this kind of thing in theory, but I hope it's being done in a cost-sustainable way.
> The Orbit add-on by Mozilla is a new AI-powered tool that summarizes and answers queries about web content, including articles and videos. It uses a Mistral LLM (Mistral 7B) hosted on Mozilla's GCP instance. The add-on is free to use and works on various websites, including Gmail, Wikipedia, NY Times, YouTube, and more. However, some users have raised concerns about the size of the model and its privacy implications, as well as the fact that it requires an internet connection to function. Additionally, some users have suggested that Mozilla should focus on improving the browser itself rather than developing new add-ons.
https://connect.mozilla.org/t5/discussions/try-orbit-by-mozi...
I don't fancy having a random floaty object in the way of my webpages, no thank you.
Edit: It appears to go away occasionally. This UX is unclear to me.
javascript:window.open('https://chatgpt.com/?q=summarize this page in 100 words: '+encodeURIComponent(window.location.href),'_blank');
Tested on Firefox and Chrome.
Some websites block ChatGPT and can't be summarized this way.
Works in incognito/anonymous mode and doesn't require a ChatGPT account.
You can probably use another AI service with this idea.
javascript:window.open('https://chatgpt.com/?q=summarize this page in 100 words: '+encodeURIComponent(document.body.innerText),'_blank');
Personally I’d worry about using this accidentally and with some sensitive data (eg logins).
I do like the idea though, I’d use this with a local model.
I just wonder if browsers will limit the amount of characters in URLs.
If memory serves me, there was a limit. But it might be high enough to work fro most pages.
I cannot access external links directly. However, if you provide the text or key points from the page, I can help summarize it for you!
and clicking the model selection drop-down produced "Log in to try advanced features"
edit: you're right it requires the user to be logged in to crawl websites. Somehow in my test while writing it I was logged in. My bad.
So while it's handy, it's not perfect.
Hopefully they continue to iterate on this with better integration (for instance, moving to a toolbar icon instead of persistent badge on every page) and then make it ~truly privacy respecting by moving locally.
[0] https://github.com/Mozilla-Ocho/llamafile [1] https://github.com/mozilla/translations
> Commitment to privacy
Buried as the last sentence in a collapsed box at the bottom of the page:
> For the current version, we are using a Mistral LLM (Mistral 7B) hosted within Mozilla’s GCP instance.
And why is it "...Mozilla's GCP instance", not "We quietly send all your data to Google servers, and everyone pinkie-swears that's totally privacy-respecting"?
We don't know whether this is another time that Mozilla execs have sold out users, or shipped something half-baked and vulnerable.
I'm not saying they're leaking the data (by agreement, or negligence), but Mozilla has mediocre credibility in recent years, and there's nothing on this page that improves that reputation.
Regarding Google, for a long time, their thinking seemed to be "We're Google, so of course anything we do is privacy-respecting", not as guidance, but to justify whatever they wanted to do. Also, every time Google gets caught with their hand in the private information cookie jar, it just mints a new industry standard practice.
Are there long and vague terms of service documents backed by a pile of lawyers?
There you go, incentive and means. I'm not even confident companies would see that as a problem when it was raised with them directly, in much the same way that Microsoft hosting all the corporate email seems to be just fine.
Also, does anyone know if we'd be able to point it to our own LLM instance for the guarantee of our data being secure?
Besides that I'm using AI Summary Helper plugin for Chromium-based browsers https://philffm.github.io/ai-summary-helper/ which also allows using Ollama (or OpenAI / Mistral), asking questions to articles and inserting summaries right into the DOM (which is perfect for hoarding articles / forwarding them to Kindle)
Also, while it's nice to have a service option for those without any spare compute, I think it's a bit of a shame on the model considering how even at the 7B class, models like Llama 3.1 8B, Qwen 2.5 8B or Tulu 3 8B, Falcon 3 7B, all clearly outclass Mistral 7B (Mistral 7B is also very bad at multilingual, and is particularly inefficient at multilingual tokenization).
The current best fully open weights (Apache 2.0 or similar) small models currently are probably: OLMo 2 7B, Qwen 2.5 7B, Granite 3.1 8B, and Mistral Nemo Instruct (12B)
There's been a recent launch of a "GPU-Poor" Chat Arena for those interested in scoping out some of the smaller models (not a lot of ratings so very noisy, take it with a grain of salt): https://huggingface.co/spaces/k-mktr/gpu-poor-llm-arena
It's a shame Brave is so far ahead of the game but no one seems to notice.
Disabled it.
After pinning the extension, click on it and deselect the "enabled" option (the line with the purple X). This will kill the floating orb UI but you can still click on the extension in the toolbar to use it.
Other than that, I'm with you. The need for summarizing is a symptom of our increasingly poor communication and degrading writing skills. That, and SEO optimization which attempts to hit as many keywords as possible.
We're heading in a direction where people will use LLMs to pad their writing, so it will appear more substantial then it actually is. The receiver will then parse it though another LLM, because the writing has now become to convoluted, or they simply don't have the time for a ten page essay (in which case the none padded draft would have sufficed).
Admittedly I have found a few useful cases for LLMs, mostly related to text parsing and information extraction, which can be seen as summarizing I suppose, but mostly I have a pretty negative view of LLMs. Part of it may be me getting older and not fully understanding how they work, partly it's also their deployment in areas where I believe communication should be human to human.
Whilst I agree there is a societal decline, there has also always been a corporate tendency towards verbosity and opaqueness.
The old rule from the days of memos still seems to apply: ignore first paragraph, get positive / negative gist from second, learn about the impact in the third, ignore remainder.
Anything that can be expressed in a few sentences, should be, and I tend not to read media that doesn't abide by that rule. If I'm reading long-form content it's because I am looking for detail and nuance that would be lost in a summary.
I do think a reliable video summary generator could be useful occasionally. Interestingly Orbit seems to work on YouTube, presumably by parsing YT's auto-generated transcript.
> Orbit is a Firefox add on that uses AI to summarize and answer queries about web content such as articles and videos.
> When a user asks Orbit to summarize or query content, Orbit gathers the context (eg. text, images, videos, etc.) of the page the user is viewing and provides a summary or answer. Orbit works on websites including Gmail, Wikipedia, NY Times, YouTube, and more.
> For the current version, we are using a Mistral LLM (Mistral 7B) hosted within Mozilla’s GCP instance.
It's interesting they're going with Mistral 7B. Is anybody else using Mistral 7B in production? And in what role?
I've considered using it for general knowledge type questions, and as a way to classify information, but would have never considered it for summarization type tasks due to it's limited context size (8k).
I would never use anything smaller than a 70B model for anything even vaguely medical related!
Everything else is a waste of time and money and energy.
The fact it doesn't even work without www. feels suspicious as well
What's a better browser that isn't basically advertising spyware (i.e. chrome, edge) ?
Not Firefox but forks based on it such as Tor Browser, Mullvad Browser or LibreWolf.
- It's clear from user share graphs that Firefox as just a browser is tending towards irrelevance. No amount of "improving the browser" is going to solve the problem.
- More fundamentally, the browser is just one portal to the internet / world wide web. With technology outside the browser getting increasingly sophisticated, Mozilla necessarily needs to expand their mandate beyond Firefox in order to serve user needs and influence the landscape. Otherwise we might easily end up in a future where the browser becomes irrelevant and everybody interacts with proprietary large models.
- As far as innovation in browser features goes, this seems like a breath of fresh air. Internet users at large deserve access to AI services in a secure and privacy-friendly, and as a pillar of the free web Mozilla is well-placed as a distribution channel to serve these needs. Therefore, this seems like a very good stepping stone / experiment for Mozilla.
There will be execution challenges that need to be figured out. AFAIK Mozilla doesn't have the talent+budget for training large AI models, or even for doing intensive product research. So they're going to have to team up with some other AI expertise -- either explicitly or implicitly, by depending on open source models. Regardless, IMHO this is a risk they have to take and figure it out as they go along.
I don't think I've ever encountered a typo in any of the LLM output I've seen, seems like the exact sort of thing an LLM would be more or less perfect at. Am I wrong to take this as an indication that this text is actually written by a human as a concise marketing example?
Gee, fucking wow, it's almost as if it's plain a day why they've sucked as a corp, non profit, and culturally for a decade at this point.
And while not all people are fond of AI, there are shit tons of people out there who do. Which means you automatically diminish your market share if you don't (because your most important competitors do)
If they aren’t trying things, they would also then be accused of languishing in obscurity.
AI being built into browsers isn’t new. Summarization isn’t novel. It’s not early in the game where resources are crazy high.
Summarization could run with a basic low powered model privately hosted.
Market share changes based on what browsers do well.
They are languishing into obscurity not because they aren't trying things, but because their browser functionality is languishing behind the others.
My guess is this could be useful to many "knowledge workers" who constantly have to crawl, translate and find the meaning of the sugar coated landfill that has become most of the web.
We are right in the middle of the Tower of Babel story.
Seriously if it works reasonably well on legal fine prints I am in.
If you want privacy first AI in the browser here are the tools
I have not studied either product in depth, so I am unable to comment on commonalities or differences.
Jolla has a mixed track record: They supported some phones over 10 years with decently working software (typing on one of those). They also failed at least once to deliver a crowd-sourced tablet to most of the backers. Not a risk-free choice, but at least someone trying to do the right thing.
Is that the right flow? FAQ link is broken so I can't tell.
why not use their technical expertise to built an in-browser "https://big-agi.com/" of sorts where users can paste in their API key and use bleeding edge models in combination with the browser's data which they could expose and manipulate as the creators of Firefox!
this product seems really random and quite frankly weird.
I really don't get the comments that they should not focus on anything else than the current browser.
If other browsers start adding llms, I bet those same people will start complaining that Firefox is outdated compared to those browsers in about a year.
https://www.ghacks.net/2024/12/03/how-to-enable-tab-groups-i...
I've been using Simple Tab Groups [1] for many, many years.
1. https://addons.mozilla.org/en-GB/firefox/addon/simple-tab-gr...
I can't think of a single Firefox feature that's better than "I don't have to faff around with passwords". Maybe if they allowed adblock on mobile, but last I checked they were fannying around requiring nightly builds and whitelisting extensions...
Yes, the default Browser wins (which BTW is not always Chrome), and Mozilla does not put up a fight to change it.
uBlock origin has been one of the allowed extensions for years. As far as I know it's the only browser allowing extensions on phones. It's a shame the allowed extensions is limited compared to desktop but I use uBlock everywhere anyway. What adblocker are you trying to install?
You shouldn't use your browser as your password manager, sometimes you might need them in another context and that creates friction.
I tried Firefox again about once a year, up until about 2022, and left disappointed again each time. Rust was important. Servo too. I'm not just hopping on the 'Mozilla blows money' train for no reason, I'm sad to see the browser languish while they blow money on things that don't matter. This doesn't matter.
I use Brave now, which has a whole host of haters, some for good reason. But it blocks ads and runs circles around Firefox on every platform that I use.
But you notice in all these threads, everyone who theoretically ought to use Firefox comes up with their own little list of nitpicks that justify them not using it.
"I can't use it because I was disgusted when they dropped feature x"
"I won't use it because they spent their money on feature y instead of just doing z"
Meanwhile Chrome doesn't give a fuck what you think and does whatever it wants and people keep using it regardless.
Firefox is doomed to be left with the niche audience of people who ignore the 95% of what it does right to focus on the 5% that it does wrong.
As a Firefox user since 2002 who has never switched away, this part of the situation feels insane. People nitpick over Mozilla and decide they'd rather be steamrolled by Google. What?
1. Using latest models (gpt-4o & claude-3.5) which show major improvements in quality
2. Extensive prompt engineering focused on distilling surprising insights vs. pure summarization
3. Structured output format that's easier to parse
Re: transparency - you're right, we need to add more details. I'm new to this and still learning how to communicate this.
Btw, it's free to use 1x/day, and here's an example: https://nuggetize.com/n/claude-shannon-wikipedia-47ca5e5b
It shocks me every time just how fast Chrome is. It is legitimately a superb piece of software. Going to Librewolf after feels like going back ten years in hardware.
Can we please start spending some money to make Firefox better? Instead of whatever Mozilla is currently doing?
Firefox Quantum was great. But why stop? Just keep doing that! It's the only thing you should be doing!
uBlock Origin on both (wouldn't browse the web without it). Vimium and Dark Reader on Librewolf. But turning Dark Reader off does speed it up by _a lot_.
Chrome still seems faster but now they're both playing in the same league.
Never thought about it.
Mistral has released Codestral under a new license, but that's not the one used here. https://mistral.ai/news/mistral-ai-non-production-license-mn...
In fact, they should have done that a decade a ago.
Mozilla has been around since the late '90s and should have evolved beyond just being a browser company. They launched a VPN service when VPNs were already everywhere, and they did the same with a bookmark manager when others were already offering similar solutions. Mozilla is always catching up, never leading, and that's a common issue with many big open-source and free software companies. They often pretend to be a business that isn't heavily propped up by big tech donations.
If I were leading a browser company, my focus would have been aggressively directed towards small business software. I’d create an internet and privacy-focused affordable minimal business software suite that lives within the browser — a combination of Proton and Zoho. And I’d strongly avoid building things that should be browser extensions.
If they shipped vertical tabs I'd probably switch back to Firefox. But that would require focusing on actual browser UX instead of random offshoots. What's their actual product differentiation from Chrome these days?
Allowing Backspace to go back a page.
The built-in screenshot tool.
Being able to turn off smooth scrolling.
Support for a menubar, so that I can navigate the features I want quickly.
Being able to choose page encodings (I use a non-Latin charset language).
A usable/useful bookmark manager with things like sorting, tags/labels, timestamps, etc.
"Restore Previous Session" feature.
These are just a few features off the top of my head, I know there are many more.
Of course, for certain sites like Google Docs and YouTube, I have to use Chrome for obvious reasons, but for most browsing I use Firefox (and qutebrowser.)
Easily solved by https://chromewebstore.google.com/detail/go-back-with-backsp... their first party webext.
> Being able to turn off smooth scrolling.
Can disable as easily as Firefox -- chrome://flags/#smooth-scrolling
Actually, Firefox lately introduced a bug that will reset smooth scrolling everytime I remote desktop to my Windows machine. Which is annoying AF and they haven't fixed it for months.
https://bugzilla.mozilla.org/show_bug.cgi?id=1912246
> Being able to choose page encodings (I use a non-Latin charset language).
https://chromewebstore.google.com/detail/set-character-encod...
> "Restore Previous Session" feature.
You can do it since.. forever?
Settings -> On Startup -> Continue where you left off
Or just press Ctrl+shift+T when you restart Chrome to restore it manually.
There are some more feature-rich session manager extensions, but they're usually available across Firefox and Chrome.
I mostly agree, but whoever came up with this shortcut should get choked.
I think it’s pretty clear why they keep on doing this type of side project. They are trying anything they can think of to diversify revenue.
Providing monopoly protection to Google.
They have no interest in actually competing for market share.
about:config, sidebar.revamp = true, sidebar.verticalTabs = true
It's getting there.
- continued support for manifest V2 (primarily because ublock origin would stop working if forced to V3 only)
- the firefox address bar is way smarter for any given string i type in than chrome's. it's ability to surface the most relevant deeplinks from my history, vs top level site, vs perform a web search, is night and day difference from the randomness that other browser search bars offer.
- I have the opportunity to use Zen (a Firefox fork) [0] and it's 100% interoperable with my vanilla Firefox instances across devices -- i can even send tabs from my Firefox Nightly on Android to my Zen instances on Windows or Mac. BTW Zen has vertical tab support, (more) first-party multi-profile support, and preserves the address bar behaviors of vanilla Firefox.
You could build an AI Assistant, or you could spend a month bikeshedding some design details of vertical tabs.
Because some project manager had "AI" on his performance goals this year.
Firefox is buggy as hell - which is incomprehensible given its age, but older brother Netscape had the same problem 20-ish years ago. The Netscape 4.x days were absolute hell and you could go hardly a day without the browser crashing.
Despite this, it's packed with nonsense no one asked for like Pocket. Which is a coincidence because "AI assistant for Firefox" is the dictionary definition for redundant things no one asked for, with better alternatives preexisting.
At this point Firefox needs to die and something new and manageable - with energetic developers - needs to take its place. Maybe Microsoft could open source the original Edge engine? The one before they bent over for the long dick of Google Chrome.
Not being the progenitor and linchpin of surveillance capitalism.
Agreed that it would be nice if they had better focus though.
Mozilla's brand is "pro privacy", it does make sense for them to launch an AI product with that brand position. I doubt it'll be successful because I don't think enough people actually care about privacy, but still.
I feel like it's a common HN sentiment to say "why don't Mozilla just focus on the browser?!"... the answer is because barely anyone is using it and there's very little they can do to move the needle on that. IMO they're an organization looking for a purpose.
Firefox is like the shitty best option that camps out in its niche, it sucks but it is really hard to push out of the way.
Of course, no option to use a local model even though the one they're using is small enough that its perfectly reasonable to use locally. Even on a cell phone.
> basic utility program decided to become political
looks like they wanted to avoid it, isn't it?
