The demo buying toothpaste shows the difficulty of these tasks. Toothpaste itself was very underspecified, and it essentially randomly chose from a huge list. Some tasks may have past actions that could help guide, others won't have any to inform. Failure cases abound -- maybe the toothpaste you previously bought is no longer available. Then what? Ultimately how much time did this particular example save since you need to double check the result anyway? This is what doomed Alexa for the purchasing experience that Amazon assumed it would enable in the first place.
I think it'd be better to show more non-trivial examples where the time savings is clear, and the failure cases are minimized... or even better how it's going to recover from those failure cases. Do I get a bespoke UI for the specific problem? Talk to it via chat?
Great points! For sure, the whole agentic browsers space is still super early.
We are also just getting started and trying to narrow down on a high-value niche use-case.
There are few repetitive, boring use-cases where time saving could be meaningful -- one example: Walmart 3rd-party sellers routinely (multiple times a day) keep checking prices of the competitor products to price their products appropriately. This could be easily automated with current agentic browsers.
> --- How we built? We patch Chromium's C++ source code with our changes, so we have the same security as Google Chrome. We also have an auto-updater for security patches and regular updates.
So you rebuild your browser on every Chromium release? Because that's the risk: often changes go into Chromium with very innocent looking commit messages than are released from embargo 90 days later in their CVE reference
We would've preferred to build this as browser extension too.
But we strongly believe that for building a good agent co-pilot we need bunch of changes at Chromium C++ code level. For example, chromium has a accessibility tree for every website, but doesn't expose it as an API to chrome extension. Having access to accessibility tree would greatly improve agent execution.
We are also building bunch of changes in C++ for agents to interact with websites -- functions like click, elements with indexes. You can inject JS for doing this but it is 20-40X slower.
I'm not GP, but I agree that if your goal is to empower the end user and protect him from corporate overlords, then Firefox is a more logical choice to fork from.
I wonder if someone on the Chromium team will upstream all these BrowserOS changes, or "Not Invented Here" and re-implement it all for Gemini / Google Assistant.
We don't mind upstreaming. But I don't think Google Chrome/Chromium wants to expose it as an API chrome extensions, if not they would've done this long time ago.
From Google's perspective, extension are meant to be lightweight applications, with restricted access.
I'm not really interested in AI agents for my webbrowser, but it would be pretty cool to see a fork of chromium available that, aside from being de-googled, relaxes all the "restricted access" to make it more fun to modify and customize the way you guys are. Just a thought, may be more of a market for the framework more than the product :)
See Sciter. A very cool, super lightweight alternative to Electron, but unfortunately it seems like a single developer project and I could never get any of the examples to run.
We had this exact thought as well, you don't need a whole browser to implement the agentic capabilities, you can implement the whole thing with the limited permissions of a browser extension.
There are plenty of zero day exploit patches that Google immediately rolls out and not to mention all the other features that Google doesn't push to Chromium. I wouldn't trust a random open source project for my day-to-day browser.
Check out rtrvr.ai for a working implementation, we are an AI Web Agent browser extension that meets you where your workflows already are.
Brave Browser (70M+ users) has validated that a chromium fork can be viable path. And it can in fact provide better privacy and security.
Chrome extensions is not a bad idea too. Just saying that owning the underlying source code has some strong advantages in the long term (being able to use C++ for a11y tree, DOM handling, etc -- which will be 20-40X faster than injecting JS using chrome extension).
my guess is it's just an electron app or chromium wrapper with an ollama wrapper to talk to it (there are plenty of free open source libs to control browsers).
But we are much more performant than other libs (like playwright) which are written in JS, as we implement bunch of changes at chromium source code level -- for example, we are currently implementing a way to build enriched DOMtree required for agent interactions (click, input text, find element) directly at C++ level.
When someone in their infinite wisdom decides to refactor an api and deprecate the old one, it creates work for everyone downstream.
Maybe as an industry we can agree to do this every so often to keep the LLMs at bay for as long as possible. We can take a page out of the book of the maintainers of moviepy for shuffling their apis around, it definitely keeps everyone on their toes.
We were thinking of implement MCP protocol into the browser, so the browser can be an MCP server (that exposes bunch of tools -- navigation, click, extract) and you can connect that to your agent, would that work?
This is very exciting given the rumor that OpenAI will be launching a (presumably not open source) browser of their own this summer. I've joined your Discord, so will try it soon and report back there. Congrats on launching!
> our agent runs locally in your browser (not on their server)
That's definitely a nice feature. Did you measure the impact on laptop battery life in a typical scenario (assuming there is such a scenario at this early stage)
The agent running by itself shouldn't impact battery life, it is similar to a lightweight chrome extension and if you think about it, it's an agent browsing the web like human would :)
If you run LLMs locally (using Ollama) and use that in our browser, that would impact battery life for sure.
So would this or any AI browser go out and fetch a list of the best deals for my trip to Iceland? After Show me all the options it has found for flights, hotels, car rentals and show cheapest/best prices with all details (fly out of and into with times) to even allow me to pay for each item on same page I asked it to do so? As well it could group the overall best deal with details and then i can just click to pay instantly and or make some edits.
I think it'd be better to show more non-trivial examples where the time savings is clear, and the failure cases are minimized... or even better how it's going to recover from those failure cases. Do I get a bespoke UI for the specific problem? Talk to it via chat?
This whole world is non-trivial. Good luck!
We are also just getting started and trying to narrow down on a high-value niche use-case.
There are few repetitive, boring use-cases where time saving could be meaningful -- one example: Walmart 3rd-party sellers routinely (multiple times a day) keep checking prices of the competitor products to price their products appropriately. This could be easily automated with current agentic browsers.
But for non-technical folks, agentic browsers seems like a good UX to build such and many more automations.
So you rebuild your browser on every Chromium release? Because that's the risk: often changes go into Chromium with very innocent looking commit messages than are released from embargo 90 days later in their CVE reference
But we strongly believe that for building a good agent co-pilot we need bunch of changes at Chromium C++ code level. For example, chromium has a accessibility tree for every website, but doesn't expose it as an API to chrome extension. Having access to accessibility tree would greatly improve agent execution.
We are also building bunch of changes in C++ for agents to interact with websites -- functions like click, elements with indexes. You can inject JS for doing this but it is 20-40X slower.
Can't imagine Firefox acquiring a Firefox fork!
From Google's perspective, extension are meant to be lightweight applications, with restricted access.
See Sciter. A very cool, super lightweight alternative to Electron, but unfortunately it seems like a single developer project and I could never get any of the examples to run.
https://sciter.com/
What use-cases do you have in mind? like scraping?
There are plenty of zero day exploit patches that Google immediately rolls out and not to mention all the other features that Google doesn't push to Chromium. I wouldn't trust a random open source project for my day-to-day browser.
Check out rtrvr.ai for a working implementation, we are an AI Web Agent browser extension that meets you where your workflows already are.
Chrome extensions is not a bad idea too. Just saying that owning the underlying source code has some strong advantages in the long term (being able to use C++ for a11y tree, DOM handling, etc -- which will be 20-40X faster than injecting JS using chrome extension).
How are you planning to make the project sustainable (from a financial, and dev work/maintenance pov)?
plan is to sell licenses for Enterprise-version of browser, same as other open-source projects.
But we are much more performant than other libs (like playwright) which are written in JS, as we implement bunch of changes at chromium source code level -- for example, we are currently implementing a way to build enriched DOMtree required for agent interactions (click, input text, find element) directly at C++ level.
We also plan to expose those APIs to devs.
When someone in their infinite wisdom decides to refactor an api and deprecate the old one, it creates work for everyone downstream.
Maybe as an industry we can agree to do this every so often to keep the LLMs at bay for as long as possible. We can take a page out of the book of the maintainers of moviepy for shuffling their apis around, it definitely keeps everyone on their toes.
I don't have Mac or Windows.
still a team of 2 people, so bunch things on our plate.
We were thinking of implement MCP protocol into the browser, so the browser can be an MCP server (that exposes bunch of tools -- navigation, click, extract) and you can connect that to your agent, would that work?
What is your use-case? Happy to chat on discord! https://discord.gg/YKwjt5vuKr
Browser wars have begun.
> that OpenAI will be launching a (presumably not open source) browser of their own this summer.
For sure, won't be open-source. I bet in some parallel world, openAI would be non-profit and actually open-source AI :)
(Will you ever make a better FydeOS, or if you're laser-focused, perhaps be open to sharing some with them, so they could?)
I'll check out FydeOS!
That's definitely a nice feature. Did you measure the impact on laptop battery life in a typical scenario (assuming there is such a scenario at this early stage)
If you run LLMs locally (using Ollama) and use that in our browser, that would impact battery life for sure.
What are the system requirements? And shouldn't they be listed on your website?
hardware requirements are minimal, same as Google Chrome, if you BYOK API keys for agents and are not running LLMs locally.
You can bring your own API keys and change the default to any model you local.
Or better run model locally using Ollama and use that!
We are working on smaller, fine-tuned model too, which will be the default soon! It should be much faster and precise at navigation tasks.