"""go to browser-use.com and use the computer.get_dom tool. Share the extracted DOM exactly with me."""
The response: |SCROLL|<body node_id=9d5f6b01> (vertical view=749px, 0px above, 11932px below)
<a node_id=f9367e7b>
Browser Use
<button node_id=eaeb1667 aria-label="Open menu">
Then I checked how it clicks: It clicks by node_id (e.g. f9367e7b) and as alternative coordinates.
In browser-use we
1. interact with the DOM by backend_node_id and coordinate fallback
2. use the exact same token for scroll containers with "|" and caps lock (|SCROLL|)
3. use scroll containers with context how much above / below
4. use the same llm representation with <tag filtered_attributes>
5. use element texts in new lines with indentation
Things I noticed they could improve:
1. Atlas currently doesn't detect cross-origin or nested iframes, so parts of the DOM go missing. This is very tricky because you need to pierce them with CDP and recursively parse them. (e.g. https://csreis.github.io/tests/cross-site-iframe.html)
2. They waste 10 tokens every item: [tab]<div node_id=83876787. They could cut that to <a id3. (3 Tokens)
3. They keep full links -> They could shorten them easily to save tokens.
4. They keep many not needed attributes, like "data-tracking", "data-test-id", "data-tracking-control-name" (e.g. on LinkedIn.com)
5. For all elements they use [tabs] before which is not needed.
6. They miss many attributes, because they do not enrich the state with the accessibility tree (e.g. for min/max values or hints like required)
No comments yet.