Skip to Content

2026-10-12

An AI Browser Agent as a Low-Vision Computer Tool: What Actually Helped Me

No more hunting for hamburgers and three dots, which probably may or may not, and might, be in the upper or perhaps lower extreme left or right, depending on your version, maybe.

If you have never had to hunt for them: the “hamburger” is the ☰ menu icon, and the “three dots” are the ⋮ menu. Every website puts them somewhere different, and every update moves them. I use ZoomText. When the screen is magnified, I see a small piece of the page at a time, so finding a tiny icon in a corner means sweeping the whole screen, piece by piece. Then the form has twenty boxes, a dropdown that will not open, and a code that arrived by email.

For the last few days an AI has been doing that hunting for me, inside my own everyday browser, while I watch. It has changed my life. This post is about how it works, what it actually did, and the rules I use to keep it safe.

The Problem: Several Hundred Times a Day

I already worked with an AI assistant, Claude Code, in a terminal. It could write and run things for me, but the moment a web page was involved, I became its eyes. It would ask me what was on the screen, ask me to scroll, ask me to zoom. One evening I told it: you can read the page and shrink it yourself instead of asking me several hundred times a day.

So we built that. It took one day.

How It Works, in Plain Words

Every Chromium-based browser (Chrome, Edge, Brave, Opera and Vivaldi) has a built-in remote control called the Chrome DevTools Protocol. It is the same channel the browser’s own developer tools use. When the browser is started with a debugging port switched on, a program can connect to it and do what you could do: read the page, click a button, type in a box, attach a file.

My setup:

  • My desktop runs NixOS Linux, and my browser is Vivaldi.
  • The AI runs on a Linux server. My other servers run Linux and FreeBSD.
  • The AI reaches my browser through a private SSH tunnel. The debugging port listens only on my own computer, never on the network.
  • One command turns remote control on, and the same command turns it off. Each switch restarts the browser in about fifteen seconds, and my tabs come back.
  • A small Python script, about 300 lines, gives the AI its commands.

The most important command is read. It turns the page into an outline of headings, buttons, links and boxes: the same information a screen reader announces. The AI does not need a screenshot to know what is on my screen. Another command lists every box in a form, with its label and whether it is filled, without ever printing what is typed inside. Others click a button by its text, type into a box, attach a file, save a page as a PDF, or make one page bigger right away.

Some modern web forms ignore text that a script simply drops into a box. The fix was to send real mouse clicks and real key presses, the same events my own hands would make. After that, even the stubborn custom dropdowns opened.

What It Actually Did

These are real jobs from the first three days.

  • A government registration form. The address lookup quietly filled a hidden “street number suffix” box, which turned my address into a wrong one. A screenshot could not show which box was wrong. The list of boxes found it in one step.
  • An online store, start to finish. Store settings, the product page, and the domain’s DNS records at the registrar. The AI read each page, filled the boxes, and told me what was left.
  • A business payment account, including the identity check. The AI attached my ID and a utility bill. I checked them and pressed Submit myself.
  • Email verification codes. The AI fetched the six-digit code from my mailbox and typed it in. I never had to read a changing number off the screen.
  • Official paperwork. It downloaded a registration certificate from a government email link and filed it with my other documents.
  • Research for this post. The keyword research behind this page was read from Google Trends through the same tool.

Is an AI Browser Safe?

Not by default. The debugging port has no password. While it is open, any program on that computer could take control of the browser, including every site I am logged into. Google takes this seriously enough that, since Chrome 136, Chrome ignores the debugging switch on your normal profile, because attackers were using it to steal cookies. That is a sensible decision, and it tells you how much is at stake.

These are my rules:

  • Off by default. Remote control is on only while the AI is helping me with a page. Then it goes off.
  • My computer only. The port listens on my own machine, and the AI reaches it through an encrypted SSH tunnel. Nothing is exposed to the internet.
  • I press the binding buttons. Pay, Submit, Agree and Create account are always my clicks. The AI can find the button and tell me where it is. It does not press it.
  • No bank or government logins. The AI does not drive sign-ins to my bank or to government accounts. For those, I save the page or paste the text.
  • Secrets stay hidden. When a password is needed, the tool reads it from a file and types it. It never appears on a command line or on the screen.
  • My tabs are mine. The tool closes only the tabs it opened itself.

It is not perfect, and I would rather say so. On the third day the AI ran some research in my browser under the wrong Google account, someone else’s in my household, because that account happened to be first in the list. Nothing was harmed, and we checked that nothing was saved to that account. The tool now always opens Google pages under my own account. Mistakes like that are the reason a human watches.

Other Browsers

  • Chrome, Edge, Brave, Opera, Vivaldi: all Chromium, all the same protocol. Chrome 136 and later need a separate profile for debugging, not your everyday one. Vivaldi still allowed it on my own profile.
  • Firefox: switched its version of this protocol off by default in Firefox 129 and removed it in Firefox 141. Firefox now uses WebDriver BiDi, a newer cross-browser standard.
  • Safari: has its own driver for classic WebDriver. WebDriver BiDi support has not shipped yet.

There are also ready-made AI browser agents: Claude in Chrome, ChatGPT’s agent and the Atlas browser, Perplexity’s Comet, Opera’s Browser Operator, and open-source projects such as Browser Use. Some of them work in a separate browser, and some can act on their own without anyone watching. If you have low vision, I would look for two things: does it use your real logged-in browser, and does it stop and let you make the final click?

Is This the Best AI for Low Vision?

Most “AI for low vision” advice today is about smart glasses and phone apps that describe what a camera sees. That is useful, but it does not help with the web forms that run my life: bills, government services, banking, shopping. An AI browser agent fills that gap.

Researchers are seeing the same thing. A 2026 case study, “Zooming In” on Agentic Web Browsers as Assistive Technologies, by Laura Colazzo and Giuseppe Anzillotti, worked with a low-vision technology expert. They found that “despite the current limitations, the navigation experience is notably fluid and flexible.” That matches my experience exactly.

If You Want to Try It

  • Start with a ready-made agent that asks before it acts, and try it on a page that does not matter.
  • If you build your own, keep the debugging port on your own computer only, and switch it off when you are done.
  • Never leave a debugging port open on a browser that is logged into your bank.
  • Keep the final click for yourself.

Closing

I still use ZoomText every day. What changed is that I no longer spend my energy finding things. I spend it deciding things. The AI reads the page, finds the box, and tells me where the button is. I make the choice and press it. For someone with low vision, that is the difference between dreading a web form and just getting it done.