CLI-Anything Gives Desktop Software a Command Line So Agents Can Use It

There is a specific kind of despair that comes from watching an AI agent try to use desktop software. It takes a screenshot, guesses where the menu is, clicks, misses, takes another screenshot. Twenty seconds later it has moved a slider by two pixels and learned nothing.

CLI-Anything, from the University of Hong Kong's Data Intelligence Lab, is an argument that the whole approach is wrong. Don't teach the agent to see the screen. Give the software a command line, then let the agent type.

The pitch is one line: making all software agent-native. The mechanism is less glamorous and more interesting — point a command at any application whose source you have, and a seven-stage pipeline generates a structured CLI wrapper for it, complete with tests, docs, and a package you install with pip.

The seven stages

The pipeline runs inside a coding agent, invoked as /cli-anything <source path or repo>. Analyze: scan the source and map GUI operations onto API calls. Design: decide command groups, state model, output formats. Implement: build a Click-based CLI with a REPL, JSON output, and undo/redo. Plan tests: write a test plan into TEST.md. Write tests: implement the suite. Document: record results back into the same file. Publish: create a setup.py and install the command onto your PATH.

There is also a stage 6.5 that generates a SKILL.md, so the resulting CLI is discoverable by an agent as a skill rather than something a human has to wire up. And a /cli-anything:refine command that does gap analysis and extends coverage incrementally.

The naming is rigid on purpose: commands live under the cli_anything.<software> namespace, packages are called cli-anything-<software>. Boring conventions are what make tooling composable, and this project is clearly thinking about a world where there are hundreds of these.

Real backends, not toy reimplementations

The detail that separates this from a weekend project: the wrappers drive the actual software. LibreOffice produces a real PDF. Blender renders a real 3D scene. GIMP does whatever GIMP does to an image. The alternative — reimplementing a subset of the app's behaviour in Python — produces something that looks like it works until you need one feature the reimplementation skipped.

The cost is that you need the applications installed. The benefit is that the output is indistinguishable from what a human operator would have produced, because it was produced by the same engine.

Every command emits JSON when asked, which is the difference between a CLI that an agent can parse and a CLI that an agent has to squint at. There is a stateful REPL for interactive work and plain subcommands for scripts, plus persistent project state and undo/redo.

What it covers

The scope is deliberately broad, and it reads like a list of software that people have been trying and failing to automate: GIMP, Blender, and Kdenlive for creative work; LibreOffice for office documents; Godot and s&box for 3D and games; QGIS and FreeCAD for science and engineering; ComfyUI and Ollama on the AI side; LLDB, RenderDoc, and Nsight for debugging.

ComfyUI is the one worth pausing on if you work with images. ComfyUI already has an API, but it is a graph API — nodes, wires, and a JSON blob whose schema depends on which custom nodes you have installed. A CLI wrapper that exposes intent-level commands, with a defined state model and JSON output, is exactly what an agent needs to run an image pipeline without reverse-engineering someone's workflow file.

The testing claim

The repository reports roughly 2,461 tests: 1,732 unit, 579 end-to-end, and 19 Node.js, with a claimed 100 percent pass rate. The E2E layer runs against the real backend, which is the only kind of E2E that matters here. If LibreOffice stops producing a PDF, the test fails, and you find out before your agent generates forty broken files.

Treat the "100 percent" as a snapshot, not a guarantee. Generated test suites have a habit of encoding the generator's assumptions. But the four-layer structure — unit, native E2E, real-backend E2E, and CLI subprocess — is the right shape.

The honest limitations

The README lists them, which is a good sign.

It depends on frontier models. Claude Opus 4.6 or Sonnet 4.6, GPT-5.4, that class. A weaker model produces incomplete or wrong CLIs that need heavy manual repair, which means the tool's usefulness is tied to a subscription you may not have.

It depends on source code. The pipeline analyses source; if all you have is a compiled binary, quality and coverage drop sharply. That quietly excludes most commercial software, which is most of the software people actually want to automate.

One run rarely covers everything. Expect to run /refine several times before the harness is production-grade. Some CLIs require the upstream application installed. Platform maturity varies — Goose, OpenCode, Codex, Hermes, and Reasonix are marked experimental, and Windows users will meet bash and cygpath compatibility problems.

The maintenance question nobody wants to ask

A generated wrapper is a commitment. The moment you adopt cli-anything-libreoffice, you have added a layer between your agent and the application, and that layer has to keep working when LibreOffice changes its document model or Blender renames an operator.

In practice this means three things. You should keep the generated harness in your own repository rather than depending on someone else's, so you can patch it. You should re-run the pipeline when you upgrade the underlying app, and diff the result. And you should treat TEST.md as the contract — the tests are the only reason you will notice a break before your agent produces forty malformed files.

The economics work because regeneration is cheap. That is the actual insight: the wrapper does not have to be perfect forever, because the thing that produced it can produce it again. Traditional SDKs are expensive to maintain because humans write them. This is not that.

The CLI-Hub bet

There is a package manager, cli-anything-hub, for browsing and installing community-built wrappers. That is the part of the project with the most upside and the most obvious risk. A hub of generated CLIs is a hub of code that runs with your permissions and touches your files.

Whether it becomes a useful ecosystem or a supply-chain story depends entirely on whether the project treats review as part of publishing. Right now the answer is "partly": the tests are generated, the docs are generated, and the review is whatever the community does.

Why this matters beyond the repo

Computer use has been the industry's favourite demo and least favourite product for two years. The demos work because someone carefully scripted a task. The products stall because pixel-level control is brittle in a way that never quite goes away.

CLI-Anything proposes a different division of labour: instead of making agents better at being humans, make software better at being machines. Every application with a source tree becomes a candidate for a generated interface, and the interface is testable in a way that a screenshot loop never is.

The trade-off is real. Generated CLIs are only as good as the model that wrote them, and someone has to own the wrapper when the upstream app changes. But the direction is right. An agent that types cli-anything-blender render --scene scene.blend --out frame.png and gets deterministic output is doing something fundamentally more useful than an agent that clicks around a viewport and hopes.