Why Qontinui models the GUI as a state machine instead of recording a script
Most automation tools record a fixed sequence of steps: click here, type there, wait, click again. That script works until the interface shifts — a dialog appears, a page loads slowly, an element moves — and then the whole sequence breaks. Model-based automation takes a different approach. Instead of describing what to do, you describe what the application looks like: its distinct screens (states) and the ways you can move between them (transitions). The engine then figures out how to reach your goal from wherever it currently is.
You give Qontinui a model of the application — a graph where states are the nodes and transitions are the edges. Automation becomes navigation: name a destination, and the engine finds a path there, recovering automatically when the application is not where it expected.
A distinct screen, dialog, or condition in the application — identified visually rather than by hardcoded coordinates or element IDs. States are the nodes of the model graph.
State Machines →A connection between states describing how to get from one to another, and which states become active or inactive afterward. Transitions are the edges of the graph.
Transitions & Pathfinding →Template matching against the live screen determines which states are currently active. This is how the engine knows where it is before deciding where to go next.
Visual Recognition →Because the engine verifies the current state before each step, it can re-plan when the application drifts — handling a stray dialog or a slow load instead of failing on a missing click.
A single model serves many goals. Once states and transitions exist, any reachable screen becomes a valid destination — no need to script each path by hand.
Visual identification works across screen sizes and UI toolkits. The model describes appearance, not DOM structure, so it applies to web, desktop, and other GUI applications alike.
A model can be reasoned about and exercised without driving a real screen. Qontinui Web includes mock execution so you can validate the state machine logic before running against a live application.
Qontinui's model-based approach descends from Brobot, a Java GUI-automation library developed from 2018 to 2024; Qontinui is its Python successor. The underlying method was formalized in the paper "Model-based GUI Automation", published in Springer's Software and Systems Modeling journal in October 2025.
At a high level, executing a model-based automation is a loop of observe, decide, act:
Note: This page explains the theory. To build a model in the visual editor, see the Qontinui Web how-to guides.
