Actions are the atomic operations of automation — clicking a button, typing text, finding an image, waiting for an element to vanish. Where states and transitions describe the structure of an application, actions are what the engine actually does when traversing a transition.
An action does not point at a fixed coordinate or a brittle selector. It targets an element by its visual appearance — the engine first finds the element on screen with template matching, then acts at the matched location. This keeps actions robust as the layout shifts.
Conceptually, actions fall into a few families. Together they let a transition observe the screen, manipulate input, branch on conditions, and orchestrate other work.
Locate an element on screen, or wait for one to appear or disappear. Find is the foundation other actions build on — it resolves the target before any interaction happens.
Drive real input: click, move, drag, scroll, type text, and press key combinations. These produce the actual interactions a user would perform.
Branch on conditions, loop over items, and read or write variables — so a transition can express logic, not just a flat list of clicks.
Coordinate the model itself (navigate to a state, run another workflow) and reach beyond the GUI when needed (run a script, invoke an AI prompt).
Most interactions follow the same shape: resolve where the target is, then act there. Understanding this pattern explains why visual automation tolerates layout changes that break coordinate-based scripts.
When an element cannot be found at the expected confidence, the engine does not have to fail outright. Qontinui includes an optional self-healing system that attempts recovery before giving up.
Successful element locations are remembered, so a repeat lookup can replay instantly instead of searching from scratch.
When exact matching fails, the engine retries at lower similarity thresholds and multiple scales to find a near match.
A vision model — local or cloud — can locate an element from a natural-language description when template matching alone is not enough.
Note: This page covers the concepts. For the complete catalog of action types you can drop into a workflow, see the Action Types reference.
