Action Types

Complete reference for all automation actions in Qontinui

What are Actions?

Actions are the atomic operations performed during automation. They represent specific tasks like clicking a button, typing text, or finding an image on screen. Actions are organized into processes and executed during state transitions.

Key Concepts

•Actions execute sequentially within a process
•Failed actions can be retried automatically
•Actions can target images, coordinates, or the last found image
•Pauses can be added before and after each action

Action Categories

Mouse Actions

Control mouse movement and clicks

CLICK

Left-click at the target location

Parameters:

target: Image, coordinates, or 'Last Find Result'
offset: Optional x/y offset from target

Example: Click the login button found via image recognition

DOUBLE_CLICK

Double-click at the target location

Parameters:

target: Image, coordinates, or 'Last Find Result'

Example: Double-click to open a file

RIGHT_CLICK

Right-click to open context menu

Parameters:

target: Image, coordinates, or 'Last Find Result'

Example: Right-click to show context menu

DRAG

Drag from source to destination

Parameters:

source: Starting location
destination: Ending location

Example: Drag a file to trash

MOUSE_MOVE / MOVE

Move mouse without clicking

Parameters:

target: Destination coordinates or image

Example: Hover over an element to reveal tooltip

MOUSE_DOWN / MOUSE_UP

Press or release mouse button

Parameters:

button: Mouse button: left, right, middle

Example: Hold mouse button for custom drag operations

SCROLL / MOUSE_SCROLL

Scroll mouse wheel

Parameters:

amount: Scroll amount (positive = down, negative = up)
direction: Optional: vertical or horizontal

Example: Scroll down to load more content

Keyboard Actions

Type text and press keys

TYPE

Type a text string

Parameters:

text: Text to type
interval: Optional delay between keystrokes (ms)

Example: Type username into login field

KEY_PRESS

Press and release a key

Parameters:

key: Key name (e.g., 'enter', 'tab', 'ctrl+c')

Example: Press Enter to submit form

KEY_DOWN / KEY_UP

Press or release a specific key

Parameters:

key: Key name to press or release

Example: Hold Shift while clicking for multi-select

Vision Actions

Find and verify visual elements

FIND

Locate an image on screen and store its location

Parameters:

target: Image to find
similarity: Match threshold (0.7-0.95)

Example: Find the submit button before clicking it

Important: The found location is stored and can be used by subsequent actions with target: 'Last Find Result'

EXISTS

Check if an image exists (returns true/false)

Parameters:

target: Image to check for
similarity: Match threshold

Example: Verify a confirmation dialog appeared

VANISH

Wait for an image to disappear from screen

Parameters:

target: Image to wait for disappearance
timeout: Maximum wait time (ms)

Example: Wait for loading spinner to disappear

Navigation Actions

Navigate states and execute processes

GO_TO_STATE

Navigate to a target state using pathfinding

Parameters:

stateId: ID of target state

Example: Navigate to the settings page from anywhere

Important: Uses automatic pathfinding through the state graph

RUN_WORKFLOW

Execute another workflow by ID

Parameters:

workflowId: ID of workflow to execute

Example: Run a reusable login workflow

Utility Actions

Timing and debugging utilities

WAIT

Pause execution for a specified duration

Parameters:

duration: Wait time in milliseconds

Example: Wait 2 seconds for page to load

SCREENSHOT

Capture the current screen

Parameters:

filename: Optional filename for screenshot

Example: Take screenshot for debugging or verification

Common Action Parameters

target

Object or String

Specifies where the action should be performed

{"type": "image", "imageId": "..."} - Find and target an image

{"type": "coordinates", "x": 100, "y": 200} - Use fixed coordinates

"Last Find Result" - Use location from last FIND action

similarity

Number (0.0 - 1.0)

Image matching threshold for visual actions

0.7 - 0.8 - Fuzzy matching (more tolerant)

0.85 - 0.9 - Standard matching (recommended)

0.95+ - Exact matching (strict)

retry_count

Integer

Number of retry attempts if action fails (default: 3)

timeout

Integer (milliseconds)

Maximum time to wait for action completion (default: 5000)

continue_on_error

Boolean

If true, process continues even if this action fails (default: false)

pause_before_begin

Integer (milliseconds)

Wait time before executing action

pause_after_end

Integer (milliseconds)

Wait time after action completes

offset

Object {x, y}

Offset from target location in pixels

Best Practices

Use FIND before CLICK for reliability

Always use a FIND action before clicking an image target. This ensures the element exists and updates the 'Last Find Result' location.

1. FIND (target: login_button) → 2. CLICK (target: "Last Find Result")

Set appropriate similarity thresholds

Start with 0.85 and adjust based on results. Lower for elements that change slightly, higher for exact matches.

Add pauses for slow applications

Use pause_after_end for actions that trigger slow operations (e.g., page loads, animations).

Use continue_on_error sparingly

Only use for truly optional actions. Most actions should fail the process if they fail.

Combine actions into logical processes

Group related actions into named processes for reusability and clarity (e.g., 'login_sequence', 'submit_form').

Next Steps

State Transitions

Learn how to connect states using processes

Working with States

Understand state identification and structure

What are Actions?

Key Concepts

•Actions execute sequentially within a process
•Failed actions can be retried automatically
•Actions can target images, coordinates, or the last found image
•Pauses can be added before and after each action

Action Categories

Mouse Actions

Control mouse movement and clicks

CLICK

Left-click at the target location

Parameters:

target: Image, coordinates, or 'Last Find Result'
offset: Optional x/y offset from target

Example: Click the login button found via image recognition

DOUBLE_CLICK

Double-click at the target location

Parameters:

target: Image, coordinates, or 'Last Find Result'

Example: Double-click to open a file

RIGHT_CLICK

Right-click to open context menu

Parameters:

target: Image, coordinates, or 'Last Find Result'

Example: Right-click to show context menu

DRAG

Drag from source to destination

Parameters:

source: Starting location
destination: Ending location

Example: Drag a file to trash

MOUSE_MOVE / MOVE

Move mouse without clicking

Parameters:

target: Destination coordinates or image

Example: Hover over an element to reveal tooltip

MOUSE_DOWN / MOUSE_UP

Press or release mouse button

Parameters:

button: Mouse button: left, right, middle

Example: Hold mouse button for custom drag operations

SCROLL / MOUSE_SCROLL

Scroll mouse wheel

Parameters:

amount: Scroll amount (positive = down, negative = up)
direction: Optional: vertical or horizontal

Example: Scroll down to load more content

Keyboard Actions

Type text and press keys

TYPE

Type a text string

Parameters:

text: Text to type
interval: Optional delay between keystrokes (ms)

Example: Type username into login field

KEY_PRESS

Press and release a key

Parameters:

key: Key name (e.g., 'enter', 'tab', 'ctrl+c')

Example: Press Enter to submit form

KEY_DOWN / KEY_UP

Press or release a specific key

Parameters:

key: Key name to press or release

Example: Hold Shift while clicking for multi-select

Vision Actions

Find and verify visual elements

FIND

Locate an image on screen and store its location

Parameters:

target: Image to find
similarity: Match threshold (0.7-0.95)

Example: Find the submit button before clicking it

Important: The found location is stored and can be used by subsequent actions with target: 'Last Find Result'

EXISTS

Check if an image exists (returns true/false)

Parameters:

target: Image to check for
similarity: Match threshold

Example: Verify a confirmation dialog appeared

VANISH

Wait for an image to disappear from screen

Parameters:

target: Image to wait for disappearance
timeout: Maximum wait time (ms)

Example: Wait for loading spinner to disappear

Navigation Actions

Navigate states and execute processes

GO_TO_STATE

Navigate to a target state using pathfinding

Parameters:

stateId: ID of target state

Example: Navigate to the settings page from anywhere

Important: Uses automatic pathfinding through the state graph

RUN_WORKFLOW

Execute another workflow by ID

Parameters:

workflowId: ID of workflow to execute

Example: Run a reusable login workflow

Utility Actions

Timing and debugging utilities

WAIT

Pause execution for a specified duration

Parameters:

duration: Wait time in milliseconds

Example: Wait 2 seconds for page to load

SCREENSHOT

Capture the current screen

Parameters:

filename: Optional filename for screenshot

Example: Take screenshot for debugging or verification

Common Action Parameters

target

Object or String

Specifies where the action should be performed

{"type": "image", "imageId": "..."} - Find and target an image

{"type": "coordinates", "x": 100, "y": 200} - Use fixed coordinates

"Last Find Result" - Use location from last FIND action

similarity

Number (0.0 - 1.0)

Image matching threshold for visual actions

0.7 - 0.8 - Fuzzy matching (more tolerant)

0.85 - 0.9 - Standard matching (recommended)

0.95+ - Exact matching (strict)

retry_count

Integer

Number of retry attempts if action fails (default: 3)

timeout

Integer (milliseconds)

Maximum time to wait for action completion (default: 5000)

continue_on_error

Boolean

If true, process continues even if this action fails (default: false)

pause_before_begin

Integer (milliseconds)

Wait time before executing action

pause_after_end

Integer (milliseconds)

Wait time after action completes

offset

Object {x, y}

Offset from target location in pixels

Best Practices

Use FIND before CLICK for reliability

Always use a FIND action before clicking an image target. This ensures the element exists and updates the 'Last Find Result' location.

1. FIND (target: login_button) → 2. CLICK (target: "Last Find Result")

Set appropriate similarity thresholds

Start with 0.85 and adjust based on results. Lower for elements that change slightly, higher for exact matches.

Add pauses for slow applications

Use pause_after_end for actions that trigger slow operations (e.g., page loads, animations).

Use continue_on_error sparingly

Only use for truly optional actions. Most actions should fail the process if they fail.

Combine actions into logical processes

Group related actions into named processes for reusability and clarity (e.g., 'login_sequence', 'submit_form').