Automation actions

The following actions selected in the automation sheet column (B) drive the automation steps: 

• Window select
• Browser open
• Item select
• Mouse click
• Text enter
• Value retrieve
• (OCR) Window scan
• (OCR) Text locate
• Color check
• Excel function
• Counter
• User input
• Windows command
• Email Process
• Web services
• Process pause
• Process end

Most of the actions have a supporting parameter window accessisble through the “Action parameters” menu button. 

With the exception of numeric positions and drop-down values, [variables] can be used as parameters. See Application sub-menu “New Excel instance” section for additional details when using actions that interact with Excel workbooks.

Action - Window select

This action selects a window (application, folder, Windows taskbar and desktop) and gives it focus. The following actions require parent window selection prior to their usage:

• Item select, Mouse click, Value retrieve, (OCR) Window scan, (OCR) Text locate, Color check

Automation sheet parameters:

• Name – name and type of the window. Use the * wild symbol to specify a window by partial name. For example “*Outlook <window>” selects any window where the window name ends with “Outlook”
• Maximize – select Yes to maximize the window or No to keep it in the current state

Action parameters window:

• Displays a list of all open windows including the desktop and the task bar
• To have the system wait for up to 6 seconds for the selected window to appear during process execution, check-off the “Extended detection” option
  • Selecting the extended detection adds a + symbol in the window name\type parameter
  • Standard detection scans for the existance of a window only once
• Select the desired window and click the OK button to update the action-step parameter

Action - Browser open

This action opens a new browser window or a new browser tab if the specified browser is already open. To select an existing browser window, use the “Window select” action instead.

Each browser slightly differs in automation-ability and if automating in one browser proves to be a challenge, try automating the process in another browser.

This action automatically selects the browser window and action “Window select” is not required afterwards unless there are multiple browsers of the same type open at the same time. Advanced users can optimize browser functionality using browser-specific parameters in the application settings.

Automation sheet parameters:

• Browser – select one of the 4 main browser types (Edge, Google Chrome, Firefox, Internet Explorer)
• URL – enter the starting URL or leave the field empty to open the default website
  • When URL is specified, application automatically waits for the page to load (Same as using action “Process pause – While header motion detected”)

Action parameters window: NA

Action - Item select

This action selects an item (button, text box, link, etc.) in the selected window. The selected item can be clicked or selected for interaction by the following actions:

• Text enter, Value retrieve, Mouse click

Automation sheet parameters:

• Option– select or click the selected item
  • Difference between “Left-click” and “Left-click+” is that the 1st option is a virtual click (faster) and the second is a physical click. In case the 1st option is not supported in the selected application, use the 2nd option
• Name – name and type of the item. Use the * wild symbol to specify an item by partial name. For example “A* <button>” selects the first button that has a name that starts with an “A”

Action parameters window:

Selection of an item can be done using two methods. In some cases, the click search method may not work (hidden by other invisible items or when the item is not properly named within the automated application) and in such cases, use the item search method instead. Once an item is detected using either method, select it in the table and click the OK button to update the action-step parameter. The two detection methods are: 

• Click search method
  This method allows direct object selection using a mouse.
  To start, click the search button and while holding the CTRL key, click the item to be automated. Items successfully selected are highlighted in a red rectangle.
  Selection process instructions are displayed in the notification popup. Wait for a sound notification after each key press and mouse click before proceeding to the next step. The item selection mode will time out after 10 seconds if no action is taken.
  The search result table displays the selected item and if applicable, the item’s parent and child.
• Item search by item type method
  This method displays all items in the selected window for a specific item type. Select the desired object type from the list of radio buttons (user input types include text input boxes, list boxes and drop-down boxes) and click the search button. By default, only the on-screen objects are listed. To show all objects including those off-screen, check off the “Include off-screen elements” checkbox.
  The search results table displays the name of an object and the <object type> next to the name. If an object is off-screen (using the off-screen search option), the object appears in gray text. Use the “Filter by partial name” text box to filter results for specific objects.

Item selection optimization:

• In case the same object name and type appear more than once, there are two options to select the correct item:
  • Using the item x & y coordinates:
    • For example: “Close <button-1864,0>”
    • The coordinates are automatically added when using the item search by item type method
  • Using the item order as it appears in the selected window:
    • For example: “Close <button-1>”
    • This option can be set using the “Add item position no” checkbox
• To have the system wait for up to 4 seconds for the selected item to become available, check-off the “Extended detection” option
  • Selecting the extended detection adds a + symbol in the item name\type parameter
  • Standard detection scans for the existence of an item only once

Action - Mouse click

This action mouse-clicks the selected item or position.

Automation sheet parameters:

• Target– identifies the starting position to be clicked:
  • Selected object – starting position is the middle of the item selected with preceding “Item select” action
  • OCR text position – starting position is the middle of the text position selected with the preceding “(OCR) Text locate” action
  • One of four corners of the selected window
    • Top-right can be used to close the window, bottom right can be used to minimize all windows
• Button– select the mouse button and click-type
  • Difference between “Left-click” and “Left-click+” is that the 1st option is a virtual click (faster) and the second is a physical click. In case the 1st option is not supported in the selected application, use the 2nd option
  • Drag & drop operation can be performed using the mouse-down and mouse-up actions
• Position – offset x, y position from the target position
  • The x & y coordinates can be positive or negative
  • Offset is useful when a position that is to be clicked is not selectable or when a nearby position needs to be clicked

Action parameters window:   

•  Automates the selection of the offset coordinates. To start, hold the CTRL key and click the middle of the starting item and the offset position subsequently
• Selection process instructions are displayed in the notification popup. Wait for a sound notification after each key press and mouse click before proceeding to the next step. The item selection mode will time out after 10 seconds if no action is taken

Action - Text enter

This action is used to enter text into the selected window.

Automation sheet parameters:

• Text – text to be entered

Action parameters window:   

• Use the “special keys” drop-down to select a special key such as CTR, ALT, ENTER etc. and the plus button to add the key to the main text
• Use the encryption button to convert passwords and other sensitive data. AutomationDock will automatically decrypt the values during execution. Decryption works only when the process is executed by the encryption creator on the original system!
• Enter standard text directly into the “Actual text” window
• Click the OK button to update the action-step parameter

Action - Value retrieve

This action retrieves text from the current window, OCR position or from the clipboard.

Automation sheet parameters:

• Source of the value:
  • Selected item – retrieves the value from the current item
  • OCR text position – retrieves the value from the position identified by preceding “(OCR) Text locate“ action
  • OCR mouse position – retrieves the value from the OCR position identified by preceding “Mouse click“ action
  • Clipboard – retrieves the clipboard value
  • Clear clipboard – clears the clipboard and returns no value
  • Value (output field) – displays the retrieved value. The value is stored in the [value] variable

Action parameters window: NA

Action - (OCR) Window scan

This action scans the selected window for text so that “(OCR) Text locate” action can be applied in subsequent steps. The scanned text is automatically saved in vriable [ocr].

OCR is used when working with images, scanned documents and Citrix-based applications. For example, SAP and Citrix applications do not always function as standard object-based applications and are actually images that appear as applications.

Automation sheet parameters:

• Zoom % – allows magnification of the scanned window to enhance OCR capability
• Position – identifies coordinates (top-left and bottom-right) of the scan area. Limiting the scan area vs scanning the whole window (0,0,0,0) will reduce scan time and ignore unwanted text
• Options – Tesseract OCR commands used to optimize OCR scan and to select the scan language. List of commands can be found at this link: Complete list of Tesseract options

Action parameters window:

• Automates the selection of the scan area coordinates. To start, hold the CTRL key and click the top-left and subsequently the bottom-right of the scan area
• Selection process instructions are displayed in the notification popup. Wait for a sound notification after each key press and mouse click before proceeding to the next step. The item selection mode will time out after 10 seconds if no action is taken

Action - (OCR) Text locate

This action locates specific text in a window that was OCR scanned. Once the expected text has been located, actions including “Mouse click”, “Value retrieve” and “Color check” can be applied.

Automation sheet parameters:

• Text – Text to be located
• Result (output field) – displays the scan true\false result. The value is stored in the [result] variable

Action parameters window: NA

Action - Color check

This action checks for the existence of a specific color at a specified location. The location that is checked is based on the most recent preceding “Item select” or “(OCR) Text locate” position.

Automation sheet parameters:

•  Color – defines the RGB color to be identified and the allowed variance percentage
• Position – defines the x, y, width, height coordinates
• Result (output field) – displays the check true\false result. The value is stored in the [result] variable

Action parameters window:

• Select a color section
  • Clicking the mouse icon button automates the color selection using a mouse. To start, hold the CTRL key and click the color area in the target window subsequently
  • Selection process instructions are displayed in the notification popup. Wait for a sound notification after each key press and mouse click before proceeding to the next step. The item selection mode will time out after 10 seconds if no action is taken
  • Clicking the paint icon button allows the selection of a color from the Windows color palette
  • The selected color appears in the rectangle on the right side of the window
• Select position and area section
  • Clicking the mouse icon button automates the color-check area coordinate selection using a mouse. To start, hold the CTRL key and click the middle of the starting-reference object, top-left and bottom-right of the color check area in the subsequent steps
  • Selection process instructions are displayed in the notification popup. Wait for a sound notification after each key press and mouse click before proceeding to the next step. The item selection mode will time out after 10 seconds if no action is taken
• Click the OK button to update the action-step parameters

Action - Excel function

This action allows the user to create functions that combine existing Excel functions with AutomationDock variables and additional features.  

Automation sheet parameters:

• Function – function definition
• Result (output field) – displays the function result. The value is stored in the [result] variable

Action parameters window:

The window is divided into four tabs\function types:

• Assign a value function (Excel)
  Assigns a value from the right-hand equation to the left-hand equation using the equals “=” symbol. Examples of assign functions:
  • [x]=3.14
  • [x]=[y]+7
  • [s]=”Test message”
  • [x]=length(“[s]”)
  • sheet1'!A1=500
• Compare function (Excel)
  Compares the left-hand equation with the right-hand equation using compare symbols (<, <=, >, >=, <>). To compare two values using the equals symbol, it is necessary to use a two-character symbol “?=” to differentiate between assignment and comparison! Examples of compare functions:
  • 5>6
  • [x]?=3.14
  • [a]<>len(“[s]”)
  • sheet1'!A1?= sheet2'!B[row]
• Select Range (Excel)
  Allows the selection of an excel range in a specific sheet and workbook. When working with a Sharepoint-based workbook, enter only the file name. Examples of range functions:
  • sheet1'!a1:d12
  • sheet1'!a1:[col][row]
• User-defined functions (UDF)
  Allows execution of a user-defined function. Select the file name and enter parameter(s) listed in the selected function if applicable.

Once a function has been setup, click the OK button to update the action-step parameter.

User interface features common to the assign and compare tabs (tab 1 and 2):

• Use the radio buttons for both left & right equations to select if the function is a value\variable, a cell in the current workbook or a cell in another workbook. Based on this selection, sheet and workbook textboxes become available
• If applicable, the value drop-down field displays a list of common Excel functions that can be used in AutomationDock functions
• Validate the functions using the validate buttons (image of a calculator)
• When working with a Sharepoint-based workbook, enter only the file name
• A complete list of Excel functions can be found here: Complete list of Excel functions

Addition Excel function tips and features:

• In some cases, Excel automatically adds a single quote in the function parameter cell which could cause the function to fail. The single quote is visible only when the cell is in edit mode (double-clicked). To remove the single quote, copy\paste an empty cell into the parameter cell

Variables

A variable is a named value that can be referred to in other functions and action-steps. Variables are enclosed in [brackets]. In the famous Pythagoras' theorem “a2 + b2 = c2”, abc are all variabes that hold a specific value.  

The variables can be set only within Excel function action-steps, but used in most of the other action-step parameters. This is demonstrated in the next example where a variable [value] is populated with text from a spreadsheet and the text in this variable is then entered into the selected application using the “Text enter” action:

When using variables with text values in a function, it is necessary to use double quotes around the variable name. This is demonstrated in the 3rd step of the below example where we use variables to store and combine names (see the values in the blue result field).  

Action - Counter

This action creates a numeric counter that increments or decrements each time the action-step is executed.  

Automation sheet parameters:

• Start at – starting numeric value
• Increment – increment\decrement numeric value
• Counter (output field) – displays the current counter value and stores it in the [counter] variable

Action parameters window: NA

Action - User input

This action is used to provide\request feedback to\from the user, notify user by voice message or to read global variables from the database.

Automation sheet parameters:

• Option
  • Set shared variables – sets the selected global variables for use in the current process
  • Notification – voice message – speaks the selected message (Windows speech must be enabled)
  • Input box – simple popup message (Automation stops until a user confirms the message)
  • Dialog box – allows the user to enter a value (Automation stops until a user confirms the message)
• Message – Message used by the selected option
• Result (output field) – displays the value (True\False with the Dialog box option) entered by the user. The value is stored in the [result] variable

Action parameters window: NA

Action - Windows command

This action executes a standard Windows command to open or manipulate with an application, file or folder. To access a window opened by a command action, add a subsequent “Window select” action.

Automation sheet parameters:

• Command – Enter the Windows command
• Application – If applicable, enter the application used to execute the command

Action parameters window:   

• Sample Windows commands – Windows command examples used to open applications, manipulate files etc.

Action - Email Process

This action interact with the current Outlook account. MS Outlook must be setup on the system.  

Automation sheet parameters:

• Option:
  • Send email – This action automatically sends an email using the current Outlook account
  • Find email – Find number of emails with specified parameters
  • Email action – Interact with the selected email – available after ‘Find email’ execution
• Parameters – Parameters for specific option
• Result (output field) – displays result value per selected option. The value is stored in the [result] variable

Action parameters window:

• Send email option – Populate the email fields and click the OK button to update the action-step parameters. Separate multiple recipients by a semicolon
• Find email option – Populate the parameters of the email(s) to find and click the OK button to update the action-step parameters
• Email action option – Select the desired task and click the OK button to update the action-step parameters

Action - Web services

This action automatically calls a Web Service.  

Automation sheet parameters:

• As the parameter fields (Connection and Parameters) combine multiple values, use the Action parameters window to populate the parameters properly
• Result (output field) – displays the value returned by the Web Service and stores it in the [result] variable

Action parameters window:

Populate the Web Service fields and click the OK button to update the action-step parameters.

Action - Process pause

This action pauses the execution pending target application motion. Note that all pause types are limited to 30 seconds after which the process resumes regardless of motion detection outcome.

Automation sheet parameters:

• Option:
  • While header motion detected – pauses while top part of the selected window is moving. This type of a pause is ideal when using browsers as the header section includes progress spinners
  • While window motion detected – pauses while motion is detected anywhere within the selected window. Using this type of a pause in browsers is not recommended as a webpage can have advertisement or moving images
  • Until window motion detected – Pauses while no motion within the selected window is detected. Once motion is detected, the process resumes
• Pause (output field) – displays the length of the pause in seconds

Action parameters window: NA

Action - Process end

This action ends the current process even if there are additional steps past it. Action also serves as a placeholder for potential process-ending redirects. If this action is a part of a sub-process during process redirection, the sub-process will end and the main process will resume.

Automation sheet parameters:

• Save – Saves the current workbook once the process execution is completed

Action parameters window: NA