FlaUI.WebDriver

FlaUI.WebDriver is a W3C WebDriver2 implementation using FlaUI's automation. It currently only supports UIA3.

Important

This WebDriver implementation is EXPERIMENTAL. It is not feature complete and may not implement all features correctly.

Motivation

• Microsoft's WinAppDriver used by Appium Windows Driver has many open issues, is not actively maintained and is not yet open source after many requests. It implements the obsolete JSON Wire Protocol by Selenium and not the new W3C WebDriver standard. When using it I stumbled upon various very basic issues, such as that click doesn't always work.
• kfrajtak/WinAppDriver is an open source alternative, but its technology stack is outdated (.NET Framework, UIAComWrapper, AutoItX.Dotnet).
• W3C WebDriver is a standard that gives many options of automation frameworks such as WebdriverIO and Selenium. It allows to write test automation in TypeScript, Java or other languages of preference (using FlaUI requires C# knowledge).
• It is open source! Any missing command can be implemented quickly by raising a Pull Request.

Capabilities

The following capabilities are supported:

Capability Name	Description	Example value
platformName	Must be set to windows (case-insensitive).	windows
appium:automationName	Must be set to FlaUI (case-insensitive).	FlaUI
appium:app	The path to the application, or in case of an UWP app, <package family name>!App. It is also possible to set app to Root. In such case the session will be invoked without any explicit target application. Either this capability, appTopLevelWindow or appTopLevelWindowTitleMatch must be provided on session startup.	C:\Windows\System32\notepad.exe, Microsoft.WindowsCalculator_8wekyb3d8bbwe!App
appium:appArguments	Application arguments string, for example /?.
appium:appWorkingDir	Full path to the folder, which is going to be set as the working dir for the application under test. This is only applicable for classic apps. When this is used the appium:app may contain a relative file path.	C:\MyApp\
appium:appTopLevelWindow	The hexadecimal handle of an existing application top level window to attach to, for example 0x12345 (should be of string type). Either this capability, appTopLevelWindowTitleMatch or app must be provided on session startup.	0xC0B46
appium:appTopLevelWindowTitleMatch	The title of an existing application top level window to attach to, for example My App Window Title (should be of string type). Either this capability, appTopLevelWindow or app must be provided on session startup.	My App Window Title or My App Window Title - .*
appium:newCommandTimeout	The number of seconds the to wait for clients to send commands before deciding that the client has gone away and the session should shut down. Default one minute (60).	120

Getting Started

This driver currently can be downloaded as an executable. Start the web driver service with:

./FlaUI.WebDriver.exe --urls=http://localhost:4723/

After it has started, it can be used via WebDriver clients such as for example:

• Appium.WebDriver
• Selenium.WebDriver
• WebdriverIO

Using the Appium.WebDriver C# client:

using OpenQA.Selenium.Appium.Windows;

public class FlaUIDriverOptions : AppiumOptions
{
    public static FlaUIDriverOptions ForApp(string path)
    {
        return new FlaUIDriverOptions()
        {
            PlatformName = "windows",
            AutomationName = "flaui",
            App = path
        };
    }
}

var driver = new WindowsDriver(new Uri("http://localhost:4723"), FlaUIDriverOptions.ForApp("C:\\YourApp.exe"))

Using the Selenium.WebDriver C# client:

using OpenQA.Selenium;

public class FlaUIDriverOptions : DriverOptions
{
    public static FlaUIDriverOptions ForApp(string path)
    {
        var options = new FlaUIDriverOptions()
        {
            PlatformName = "windows"
        };
        options.AddAdditionalOption("appium:automationName", "flaui");
        options.AddAdditionalOption("appium:app", path);
        return options;
    }

    public override ICapabilities ToCapabilities()
    {
        return GenerateDesiredCapabilities(true);
    }
}

var driver = new RemoteWebDriver(new Uri("http://localhost:4723"), FlaUIDriverOptions.ForApp("C:\\YourApp.exe"))

Using the WebdriverIO JavaScript client:

import { remote } from 'webdriverio'

const driver = await remote({
    capabilities: {
        platformName: 'windows',
        'appium:automationName': 'flaui'
        'appium:app': 'C:\\YourApp.exe'
    }
});

Selectors

On Windows, the recommended selectors, in order of reliability are:

Selector	Locator strategy keyword	Supported?
Automation ID	"accessibility id"	✅
Name	"name"	✅
Class name	"class name"	✅
Link text selector	"link text"	✅
Partial link text selector	"partial link text"	✅
Tag name	"tag name"	✅
XPath selector	"xpath"	✅
CSS selector	"css selector"	Only ID, class or name attribute selectors. IDs are interpreted as automation IDs.

Using the Selenium C# client, the selectors are:

driver.FindElement(By.Id("TextBox")).Click(); // Matches by automation ID
driver.FindElement(By.Name("TextBox")).Click();
driver.FindElement(By.ClassName("TextBox")).Click();
driver.FindElement(By.LinkText("Button")).Click();
driver.FindElement(By.PartialLinkText("Button")).Click();
driver.FindElement(By.TagName("RadioButton")).Click();
driver.FindElement(By.XPath("//RadioButton")).Click();

Using the WebdriverIO JavaScript client (see WebdriverIO Selectors guide:

await driver.$('~automationId').click();
await driver.$('[name="Name"]').click();
await driver.$('.TextBox').click();
await driver.$('=Button').click();
await driver.$('*=Button').click();
await driver.$('<RadioButton />').click();
await driver.$('//RadioButton').click();

Windows

The driver supports switching windows. The behavior of windows is as following (identical to behavior of e.g. the Chrome driver):

• By default, the window is the window that the application was started with.
• The window does not change if the app/user opens another window, also not if that window happens to be on the foreground.
• All open window handles from the same app process (same process ID in Windows) can be retrieved.
• Other processes spawned by the app that open windows are not visible as window handles. Those can be automated by starting a new driver session with e.g. the appium:appTopLevelWindow capability.
• Closing a window does not automatically switch the window handle. That means that after closing a window, most commands will return an error "no such window" until the window is switched.
• Switching to a window will set that window in the foreground.

Running scripts

The driver supports PowerShell commands.

Using the Selenium or Appium WebDriver C# client:

var result = driver.ExecuteScript("powerShell", new Dictionary<string,string> { ["command"] = "1+1" });

Using the WebdriverIO JavaScript client:

const result = driver.executeScript("powerShell", [{ command: `1+1` }]);

Windows extensions

To enable easy switching from appium-windows-driver, there is a rudimentary implementation of windows: click, windows: hover, windows: scroll, windows: keys, windows: getClipboard and windows: setClipboard.

Supported WebDriver Commands

Method	URI Template	Command	Implemented
POST	/session	New Session	✅
DELETE	/session/{session id}	Delete Session	✅
GET	/status	Status	✅
GET	/session/{session id}/timeouts	Get Timeouts	✅
POST	/session/{session id}/timeouts	Set Timeouts	✅
POST	/session/{session id}/url	Navigate To	N/A
GET	/session/{session id}/url	Get Current URL	N/A
POST	/session/{session id}/back	Back	N/A
POST	/session/{session id}/forward	Forward	N/A
POST	/session/{session id}/refresh	Refresh	N/A
GET	/session/{session id}/title	Get Title	✅
GET	/session/{session id}/window	Get Window Handle	✅
DELETE	/session/{session id}/window	Close Window	✅
POST	/session/{session id}/window	Switch To Window	✅
GET	/session/{session id}/window/handles	Get Window Handles	✅
POST	/session/{session id}/window/new	New Window
POST	/session/{session id}/frame	Switch To Frame	N/A
POST	/session/{session id}/frame/parent	Switch To Parent Frame	N/A
GET	/session/{session id}/window/rect	Get Window Rect	✅
POST	/session/{session id}/window/rect	Set Window Rect	✅
POST	/session/{session id}/window/maximize	Maximize Window
POST	/session/{session id}/window/minimize	Minimize Window
POST	/session/{session id}/window/fullscreen	Fullscreen Window
GET	/session/{session id}/element/active	Get Active Element	✅
GET	/session/{session id}/element/{element id}/shadow	Get Element Shadow Root	N/A
POST	/session/{session id}/element	Find Element	✅
POST	/session/{session id}/elements	Find Elements	✅
POST	/session/{session id}/element/{element id}/element	Find Element From Element	✅
POST	/session/{session id}/element/{element id}/elements	Find Elements From Element	✅
POST	/session/{session id}/shadow/{shadow id}/element	Find Element From Shadow Root	N/A
POST	/session/{session id}/shadow/{shadow id}/elements	Find Elements From Shadow Root	N/A
GET	/session/{session id}/element/{element id}/selected	Is Element Selected	✅
GET	/session/{session id}/element/{element id}/displayed	Is Element Displayed	✅ 1
GET	/session/{session id}/element/{element id}/attribute/{name}	Get Element Attribute	✅ 2
GET	/session/{session id}/element/{element id}/property/{name}	Get Element Property	✅
GET	/session/{session id}/element/{element id}/css/{property name}	Get Element CSS Value	N/A
GET	/session/{session id}/element/{element id}/text	Get Element Text	✅
GET	/session/{session id}/element/{element id}/name	Get Element Tag Name	✅
GET	/session/{session id}/element/{element id}/rect	Get Element Rect	✅
GET	/session/{session id}/element/{element id}/enabled	Is Element Enabled	✅
GET	/session/{session id}/element/{element id}/computedrole	Get Computed Role
GET	/session/{session id}/element/{element id}/computedlabel	Get Computed Label
POST	/session/{session id}/element/{element id}/click	Element Click	✅
POST	/session/{session id}/element/{element id}/clear	Element Clear	✅
POST	/session/{session id}/element/{element id}/value	Element Send Keys	✅
GET	/session/{session id}/source	Get Page Source	N/A
POST	/session/{session id}/execute/sync	Execute Script	✅
POST	/session/{session id}/execute/async	Execute Async Script
GET	/session/{session id}/cookie	Get All Cookies	N/A
GET	/session/{session id}/cookie/{name}	Get Named Cookie	N/A
POST	/session/{session id}/cookie	Add Cookie	N/A
DELETE	/session/{session id}/cookie/{name}	Delete Cookie	N/A
DELETE	/session/{session id}/cookie	Delete All Cookies	N/A
POST	/session/{session id}/actions	Perform Actions	✅
DELETE	/session/{session id}/actions	Release Actions	✅
POST	/session/{session id}/alert/dismiss	Dismiss Alert
POST	/session/{session id}/alert/accept	Accept Alert
GET	/session/{session id}/alert/text	Get Alert Text
POST	/session/{session id}/alert/text	Send Alert Text
GET	/session/{session id}/screenshot	Take Screenshot	✅
GET	/session/{session id}/element/{element id}/screenshot	Take Element Screenshot	✅
POST	/session/{session id}/print	Print Page

WebDriver Interpretation

There is an interpretation to use the WebDriver specification to drive native automation. Appium does not seem to describe that interpretation and leaves it up to the implementer as well. Therefore we describe it here:

WebDriver term	Interpretation
browser	The Windows OS on which the FlaUI.WebDriver instance is running
top-level browsing contexts	Any window of the app under test (modal windows too)
current top-level browsing context	The current selected window of the app under test
browsing contexts	Any window of the app under test (equal to "top-level browsing contexts")
current browsing context	The current selected window of the app under test (equal to "current top-level browsing context")
window	Any window of the app under test (modal windows too)
frame	Not implemented - frames are only relevant for web browsers
shadow root	Not implemented - shadow DOM is only relevant for web browsers
cookie	Not implemented - cookies are only relevant for web browsers
tag name	Control type in Windows
attribute	UI automation element property in Windows

Deviations from W3C WebDriver Spec

https://www.w3.org/TR/webdriver2/#element-send-keys says:

Set the text insertion caret using set selection range using current text length for both the start and end parameters.

This is impossible using UIA, as there is no API to set the caret position: text instead gets inserted at the beginning of a text box. This is also WinAppDriver's behavior.

Element Attributes

Attributes are mapped to UI automation element properties. Attributes without a period (.) are mapped to Automation Element Properties. For example to read the UIA_ClassNamePropertyId using Selenium or Appium WebDriver:

var element = driver.FindElement(By.Id("TextBox"));
var value = element.GetDomAttribute("ClassName");

Attributes with a period are treated as Control Pattern Properties with the form Pattern.Property. For example to read the UIA_ToggleToggleStatePropertyId using Selenium WebDriver:

var element = driver.FindElement(By.Id("ToggleButton"));
var value = element.GetDomAttribute("Toggle.ToggleState");

Next Steps

Possible next steps for this project:

• Distribute as Appium driver

Footnotes

1. In Selenium WebDriver, the Displayed property converts to javascript. Use Appium WebDriver to use this functionality. It uses the IsOffscreen property that however does not seem to take it into account if the element is blocked by another window. ↩

2. In Selenium WebDriver, use GetDomAttribute because GetAttribute converts to javascript. ↩