Swingin' it

Here's a really simple demo, or tutorial if you will, about using the Robotframework SwingLibrary.

Download the latest demo and extract it to the desired directory. Oh, and you should have the Robotframework and Jython installed. Open your command line and navigate to the extracted demo directory. Try to run the finnished demo to verify that everything is installed and working ok. Execute:

run_demo.py example

The SwingLibrary jar needs to be included in the classpath in order to be found by the Robotframework. The run_demo.py does this for you and executes the test suite provided as an argument.

Examine the SUT i.e. Todo List Application. Start the application by running the following in the command line at the demo root:


java -cp lib/swinglibrary-1.1.2-SNAPSHOT-jar-with-dependencies.jar \ org.robotframework.swing.testapp.examplesut.TodoListApplication

As you can see, this a really basic todo list Swing application written in Java. Have a look around and close the application when your done.

Open your favourite text editor. Create a new test suite file (e.g. test.txt) and save it to the demo root. Take the SwingLibrary into use in the test suite, type:

*** Settings ***
Library  SwingLibrary

*** Test Cases ***
Test Add Todo Item

(Note! In this plain text format the cell separator is 2 spaces)

Run the test.

run_demo.py example.txt

The test should execute and fail with an error message: "The test case contains no keywords". Ok let's add some and start the application and select it's main window.

*** settings ***
Library  SwingLibrary

*** Test Cases ***
Test Add Todo Item
    Start Test Application

*** User Keywords ***
Start Test Application
    Start Application  org.robotframework.swing.testapp.examplesut.TodoListApplication
    Select Main Window

Run the test again and now the application should start and test still fails.

Ok, progress. Let's move the start-up stuff into Suite Setup. Then insert text "Buy some milk" into the text field, that the developer has named logically "description". While at it, let's push the "Add Todo Item"-button with a logical name "add".

*** settings ***
Library  SwingLibrary
Suite Setup  Start Test Application

*** Test Cases ***
Test Add Todo Item
    Insert Todo Item Buy some milk

*** User Keywords ***
Start Test Application
    Start Application  org.robotframework.swing.testapp.examplesut.TodoListApplication
    Select Main Window

Insert Todo Item ${arg}
    Insert Into Text Field  description  ${arg}
    Push Button  add

Run the test and enjoy the greenness. Note, that the user keyword: Insert Todo Item, uses embedded variable syntax. So the "Buy some milk" is provided to the keyword in the same cell.

Great, but still no testing or assertions. We can easily fix that. Select the first (0th) item from the todolist and read the the selected value into an argument. The we can assert that the value is the same as we expect i.e. "Buy some milk".

*** settings ***
Library  SwingLibrary
Suite Setup  Start Test Application

*** Test Cases ***
Test Add Todo Item
    Insert Todo Item Buy some milk
    Select From List  todolist  0
    ${item}=  Get Selected Value From List  todolist
    Should Be Equal  ${item}  Buy some milk

*** User Keywords ***
Start Test Application
    Start Application  org.robotframework.swing.testapp.examplesut.TodoListApplication
    Select Main Window

Insert Todo Item ${arg}
    Insert Into Text Field  description  ${arg}
    Push Button  add

Run the test and it should pass.

Write tests for the todo item deletion...

Too much sleep is better than not enough

Timing is an important aspect of automated testing. Tests should run in minimum time so that they give rapid feedback and are not bottle necks. But when testing complex systems some operations unfortunately just take time and this has to be handled in tests. This post is about handling those situation in tests.

Usually the first solution is to use sleeps. A sleep (in Robot Framework BuiltIn.Sleep) does nothing for a specified time period and continues the test after this. A Sleep isn't usually the optimal solution as it doesn't guarantee anything else than that the give time has passed. In other words: It guarantees that the test will take longer and doesn't guarantee that the waited event has happened.

Sleeps don't always work. This results in flickering tests.

To make Sleep work always it has to be very long so that it will be enough for every environment where the system under test should work. This is most likely longer time than what it would take in average.

Another solution for waiting is polling (or busy wait). In polling (in Robot Framework BuiltIn.Wait Until Keyword Succeeds) the waited condition is checked repeatedly until the condition passes or a timeout period expires. Polling guarantees that the waited event has happened.

Polling will usually result in a less total time of test execution than using sleeps. Sometimes this doesn't happen. For example the polling check could exhaust the system under test. Repeating the keyword will also produce more test logs.

If the repeated keyword takes enough time wait until could take more time than needed.
For example keyword that would take 9 seconds, in a situation where there has to be at least 10 seconds of waiting before the keyword can succeed, it will take 27 seconds to execute the Wait Until Keyword Succeeds but it would take 19 seconds to sleep 10 seconds and then execute the keyword.

Other methods to handle time:
* If possible simulate the passing of time
* Hollywood principle
* First sleep and then use wait until keyword succeeds to minimize polling