path: root/doc/qtcreator/src/howto/creator-only/creator-squish.qdoc

// Copyright (C) 2022 The Qt Company Ltd.
// SPDX-License-Identifier: LicenseRef-Qt-Commercial OR GFDL-1.3-no-invariants-only

/*!
    \previouspage creator-autotest.html
    \page creator-squish.html
    \nextpage creator-advanced.html

    \title Using Squish

    \l{https://www.qt.io/product/quality-assurance/squish}{Squish} is an automated GUI
    testing framework for testing Android, iOS, Java, \macos, Qt, Tk, Windows, and
    XView applications, as well as HTML-based web applications running in
    browsers, such as Apple Safari, Mozilla Firefox, Google Chrome, and
    Microsoft Internet Explorer and Edge.

    The experimental Squish plugin integrates Squish into \QC. You can:

    \list
        \li Open existing Squish test suites.
        \li Create new test suites and test cases.
        \li Record test cases (in a very limited way compared to what you can do
            inside the Squish IDE).
        \li Use Squish Runner and Server to run test suites or cases and view
            the results in the \uicontrol Squish \l{Viewing Output}{output}.
        \li Set breakpoints before running tests to stop at certain locations and
            inspect the local variables, similarly to when debugging a test.
    \endlist

    When running test suites or cases, the Squish Runner instructs the Squish
    Server to start the application under test (AUT). The server can be running
    on multiple machines, and the AUT can be located on a different path on each
    of them. Therefore, you must either map AUTs to their corresponding paths or
    specify AUT paths to search from in the server settings.

    In addition, you can test an already running application by attaching to it.
    This enables you to test your application using a Squish Server running on
    another machine. However, you can have only one server attached to your
    application at a time. Also, the attached application is not closed when the
    test case is completed.

    To use the plugin, you must download and install Squish, create a connection
    to the Squish Server, and specify AUTs to run.

    \section1 Enabling the Squish Plugin

    To enable the Squish plugin:

    \list 1
        \li Select \uicontrol Help > \uicontrol {About Plugins} >
            \uicontrol Utilities > \uicontrol Squish to enable the plugin.
        \li Select \uicontrol {Restart Now} to restart \QC and load the plugin.
    \endlist

    \section1 Specifying a Squish Server

    To specify a Squish Server to run:

    \list 1
        \li Select \uicontrol Edit > \uicontrol Preferences > \uicontrol Squish.
            \image qtcreator-squish-preferences.png "Squish general preferences"
        \li In the \uicontrol {Squish path} field, specify the path to the Squish
            installation directory.
        \li In the \uicontrol {License path} field, specify the path to your
            Squish license file if it is not located in your home folder. For
            example, if you have a global installation with several users, where
            the license key is installed in the global folder.
        \li Select the \uicontrol {Local server} check box to use a locally
            installed \c {squishserver.exe}. To use a server running in another
            machine, deselect the check box and specify the server address in the
            \uicontrol {Server host} field and the port number in the
            \uicontrol {Server port} field. If no port is specified, \QC starts
            \c squishserver in a way that enables it to automatically select an
            open port.
        \li Select the \uicontrol {Verbose log} check box to include additional
            logging levels in the log output.
        \li Select the \uicontrol {Minimize IDE} check box to automatically
            minimize \QC when running or recording test cases.
    \endlist

    \section1 Specifying AUTs

    To specify applications to test using Squish, select \uicontrol {Tools} >
    \uicontrol {Squish} > \uicontrol {Server Settings}.

    \image qtcreator-squish-server-settings.png "Squish Server Settings"

    \section2 Mapping AUTs

    To specify the path to an AUT to run, select \uicontrol {Mapped AUTs} >
    \uicontrol Add and locate the AUT.

    The Squish server checks whether the name of the AUT to run is mapped to a
    path and starts the AUT using the mapped path. This way, it does not need
    to search from all the specified AUT paths.

    Mapping AUTs prevents the server from accidentally executing the wrong AUT
    if two different executables have the same name, as the server executes the
    first matching AUT it finds in the \uicontrol {AUT Paths} list.

    To change the path to the selected AUT, select \uicontrol {Edit}.

    To remove the mapping to the selected AUT, select \uicontrol {Remove}.

    \section2 Specifying AUT Paths

    To specify a path to search AUTs from, select \uicontrol {AUT Paths} >
    \uicontrol Add.

    The Squish Server searches for the executable to run from the specified
    AUT paths and runs the first one with the specified name that it finds in
    any path.

    To change the selected AUT path, select \uicontrol {Edit}.

    To remove the selected AUT path, select \uicontrol {Remove}.

    \section2 Adding Attachable AUTs

    To specify the path to a running AUT, select \uicontrol {Attachable AUTs} >
    \uicontrol Add. In the \uicontrol {Add Attachable AUT} dialog, specify a
    connection to a running application to register an attachable AUT.

    \image qtcreator-squish-server-settings-add-attachable-aut.png "Add Attachable AUT dialog"

    To change the connection to the selected AUT, select \uicontrol {Edit}.

    To remove the connection to the selected AUT, select \uicontrol {Remove}.

    \section1 Managing Test Suites and Cases

    You can manage Squish test suites and cases in the \uicontrol Squish
    \l {Working with Sidebars}{view}.

    \image qtcreator-squish-view.png "Squish sidebar view"

    To show existing test suites in \uicontrol {Test Suites}, select
    \uicontrol {Open Squish Suites} in the context menu.

    \image qtcreator-squish-view-open-squish-suites.png "Open Squish Test Suites dialog"

    You can open the \uicontrol {Squish Test Suite} wizard for creating a new
    test suite by selecting \uicontrol {Create New Test Suite} in the context
    menu.

    To add a test case to a test suite, select it and then select
    \uicontrol {Add New Test Case} in the context menu.

    To close all test suites, select \uicontrol {Close All Test Suites} in the
    context menu.

    To add a shared folder to \uicontrol {Shared Folders}, select
    \uicontrol {Add Shared Folder} in the context menu. To remove all
    shared folders, select \uicontrol {Remove All Shared Folders}.

    Double-click a test suite in \uicontrol {Test Suites} to open the
    \c {suite.conf} configuration file for editing.

    \section2 Creating Test Suites

    To create a new test suite:

    \list 1
        \li Select \uicontrol File > \uicontrol {New Project}
            > \uicontrol {Squish} > \uicontrol {Squish Test Suite} >
            \uicontrol Choose.
            \image qtcreator-squish-create-test-suite.png "Create Squish Test Suite wizard"
        \li On the \uicontrol {Location} page, in \uicontrol {Test Suite Name},
            enter a name for the test suite.
            \image qtcreator-squish-create-test-suite-location.png "Location page"
        \li In \uicontrol {Test Suite folder's parent folder}, enter the path to
            the folder to create the test suite folder, and then select
            \uicontrol Next.
        \li On the \uicontrol Setup page, select the GUI toolkit used by the AUT,
            and then select \uicontrol Next.
            \image qtcreator-squish-create-test-suite-setup.png "Setup page"
            Currently, only desktop GUI toolkits are supported.
        \li On the \uicontrol {Script Language} page, select the scripting
            language to use for the test suite's test script, and then select
            \uicontrol Next.
            \image qtcreator-squish-create-test-suite-language.png "Languages page"
        \li On the \uicontrol {AUT} page, select the AUT to test, and then select
            \uicontrol Next.
            \image qtcreator-squish-create-test-suite-aut.png "AUT page"
        \li On the \uicontrol {Summary} page review the test suite settings, and
            then select \uicontrol Finish to create the test suite.
    \endlist

    The test suite is listed in \uicontrol {Test Suites} in the \uicontrol Squish
    sidebar view.

    \section2 Recording Test Cases

    Squish records tests using the scripting language that you specified for the
    test suite. Recordings are made into existing test cases.

    In \uicontrol {Test Suites}, select the \inlineimage icons/recordfill.png
    (\uicontrol {Record Test Case}) button next to the test case name. The AUT
    that you selected for the test suite is displayed and you can start recording
    the test case.

    \image qtcreator-squish-control-bar-record-test-case.png "Squish control bar for recording test cases"

    When you are done, select the \inlineimage icons/stop_small.png
    (\uicontrol {Stop}) button in the \uicontrol {Control Bar}.

    You can edit recorded test scripts or copy parts of them into manually
    created test scripts.

    \image qtcreator-squish-test-script-edit.png "A test script open in the editor"

    \section2 Running Test Suites

    You can run a recorded test case to have Squish repeat all the actions that
    you applied when recording the test, but without the pauses that humans are
    prone to but which computers don't need.
    To run a test case, select the \inlineimage icons/run_small.png
    (\uicontrol {Run}) button next to the test case in \uicontrol {Test Suites}.

    \image qtcreator-squish-control-bar-run-test-case.png "Squish control bar for running test cases"

    While the test is running, you can view test results as well as interrupt and
    stop tests in the \uicontrol {Control Bar}.

    \section2 Mapping Symbolic Names

    When Squish records a test, it uses \e {symbolic names} to identify the UI
    elements. Symbolic names are stored in an object map that can be either
    text-based or script-based. Text-based symbolic names are plain strings
    starting with a colon (:), whereas script-based symbolic names are script
    variables.

    Squish generates symbolic names programmatically, but you can use them in
    hand-written code, or when you edit test cases or use extracts from recorded
    test cases.

    Symbolic names have one major advantage over real names: if a property
    that a real name depends on changes in the AUT, the real name becomes
    invalid, and you must update all occurrences of it in test scripts.
    When using symbolic names, you only need to update the real name in the
    object map. You do not need to make any changes to the tests.

    To edit the object map of a test suite, select the
    \inlineimage icons/objectsmap.png
    (\uicontrol {Object Map}) button next to the test
    suite in \uicontrol {Test Suites}.

    \image qtcreator-squish-symbolic-names.png "Symbolic Names view"

    You can filter the symbolic names in the \uicontrol {Symbolic Names} view.
    To edit a symbolic name or the names or values of its properties,
    double-click the name or value in the view and enter a new one.

    To add a new symbolic name, select \uicontrol New. Double-click the
    placeholder for the name and enter a new name. Then select \uicontrol New
    next to \uicontrol Properties to enter properties for the symbolic name.

    To remove the selected symbolic name or property, select \uicontrol Remove.

    To jump to the symbolic name associated to the selected
    property, select  \inlineimage icons/jumpto.png
    .

    \section2 Inspecting Local Variables

    If you set breakpoints in the test code before running the test, the test
    execution is automatically interrupted when a breakpoint is hit. You can
    inspect the contents of local variables in the \uicontrol {Squish Locals}
    view.

    \image qtcreator-squish-locals.png "Squish Locals view"

    Use the \uicontrol {Step Into}, \uicontrol {Step Over}, and
    \uicontrol {Step Out} buttons in the \uicontrol Squish debugging
    view to step through the code.

    \image qtcreator-squish-debugging-view.png "Squish debugging view"

    \section2 Specifying Settings for Running Tests

    To specify settings for running tests, select \uicontrol Tools >
    \uicontrol Squish > \uicontrol {Server Settings}:

    \list
        \li In the \uicontrol {Maximum startup time} field, set the maximum time
            to wait for the AUT to start before throwing an error.
        \li In the \uicontrol {Maximum response time} field, set the maximum time
            to wait for the AUT to respond before throwing an error.
        \li In the \uicontrol {Maximum post-mortem wait time} field, set the
            maximum time to wait after the main AUT has exited. This is useful
            for AUTs invoked through launcher applications, such as shell scripts
            or batch files.
        \li Select the \uicontrol {Animate mouse cursor} check box to animate
            the mouse cursor when playing back a test.
    \endlist

    \section1 Viewing Test Results

    Squish uses compare, verify, and exception functions to record the results of
    tests applied to a running AUT in the test log as \e {passes} or \e {fails}.
    In addition, any kind of test results can be recorded in the test log.

    You can view the test log in the \uicontrol Squish output,
    \uicontrol {Test Results} tab.

    \image qtcreator-squish-output-test-results.png "Test Results output"

    The \uicontrol Result column displays the time when each test run started
    and finished, log information and warnings, and test result status:

    \list
        \li \uicontrol Pass - The test passed.
        \li \uicontrol Fail - The test failed.
        \li \uicontrol ExpectedFail - The test failed, as expected. For example,
            a known bug in the AUT caused a comparison or verification to fail.
        \li \uicontrol UnexpectedPass - The test passed unexpectedly. For
            example, a comparison or verification succeeded even though it was
            expected to fail.
    \endlist

    The \uicontrol Message column displays log messages and information about
    the type of the operation that was performed: comparison, verification, or
    exception.

    \section1 Viewing Squish Runner and Server Logs

    You can view the Squish Runner and Server logs in the \uicontrol Squish
    output, \uicontrol {Runner/Server Log} tab.

    \image qtcreator-squish-output-runner-server-log.png "Runner and Server Log output"
*/