| Previous | 14.1 Specifics of measurement server types | Next |
|---|
14.1.19 Application monitoring measurement servers
With the help of the measurement server PVSR can monitor thick client Windows applications by running their graphical user interface and simulating user interactions. PVSR uses the free Sikuli engine for achieving this goal. The tests prepared with Sikuli can be uploaded into PVSR on the PVSR WEB GUI and then they can be assigned to PVSR equipments representing the actual testing machines. Currently only Windows 7 and XP is supported as test machines.
Each equipment has the following additional parameters (the code of the parameter is given in parenthesis; see Subsection 7.5.6 on Parameters of non-SNMP data collectors):
· Hostname (APPMON EQ 1 IP): The IP address or hostname of the machine running the tests
· Username (APPMON EQ 2 USER): User named used to log into the machine
· Password (APPMON EQ 3 PASSWORD): User password. This is an optional parameter, because PVSR can also use key based authentication
The measurement system consist of two parts: the measurement server and the agent running on the remote Windows machine. For security reasons a Windows service cannot start an application which has a graphical interface, so someone has to start manually the PVSR agent on the monitored machine or it must be started automatically as a Startup application. The agent communicates with PVSR through files and does not open any port. The measurement server handles these file using the SFTP protocol: it uploads, checks and if needed refreshes the test files used for that equipment and also controls the agent through files. The agent and the Sikuli application must be installed manually on the monitored server.
The measurement server uses the log files in the PVSR logs directory: The APPMON_MAIN is for the whole collector, while the APPMON_EQ.<ID> contains test results for a specific equipment.
14.1.19.1 Creating tests
The tests must be created with the Sikuli software. The application works by looking for images on the screen. These images are saved during the test configuration and the user can also configure that he is looking for an exact match or for a similar image. When the image is found then Sikuli can issue a user interaction, like clicking somewhere on the image or filling an input text box. The PVSR Sikuli environment contains an add-on named pvsrappmon which simplifies several common task and also enables the PVSR integration. The tests are using the Python programming language, its documenatition can be found here: https://docs.python.org/2/
The PVSR specific funtions are the following:
· PVSRstartApp: Start the application specified by the parameters
o Parameters:
§ appFile: name of the file to run
§ appParam: optional command line parameters
§ appPath: optional path to the file
o Return value: Sikuli App object
· PVSRcloseApp: it tries to close the application, first by sending the ALT + F4 key stroke and then calling the close function on the optional object
o Parameters:
§ appObj: Sikuli App object, optional
o Return value: no
· PVSRrun: the same as the built-in run, except the command to run and its output is logged
o Parameters:
§ runcmd: the command to run
o Return value: the output of the command
· PVSRtaskkill: uses the taskkill command to kill a process
o Parameters:
§ imagename: the name of the command image to kill (for example cmd.exe)
o Return value: the output of the taskkill command
· PVSRabortTest: stops the test immediatelly
o Parameters: no
o Return value: no
· PVSRreadParamFile: reads a file containing name=value lines. During the file processes the call converts the names to lowercase. The file might contain remark lines starting with #
o Parameters:
§ filepath: the name of the file
o Return value: no
· PVSRgetParam: gives back the parameter from the parameter file
o Parameters:
§ paramName: the name of the parameter asked for. PVSR converts the parameter to lowercase
o Return value: value of the parameter or None if the parameter wasn’t present in the file
· PVSRgetParamRaw: the same as PVSRgetParam, except the call does not convert the parameter name to lowercase
· PVSRdumpParams: writes the parameters into the log file
o Parameters: no
o Return value: no
· PVSRshowDesktop: hides all open window by using the WIN + M key stroke
o Parameters: no
o Return value: no
· PVSRwait: an extension to the built-in wait function, including logging
o Parameters:
§ picture: the image to search for
§ timeout: optional wait time in seconds
o Return value: the best match or 0 is there was no result
· PVSRpaste: calls the built-in paste function
o Parameters:
§ text: the text to paste
o Return value: no
· PVSRtype: calls the built-in type function
o Parameters:
§ text: the text to paste
o Return value: no
· PVSRpasteWE: calls the built-in paste function and also sends ENTER
o Parameters:
§ text: the text to paste
o Return value: no
· PVSRtypeWE: calls the built-in type function and also sends ENTER
o Parameters:
§ text: the text to paste
o Return value: no
· PVSRclick: calls the built-in click function
o Parameters:
§ picture: the image to search for
§ modifiers: optional modifier key, for example ALT, CTRL, SHIFT
o Return value: 0 if unsuccessful, otherwise 1
· PVSRexistsClick: clicks on the image if it exists
o Parameters:
§ picture: the image to search for
§ timeout: timeout to wait for the image
o Return value: 0 if unsuccessful, otherwise 1
· PVSRdoubleClick: calls the built-in doubleClick function
o Parameters:
§ picture: the image to search for
§ modifiers: optional modifier key, for example ALT, CTRL, SHIFT
o Return value: 0 if unsuccessful, otherwise 1
· PVSRrightClick: calls the built-in rightClick function
o Parameters:
§ picture: the image to search for
§ modifiers: optional modifier key, for example ALT, CTRL, SHIFT
o Return value: 0 if unsuccessful, otherwise 1
· PVSRhover: calls the built-in hover function
o Parameters:
§ picture: the image to search for
o Return value: 0 if unsuccessful, otherwise 1
· PVSRdragDrop: it moves the mouse cursor using the first image then with the left mouse button held down moves the cursor to the second image (buil-in dragDrop function).
o Parameters:
§ picture1: start image
§ picture2: end image
o Return value: 0 if unsuccessful, otherwise 1
· PVSRclickList: looks through the provided images and clicks on the first one found
o Parameters:
§ args: image list
o Return value: 0 if no picture was found, otherwise the result of the click function
· PVSRfindList: looks through the provided images and returns the first one found
o Parameters:
§ args: image list
o Return value: the image found as a Pattern object
· PVSRstartTiming: start an internal timer
o Parameters:
§ key: text, this identifies the timer
o Return value: no
· PVSRendTiming: stop the timer
o Parameters:
§ key: text, this identifies the timer
o Return value: the time passed since PVSRstartTiming
· PVSRcheckPoint: set the value for a checkpoint
o Parameters:
§ name: name of the checkpoint
§ result: value, 0 means unsuccessful, 1 means successful
o Return value: result
· PVSRexistsCheckPoint: set the value of a checkpoint based on image search
o Parameters:
§ name: name of the checkpoint
§ picture: the image to search for
§ timeout: optional wait time in seconds
o Return value: result
· PVSRcustomValue: create own measurement
o Parameters:
§ key: text, this identifies the timer
§ value: value, only number is supported
o Return value: no
· PVSRsaveScreenShot: takes a screenshot and stores it in the PVSR database. The screenshots can be viewed using the Test screenshots collector page. Some of the calls, like the PVSRclick and the PVSRexists take a screenshot if they detect a problem. The screenshot taking can globally disabled/re-enabled with the setCreatePVSRsaveScreenShot call while the screenshot taking during a PVSRexists error can be disabled/re-enabled with the setPVSRexistsScreenshot call
o Parameters:
§ postfix: optional short postfix, default value is „appmon”
§ comment: custom comment, maximum 500 long
o Return value: no
· setPVSRexistsScreenshot: disable/re-enable screenshots during a PVSRexists error
o Parameters:
§ enable/disable screenshots, True or False
o Return value: no
· setCreatePVSRsaveScreenShot: globally disable/re-enable screenshots
o Parameters:
§ enable/disable screenshots, True or False
o Return value: no
· PVSRgetFreeSpace: returns the free space on a drive
o Parameters:
§ filename: optional parameter, the default is c:
o Return value: free space
· PVSRlogInfo: log an information message
o Parameters:
§ str: message
o Return value: no
· PVSRlogError: log an error message
o Parameters:
§ str: message
o Return value: no
· PVSRlogDebug: log a debug message
o Parameters:
§ str: message
o Return value: no
· PVSRsetDebugLog: enable debug level logging
o Parameters:
§ enable: optional, either true or false
o Return value: no
· PVSRsetDebugComputerName: sets on which machine the PVSRshowDebugMessage call should bring up the popup window
o Parameters:
§ DebugComputerName: the PVSRshowDebugMessage call shows the popup window only on that machine which has a machine name matching the name set in this parameter
o Return value: no
· PVSRshowDebugMessage: shows the popup window with the specified message if the computer’s name matches the name set in the PVSRsetDebugComputerName call. The use of this call is recommended when we want to manually test a script on a development machine
o Parameters:
§ msg: message to be shown
o Return value: no
14.1.19.2 Handling the tests
Tests can be managed on the PVSR WEB GUI. For every Appmon equipment the Operation drop-down list contains an item called “Application test script”. Users can manage every test in the system, not just the ones used for the particular equipment: create new ones, download, change and delete existing ones. A restricted administrator can modify or use all the test from one Appmon equipment. No test level authorization. Every test has a version number and the last modification date. The tests created in Sikuli must be first exported using the File => “Export executable…” menu in Sikuli and then uploaded into PVSR using this page.
14.1.19.3 Measurements
The measurements are mainly the successfulness of checkpoints and the time it took to reach that checkpoint.
List of measurements:
· Checkpoint result: the given checkpoint was successful or not
· Checkpoint time: the time between the current and the previous checkpoint (or the start of the test for the first checkpoint) in seconds
· Checkpoints success rate: the percentage of successful checkpoints between two checkpoints, including these boundaries. There are two built-in checkpoint names, AM_BEGIN and AM_END, these represent the begin and the end of the test and they are successful if there was a result file containing the measurement values for the test
· Checkpoints time difference: the time between two checkpoints. There are two built-in checkpoint names, AM_BEGIN and AM_END, these represent the begin and the end of the test and they are successful if there was a result file containing the measurement values for the test
· Full test time: the total test completition time in seconds
· Timing: the value of a timing identified by its name
· Custom value: the value of a custom parameter identified by its name
A part of the measurements can be discovered by PVSR based on the tests. The discovery looks at all of the tests and searches for PVSR functions in the main test Python file:
· Discovers “Timing” measurements based on PVSRendTiming calls
· Discovers “Checkpoint result” and “Checkpoint time” measurements based on PVSRcheckPoint and PVSRexistsCheckPoint calls
· Discoveres “Custom value” measurements based on PVSRcustomValue calls
· Discoveres the “Full test time” measurement
14.1.19.4 Preparing the test server
The following components are needed on the test server:
· Cygwin: the Cygwin MUST be installed into the C:\cygwin directory
· Java JRE 1.6 32 bit
· Sikuli + PVSR appmon agent (appmon.zip)
After the Cygwin installation the SSH service must be configured. It is important that the Java 1.6 32 bit version be available, otherwise Sikuli won’t work. The bin directory of this Java version must be present in the PATH environment variable.
The content of the appmon.zip MUST be extracted into the C:\cygwin\opt\pvsr\appmon directory. To start the agent one must manually start start_pvsr_appmon.bat batch file in the C:\cygwin\opt\pvsr\appmon directory.
The tests generate log files into the C:\cygwin\opt\pvsr\appmon\logs directory and the system keeps the log files for the last two days.
14.1.19.5 Test logs
This page shows the detailed log for every application monitoring tests up to 30 days by default. This retention policy can be changed with the PVSR_APPMON_LOG_MAX_DATE parameter in the CONFIG_INI.pm file. The initial table on the page shows the name of the equipment, the name of the test and the time of the test. The details page shows a table with the checkpoints in the test log, their status and the time it took to get to that point in the test. The content of the test log file includes the important PVSR* function calls, the images the test is looking for and every screenshot taken during the test, which are still keept in the database. The page shows the screenshots with a size so they fit on the page. The user can click on a screenshot and open the original image in a separate browser window.
It is important to note that the application keeps only the last 5GB of pictures by default, this retention policy can be changed with the PVSR_APPMON_PIC_MAX_SIZE_MB parameter in the CONFIG_INI.pm file.
14.1.19.6 Test screenshots
This page shows the screenshots taken during the test. It is important to note that the application keeps only the last 5GB of pictures by default, this retention policy can be changed with the PVSR_APPMON_PIC_MAX_SIZE_MB parameter in the CONFIG_INI.pm file. The page shows the name of the equipment and the name of the test along with these screenshot informations:
· The exact time the screenshot was taken
· The line in the test file where the screenshot was taken
· Category and remark: the two parameters of the PVSRsaveScreenShot function
The details page shows the screenshots with a size so they fit on the page. The user can click on a screenshot and open the original image in a separate browser window.