Welcome to the Cyara Knowledge Center

This article will go through the entire process of setting up and validating a Rally Integration. The main sections of the following article are as follows:  

1. Ensure Rally Integration is enabled
2. Run Velocity Campaign. Review Velocity campaign report and open Rally defects for Test Case results that did not produced desired results
3. Verify Rally defect is logged in to Rally correctly
4. Run an individual Test Case and open Rally defects for Test Case results that did not produced desired results
5. Overview of configuring Rally Integration
6. Rally Integration Setup Walkthrough Video Tutorial

1. Ensure the Rally Integration is enabled

Before we being with creating Rally Issue Tickets, first navigate to “Integrations” under “Tools” from the toolbar:

Ensure that you have an active Integration with Rally and that is set to Enabled. Below the "Rally Integration" is set to “Yes” under Enabled: (For more information on how to first build the Rally Integration, consult the Cyara User Guide article on "Creating a Rally Integration") 

2. Submit Rally defects from the results of the Velocity Campaign

Next, we will run a Velocity Campaign that we have designed to have a range of Test Case results, including failures. Navigate to the Campaigns Tab.

Run a Velocity Campaign. For the purposes of this demonstration, we'll run a quick regression test.

• This campaign will three very small test cases that will generate one of each test results:
  • Success
  • Failed
  • Satisfactory

The campaign will take about 30 seconds to run. Once finished, navigate to the Reports Tab and Select Velocity

Click view next to the the latest report for the Velocity Campaign that was just run.

You will see a report depicting the results of your Velocity Campaign. In our demonstration test we placed a total of 3 calls, of which one was Success, one Failed and one Satisfactory

Scroll down to the list of the individual Test Case results and select the result for the “Test – Fail” Test Case

Looking at the detailed result you will see that the test case had failed due to the Expect to Hear No Match in Step 1

Now that we have found an unexpected behavior, we want to open a Rally defect so that development could look at it.

To open Rally Defect you will need to click on “Create” button in the upper hand corner section of the Test Case Result page.

Once clicked, a “Create New Ticket” dialog box will open.

You will notice how all the relevant data fields have been pre-populated with the data.

You can also extend the “Description” field (by dragging lower right-hand corner of the text box) to show all test case results data that was configured to be submitted with the defect.

At this time (depending on the configuration) the “tester” can review and update any information that is being submitted.

More on how this data gets mapped into the data fields is to be discussed when we review the configuration of the Rally integration.

3. Verify Rally defect is logged in to Rally correctly

Once “tester” clicks the “Create” button, the Rally defect ticket is created, and the Test Case result is permanently linked to the Rally defect number.

At this time, you can click on the “Rally-XX” defect link or on the “View” button to view the defect in Rally.

If you are currently not logged into Rally, you will be presented with the Rally Portal Login dialog box. Please use the credentials listed in the Access Information section.

Note: Typically, after you logged in once, Rally creates a cookie which will enable browser to skip login part for all subsequent Rally sessions.

The Rally Defect with all the information from the Test Case will now be displayed:

Rally also lists all other project defects by default as you can see to the left of the newly created defect above.

4. Submit Rally defects from the results of the manual Test Case validation

Rally defects do not have to be only created from Velocity Campaign Test Result Failures, they can also be created from any manual Test Case Validation, see below for details.

First navigate to the Test Cases Tab and Select “Test Cases & Blocks”

In this example, we're going to validate a Test Case that we know will fail.

Looking at the detailed result you will see that the test case had failed due to the Expect to Hear No Match in Step 1

Now that we have found an unexpected behavior, we want to open a Rally defect so that development could look at it.

Similarly to how the Rally Defect was created earlier in the Velocity Campaign, on the Test Case Validation Results screen, click the “Create” button in the upper hand corner to start the "Create New Ticket" dialog box process.

You will notice how all the relevant data fields have been pre-populated with the data.

I have extended the “Description” field (by dragging lower right-hand corner of the text box) to show all test case results data that was configured to be submitted with the defect.

At this time (depending on the configuration) the “tester” can review and update any information that is being submitted.

5. Overview of configuring Rally Integration

Further to the instructions on how to create a Rally Integration for the first time as covered in the Cyara User Guide article on "Creating a Rally Integration"
The following section contains additional information needed for establishing connectivity.

Note: When configuring the Rally integration from Cyara Cloud Portal to Rally deployed on customer premise and Rally portal is not accessible from the internet, the customers IT organization must enable firewall access from the Cyara Cloud to premise instance of Rally by whitelisting the Cyara integration FQDN to allow connections from the Cyara Cloud. We communicate to Rally using HTTPS.

• US FQDN: egress-ips.cyaraportal.us
• AU FQDN: egress-ips.cyaraportal.com
• UK FQDN: egress-ips.cyaraportal.co.uk

The following section allows for configuring mapping of Cyara Test Case Results to Rally data fields.

Known Rally fields are selected from the “New Rally Field” list and new fields can be defined by selecting “New Custom Field”. Selecting the ? Icon will display list of Cyara variables that will contain Test Case results information.

As an example, the Rally “description” field contains the complete list of all variables so that once Rally defect is created you can see what that data looks like (see below). Multiple variables can be used in the same text string to comprise complex information.

The following settings in the filed selection are used to control what fields are truly required and if they could be overwritten by the tester when they are submitting the Rally defect (remember the pop-up that is displayed when Rally defect is submitted form the test case result). Some fields can be pre-populated with Cyara $variables, while some can be just predefined fields with no data, that must be completed by the tester. Each flag is a selection between “Yes” or “No”

6. Rally Integration Setup Walkthrough Video Tutorial

For a detailed walkthrough of how to configure the Integration between Rally and your Cyara Platform, watch the tutorial video below.