How to Manage Test Cases and Bugs Using JIRA

My determination of writing this blog is to describe how we can use JIRA as a TC (Test cases) and bug management with the help of add-on JIRA Cli. Also, this document will help in analyzing the benefits & drawbacks of using JIRA as a test case management.

Previously, I had to depend on an Excel spreadsheet for logging test cases and JIRA for tracking bugs in my project. I needed a tool by which we can accomplish both test cases and bugs with using a single tool. Now I am using JIRA for Test cases and bug management in spite of some constraints as outlined in the later part of this blog.

Modification in JIRA

Modifications are required to use existing flow of JIRA as a Test Case  management with the help of a JIRA administrator. Below are the steps to modify the existing JIRA workflow:

• Create/Add new Issue type
• Create/Add new Status
• JIRA default workflow
• Test case work flow after modification
• Status in Test case workflow

Here are additional detail explaining each of the steps below:

How to create issue type?

Add a new issue type in existing flow and name it “Test case.” This should be a top- level (standard) task (for this go to Administration > Issues > Issues Types and use  Add Issue Type option).

How to create status?

Add new status (for this go to Administration > Status > Status Types and use Add    Status Type option)

JIRA default workflow

JIRA workflow is the set of status and transitions through which an issue goes during its lifecycle.

Test case work flow after modification

After modification with the help of JIRA administrator below is the Test case workflow with different status:

Status in Test case workflow:

TC awaiting review: Default status when a Test Case is created and awaiting for Review from the QA lead.

TC Template: An approved TC that has not yet been scheduled yet.

TC-Ready to Run: TC that has been scheduled.

In Progress: TC that has been in-progress state

TC-Failed: After execution when TC fail status update to TC-Failed

TC-Passed: After execution when TC Pass status update to TC-Passed

TC-Invalid: A test that has become obsolete or cannot be run

Closed: This is the final status of the TC.

About JIRA CLI

One of the major challenge was how to import the TCs in JIRA because designing test cases one by one in JIRA was time consuming. A tool was needed by which I can upload the TC in JIRA from file (.csv, Excel, etc.). Finally, I found Add-on JIRA CLI (free tool ~Till version 2.6.0) to import the Test case’s from .csv

How to import Test case using JIRA CLI?

-Download jira-cli-2.6.0 from this link

-Save the Test cases file with name import.xls and convert into .csv format then copy import.Csv file under downloaded folder (jira-cli-2.6.0) .csv format given below

~In above import.csv file Description column consist  merge data of (Prerequisites, Steps, and Expected Results)

-Under downloaded folder (jira-cli-2.6.0) there is a file with name import.properties

-Map column name for which we want to import the data in the properties file

field.Type = Type

field.Component = Components

field.Project = Project

field.Objective = Summary

field.Description = Description

File (~import.properties file format as seen above)

-Run the following single line command in command prompt

c://Jira-cli-3.8.0>jira.bat –server <URL of JIRA> -user kailash.pathak – password “xxxxxxx” –action runFromCsv –file import.csv –propertyFile import. Properties –common “–project SSPQA”               — continue

(~Here “SSPQA” Is Project Key)

-Test case data is imported in JIRA against the particular components successfully

Process for Test case creation and allocation in JIRA

Below are the steps for test case creation and allocation in JIRA:

Test Case creation steps:

• Create a new project with the help of JIRA administrator.
• Create sprint Task, sub task for QA e.g. Design Test case and Execute Test case of particular module    
• Create Components against the Project with the help of JIRA administrator
• Create Test Case In Excel
• Convert TC in .csv file with giving name as import.csv (its mandatory csv with this  name) for to import in JIRA. csv file format attached above
• Upload .csv into JIRA using JIRA CLI using the (above mentioned steps)

Create Test Case in excel sheet in the format which is acceptable by JIRA CLI (see below for example)

Test Case Allocation steps :

We can allocate Test Cases for execution using bulk change functionality. During the bulk change we can assign the user name to the Test Cases.

First search the TCs then assign user name in bulk by clicking on Assign to me link see below screen shot for more detail

Test Case Execution

Once the TCs are allocated next step is to execute the Test Cases in JIRA with updating their status with Pass, Fail etc.

Prerequisites:

• We are executing TC in three browsers (Chrome, IE, and FireFox)
• We are executing each of TC in two cycles (Cycle 1, Cycle 2)

After uploading and allocating the test cases next step is to execute the test cases. Follow these steps to execute it:

• Bring all Test cases we want to execute in “IN PROGRESS” state using Bulk change functionality
• Execute TCs one by one

In case of Test case FAIL

• In case of a FAIL a New Transition comments screen (See screen shot below) open
• Enter actual result.
• Enter Sprint<Sprint Number>-Cycle< Test Cycle No. >-<Browser Name> <Fail>

E.g. Sprint26-Cycle1-IE-Fail as Comment

• Click on the Fail button
• Particular test case is Fail
• Report bug for Fail Test Case
• Link Fail Test Case with Bug Number (Explain in next point)

Linking of Fail Test case with Bug ID

To link the test case with Bug ID Open Particular Failed TC and Click on “Link” from menu following screen open to link the Test Case with Bug ID after clicking on link button Fail TC link with the Bug ID

In case of Test case PASS

If Test Case Pass in first time or in second cycle in that case update the comments with Pass status and with respective Cycle Below the screenshot when TC pass in second cycle

*Note:

Entered above comments help us to pull out the report.

In case of Test case INVALID

In case of invalid test case that has become obsolete or cannot be run. We will update the status as shown below:

*Note:

Same transition screen open to enter comments in case of other status (Closed, TC-Template, TC-Ready to Run etc.)

Report in JIRA of Test Cases

We can generate execution report using different criteria (We can use the entered comments during execution of TC as search criteria).

Search Criteria:

Sprint Wise Report

    1. Complete Sprint wise report (Sprint26-*)

    Example:

Enter Sprint26-* in the Comment window after selecting Project, Components as shown in the screen below:

End Result:

This search criteria displays all executed TCs (Pass, Fail, Invalid etc.)  with a consolidated list as shown below:

We can convert this report data (in Two Dimensional and Pie Chart illustrations as well)

See two dimensional and pie chart report for the above documented criteria

*Note:

For each of the below mentioned search criteria we can create a similar report

2. Sprint with cycle wise report Sprint26-Cycle1-*

End Result:

This search criteria display all executed TC of Cycle1

We can convert this report data in (Two Dimensional and Pie Chart

3.Sprint with Cycle and Browser wise (Firefox, IE, Chrome) report

• Search by criteria Sprint26-Cycle1-FF-*
• Search by criteria Sprint26-Cycle1-IE-*
• Search by criteria Sprint26-Cycle1-Chrome-*

End Result:

This search criteria displays all executed TC of Cycle1 for different browser

We can convert this report data in (Two Dimensional and Pie Chart)

4. Sprint with (Cycle, Browser and Test case Pass) wise report

Search by criteria Sprint26-Cycle1-IE-Pass

End Result:

This search criteria display all Passed TC of Cycle1 for Browser IE

We can convert this report data in (Two Dimensional and Pie Chart)

5. Sprint with (Cycle, Browser and Test case Fail) wise report

Search by criteria Sprint26-Cycle1-IE-Fail

End Result:

This search criteria display all Failed TC of Cycle1 for Browser IE

We can convert this report data in (Two Dimensional and Pie Chart)

*Note:

Similarly we can generate report for Cycle2 #

JIRA – Pros

• Bulk clone is possible
• Possibility to update the status of multiple test cases at once
• Custom searches can be saved as filters and shared through products/users
• Test plan can be easily create in JIRA by putting together all test cases to be executed in that iteration (Sprint)
• Easily linking between Test Cases and Bugs
• Provides an advanced search mechanism and a way to create custom search queries

JIRA – Cons:

• The “look and feel” is the same as for bugs not as for a normal test case management tool (like Test Link or Quality Center)
• Initial set up is bit time consuming but once design user can easily manage the Test Cases

 Conclusion

• With this blog, we have understood how we can use JIRA as a single tool to create, execute Test Cases and bug management.
• We have also given a brief overview of add-on JIRA Cli and its uses when it comes to importing the test case in JIRA.