Integrate your JUnit tests with TestRail

5 min readFeb 8, 2021

Have you ever thought that would be great if all the results of test automation run will be reported and stored in TestRail automatically?

Here you are, that moment came just now! A Library, which helps to implement auto-reporting of your JUnit-based tests to your TestRail management system. You need to follow just a few steps and it will be doing all stuff by its own. Review the article and try it out!

This guide explains how to integrate JUnit-based tests with TestRail in an easily manner.

RSheremeta/TestRail-JUnit

This Library helps you to integrate your JUnit-based tests with TestRail in an easily manner.

I wrote this library in order to integrate your JUnit-based tests with TestRail just in an easily manner. Let’s see how to do that.

IMPORTANT:
For integration you need to provide TestRail link, Project Id and Credentials (username & password) into your code.

Now we can move further.

So, firstly, you need to add this Library to your project. Since it’s not pushed to the MVN central repository yet, there are two installation options:

Download already compiled and packaged TestRail-JUnit-1.0.jar file HERE and proceed to the 2nd step
Download source code HERE to your local machine, make a jar by running mvn package in the repo directory and proceed to the 2nd step

2. Put a TestRail-JUnit-1.0.jar file into your project directory (for instance — create root dir ~/libs/ and paste there)

3. After that, add this dependency to your pom.xml (systemPath value is for instance)

Mvn dependency

Now when we’re done with adding a Library to the project, we need to annotate our tests with @TestRailCase annotation and pass the TestRail id for that test respectively.

package com.github.rsheremeta;


import com.github.rsheremeta.testrail.TestRailCase;
import org.junit.Assert;
import org.junit.jupiter.api.Disabled;
import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.Test;
public class TestClassOne extends BaseTest {


  @Test
  @TestRailCase(id = "1")
  @DisplayName("This Test should be passed")
  public void oneFirst() {
    Assert.assertTrue(true);
    System.out.println("Passed");
  }

  @Test
  @TestRailCase(id = "2")
  @Disabled()
  @DisplayName("This Test should be skipped")
  public void oneSecond() {
    System.out.println("Skipped");
  }

}

package com.github.rsheremeta;


import com.github.rsheremeta.testrail.TestRailCase;
import org.junit.Assert;
import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.Test;
public class TestClassTwo extends BaseTest {


  @Test
  @TestRailCase(id = "3")
  @DisplayName("This Test should be passed")
  public void twoFirst() {
    Assert.assertTrue(true);
    System.out.println("Passed");
  }

  @Test
  @TestRailCase(id = "4")
  @DisplayName("This Test should be failed")
  public void twoSecond() {
    int a = 3;
    int b = a/0;
    Assert.assertEquals(a, b); // division by zero
    System.out.println("Failed with exception");
  }

}

Once you’re done with annotating your tests with corresponding TestRail Ids, you need to create a TestRailListener class that implements BeforeAllCallback, TestWatcher interfaces and override their methods:

package com.github.rsheremeta;

import org.junit.jupiter.api.extension.*;
import java.util.Optional;


public class TestRailListener implements BeforeAllCallback, TestWatcher {

  @Override
  public void beforeAll(ExtensionContext extensionContext) {

  }

  @Override
  public void testDisabled(ExtensionContext extensionContext, Optional<String> reason) { }

  @Override
  public void testSuccessful(ExtensionContext extensionContext) {

  }

  @Override
  public void testAborted(ExtensionContext extensionContext, Throwable cause) {

  }

  @Override
  public void testFailed(ExtensionContext extensionContext, Throwable cause) {

  }

}

Now we’ll be implementing the key logic for all these methods one-by-one. Let’s start with beforeAll() method, which is needed for creation a new test run and setting up all needed data like User creds, Project id etc. Let’s define your TestRail Project Id and Name for automatically generated runs.

Please note, Test run name will be appended by a timestamp in format MM-dd-yyyy HH:mm:ss (For instance — 02–07–2021 13:21:11).

We’ll use a TestRailCreds builder class for creating TestRail data object for further passing as a parameter for the static method createNewTestRailRun() of TestRailHelper class, like this:

beforeAll() method

Now we’ll be implementing listener methods for the corresponding test results. Setting up statuses will be done by calling static method setTestRailStatus() of the well-known TestRailHelper class.

This setTestRailStatus() method accept two parameters: instance of ExtensionContext and TestResult enumeration with one of the corresponding values — PASSED, FAILED, BLOCKED.

listener methods

Please pay attention on the testDisabled() method, which is empty, as TestRail does not allow to add result as “Untested”, which is a default value for each and every test in a run, so there is no need to define and set it explicitly.

Now we’re all set with setting statuses for our tests and the last one step is to annotate our test classes with our Listener written just now. If all your test classes are inherited from BaseTest (as in code mentioned above), you can annotate only parent class like this:

BaseTest.class

So finally, we’re done with code-related activities! Now we just need to run our test suite and navigate to the TestRail runs listing page and see what the magic happened :)

Run your test suite, in my case it’s done by mvn clean test command in Terminal.
Navigate to the TestRail in browser, switch to the TEST RUNS & RESULTS tab and locate just created run.

Let’s take a look on my results.

Here is a listing page with all my test runs generated automatically: