NatRitmeyer.com

Reporting aggregated unit and integration test coverage with jacoco

2019-06-26T00:00:00+01:00

The higher your test coverage, the less your fear.

Robert C. Martin, Clean Code (2009), 124.

Jacoco is a pretty popular code coverage tool in the java world. Configuring it in the pom.xml to generate coverage reports for unit and integration is easy enough. But, there are times when you want a report that takes both unit and integration tests into account.

In this post we’ll go through the configuration options required to generate unit and integration test coverage reports as well as a report that integrates the merged coverage data. I’m assuming the following setup:

Java 8+
Maven 3.3+

Unit test coverage report

Let’s start by configuring jacoco to create a unit test coverage report. The first step is adding jacoco to your project.

Include the Jacoco plugin

To add jacoco to your project add the following to the <build> section of your pom.xml:

<build>
  <plugins>
    <plugin>
      <groupId>org.jacoco</groupId>
      <artifactId>jacoco-maven-plugin</artifactId>
      <version>0.8.4</version>
      <executions>
      </executions>
    </plugin>
  </plugins>
</build>

With that it’s time to add a load of <execution> blocks to the <executions> section.

Configuring jacoco unit test coverage data

Jacoco splits collection of test coverage data from the generation of the coverage report. There are therefore <execution> blocks required for each stage in the process.

This first execution block tells jacoco where to store the unit test coverage data it will collect during unit test execution:

<execution>
  <id>before-unit-test-execution</id>
  <goals>
    <goal>prepare-agent</goal>
  </goals>
  <configuration>
    <destFile>${project.build.directory}/jacoco-output/jacoco-unit-tests.exec</destFile>
    <propertyName>surefire.jacoco.args</propertyName>
  </configuration>
</execution>

The <destFile> element contains the path the data will be written to. The <propertyName> contains the name of a variable that will be populated with some arguments which will be passed to surefire (explained later) to point it at the coverage data collection file defined in <destFile>.

Generating the unit test coverage report

The next execution block is used to tell jacoco where to write the unit test report, and which coverage data file to generate the report from.

<execution>
  <id>after-unit-test-execution</id>
  <phase>test</phase>
  <goals>
    <goal>report</goal>
  </goals>
  <configuration>
    <dataFile>${project.build.directory}/jacoco-output/jacoco-unit-tests.exec</dataFile>
    <outputDirectory>${project.reporting.outputDirectory}/jacoco-unit-test-coverage-report</outputDirectory>
  </configuration>
</execution>

The report generator will look for its input (the coverage data file populated during unit testing) in the path defined in the <dataFile> element, and will save the generated report in the path specified in <outputDirectory>. Note that in the above example uses the project.reporting.outputDirectory property which points at target/site.

Configuring surefire

Maven uses surefire to execute unit tests, and for coverage data to be collected jacoco configures surefire to use a java agent that instruments the classes under test to enable collection of execution data.

Here’s the how surefire needs to be configured:

<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-surefire-plugin</artifactId>
  <version>2.22.2</version>
  <configuration>
    <argLine>${surefire.jacoco.args}</argLine>
  </configuration>
</plugin>

Now, next time you run mvn test you should see the following:

[INFO] --- jacoco-maven-plugin:0.8.4:prepare-agent (before-unit-test-execution) @ jacocoexample ---
[INFO] surefire.jacoco.args set to -javaagent:/Users/nat/.m2/repository/org/jacoco/org.jacoco.agent/0.8.4/org.jacoco.agent-0.8.4-runtime.jar=destfile=/Users/nat/dev/jacocoexample/target/jacoco-output/jacoco-unit-tests.exec
[INFO] 
...
[INFO] 
[INFO] --- maven-surefire-plugin:2.22.2:test (default-test) @ jacocoexample ---
[INFO] 
[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO] Running com.natritmeyer.jacocoexample.unit.ThingTest
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.047 s - in com.natritmeyer.jacocoexample.unit.ThingTest
[INFO] 
[INFO] Results:
[INFO] 
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0
[INFO] 
[INFO] 
[INFO] --- jacoco-maven-plugin:0.8.4:report (after-unit-test-execution) @ jacocoexample ---
[INFO] Loading execution data file /Users/nat/dev/jacocoexample/target/jacoco-output/jacoco-unit-tests.exec
[INFO] Analyzed bundle 'jacocoexample' with 3 classes

Hopefully you can make out that the surefire.jacoco.args property is set in the first step. Next comes the test execution, followed by the generation of the unit test coverage report. If you look in your target/site/jacoco-unit-test-coverage-report/ directory you should see an index.html file. Open it and you should see your unit test coverage report:

Integration test coverage report

Now that we’ve got our build generating a coverage report for the unit tests it’s time to set up matching config for the integration tests.

In principle this is exactly the same as setting up coverage for unit tests with only one difference: instead of configuring surefire, the unit tests execution tool, we’re going to configure failsafe, maven’s integration test execution tool.

Configuring jacoco’s integration coverage report

Here are the two <execution> blocks required to set up integration test coverage:

<execution>
  <id>before-integration-test-execution</id>
  <phase>pre-integration-test</phase>
  <goals>
    <goal>prepare-agent</goal>
  </goals>
  <configuration>
    <destFile>${project.build.directory}/jacoco-output/jacoco-integration-tests.exec</destFile>
    <propertyName>failsafe.jacoco.args</propertyName>
  </configuration>
</execution>

<execution>
  <id>after-integration-test-execution</id>
  <phase>post-integration-test</phase>
  <goals>
    <goal>report</goal>
  </goals>
  <configuration>
    <dataFile>${project.build.directory}/jacoco-output/jacoco-integration-tests.exec</dataFile>
  <outputDirectory>${project.reporting.outputDirectory}/jacoco-integration-test-coverage-report</outputDirectory>
  </configuration>
</execution>

Configuring failsafe

Finally, failsafe needs to be configured to use the contents of the failsafe.jacoco.args property as arguments:

<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-failsafe-plugin</artifactId>
  <version>2.22.2</version>
  <configuration>
    <argLine>${failsafe.jacoco.args}</argLine>
  </configuration>
  <executions>
    <execution>
      <goals>
        <goal>integration-test</goal>
        <goal>verify</goal>
      </goals>
    </execution>
  </executions>
</plugin>

If you run mvn verify you should see the following:

[INFO] --- jacoco-maven-plugin:0.8.4:prepare-agent (before-integration-test-execution) @ jacocoexample ---
[INFO] failsafe.jacoco.args set to -javaagent:/Users/nat/.m2/repository/org/jacoco/org.jacoco.agent/0.8.4/org.jacoco.agent-0.8.4-runtime.jar=destfile=/Users/nat/dev/jacocoexample/target/jacoco-output/jacoco-integration-tests.exec
[INFO] 
[INFO] --- maven-failsafe-plugin:2.22.2:integration-test (default) @ jacocoexample ---
[INFO] 
[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO] Running com.natritmeyer.jacocoexample.integration.IntegratedThingsIT
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.045 s - in com.natritmeyer.jacocoexample.integration.IntegratedThingsIT
[INFO] 
[INFO] Results:
[INFO] 
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
[INFO] 
[INFO] 
[INFO] --- jacoco-maven-plugin:0.8.4:report (after-integration-test-execution) @ jacocoexample ---
[INFO] Loading execution data file /Users/nat/dev/jacocoexample/target/jacoco-output/jacoco-integration-tests.exec
[INFO] Analyzed bundle 'jacocoexample' with 3 classes

Just like what happened for the unit tests, the first thing that happened was the population of the failsafe.jacoco.args property. Next came test execution, and finally the report was generated. The result should be an index.html file in thetarget/site/jacoco-integration-test-coverage-report directory. Open it and it should look something like this:

Merged test coverage report

We’ll now configure jacoco to create a report based on both unit and integration test coverage. Achieving this will require first merging

Merging unit and integration test coverage data

Jacoco generates reports from .exec files. Thus far we have two of them. If we want jacoco to generate a report based on the data in both files we first need to merge the data from both .exec files into a new .exec file. Here’s The <execution> that will make that happen:

<execution>
  <id>merge-unit-and-integration</id>
    <phase>post-integration-test</phase>
      <goals>
        <goal>merge</goal>
      </goals>
      <configuration>
      <fileSets>
        <fileSet>
          <directory>${project.build.directory}/jacoco-output/</directory>
          <includes>
            <include>*.exec</include>
          </includes>
        </fileSet>
      </fileSets>
      <destFile>${project.build.directory}/jacoco-output/merged.exec</destFile>
    </configuration>
  </execution>
<execution>

The <fileSet> tells jacoco which directory to look for input to its merge goal. The <include> element tells jacoco which files to read in. In the case above that’s *.exec*. Finally, the <destFile> is where the merged coverage data will be written.

Generating the merged unit and integration test coverage report

The final step is to configure jacoco to generate a coverage report from the merged.exec data we created in the previous step. We’ve already seen how to do that - it’s the same as generating a unit or integration test report:

<execution>
  <id>create-merged-report</id>
  <phase>post-integration-test</phase>
  <goals>
    <goal>report</goal>
  </goals>
  <configuration>
    <dataFile>${project.build.directory}/jacoco-output/merged.exec</dataFile>
    <outputDirectory>${project.reporting.outputDirectory}/jacoco-merged-test-coverage-report</outputDirectory>
  </configuration>
</execution>

There’s an important thing to notice: the <phase>post-integration-test</phase> element. Though this is the same phase as the previous step that created the merged coverage data file, because maven respects the sequence of <execution> blocks this last report-generation step will always follow the preceding coverage-data-file-generation step.

If you run mvn verify now you should see the following:

[INFO] --- jacoco-maven-plugin:0.8.4:merge (merge-unit-and-integration) @ jacocoexample ---
[INFO] Loading execution data file /Users/nat/dev/jacocoexample/target/jacoco-output/jacoco-integration-tests.exec
[INFO] Loading execution data file /Users/nat/dev/jacocoexample/target/jacoco-output/jacoco-unit-tests.exec
[INFO] Writing merged execution data to /Users/nat/dev/jacocoexample/target/jacoco-output/merged.exec
[INFO] 
[INFO] --- jacoco-maven-plugin:0.8.4:report (create-merged-report) @ jacocoexample ---
[INFO] Loading execution data file /Users/nat/dev/jacocoexample/target/jacoco-output/merged.exec
[INFO] Analyzed bundle 'jacocoexample' with 3 classes

The first block explains what happened in the merge, the second block tells you that the merged coverage report was generated.

In your target/site/jacoco-merged-test-coverage-report directory you should find an index.html file. Open it and you’ll see something like the following:

There you go. A test coverage report based on the execution of both unit and integration tests.

Failing the build

Putting aside religious wars about what level of test coverage is acceptable, here’s how to get jacoco to fail the build based on the aggregate coverage of the unit and integration tests (as opposed to the unit and integration test phases individually).

We’ll use a jacoco check <execution> block to do the job. This example will fail the build if coverage, taking both unit and integration tests, drops below 100%:

<execution>
  <id>check</id>
  <phase>verify</phase>
  <goals>
    <goal>check</goal>
  </goals>
  <configuration>
    <rules>
      <rule>
        <element>CLASS</element>
        <excludes>
          <exclude>*Test</exclude>
          <exclude>*IT</exclude>
        </excludes>
        <limits>
          <limit>
            <counter>LINE</counter>
            <value>COVEREDRATIO</value>
            <minimum>100%</minimum>
          </limit>
        </limits>
      </rule>
    </rules>
    <dataFile>${project.build.directory}/jacoco-output/merged.exec</dataFile>
  </configuration>
</execution>

Just one thing to note… the check is being executed against the merged.exec coverage data file created further up by the merge <execution> block.

Hopefully the above makes sense and was easy enough to follow. For those who’re here just to copy and paste the whole lot into your pom.xml, here’s what you need:

TL;DR

If you want the whole thing in one hit, here it is:

<build>
  <plugins>
    <plugin>
      <groupId>org.jacoco</groupId>
      <artifactId>jacoco-maven-plugin</artifactId>
      <version>0.8.4</version>
      <executions>
        <execution>
        <id>before-unit-test-execution</id>
        <goals>
          <goal>prepare-agent</goal>
        </goals>
        <configuration>
          <destFile>${project.build.directory}/jacoco-output/jacoco-unit-tests.exec</destFile>
          <propertyName>surefire.jacoco.args</propertyName>
        </configuration>
      </execution>
      <execution>
        <id>after-unit-test-execution</id>
          <phase>test</phase>
          <goals>
            <goal>report</goal>
          </goals>
          <configuration>
            <dataFile>${project.build.directory}/jacoco-output/jacoco-unit-tests.exec</dataFile>
            <outputDirectory>${project.reporting.outputDirectory}/jacoco-unit-test-coverage-report</outputDirectory>
          </configuration>
        </execution>
        <execution>
          <id>before-integration-test-execution</id>
          <phase>pre-integration-test</phase>
          <goals>
            <goal>prepare-agent</goal>
          </goals>
          <configuration>
            <destFile>${project.build.directory}/jacoco-output/jacoco-integration-tests.exec</destFile>
            <propertyName>failsafe.jacoco.args</propertyName>
          </configuration>
        </execution>
        <execution>
          <id>after-integration-test-execution</id>
          <phase>post-integration-test</phase>
          <goals>
            <goal>report</goal>
          </goals>
          <configuration>
            <dataFile>${project.build.directory}/jacoco-output/jacoco-integration-tests.exec</dataFile>
            <outputDirectory>${project.reporting.outputDirectory}/jacoco-integration-test-coverage-report</outputDirectory>
          </configuration>
        </execution>
        <execution>
          <id>merge-unit-and-integration</id>
          <phase>post-integration-test</phase>
          <goals>
            <goal>merge</goal>
          </goals>
          <configuration>
            <fileSets>
              <fileSet>
                <directory>${project.build.directory}/jacoco-output/</directory>
                <includes>
                  <include>*.exec</include>
                </includes>
              </fileSet>
            </fileSets>
            <destFile>${project.build.directory}/jacoco-output/merged.exec</destFile>
          </configuration>
        </execution>
        <execution>
          <id>create-merged-report</id>
          <phase>post-integration-test</phase>
          <goals>
            <goal>report</goal>
          </goals>
          <configuration>
            <dataFile>${project.build.directory}/jacoco-output/merged.exec</dataFile>
            <outputDirectory>${project.reporting.outputDirectory}/jacoco-merged-test-coverage-report</outputDirectory>
          </configuration>
        </execution>
        <execution>
          <id>check</id>
          <phase>verify</phase>
          <goals>
            <goal>check</goal>
          </goals>
          <configuration>
            <rules>
              <rule>
                <element>CLASS</element>
                <excludes>
                  <exclude>*Test</exclude>
                  <exclude>*IT</exclude>
                </excludes>
                <limits>
                  <limit>
                    <counter>LINE</counter>
                    <value>COVEREDRATIO</value>
                    <minimum>100%</minimum>
                  </limit>
                </limits>
              </rule>
            </rules>
            <dataFile>${project.build.directory}/jacoco-output/merged.exec</dataFile>
          </configuration>
        </execution>
      </executions>
    </plugin>
    <plugin>
      <groupId>org.apache.maven.plugins</groupId>
      <artifactId>maven-surefire-plugin</artifactId>
      <version>2.22.2</version>
      <configuration>
        <argLine>${surefire.jacoco.args}</argLine>
      </configuration>
    </plugin>
    <plugin>
      <groupId>org.apache.maven.plugins</groupId>
      <artifactId>maven-failsafe-plugin</artifactId>
      <version>2.22.2</version>
      <configuration>
        <argLine>${failsafe.jacoco.args}</argLine>
      </configuration>
      <executions>
        <execution>
          <goals>
            <goal>integration-test</goal>
            <goal>verify</goal>
          </goals>
        </execution>
      </executions>
    </plugin>
  </plugins>
</build>

Remember, if you don’t want low test coverage to fail the build, remove the check <execution> block.

Updating a BrowserStack session’s name and result

2019-03-30T00:00:00+00:00

Intro

BrowserStack’s REST API provides the ability to update various aspects of a session, e.g. test result, test name, etc. They do seem to provide a client library called browserstack-automate-java for interacting with the API (GitHub, maven-central), but it doesn’t seem to get much love and I can’t find any documentation for it. Its basic functions appear to work but using it to update anything in browserstack seems to fail.

Solving the problem

Wanting to update the session name and result, I wrote my own client based on Spring-Flux’s WebClient. Here’s what you need if you want to do the same.

Some dependencies

First, you’ll need some dependencies:

<dependencies>
  <dependency>
    <groupId>org.springframework</groupId>
    <artifactId>spring-webflux</artifactId>
    <version>5.1.5.RELEASE</version>
  </dependency>
  <dependency>
    <groupId>com.fasterxml.jackson.core</groupId>
    <artifactId>jackson-databind</artifactId>
    <version>2.9.4</version>
  </dependency>
  <dependency>
    <groupId>io.projectreactor.netty</groupId>
    <artifactId>reactor-netty</artifactId>
    <version>0.8.5.RELEASE</version>
  </dependency>
</dependencies>

Getting the BrowserStack Session ID

To update a BrowserStack session you need its ID; you can get that from your RemoteWebDriver instance:

RemoteWebDriver driver = ...;

// do some navigation, testing, etc.

String sessionId = driver.getSessionId().toString();

// sessionId is now "d430d61935a66ddd9c5c9a9e15362c520414b082"

Preparing the payload

Next, because you’re going to be sending JSON you’ll need a simple class that when converted to JSON matches what the API expects. Given we’re updating the session name and status the following is enough for our needs:

public class SessionDetails {
  private String name;
  private String status;

  private SessionDetails(String name, String status) {
    this.name = name;
    this.status = status;
  }

  public String getName() {
    return name;
  }

  public String getStatus() {
    return status;
  }
}

Assuming you’re using cucumber-jvm you’ll probably be making calls to BrowserStack’s API in an @After step. You’ll therefore have access to the scenario’s name as well as its result and can set the BrowserStack session’s result accordingly:

@After
public void setBrowserStackSessionDetails(Scenario scenario) {
  String testName = scenario.getName();
  
  // The API expects either "passed" or "failed". Here's a
  // rubbish way to ensure that. Maybe use an enum instead.
  String testResult = scenario.isFailed() ? "failed" : "passed";
}

We can now create an instance of the SessionDetails with the details we want to send:

@After
public void setBrowserStackSessionDetails(Scenario scenario) {
  String testName = scenario.getName();
  String testResult = scenario.isFailed() ? "failed" : "passed";
  
  SessionDetails sessionDetails = new sessionDetails(testName, testResult);
}

Making the call to BrowserStack

Now that we’ve got something we want to send to BrowserStack we need to get a HTTP client ready.

Preparing the WebClient

The old resttemplate way of doing things will be deprecated in the near future and replaced with WebClient. Its API isn’t all that intuitive but it makes sense in the end. Here’s how to set up a WebClient`:

import org.springframework.web.reactive.function.client.ExchangeFilterFunctions;
import org.springframework.web.reactive.function.client.WebClient;

//...

String browserStackUsername = "MY_BROWSERSTACK_USERNAME";
String browserStackAccessKey = "MY_BROWSERSTACK_ACCESS_KEY";

WebClient client = WebClient.builder()
  .baseUrl("https://api.browserstack.com")
  .filter(ExchangeFilterFunctions.basicAuthentication(browserStackUsername, browserStackAccessKey))
  .build();

Updating BrowserStack’s session details

The final thing to do is make the call to browserstack. Here’s how it’s done:

import org.springframework.http.MediaType;
import reactor.core.publisher.Mono;

//...

client.put().uri(builder -> builder.path("/automate/sessions/" + sessionId + ".json").build())
  .contentType(MediaType.APPLICATION_JSON)
  .body(Mono.just(sessionDetails), SessionDetails.class)
  .retrieve()
  .bodyToMono(String.class)
  .block();

That takes the sessionDetails containing the test name and status (passed/failed), converts it to JSON, and makes a PUT call to the session ID extracted from the RemoteWebDriver earlier.

Given your scenario name was “My Scenario Name” and the scenario failed, here’s what you should see:

Here’s what you should see:

Hope that helps.

Cucumber-JVM Dependency Injection with Spring

2019-01-13T00:00:00+00:00

Introduction

One of the common challenges with using cucumber is passing around the WebDriver instance, as well as page object instances. I’ve seen all sorts of weird ways people have dealt with the problem, but all these solutions have one thing in common: they get tied in knots. The pattern established early in the project doesn’t work with a new scenario, so a hack is put in place to keep things working. After a few more similar incidents the framework falls in a heap. Testing gets out of sync with dev, falls further and further behind, and it’s not long before test automation is abandoned. This is not a rare occurrence; I’ve seen it time and time again.

The solution to this problem is Dependency Injection.

What does that mean? Well, instead of creating and managing instances of WebDriver yourself, use a library to do all that for you, including making sure a WebDriver instance ends up where you need it. Same goes for Page Objects. Instead of creating an instance of a page object, somehow getting your driver into it and then getting the page object instance where it needs to be, let the same library do that for you too.

The library we’re going to do this with is Spring and the pattern we’re going to use is simple.

In this post we’ll go through what’s required to put together a test automation framework with Cucumber-JVM that utilises dependency injection using Spring. We’ll go through each step, starting from nothing, explaining what the various moving parts are. As such this tutorial is aimed more at beginners than those with more experience. There’s going to be some fudging of the detail when it comes to some of spring’s more involved concepts - those with more experience will be able to point out the hand waving. YMMV.

Also, the finished product lives here: https://github.com/natritmeyer/cucumber-jvm-dependency-injection-with-spring.

Framework overview

To keep things focused we’re going to keep the “feature” tested in our framework very simple and uninteresting: It’ll search Bing for the word “cucumber”, and then verify that the link to the first hit includes the word “Cucumber”. Here’s the contents of the feature file:

Feature: Search for content

  Scenario: Search for information
    Given I am on the bing search engine
    When I enter a search term
    Then relevant results for that search term are displayed

Our aim is that our single scenario will be able to test both desktop and mobile implementation of Bing’s search function, and that we’ll use dependency injection to make sure the right WebDriver instance and implementation-specific page objects are used in the step definition class.

Ingredients

The framework we’re going to build is based on the following:

Java
Maven
Cucumber-JVM
Spring
WebDriver
Chrome

I’m assuming you’ve got at least a vague notion of what those are.

OK, let’s start putting the pieces together.

Step by step

We’ll start by creating an ordinary, bare bones cucumber project that’ll be just enough to execute the feature file. Once that’s in place we’ll add what we need to integrate spring, and finally implement the springified page objects and browser management classes.

We’ll begin by creating the root directory of our project; let’s call it cucumber-jvm-dependency-injection-with-spring

Setting up a basic cucumber project

Maven projects have a pom.xml file at their root. Amongst other things the pom.xml file describes the project, lists the project’s dependencies, and lists and configures any maven plugins that assist in building the software. Use your IDE to create an empty Maven project, and open the autogenerated pom.xml file in the root (feel free to copy and paste from the one in the github repo linked to at the end of this post).

The `pom.xml` file

Let’s start by adding the usual dependencies required of all cucumber projects:

<dependency>
  <groupId>io.cucumber</groupId>
  <artifactId>cucumber-junit</artifactId>
  <version>4.2.1</version>
  <scope>test</scope>
</dependency>
<dependency>
  <groupId>io.cucumber</groupId>
  <artifactId>cucumber-java</artifactId>
  <version>4.2.1</version>
  <scope>test</scope>
</dependency>
<dependency>
  <groupId>io.cucumber</groupId>
  <artifactId>cucumber-java8</artifactId>
  <version>4.2.1</version>
  <scope>test</scope>
</dependency>

Next we need to configure where in the maven lifecycle the tests will execute - for this example project we’ll have them run during the verify phase. To ensure that, as well as telling the compiler that we’re writing Java 8 code, we’ll add the following to the pom, just below the end of the dependencies section:

<build>
  <plugins>
    <plugin>
      <groupId>org.apache.maven.plugins</groupId>
      <artifactId>maven-compiler-plugin</artifactId>
      <version>3.8.0</version>
      <configuration>
        <source>1.8</source>
        <target>1.8</target>
      </configuration>
    </plugin>
    <plugin>
      <groupId>org.apache.maven.plugins</groupId>
      <artifactId>maven-failsafe-plugin</artifactId>
      <version>2.22.1</version>
      <executions>
        <execution>
          <goals>
            <goal>integration-test</goal>
            <goal>verify</goal>
          </goals>
        </execution>
      </executions>
    </plugin>
  </plugins>
</build>

With that done we can move on to the next stage of creating a basic cucumber project.

The feature file

The feature file should be called search.feature and placed in src/test/resources/features/. Once created set its content to be:

Feature: Search for content

  Scenario: Search for information
    Given I am on the bing search engine
    When I enter a search term
    Then relevant results for that search term are displayed

The JUnit test runner

We now need a file called RunCucumberIT.java in src/test/java/com/natritmeyer/examplecucumberjvmspringframework/testrunners/; here’s what its content needs to be:

package com.natritmeyer.examplecucumberjvmspringframework.testrunners;

import cucumber.api.CucumberOptions;
import cucumber.api.junit.Cucumber;
import org.junit.runner.RunWith;

@RunWith(Cucumber.class)
@CucumberOptions(
  features = {"classpath:features/"},
  plugin   = {"pretty"},
  glue     = {"com.natritmeyer.examplecucumberjvmspringframework.steps"})
public class RunCucumberIT {
}

That’s just telling cucumber to look for the feature files in src/test/resources/features/, and to look for the step definitions in src/test/java/com/natritmeyer/examplecucumberjvmspringframework/steps/ (you’re right, we haven’t made the second directory yet).

The step definitions class

The last file we need to complete our basic cucumber project is the step definition class. It should be called BingSearchSteps.java and put here: src/test/java/com/natritmeyer/examplecucumberjvmspringframework/steps/. Here’s what it needs to look like:

package com.natritmeyer.examplecucumberjvmspringframework.steps;

import cucumber.api.java8.En;

public class BingSearchSteps implements En {
  public BingSearchSteps() {
    Given("I am on the bing search engine", () -> {
    });

    When("I enter a search term", () -> {
    });

    Then("relevant results for that search term are displayed", () -> {
    });
  }
}

And with that we have something that will run. At the command line you should be able to run mvn verify and get some meaningful output.

We’re now going to get to the interesting part - adding spring into the mix, followed by webdriver, browser configuration, page objects, and clean step definition code.

Adding Spring to the project

To use dependency injection we need to add a few things to the project. Let’s start with a couple of dependencies.

Two more dependencies for the `pom.xml` file

Add the following dependencies to the pom.xml file:

<dependency>
  <groupId>io.cucumber</groupId>
  <artifactId>cucumber-spring</artifactId>
  <version>4.2.1</version>
  <scope>test</scope>
</dependency>
<dependency>
  <groupId>org.springframework</groupId>
  <artifactId>spring-context</artifactId>
  <version>5.1.4.RELEASE</version>
  <scope>test</scope>
</dependency>

The first of those is a cucumber library that allows the use of spring in cucumber projects; the second is the tiny portion of the massive Spring ecosystem that cucumber needs to allow dependency injection.

Now we need to configure cucumber spring.

Adding `cucumber.xml`

We now need to make a file called cucumber.xml and place it in src/test/resources/. The purpose of this file is to tell cucumber’s spring interaction where to look for things. Here’s the contents of the file:

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xmlns:context="http://www.springframework.org/schema/context"
  xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.0.xsd
  http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context-3.0.xsd">

  <context:annotation-config/>
  <bean name="Config" class="com.natritmeyer.examplecucumberjvmspringframework.config.Config"/>
</beans>

The last two lines are the important ones. The first of them tells spring to use “annotation config”, which basically means we can manage all our spring configuration in code rather than in this xml file. The second line tells spring to look for a class called Config in src/test/java/com/natritmeyer/examplecucumberjvmspringframework/config/. And it’s that class which really controls what’s happening in spring, so let’s move on to that.

The spring configuration class

Let’s create a file called Config.java in src/test/java/com/natritmeyer/examplecucumberjvmspringframework/config/ and give it the following content:

package com.natritmeyer.examplecucumberjvmspringframework.config;

import org.springframework.context.annotation.ComponentScan;
import org.springframework.context.annotation.PropertySource;

@PropertySource("application.properties")
@ComponentScan("com.natritmeyer.examplecucumberjvmspringframework")
public class Config {
}

Just to explain what’s going on here, the first line (@PropertySource) tells spring to look for configuration values in a file called application.properties, and the second line tells spring to look for things it’s interested in in src/test/java/com/natritmeyer/examplecucumberjvmspringframework/. The sorts of things spring is interested in that it’ll find in this project are classes annotated with @Component and @Configuration. We’ll get more into the detail of that when we start creating page objects and browsers a bit later.

And that’s what’s required to get spring added to the project. From now on we’re going to be spending our time writing step definitions that have their dependencies injected in, and writing page objects that do the same.

Filling out the test

We’ll start this phase by adding the last of our dependencies.

The remaining `pom.xml` dependencies

We’re going to use AssertJ for our assertions, and WebDriver to control the browser. Here’s what needs adding to the pom.xml file:

<dependency>
  <groupId>org.assertj</groupId>
  <artifactId>assertj-core</artifactId>
  <version>3.11.1</version>
  <scope>test</scope>
</dependency>
<dependency>
  <groupId>org.seleniumhq.selenium</groupId>
  <artifactId>selenium-java</artifactId>
  <version>3.141.59</version>
  <scope>test</scope>
</dependency>

Creating the AUT model

So that the step definitions will work against Bing’s mobile and desktop search engine implementations it’s important that we “code to an interface, not an implementation”. This means that the steps must only deal with interfaces, not classes. As such we need to model the application under test (i.e. Bing) as a group of interfaces.

If we remember our scenario, all that’s required is the ability to interact with Bing’s home page and its search results page. Specifically, we need to load the Bing home page and use it to search for a search term, and we need to get the title of the first link on the search results page.

We need to create a file to contain the first interface called BingHomePage.java in src/test/java/com/natritmeyer/examplecucumberjvmspringframework/aut/model/. Its contents are:

package com.natritmeyer.examplecucumberjvmspringframework.aut.model;

public interface BingHomePage {
  void load();

  void searchFor(String cucumber);
}

The other interface needs to go in a file named BingSearchResultsPage.java in the same directory as the interface above. The file’s contents should be:

package com.natritmeyer.examplecucumberjvmspringframework.aut.model;

public interface BingSearchResultsPage {
  String getFirstResultTitle();
}

With that we have our complete model for our application under test.

Injecting the model into the step definitions

We’re now at the stage where we can update our step definition so that its dependencies are injected. Here’s how that works…

First, the constructor is updated. Here’s how it looked originally:

public BingSearchSteps() {
  //...
}

We need to change it so it looks like this:

@Autowired
public BingSearchSteps(BingHomePage bingHomePage, BingSearchResultsPage bingSearchResultsPage) {
  //...
}

There are a couple of changes. Firstly, we’ve annotated the constructor with @Autowired. This tells spring that when an instance of this step definitions class is created, automatically pass in (or ‘inject’) pre-created instances of whatever’s mentioned as constructor arguments. Secondly, we’ve added one instance of each of our AUT model interfaces as constructor arguments. In reality, this means that when cucumber is executed, spring will automagically create instances of the BingHomePage and BingSearchResultsPage interfaces (how it does this we’ll get to later), stores them in a pool of managed objects (so they can be used across the spring context), and then pass those instances into the step definition class’ constructor.

We can now update the steps themselves from:

Given("I am on the bing search engine", () -> {
});

When("I enter a search term", () -> {
});

Then("relevant results for that search term are displayed", () -> {
});

…to…

Given("I am on the bing search engine", () -> {
  bingHomePage.load();
});

When("I enter a search term", () -> {
  bingHomePage.searchFor("cucumber");
});

Then("relevant results for that search term are displayed", () -> {
  assertThat(bingSearchResultsPage.getFirstResultTitle()).contains("Cucumber");
});

You’ll notice that the steps here are very simple. They’re just one line each calling only what’s available on the interfaces that represent the application under test. As a result they’re readable - a key feature of good step definition files.

Here’s the full step definition class after being springified:

package com.natritmeyer.examplecucumberjvmspringframework.steps;

import com.natritmeyer.examplecucumberjvmspringframework.aut.model.BingHomePage;
import com.natritmeyer.examplecucumberjvmspringframework.aut.model.BingSearchResultsPage;
import cucumber.api.java8.En;
import org.springframework.beans.factory.annotation.Autowired;

import static org.assertj.core.api.Assertions.assertThat;

public class BingSearchSteps implements En {
  private final BingHomePage bingHomePage;
  private final BingSearchResultsPage bingSearchResultsPage;

  @Autowired
  public BingSearchSteps(BingHomePage bingHomePage, BingSearchResultsPage bingSearchResultsPage) {
    this.bingHomePage = bingHomePage;
    this.bingSearchResultsPage = bingSearchResultsPage;

    Given("I am on the bing search engine", () -> {
      bingHomePage.load();
    });

    When("I enter a search term", () -> {
      bingHomePage.searchFor("cucumber");
    });

    Then("relevant results for that search term are displayed", () -> {
      assertThat(bingSearchResultsPage.getFirstResultTitle()).contains("Cucumber");
    });
  }
}

Creating implementations of the AUT model

Before the steps can be used we need at least one implementation of the AUT model we’ve created. In this example cucumber project we’ll create two separate implementations of the AUT model interfaces - desktop and mobile. Each of our two interfaces represents a page, so each implementation of an interface represents a page object. So, we’re going to end up with 4 new classes: DesktopBingHomePage, DesktopBingSearchResultsPage, MobileBingHomePage, and MobileBingSearchResultsPage.

Desktop page objects

We’ll begin with the desktop implementation of the AUT interfaces, starting with the home page.

We’ll start be creating a file called DesktopBingHomePage.java in src/test/java/com/natritmeyer/examplecucumnberjvmspringframework/. Here are the file’s contents:

package com.natritmeyer.examplecucumberjvmspringframework.aut.implementations.desktop;

import com.natritmeyer.examplecucumberjvmspringframework.aut.model.BingHomePage;
import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Profile;
import org.springframework.stereotype.Component;

@Component
@Profile("desktop")
public class DesktopBingHomePage implements BingHomePage {
  @Value("${aut.urls.home}")
  private String homePageUrl;

  private final WebDriver driver;

  @Autowired
  public DesktopBingHomePage(WebDriver driver) {
    this.driver = driver;
  }

  @Override
  public void load() {
    driver.get(homePageUrl);
  }

  @Override
  public void searchFor(String searchTerm) {
    driver.findElement(By.id("sb_form_q")).sendKeys(searchTerm);
    driver.findElement(By.id("sb_form_go")).click();
  }
}

There are a few things here that need explaining.

@Component tells spring to create an instance of this class and add it to the pool of objects spring manages. Wherever DesktopBingHomePage is mentioned as a constructor argument (and the constructor is annotated with @Autowired) this instance will be injected in.

The @Profile("desktop") annotation acts as a kind of filter - this is how spring chooses between multiple implementations of an interface. If there was another class that implemented BingHomePage (and there will be in a minute when we create a mobile version), spring will choose between them based on the spring.profiles.active system property. We’ll see how to go about setting that a bit later.

This page object that represents Bing’s home page for desktops implements BingHomePage. I.e. it’s an implementation of the BingHomePage interface. This allows DesktopBingHomePage to be injected into any constructor that asks for BingHomePage.

The page object has a constructor that’s annotated with @Autowired. So, when spring automagically creates an instance of this class (because it’s annotated with the @Component annotation - see above), it’ll inject it with a preexisting WebDriver instance. We’ll get to WebDriver management in a bit.

There are two methods in this class, both annotated @Override: public void load() and public void searchFor(String searchTerm). These are the methods that fulfil the requirements of the BingHomePage interface - they’re what actually get used by the step definition when the test is run.

Finally, there’s the homePageUrl variable with its @Value("${aut.urls.home}") annotation. This is how to get config data into your object. Here, the homePageUrl value is being populated with the contents of whatever aut.urls.home is set to in the src/test/resources/application.properties file. Let’s take a quick diversion into configuration…

A diversion through the `application.properties` configuration file

The @Value(...) annotation is used to get data from configuration files.

Back when we were creating the Config class we annotated it with @PropertySource("application.properties"). This instructed spring to look for any config in src/test/resources/application.properties during the course of test execution. So, create that file in that location, and set its contents to the following:

aut.urls.home=https://www.bing.com/

Now, when the tests get run, the DesktopBingHomePage’s homePageUrl string will have its value set to https://www.bing.com/. We’ll make more use of the config file when we get to configuring our desktop and mobile browsers.

The rest of the page objects

The other 3 page objects are just variations on the same theme, so here they are:

Create a file called DesktopBingSearchResultsPage.java in src/test/java/com/natritmeyer/examplecucumberjvmspringframework/aut/implementations/desktop/, and set its contents to:

package com.natritmeyer.examplecucumberjvmspringframework.aut.implementations.desktop;

import com.natritmeyer.examplecucumberjvmspringframework.aut.model.BingSearchResultsPage;
import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.annotation.Profile;
import org.springframework.stereotype.Component;

@Component
@Profile("desktop")
public class DesktopBingSearchResultsPage implements BingSearchResultsPage {
  private final WebDriver driver;

  @Autowired
  public DesktopBingSearchResultsPage(WebDriver driver) {
    this.driver = driver;
  }

  @Override
  public String getFirstResultTitle() {
    return driver.findElement(By.cssSelector("ol#b_results > li > h2 > a")).getText();
  }
}

Create a file called MobileBingHomePage.java in src/test/java/com/natritmeyer/examplecucumberjvmspringframework/aut/implementations/mobile/, and set its contents to:

package com.natritmeyer.examplecucumberjvmspringframework.aut.implementations.mobile;

import com.natritmeyer.examplecucumberjvmspringframework.aut.model.BingHomePage;
import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Profile;
import org.springframework.stereotype.Component;

@Component
@Profile("mobile")
public class MobileBingHomePage implements BingHomePage {
  @Value("${aut.urls.home}")
  private String homePageUrl;

  private final WebDriver driver;

  @Autowired
  public MobileBingHomePage(WebDriver driver) {
    this.driver = driver;
  }

  @Override
  public void load() {
    driver.get(homePageUrl);
  }

  @Override
  public void searchFor(String searchTerm) {
    driver.findElement(By.id("hc_popnow")); //wait for the trending stories - js has done loading by this point
    driver.findElement(By.id("sb_form_q")).sendKeys(searchTerm);
    driver.findElement(By.id("sbBtn")).click();
  }
}

Create a file called MobileBingSearchResultsPage.java in src/test/java/com/natritmeyer/examplecucumberjvmspringframework/aut/implementations/mobile/, and set its contents to:

package com.natritmeyer.examplecucumberjvmspringframework.aut.implementations.mobile;

import com.natritmeyer.examplecucumberjvmspringframework.aut.model.BingSearchResultsPage;
import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.annotation.Profile;
import org.springframework.stereotype.Component;

@Component
@Profile("mobile")
public class MobileBingSearchResultsPage implements BingSearchResultsPage {
  private final WebDriver driver;

  @Autowired
  public MobileBingSearchResultsPage(WebDriver driver) {
    this.driver = driver;
  }

  @Override
  public String getFirstResultTitle() {
    driver.findElement(By.cssSelector("ol#b_results")); //waits for the results to be displayed
    return driver.findElement(By.cssSelector("ol#b_results > li.b_algo > div > a > h2")).getText();
  }
}

Note that the last two are the mobile implementations of the BingHomePage and BingSearchResultsPage interfaces are are annotated with @Profile("mobile") instead of @Profile("desktop").

Browser management

You’ll have noticed that the page objects all have an @Autowired constructor that takes an instance of WebDriver. We’ll now code up what’s behind that.

When we run the tests against Bing’s mobile search page we need a webdriver session set up with a mobile user agent, and with window dimensions of a mobile browser. Conversely, when we run the tests against Bing’s desktop site we’ll want webdriver set up with desktop dimensions. To achieve this we’ll create two @Bean methods - these are methods that return instances of specific classes that can be called by spring when instances of specific classes are required. As before we’ll start with the desktop and move on to the mobile implementation.

Desktop browser

We’ll begin by creating a file called DesktopBrowser.java in src/test/java/com/natritmeyer/examplecucumberjvmspringframework/browsers/ and we’ll set its content to this:

package com.natritmeyer.examplecucumberjvmspringframework.browsers;

import org.openqa.selenium.Dimension;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.Profile;

import java.util.concurrent.TimeUnit;

@Configuration
public class DesktopBrowser {
  @Value("${browser.desktop.width}")
  private int desktopWidth;

  @Value("${browser.desktop.height}")
  private int desktopHeight;

  @Bean
  @Profile("desktop")
  public WebDriver getDriver() {
    WebDriver driver = new ChromeDriver();
    driver.manage().window().setSize(new Dimension(desktopWidth, desktopHeight));
    driver.manage().timeouts().implicitlyWait(10, TimeUnit.SECONDS);
    return driver;
  }
}

There are a few things going here - let’s go through them.

The class is annotated with @Configuration. This tells spring to look in this class for a method annotated with @Bean.

There are two int variables, desktopWidth and desktopHeight. The value of these ints will come from application.properties; we may as well add the relevant values now. Here’s what needs to go into the properties file:

browser.desktop.width=1024
browser.desktop.height=768

Back in the DesktopBrowser class, there’s a method called public WebDriver getDriver(). Because this is annotated with @Bean, spring will use this method to get its WebDriver instance, and then use that whenever any constructor asks for a WebDriver instance - this is how using dependency injection gets rid of the problem of passing driver around in weird ways leading to project collapse.

The final thing to note about the class is that the getDriver() method is also annotated with @Profile("desktop"), just like the first two page object classes we created. This is how spring avoids confusing this @Bean method that returns a WebDriver instance, and the next one we’re going to create.

Mobile browser

When we’re running the tests against Bing’s mobile site we want to make sure we’re using a webdriver instance that’s configured to look and act like a mobile browser. For the purposes of this example project it’s enough to just set the window’s width and height, along with setting the user agent to something mobile.

So, let’s create our final file, MobileBrowser.java in src/test/java/com/natritmeyer/examplecucumberjvmspringframework/browsers/ and give it the following contents:

package com.natritmeyer.examplecucumberjvmspringframework.browsers;

import org.openqa.selenium.Dimension;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.chrome.ChromeOptions;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.Profile;

import java.util.concurrent.TimeUnit;

@Configuration
public class MobileBrowser {
  @Value("${browser.mobile.width}")
  private int mobileWidth;

  @Value("${browser.mobile.height}")
  private int mobileHeight;

  @Value("${browser.mobile.useragent}")
  private String userAgent;

  @Bean
  @Profile("mobile")
  public WebDriver getDriver() {
    String userAgentString = String.format("--user-agent=%s", userAgent);

    ChromeOptions chromeOptions = new ChromeOptions();
    chromeOptions.addArguments(userAgentString);

    WebDriver driver = new ChromeDriver(chromeOptions);
    driver.manage().window().setSize(new Dimension(mobileWidth, mobileHeight));
    driver.manage().timeouts().implicitlyWait(10, TimeUnit.SECONDS);
    return driver;
  }
}

Just like the DesktopBrowser class, this class needs some config adding to application.properties. Here’s what you need to add:

browser.mobile.width=400
browser.mobile.height=800
browser.mobile.useragent=Mozilla/5.0 (iPhone; CPU OS 11_0 like Mac OS X) AppleWebKit/604.1.25 (KHTML, like Gecko) Version/11.0 Mobile/15A372 Safari/604.1

The contents of MobileBrowser’s getDriver() method is different to DesktopBrowser’s, but it’s just because it has to handle the user agent string as well as the window dimensions - nothing more interesting than that.

And… that’s it. That’s everything. We’re at the stage now where everything should work…

Executing the feature file

With our feature file, step definitions, page objects, and browser management in place, we’re ready to execute the feature file.

Let’s first make sure we’re in the right place:

cd cucumber-jvm-dependency-injection-with-spring

We’re now ready to run. Let’s start with the desktop implementation. To execute the feature file against Bing’s desktop site we need to set the desktop profile. The command that executes the feature files and sets that profile is as follows:

mvn clean verify -Dspring.profiles.active=desktop

If you run that you should see a browser flash up on the screen, navigate to bing.com, search for “cucumber”, and then display the search results before closing.

As well as that you should see something like this in your console:

...
[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO] Running com.natritmeyer.examplecucumberjvmspringframework.testrunners.RunCucumberIT
Starting ChromeDriver 2.45.615355 (d5698f682d8b2742017df6c81e0bd8e6a3063189) on port 19432
Only local connections are allowed.
Jan 15, 2019 11:59:45 AM org.openqa.selenium.remote.ProtocolHandshake createSession
INFO: Detected dialect: OSS
Feature: Search for content

  Scenario: Search for information                           # features/search.feature:3
    Given I am on the bing search engine                     # BingSearchSteps.java:19
    When I enter a search term                               # BingSearchSteps.java:23
    Then relevant results for that search term are displayed # BingSearchSteps.java:27

1 Scenarios (1 passed)
3 Steps (3 passed)
0m3.515s

[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.653 s - in com.natritmeyer.examplecucumberjvmspringframework.testrunners.RunCucumberIT
...

If that’s working then go ahead and execute the feature file again, but this time with the mobile profile set:

mvn clean verify -Dspring.profiles.active=mobile

If that worked you’ll have seen a much smaller browser window open, navigate to the Bing home page, searched using the mobile interface, and displayed the mobile search results page before closing. The console output should be practically identical.

Conclusion

That’s it. We created an example cucumber project with a single feature file backed by a single step definition class. Because we set up that step definition class to use interfaces rather than classes we were able to inject in whatever implementation we liked - in this case desktop or mobile page object implementations of those interfaces. By injecting appropriately configured WebDriver instances into those implementation-specific page objects we were able with a single command line switch (spring.profiles.active) to determine at runtime which of the implementations we wanted to test.

Most importantly, by using dependency injection we got rid of the need to create and maintain WebDriver and page object instances, removing the need to carefully pass them around.

This is a massive maintenance win, as many readers will appreciate. It also increased the readability of the code (especially the step definition class), as well as divided up the project into loosely coupled code.

Note, this example project is not perfect - far from it. There are quire a few optimisations I’d ordinarily make, but for the sake of clarity I’ve left them out. However, it should be good enough to show you how cucumber and spring can work together to achieve dependency injection in most scenarios.

Finally, this example project in its complete form can be found here https://github.com/natritmeyer/cucumber-jvm-dependency-injection-with-spring.

Legacy firefox plugin demised

2018-08-21T00:00:00+01:00

It’s rather ironic that the day after restarting this site my inbox is graced with an email from Mozilla informing me that a long-forgotten firefox plugin I wrote more than 5 years ago will no longer be compatible with future versions of the browser:

You are receiving this email because you are listed as a developer of a legacy add-on on addons.mozilla.org (AMO). Mozilla will stop supporting Firefox Extended Support Release (ESR) 52, the final release that is compatible with legacy add-ons, on September 5, 2018. As no supported versions of Firefox will be compatible with legacy add-ons after this date, we will start the process of disabling legacy add-on versions on AMO. On September 6, 2018, submissions for new legacy add-on versions will be disabled. All legacy add-on versions will be disabled by early October 2018. Once this happens, users will no longer be able to find your legacy versions on AMO.

So long, test-automation-password-manager. You were once useful to many, but today you are consigned to the scrap heap of history.

If for some unfathomable reason you’re still using this plugin, sorry.

How to make maven display stack traces of failed tests

2018-08-21T00:00:00+01:00

TL;DR

If you want to see stack traces when tests executed by mvn test or mvn verify fail instead of the short summary you’re seeing, make sure you’ve got trimStackTrace set to false in your pom.xml.

E.g.

Enable stack traces for `mvn test`

<build>
  <plugins>
    <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-surefire-plugin</artifactId>
        <version>${your.maven.surefire.version.here}</version>
        <configuration>
            <trimStackTrace>false</trimStackTrace>
        </configuration>
    </plugin>
  </plugins>
</build>

Enable stack traces for `mvn verify`

<build>
  <plugins>
    <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-failsafe-plugin</artifactId>
        <version>${your.maven.failsafe.version.here}</version>
        <configuration>
            <trimStackTrace>false</trimStackTrace>
        </configuration>
        <executions>
          <!-- usual execution stuff here -->
        </executions>
    </plugin>
  </plugins>
</build>

And now some archaeology…

The history behind `trimStackTrace`

Back in the day mvn test and mvn verify would print the stack trace of any test that failed. Though the information in the stack trace was useful for debugging, stack traces took up rather a lot of space in the console output. Particularly if there were a number of failing tests it became easy to get lost in the noise.

In a wonderfully terse description of the problem, Issue 936 in surefire’s Jira project explains,

It is too hard to find out what failed at a glance. Fix this.

A bit of coding later and surefire version 2.13 was released back in December 2012. Amongst the ‘notable changes’ listed in the announcement was,

Stack trace trimming now works for JUnit4.x. The itch you didn’t know you had until it was scratched.

The previously-noisy stack trace output was replaced with a calmer 1-line error summary. And just like that everyone waved goodbye to their stack traces.

Speaking personally, it’s nice to have the option. I’d have preferred the old behaviour remained the default, but it doesn’t matter much either way.

Resurrection

2018-08-20T00:00:00+01:00

That is not dead which can eternal lie. And with strange aeons even death may die.

Lovecraft, The Call of Cthulhu (1928)

It’s time to bring this blog back to life.

NatRitmeyer.com

Reporting aggregated unit and integration test coverage with jacoco

Unit test coverage report

Include the Jacoco plugin

Configuring jacoco unit test coverage data

Generating the unit test coverage report

Configuring surefire

Integration test coverage report

Configuring jacoco’s integration coverage report

Configuring failsafe

Merged test coverage report

Merging unit and integration test coverage data

Generating the merged unit and integration test coverage report

Failing the build

TL;DR

Updating a BrowserStack session’s name and result

Intro

Solving the problem

Some dependencies

Getting the BrowserStack Session ID

Preparing the payload

Making the call to BrowserStack

Preparing the WebClient

Updating BrowserStack’s session details

Cucumber-JVM Dependency Injection with Spring

Introduction

Framework overview

Ingredients

Step by step

Setting up a basic cucumber project

The pom.xml file

The feature file

The JUnit test runner

The step definitions class

Adding Spring to the project

Two more dependencies for the pom.xml file

Adding cucumber.xml

The spring configuration class

Filling out the test

The remaining pom.xml dependencies

Creating the AUT model

Injecting the model into the step definitions

Creating implementations of the AUT model

Desktop page objects

A diversion through the application.properties configuration file

The rest of the page objects

Browser management

Desktop browser

Mobile browser

Executing the feature file

Conclusion

Legacy firefox plugin demised

How to make maven display stack traces of failed tests

TL;DR

Enable stack traces for mvn test

Enable stack traces for mvn verify

The history behind trimStackTrace

Resurrection

The `pom.xml` file

Two more dependencies for the `pom.xml` file

Adding `cucumber.xml`

The remaining `pom.xml` dependencies

A diversion through the `application.properties` configuration file

Enable stack traces for `mvn test`

Enable stack traces for `mvn verify`

The history behind `trimStackTrace`