TestNG offers inbuilt reports of the test execution. The built-in reports, despite being carrying information about the executed test cases, more information and readability are required to share this with stakeholders. Additionally, an Extent report is a customized HTML-based report integrated with Selenium using the TestNG framework. Moreover, the Extent report offers numerous advantages concerning the built-in default report of TestNG, such as visually appealing pie charts, step-wise details, screenshots, etc. It all comes in a neat and presentable UI that is easy to share with other project stakeholders.

Subsequently, in this article, we will learn how to generate an Extent report with Cucumber using the ExtentReport Plugin for Cucumber. We will use the  "grasshopper" plugin to produce the Extent report for cucumber.

• What is the Extent Report?
  • Can we use the Extent Report with Cucumber?
  • Can we use the Extent Report with TestNG?
• How to integrate the Extent Reports in a Cucumber-TestNG project?
  • How to generate the Extent Report in a Cucumber TestNG project?
• Cucumber Extent Reporter Features
  • How to customize the report folder name?
  • How to add screenshots to scenario steps?
  • How to add screenshots only for failed scenario steps?

What is the Extent Report?

Extent Report is an open-source reporting library used to create visually attractive reports for Selenium tests using JUnit and TestNG. Extent reports produce HTML-based documents that offer several advantages like pie charts, graphs, screenshots addition, and test summary. Additionally, we can easily share the generated report with stakeholders via mails. It presents several different functionalities over the in-built default report. Conclusively, it's one of the most used reporting libraries for Selenium tests due to all these features.

Can we use the Extent Report with Cucumber?

Extent report doesn't directly support the Cucumber framework. But they can be integrated quite quickly with it by using the cucumber grasshopper adapter plugin. Additionally, this will allow the Extent report library to understand and record scenarios or features from the framework. Once scenarios and features are recognized, the extent report adapter plugin will record all these in the report. The ease with which this plugin works with the Cucumber framework makes it very easy to implement and use. Overall grasshopper adapter plugin reduces the pain of implementing Extent reporting with Cucumber.

Can we use the Extent Report with TestNG?

We can use the Extent report with the Selenium TestNG framework to generate reports. But while using Cucumber-TestNG extent report cannot be directly used. To use the extent report, we will use the extent report adapter plugin and the extent report libraries. As we already discussed in the previous topic, we can use the grasshopper plugin to generate an Extent report for the Cucumber TestNG framework.

How to integrate the Extent Reports in a Cucumber-TestNG project?

Reading through all the topics mentioned above, we now know that it's possible to add an Extent Report to our cucumber TestNG framework. Subsequently, let's look at the steps to integrate the Extent report with the Cucumber TestNG project.

Before we could proceed with the Extent report plugin's configuration with the cucumber project, we need to add few dependencies. Using maven allows us to mention the dependencies in the pom.xml file instead of downloading separate libraries and adding them to the project. Consequently, let's add the following dependencies to our project.

Add Cucumber and TestNG dependencies to the project:

Before we can start working with the extent report, we need to add Cucumber dependencies to the project. As we will create a Cucumber TestNG framework, we need to ensure that all the cucumber dependencies add to the project. We need to make sure the following dependencies are present in the maven pom.xml file.

• cucumber-java
• cucumber-picocontainer
• gherkin
• cucumber-jvm
• cucumber-testng

The dependencies defined above are the necessary dependencies that are required to create a Cucumber TestNG project. Let's add these to our pom.xml file.

<!-- https://mvnrepository.com/artifact/io.cucumber/cucumber-java -->
		<dependency>
			<groupId>io.cucumber</groupId>
			<artifactId>cucumber-java</artifactId>
			<version>6.8.1</version>
		</dependency>
		<!-- https://mvnrepository.com/artifact/io.cucumber/cucumber-picocontainer -->
		<dependency>
			<groupId>io.cucumber</groupId>
			<artifactId>cucumber-picocontainer</artifactId>
			<version>6.8.1</version>
			<scope>test</scope>
		</dependency>
		<!-- https://mvnrepository.com/artifact/io.cucumber/gherkin -->
		<dependency>
			<groupId>io.cucumber</groupId>
			<artifactId>gherkin</artifactId>
			<version>15.0.2</version>
		</dependency>
		<!-- https://mvnrepository.com/artifact/io.cucumber/cucumber-jvm -->
		<dependency>
			<groupId>io.cucumber</groupId>
			<artifactId>cucumber-jvm</artifactId>
			<version>6.8.1</version>
			<type>pom</type>
		</dependency>
		<!-- https://mvnrepository.com/artifact/io.cucumber/cucumber-testng -->
		<dependency>
			<groupId>io.cucumber</groupId>
			<artifactId>cucumber-testng</artifactId>
			<version>6.8.1</version>
		</dependency>

Add Extent report and grasshopper extent report adapter plugin dependencies to the project:

To generate an Extent Report for our project, we will need two libraries; the extent report library and the extent report adapter plugin from grasshopper. Both these dependencies need to add to the project. Additionally, we can find both these dependencies at the maven repository.

• We can find the Extent report dependency at Maven Repository- Extent Reports.
• Similarly, Grasshopper extent report adapter plugin at Maven Repository- Extent reports cucumber6 adapter.

Note: To work with extent report 5, you will need to add your pom's latest available version. For earlier versions like Extent Report 4, please use version 1.2.

Let's add these dependencies to our pom.xml file.

		<dependency>
			<groupId>tech.grasshopper</groupId>
			<artifactId>extentreports-cucumber6-adapter</artifactId>
			<version>2.0.0</version>
			<scope>test</scope>
		</dependency>
		<!-- https://mvnrepository.com/artifact/com.aventstack/extentreports -->
		<dependency>
			<groupId>com.aventstack</groupId>
			<artifactId>extentreports</artifactId>
			<version>5.0.4</version>
		</dependency>

By adding these dependencies, we have added all the required libraries to our project. The next step will be to configure and generate the report.

How to generate the Extent Report in a Cucumber TestNG Project?

After adding all the required libraries, we need to make few configuration changes to start using the Extent report. Let's follow few simple steps to configure our project. Consequently, we will begin by adding the Extent report properties file to the project.

Step 1: Add Extent properties file to the project - The Extent properties file defines the Extent Report's properties. It provides an easy way for users to make any configuration changes to the extent report or to define few properties. The extent.properties file has to create at the src/test/resources folder of the project. It allows the grasshopper extent report adapter to recognize and read all the properties.

To create the file, navigate to the src/test/resources folder and right-click on the resources folder. After that, from the menu, navigate to New -> Other as shown in the image below marked by steps 1 and 2.

The wizard search for File and select File from the search result as marked in the arrow's below image. After selection, click on the Next button.

In the next wizard, make sure that your file is located in the src/test/resources folder, as shown in the image below. Give the file's name as "extent.properties" as marked by step 1 in the image below. Finally, click on the Finish button.

Let's add the following information to the properties file:

extent.reporter.spark.start=true
extent.reporter.spark.out=Reports/Spark.html

As you can see in the image below, we have created an extent.properties file in the src/test/resources folder. We can use this file to add all required properties for the Extent report. Here we are putting generate and store reports at a given location.

Here we are putting sets to generate and store reports at a given location. You can provide any location where you want your reports.

Step 2: Configure Test Runner class to Implement Cucumber Extent Reporter - The next step is to add the extent report adapter to the CucumberOption annotation present in the runner class. Moreover, this is the most vital step in the configuration and helps initiate an Extent report for cucumber. To add the Extent report, we will need to add the following text as a plugin to the CucumberOption.

@CucumberOptions(plugin = {"com.aventstack.extentreports.cucumber.adapter.ExtentCucumberAdapter:"})

Let's navigate to our Runner class and add the above text to the plugin. Be sure to keep the colon ": " at the end, as mentioned. Subsequently, after adding to our project, this is how your runner class should look:

Make sure that you have made the correct import for the CucumberOption. If you accidentally imported JUnit CucumberOption instead of TestNG, your tests will not get recognition.

All our configuration finishes by adding the plugin. Extent report will automatically detect features, scenarios, and steps. All these will automatically add to the report.

Step 3: Run Cucumber Test with TestNG to generate Extent Report - With all the setup and configuration completed. Let's execute a few simple end-to-end cucumber tests to validate the working of our extent report plugin.

Our sample feature file with two scenarios looks like below.

Feature: Page Element validation on DemoQA

Background: User is on demoQA home page
	Given user is on demoQA Home Page

Scenario: Validate text boxes on demoQA
	Given user navigates to textbox page
	When user enters full name and email
	And user enters current address and permanent address
	When user clicks on submit button
	Then validate correct name is displayed
	
Scenario: Validate radio button on demoQA
	Given user navigates to radio button page
	When user click on Yes radio
	Then validate yes radio is selected

Step definition file:

package stepDefinition;


import java.util.List;
import java.util.concurrent.TimeUnit;

import org.openqa.selenium.By;
import org.openqa.selenium.Keys;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.interactions.Actions;

import io.cucumber.datatable.DataTable;
import io.cucumber.java.After;
import io.cucumber.java.en.And;
import io.cucumber.java.en.Given;
import io.cucumber.java.en.Then;
import io.cucumber.java.en.When;
import junit.framework.Assert;


public class StepDefinition {
	 WebDriver driver;
	
	 @Given("^user is on demoQA Home Page$")
	 public void user_is_on_demoQA_Home_Page(){
	 System.setProperty("webdriver.chrome.driver","C:\\Selenium_Drivers\\chromedriver\\chromedriver.exe");
	 driver = new ChromeDriver();
	 driver.manage().window().maximize();
	 driver.manage().timeouts().implicitlyWait(10, TimeUnit.SECONDS);
	 driver.get("https://demoqa.com");
	 }
	
	@Given("^user navigates to textbox page$")
	public void user_navigates_to_textbox_page() {
		driver.navigate().to("https://www.demoqa.com/text-box");
		
	}
	
	@When("^user enters full name and email$")
	public void user_enters_full_name_and_email() {
		driver.findElement(By.id("userName")).sendKeys("Tools QA");
		driver.findElement(By.id("userEmail")).sendKeys("[email protected]");
		
	}
	
	@And("^user enters current address and permanent address$")
	public void user_enters_current_address_and_permanent_address() {
		driver.findElement(By.id("currentAddress")).sendKeys("Current Address");
		driver.findElement(By.id("permanentAddress")).sendKeys("permanent address");
		
	}
	
	@When("^user clicks on submit button$")
	public void user_clicks_on_submit_button() {
		WebElement btn = driver.findElement(By.xpath("//div/button"));
		btn.sendKeys(Keys.TAB);
		btn.click();
				
	}
	
	@Then("^validate correct name is displayed$")
	public void validate_correct_name_is_displayed() {
		WebElement name = driver.findElement(By.xpath("//p[@id='name']"));
		if(name.getText().contains("Tools QA")) {
			
		}else {
			Assert.assertTrue(false);
		}
	}
	
	
	@Given("^user navigates to radio button page$")
	public void user_navigates_to_radio_button_page() {
		driver.navigate().to("https://www.demoqa.com/radio-button");
	}
	
	@When("^user click on Yes radio$")
	public void user_click_on_Yes_radio() {
		driver.findElement(By.xpath("//label[text()='Yes']")).click();
	}
	
	@Then("^validate yes radio is selected$")
	public void validate_yes_radio_is_selected() {
		Assert.assertEquals("Yes", driver.findElement(By.xpath("//p/span")).getText());
	}
	
	
	@After
	public void quite() {
		driver.quit();
	}
}

Our runner file with the plugin for extent report added will be something like this:

package Runner;

import io.cucumber.testng.AbstractTestNGCucumberTests;
import io.cucumber.testng.CucumberOptions;

@CucumberOptions(
		features = "C:\\Users\\Admin\\git\\CucumberFramework\\Cucumber\\src\\main\\java\\features"
		,glue = {"stepDefinition"}
		,plugin = {"com.aventstack.extentreports.cucumber.adapter.ExtentCucumberAdapter:"}
		,monochrome = true
		,publish = true
		
		)
public class Runner extends AbstractTestNGCucumberTests {

}

With all our files in place, we can now execute feature files to generate reports. We have one feature file with two scenarios added to it. We will be using our DemoQA website for this example.

Run as TestNG test - To execute our feature file, we need first to navigate our Runner class. Right-click inside the Runner class and then select "Run As". From the menu, select "TestNG test" ,as shown in the image below. It will start our execution.

Note: This "Run As" >> "TestNG test" option will only appear if you have installed TestNG as a plugin in Eclipse. To install the same please see How to Install TestNG in Eclipse or IntelliJ?

Once the execution completes, the grasshopper extent report plugin will generate the extent report complete with all features, scenarios, and steps. The report will generate at the location defined in the properties file.

The plugin will generate a file named Spark.html as defined in the properties file. We can now navigate and validate if we can generate the report. In the image below, we can see that a report develops by the name of Spark.html.

Now, we will open the file to view the details that we captured. As shown from the below image, point 1 denotes the feature, and 2 represents the Scenarios inside the feature.

We can click on the scenarios to expand it to view the step details.

The Extent report also has an overview section with the details of the execution. It will contain details like pass/fail pie chart, start time, end time, date, etc., as shown in the image below.

It concludes the generation of extent report with Cucumber TestNG.

Cucumber Extent Reporter Features

Apart from the nice readable report, the Extent report also offers features to customize the report as per requirement. Let's have a look at some of the customizations on offer.

How to customize the report folder name?

Previously we have learned ways to generate an Extent report with Cucumber TestNG. But the previous approach generates reports by overriding the existing reports with new reports. Usually, we need to maintain a backup of all the test execution results. To do this, either we need to save each report with a unique name or create a folder with a unique name. Let's learn how to achieve this.

It is quite easy with an Extent report adapter. We need to add two new settings to the extent.properties file:

• basefolder.name
• basefolder.datetimepattern

The value that we provide to these settings will merge to create the folder name. The value for the basefolder.datetimepattern should be a valid date-time format. (refer to oracle formatter pattern for a different format and other details).

To add this, go to the extent.properties file and add both the settings as shown below.

basefolder.name=Reports/SparkReport
basefolder.datetimepattern=d-MMM-YY HH-mm-ss

In the above example, we have provided the name "Reports/SparkReport". It means that a folder starts with the name "SparkReport" under the "Reports" folder. The date-time pattern we have provided in another format is basis a valid pattern. It will concatenate with the folder name to generate a unique folder for each execution.

Now, let's run the report and see if all the folders and reports create. As can be seen from the below image folder has been created with date and time.

As seen in the image below, the "Reports" folder gets created inside the new folder.

If we look inside the folder, we can see that the report generates.

Now, whenever we execute the features, a unique folder will generate for each run. It will help in keeping track of all the execution results.

How to add screenshots to scenario steps?

We require Screenshots to help QA in analyzing steps and understand the execution result. A screenshot offers much-needed analysis capability to the users and better understand in the event of a failure. With an Extent report adapter, it's relatively easy to add a report to the project. Subsequently, let's see how we can use this.

1. Take a screenshot and add it to scenario steps - Firstly, to use the screenshot for the test step, we will need to add it to the @AfterStep annotation in the Hooks class. Additionally, for capturing screenshots for each step, we are using AfterStep annotation. Based on your requirement or need, you can take a screenshot with any other annotation.

Let's add the following method to Hooks class or in any class you have defined your Hooks scenarios.

@AfterStep
	public void addScreenshot(Scenario scenario) throws IOException {
		  File screenshot = ((TakesScreenshot) driver).getScreenshotAs(OutputType.FILE);
		  byte[] fileContent = FileUtils.readFileToByteArray(screenshot);
		  scenario.attach(fileContent, "image/png", "screenshot");
		
	}

In the above example, inside the addScreenshot method, we are accepting a Scenario type object. The Scenario is present inside the io.cucumber. Inside the method, we have used the common screenshot feature from Selenium. Then we want to read the file as byte[] type, as a scenario.attach method accepts byte[] type object as a parameter. Additionally, Scenario.attach attaches the screenshot with each step of the scenario.

2. Add screenshot settings to the extent.properties file - To complete the screenshots' configuration, we will need to add the settings to the extent.properties file. Consequently, we will need to add two settings

• screenshot.dir
• screenshot.rel.path

The first setting will accept the name of the folder where you want to save your screenshots. Similarly, the second setting will accept the relative path of the image from the report folder. So, our extent.properties file will be:

#extent.properties
extent.reporter.spark.start=true
extent.reporter.spark.out=Reports/Spark.html

#Adding folder name and non repeating pattern
basefolder.name=Reports/SparkReport
basefolder.datetimepattern=d-MMM-YY HH-mm-ss

#Screenshot
screenshot.dir=Screenshots/
screenshot.rel.path=../Screenshots/

Let's execute the tests to see if screenshots are present in the scenario steps. Consequently, the output folder will contain all the details.

As shown in the image below, a screenshot folder generates along with a reports folder.

We can navigate the screenshot folder to view all the screenshots taken with each step. Additionally, screenshots will generate and will get a name automatically.

Now, let's navigate to the Reports folder to view the report. As we can see in the image below, the report generates.

Let's open the report and view the report. As you can see, besides the scenario,  an attachment sign is available, which means something attaches to the scenario.

We can now expand the scenario to view the step details, as shown in the image below. Each step now contains a screenshot of the step.

To view the screenshots in the full-screen mode, we can click on the screenshot. Consequently, it will open on the full screen.

How to add screenshots only for failed scenario steps?

In the previous section, we learned how we can add screenshots to each scenario step of the extent report. But sometimes we may need to capture screenshots just for the failed steps.

In this section, we will discuss how we can attach screenshots only for failed steps. Let's have a look at the @AfterStep annotation we added in the previous step.

	@AfterStep
	public void addScreenshot(Scenario scenario){

	      final byte[] screenshot = ((TakesScreenshot) driver).getScreenshotAs(OutputType.BYTES);
	      scenario.attach(screenshot, "image/png", "image"); 
		
	}

In the above code snippet, we are capturing the screenshot after each scenario step and then attaching it to the scenario. The same is then attached to the extent report. Now, let's enclose the above code inside a conditional statement such that it only executes if the scenario fails.

This will give us the following code snippet:

	@AfterStep
	public void addScreenshot(Scenario scenario){

		//validate if scenario has failed
		if(scenario.isFailed()) {
			final byte[] screenshot = ((TakesScreenshot) driver).getScreenshotAs(OutputType.BYTES);
			scenario.attach(screenshot, "image/png", "image"); 
		}
		
	}

In the above statement, we used an "if statement" to check if the scenario is in failed state or not. If the scenario is failed then only the screenshot code block will be executed.

To check this code let's simulate a fail condition. We can do it using an assert true statement and passing a false value to it anywhere in our step definition. For example, we added the following code to our last step in the step definition file i.e. the step that we want to fail.

Assert.assertTrue(false);

Let's run the scenarios from the TestRunner file using TestNG and see the results.

In the generated report folder:

As we can see in the above image, the report and the screenshot folders have been generated. If we navigate to the "Screenshot" folder, we will be able to view all the failed scenario screenshots.

As we have only one failed step, so only one screenshot has been captured as seen in the above image. Now, let's navigate to the reports folder and open the generated report.

As we can see from the above image failed step has been logged into the report along with the screenshot. Similarly, any more failure that may occur during test execution will be logged in the report along with the screenshot of the failure.

Key Takeaways

• Extent reports generate easy and visually appealing reports. Moreover, we can easily share the HTML based report with other stakeholders.
• In addition to that, the extent report doesn't directly support Cucumber TestNG. However, we can use the adapter plugin to generate the required reports.
• Moreover, plugin integration also allows users to perform different customization on reports as per their requirements.