As discussed in the Robot class Introduction tutorial, it provides methods that can be used to simulate keyboard and mouse actions. For example, simulation on OS popups/alerts or OS applications like Calculator, Notepad, etc. In this tutorial, we are going to cover some of the methods to simulate the mouse events provided by Java Robot class (Robot Class Mouse Events).

Robot Class Mouse Events

Let us understand some of the essential techniques of the Robot class, which are used to simulate the mouse events.

• mousePress(int buttons): This method presses one or more mouse buttons.

Here, parameter buttons are the Button mask. Which, in turn, is a combination of one or more mouse button masks.

Following are standard button masks available for mousePress method :

• InputEvent.BUTTON1_DOWN_MASK :  For mouse left -click

• InputEvent.BUTTON2_DOWN_MASK  : For mouse middle button click.

• InputEvent.BUTTON3_DOWN_MASK : For mouse right-click

• InputEvent.BUTTON1_MASK

• InputEvent.BUTTON2_MASK

• InputEvent.BUTTON3_MASK

• mouseRelease(int buttons): This method releases one or more mouse buttons. For Example, robot.mouseRelease(InputEvent. BUTTON1_DOWN_MASK) will release the left click press of the mouse.

• mouseMove(int x, int y): This method moves the mouse pointer to the given screen position. Here, x is X position, and y is Y position in the coordinates. For Example, the method mouseMove(100, 50) will move the mouse pointer to the x coordinate 100 and y coordinate 50 on the screen.

How to perform mouse click in Selenium using Robot class?

Let us understand some of these methods with the help of a sample example used in Keyboards tutorial, i.e., https://demoqa.com/keyboard-events/

To select the file for upload, we will need to perform the following steps:

• Firstly, identify the screen resolution of the "filename" in terms of its X and Y position.
• Secondly, move the mouse cursor to the identified x and y coordinates.
• Finally, click on the identified x and y coordinates.

Mouse events using java Robot Class in Selenium

We will follow the following steps to automate the file upload scenario:

1. First, we will launch the web browser and start our practice page. https://demoqa.com/keyboard-events/
2. Secondly, we will click on the ‘Choose File’ button to display the Desktop Windows popup.
3. Thirdly, we will move the mouse cursor to the desired file name by identifying the coordinates of the file name.
4. Fourthly, we will click the Left Mouse button on the identified coordinate to select the file.
5. After that, we will press the Enter key to close the popup.
6. Finally, we will close the browser to end the program.

Following is the sample code:

package com.toolsqa.tutorials.actions;

import java.awt.AWTException;
import java.awt.Robot;
import java.awt.event.InputEvent;
import java.awt.event.KeyEvent;
import java.io.IOException;

import org.openqa.selenium.By;
import org.openqa.selenium.Dimension;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.firefox.FirefoxDriver;
import org.openqa.selenium.interactions.Actions;

public class RobotMouseDemo {

	public static void main(String[] args) throws InterruptedException, AWTException, IOException {

		System.setProperty("webdriver.gecko.driver","C:\\Selenium\\lib\\geckodriver-v0.24.0-win64\\geckodriver.exe");

		// Start browser
		WebDriver driver = new FirefoxDriver();

		String URL = "https://demoqa.com/keyboard-events/";
		// maximize browser
		driver.manage().window().maximize();

		driver.get(URL);
		Thread.sleep(2000);

		// This will click on Browse button
		WebElement webElement = driver.findElement(By.id("browseFile"));
		webElement.sendKeys("ENTER");

		//Create object of Robot class
		Robot robot = new Robot();
         
		//Find x and y coordinates to pass to mouseMove method 
		//1. Get the size of the current window. 
		//2. Dimension class is similar to java Point class which represents a location in a two-dimensional (x, y) coordinate space. 
		//But here Point point = element.getLocation() method can't be used to find the position 
		//as this is Windows Popup and its locator is not identifiable using browser developer tool 
		Dimension i = driver.manage().window().getSize(); 
		System.out.println("Dimension x and y :"+i.getWidth()+" "+i.getHeight()); 
		//3. Get the height and width of the screen 
		int x = (i.getWidth()/4)+20; 
		int y = (i.getHeight()/10)+50; 
		//4. Now, adjust the x and y coordinates with reference to the Windows popup size on the screen 
		//e.g. On current screen , Windows popup displays on almost 1/4th of the screen . So with reference to the same, file name x and y position is specified. 
		//Note : Please note that coordinates calculated in this sample i.e. x and y may vary as per the screen resolution settings
		robot.mouseMove(x,y); 

		//Clicks Left mouse button
		robot.mousePress(InputEvent.BUTTON1_DOWN_MASK); 
		robot.mouseRelease(InputEvent.BUTTON1_DOWN_MASK);
		System.out.println("Browse button clicked");
		Thread.sleep(2000);

		//Closes the Desktop Windows popup
		robot.keyPress(KeyEvent.VK_ENTER);
		System.out.println("Closed the windows popup");
		Thread.sleep(1000);

		// Close the main window 
		driver.close();
	}

}

Conclusion

To conclude, in this tutorial, we have covered essential methods of Robot class to control the mouse to perform mouse actions on Desktop Windows popup.

• mousePress(int buttons): Method to press one or more mouse buttons
• mouseRelease(int buttons): Method to release one or more mouse buttons
• mouseMove(int x, int y): Method to move the mouse pointer to the given screen coordinate

Therefore, we have tried to cover all the essential Keyboard and Mouse event methods in this tutorial. However, Robot class provides many other methods like mouseWheel(), createScreenCapture(), etc. Additionally, you can explore them by clicking on the link below. https://docs.oracle.com/javase/10/docs/api/java/awt/Robot.html