In some situations, it can be useful to simulate user events on a keyboard or mouse, without an actual keyboard or mouse. The original idea that made me look into this subject, was that I wanted to run RetroPie on my Raspberry Pi with different ways to control the system. Instead of a keyboard, I wanted to use GPIO inputs (buttons or joysticks), or even a web interface that I could access from my phone. We’ll use a program called xdotool to do this.
If you think about it, once you get the keyboard simulation part down, it’s only a matter of interfacing it with whatever source you want to use, whether it’s a custom GPIO controller, a web interface, or even your laptop through SSH. The possibilities are endless, but it all starts with finding a way to interact with the system by simulating keyboard events.
In this tutorial I’ll talk about xdotool, which is the method I chose to simulate events. I am using a Raspberry Pi 3 with Raspbian Jessie (Desktop version). Keep in mind that there are other options, and some may even be better (I’ll be happy to hear about them in the comments!). Also, this article applies to many Linux distributions, and I only talk about Raspberry Pi/Raspbian here because that’s what I’ve been working on; I can’t guarantee that it will be 100% identical on other OS or architectures.
If you don’t have a Raspberry Pi yet, here is a great deal on a Raspberry Pi 3 on Amazon. Or if you need a full pack with the Raspberry Pi, a power supply, a SD card, a HDMI cable and a case, check out this complete kit.
NOTE: I unfortunately discovered that it is not possible to send keyboard or mouse events to RetroPie with xdotools. I have decided to use uinput instead, with turned out to be more adapted to the problem.
There are two main tools that I’ve tried in order to simulate user input via a virtual keyboard/mouse.
The first one is python-uinput, a Python library that lets you create a virtual input device, and generate events such as a key press, or a mouse move/click. I originally couldn’t get it to work over SSH, but later found out how to to this. If you want more details about it, you can go to the Github page of python-uinput. It is also possible to use the uinput library in a C program.
The second one is xdotools. Unlike python-uinput, which creates an actual entry in the device directory /dev on the machine, xdotools is only directed to X11 environment, which is basically the graphical interface of the OS. For example, it lets you interact with windows, and generate events on the keyboard and the mouse. You can either use the command line version (xdotool), or the C library (libxdo). Feel free to visit the Github page of xdotool to get more info about it.
Command line tool
The tool is part of the standard Debian repositories, so on a Raspberry Pi running Raspbian, you can install it by running:
sudo apt-get install xdotool
As with any other program, you can check that it is correctly installed by running the command:
It will return the path to the executable.
C library: libxdo
To install the C library of xdotool, run the command:
sudo apt-get install libxdo-dev
To check if it was installed correctly, type:
ls /usr/include/ | grep xdo
Using xdotool over SSH
My Raspberry Pi is connected to my TV via HDMI, and running headless (without a keyboard or mouse). I always use SSH when I need to work it. The problem is that xdotool will not do anything unless you give it the display your Raspberry Pi is connected to. To do so, you have to export the DISPLAY variable. There are two way to do this:
# This will only set up the DISPLAY variable for one command
DISPLAY=:0 xdotool mousemove 100 100
DISPLAY=:0 xdotool click 1
# This will set up the DISPLAY variable for every command executed on this terminal,
# and child processes spawned from this terminal
xdotool mousemove 100 100
xdotool click 1
If you wonder how I found the value :0 for DISPLAY, you can find it by running xauth list.
Exporting the value of DISPLAY like this will work both for the command line tool and the C library (the C library reads this environment variable when you call xdo_new()).
Quick overview of the commands
There’s a lot of things you can do with xdotool. Let’s just focus on the command line tool for now. To get a list of available commands, type:
Most of the commands have a self-explanatory name.
- Move the mouse
- Absolute position: xdotool mousemove 200 400 – Move the mouse pointer to the position 200 400 (point 0 0 is the top left corner)
- Relative position: xdotool mousemove_relative 100 50 – Move the mouse by 100 on the x axis and 50 on the y axis, relative to its current position. To move by a negative amount, use xdotool mousemove_relative -- -20 10. This will move the mouse -10 on the x axis and +10 on the y axis.
- Get the current mouse location: xdotool getmouselocation – Gives the current x and y location of the mouse pointer, with the screen and current window ID
Note that if you need to wait until the mouse has actually moved before executing another command, you can add the option --sync to the command.
- Mouse click
- Simulate a click: xdotool click 1. The number represent which button click is simulated (1 = left, 2 = middle, 3 = right, 4 = wheel up and 5 = wheel down).
- Mouse down/up: xdotool mousedown 1 and xdotool mouseup 1 – This can be useful for a drag and drop for example: xdotool mousedown 1 mousemove_relative --sync 100 100 mouseup 1. This will drag and drop from the current position to the relative position 100 100 away from it.
- Simulate a key press: xdotool key A – Type the letter ‘A’.
- Hold a key down: xdotool keydown B and xdotool keyup B . For example you can simulate a user pressing on a key for 1 second and releasing it: xdotool keydown B sleep 1 keyup B . To get a list of supported options for this command, use xdotool keydown.
- Typing a string: xdotool type "Hello" – Type a whole string.
Interacting with windows
- Get the current window ID: xdotool getactivewindow – This will return the ID of the current window, for example 25165825.
- Get the name of a window: xdotool getwindowname 25165825 – This will print the title of the window, for example “New Tab – Chromium”.
- Change the size of a window: xdotool windowsize 25165825 1280 720 – Resize the window to 1280×720.
- Move a window: xdotool windowmove 25165825 100 100 – Move the window to the position 100 100.
- Get the position and size of a window: xdotool getwindowgeometry 25165825 – This can be useful if you want to move your mouse and keep it inside a specific window.
That’s just an overview of the most useful commands offered by xdotool. There are a lot more you can do with this tool, and the best way to learn all you need to know about it, is to experiment on your own!
As you can see, xdotool is a very powerful tool, and gives you full control on the desktop environment, without any actual keyboard or mouse. It’s a huge help if you want to automate tasks, or implement your own custom controller and need to programmatically send events to the graphical environment and get information about it. If you have any question or need help with xdotool, feel free to leave a comment below!