Controlling Mouse Movements and Clicks

Move and click

import pyautogui
 
pyautogui.moveTo(200, 200, duration=0.5)
pyautogui.click()

Drag

import pyautogui
 
pyautogui.dragTo(400, 400, duration=0.5, button="left")

Scroll

import pyautogui
 
pyautogui.scroll(-500)

Visualize it

GUI automation locates a target on screen, then moves and acts on it before checking the outcome.

  flowchart LR
    A["Locate target (coordinates or image)"] --> B["Move the mouse"]
    B --> C["Click or type"]
    C --> D["Verify the result"]

How the mouse functions work

pyautogui moves the real system cursor. Positions are pixels, and (0, 0) is the top-left corner of the primary screen. x grows to the right and y grows downward, which is the opposite of a maths graph. pyautogui.size() returns the screen width and height, and pyautogui.position() returns where the cursor is now. Use these to check that a target lies inside the screen.

The main calls:

Give moves a duration. An instant jump can be too fast for some programs to register, and a visible motion lets you see what is happening and use the failsafe.

A careful script

In the drag example, dragTo starts from wherever the mouse is, so you must move to the start point first. This helper checks bounds, moves, drags, and pauses.

import pyautogui
 
pyautogui.FAILSAFE = True
pyautogui.PAUSE = 0.3
 
 
def inside_screen(x: int, y: int) -> bool:
    width, height = pyautogui.size()
    return 0 <= x < width and 0 <= y < height
 
 
def drag_between(start: tuple[int, int], end: tuple[int, int]) -> None:
    for point in (start, end):
        if not inside_screen(*point):
            raise ValueError(f"{point} is outside the screen")
    pyautogui.moveTo(*start, duration=0.5)
    pyautogui.dragTo(*end, duration=0.8, button="left")
 
 
print(pyautogui.size(), pyautogui.position())
# drag_between((300, 300), (600, 300))

The drag call is commented out, so the script only prints the size and the position until you choose to run it. Try it first in a blank drawing program, not in a file manager, where a drag can move files.

Finding coordinates

Run pyautogui.displayMousePosition() in a terminal, which prints the cursor position live as you move it. Coordinates depend on the screen resolution, on display scaling (a 150% scale factor on Windows changes the numbers on some setups), and on which monitor is primary. A script that uses fixed coordinates on one machine often clicks the wrong place on another.

Safety

Clicking is not undoable. A click on the wrong button can delete, send, or buy. Keep FAILSAFE on: slamming the mouse into the top-left corner raises FailSafeException and stops the script. Test in a harmless app first, and do not leave a clicking script running while you use the machine, because your own mouse movements will compete with it.

Common mistakes

• Calling dragTo without moving first. The drag begins at the current position.
• Instant moves with duration=0.
• Positive scroll values scrolling up, when you wanted down.
• Fixed coordinates across machines with different resolutions.
• Using a second monitor without checking the coordinate range. Points left of or above the primary screen are negative.

Check your understanding