Python code that solves all-white jigsaw puzzles and runs an automatic Puzzle-solving robot made out of Lego Mindstorms.
This is the code for my automatic Lego Jigsaw Puzzle solving machine.
If you enjoy what I do and would like to support future creations, consider buying me a 🧱 Lego brick!
Every contribution helps fund new machines and videos on YouTube!
This project contains the complete Python codebase for a robot that can solve all-white jigsaw puzzles. The system is managed by a central controller main.py that executes a step-by-step process via a graphical user interface (GUI). It uses computer vision to scan and analyze the puzzle pieces, a matching algorithm to determine the solution, and a custom-built Lego Mindstorms robot to physically assemble the puzzle. The code is structured into modules for camera management, image processing, puzzle-solving logic, GUI, and EV3 robot control.
The project includes a multi-stage pipeline with the following features:
- Graphical User Interface: A user-friendly interface built with PyQt6 that guides the user through each stage of the process, from calibration to final assembly.
- Camera Calibration: An interactive tool to calibrate the camera and correct for perspective distortion, ensuring a perfect top-down view of the puzzle area.
- Piece Detection and Segmentation: Automatically scans the puzzle area, thresholds the image to create a binary mask, and segments each individual piece.
- Piece Analysis: For each piece, the software detects its corners, extracts the contour of each of its four sides, and classifies them as 'straight', 'tab', or 'blank'.
- Spline-Based Side Matching: A matching algorithm compares the shape of piece sides using splines to find the most likely connections.
- Tree Solver: A beam search-based solver uses the match scores to determine the final and correct layout of the entire puzzle.
- Autonomous Robotic Assembly: A two-brick Lego Mindstorms EV3 robot, operating a gantry system with suction, picks and places each piece in its correct location and orientation.
- Hardware Compensation: The robot software automatically compensates for mechanical imperfections, such as the sagging of the long X-axis beam and play in the X- and Y-axis.
- Recovery System: The process can be stopped and resumed at different stages, as the software saves its progress after key milestones (scanning, piece detection, matching, and solution).
- A custom-built Lego puzzle-solving machine containing two LEGO Mindstorms EV3 kits.
- A rectangular all-white jigsaw puzzle.
- Python 3 with libraries installed (see
requirements.txt). - A camera that can be used as an IP Webcam (e.g., a smartphone with IP Webcam by Thyoni Tech).
- Tested on a Windows 11 operating system.
- Clone this repository:
git clone https://github.com/CreativeMindstorms/LEGO-Puzzle-Robot.git
- Create a virtual environment and install the required packages.
- Adjust the variables inside
config.py, especially:IP_ADDRESSandPORTfor your IP Webcam.EV3_TOP_ADDRESSandEV3_BOTTOM_ADDRESSfor your Mindstorms bricks.- Workspace and puzzle dimensions (
STAGING_AREA_*,SOLUTION_AREA_*). - Motor parameters.
- Run the application. The GUI will appear to guide you through the calibration and solving process.
Run the main controller script after installation and configuration:
python main.pyThe Graphical User Interface will launch. Follow the on-screen instructions, starting with camera calibration and then proceeding with the main puzzle-solving workflow.
The robot uses two Mindstorms EV3 Bricks, for which you need the Bluetooth MAC addresses.
EV3_TOP (Controls X, Z, Rotation, and Suction):
| PORT | DESCRIPTION |
|---|---|
| A | Rotation motor (motor_r). |
| B | Z-axis (lift) motor (motor_z). |
| C | X-axis gantry motor (motor_x). |
| D | Suction motor (motor_s). |
| 2 | X-axis touch sensor for homing (sensor_x). |
| 4 | Z-axis color sensor for surface detection (sensor_z). |
EV3_BOTTOM (Controls Y-axis):
| PORT | DESCRIPTION |
|---|---|
| A | Left Y-axis motor (motor_y). |
| D | Right Y-axis motor (motor_y). |
| 4 | Y-axis touch sensor for homing (sensor_y). |
- Establish a bluetooth connection at least once via the EV3 Classroom app, so that the ev3 device is paired to the laptop and then disconnect it.
- Right click the windows logo or press
Windows + X>Device Manager>Bluetooth>EV3(or the name of your brick) >Properties>Details>Bluetooth Device Address - Replace the placeholder in config.py with the letters and numbers and a colon between every two characters, such that it becomes "AB:12:CD:34:EF:56".
- Ensure your EV3 brick is turned on with the regular operating system (no ev3dev, just any official firmware) and Bluetooth is turned on. Then you should be able to run the code.
- If the code runs into an error or, during testing, you terminate the code unsafely, the code won't properly disconnect the Bluetooth, which will cause the
No EV3 device founderror to appear the next time you run the code. In this case, simply turn the Bluetooth on and off on your EV3, after which you can run the code again.
This project is licensed under the GPLv3 License. Contributions and modifications are welcome, but they must remain open-source and credit the original author.
Contributions are welcome, and I appreciate your interest in improving this project! However, I want to keep this easy to manage, as it is a personal project and a learning experience for me.
If you’d like to suggest changes or improvements, here’s how you can contribute:
- Fork the Repository: Create a personal copy of the repository by clicking the "Fork" button at the top.
- Make Changes: Implement your changes in your forked repository. Please keep changes focused and well-documented.
- Submit a Pull Request (PR): Open a pull request with a clear explanation of your changes. Include why the change is beneficial and how it affects the project.
This project is a hobby, and while I enjoy working on it, I can’t provide consistent support or assistance. Please feel free to reach out via email for questions or feedback, but responses may be delayed depending on my availability.
The Python files in this repository have been automatically refactored using an AI assistant with the primary goal of improving documentation and ensuring PEP8 compliance.
While the code has been tested and is believed to be functionally correct, the generation process was largely automated. As such, it is possible that subtle, unforeseen bugs or logical inconsistencies may exist.
If you encounter any unexpected behavior or suspect an error, please do not hesitate to open an issue. This will help improve the stability and reliability of the project for everyone.