Hand-Tracking-Canvas

Hand Tracking Canvas is a Python desktop drawing application that leverages MediaPipe for hand gesture detection and landmark tracking, and OpenCV for rendering drawings, enabling intuitive, touch-free interaction. It supports three painting modes: index and middle fingers raised for selection, thumb and index fingers raised for drag-and-drop, and index finger raised for drawing.

Selection Mode

Paintin Mode

Drag and Drop Mode

Installation

Download the source code to run this project.
However, there are some prerequisites that are needed to be done first.
You need to install the required libraries by running the following command lines in the project's directory:

pip install -r requirements

now you can run the project directly from your terminal or IDE:

cd src
python app.py

If everything has gone right, you can use different hand gesture modes as demo above to paint:

Using Tips

• When both index and middle fingers are up, then turn into selection mode, and make a selection.
• Paint when only the index finger is up.
• After selecting a shape to draw, use your thumb and index finger to drag. When all fingers are up, then the shape would be fixed on the canvas.
  
  

Table of Contents

• Main Idea and Features
• Architecture
• Technique
  • Key Points
  • Environment
  • Version Control
• Contact

Main Idea and Features

• Using OpenCv to process video frames captured from a camera, turn it into a .jpeg file
• Using Flask framework to keep sending responses to the front-end displayed image
• Using MediaPipe to detect the hand and finger tip location, as the image below

Architecture

• Server Architecture

  

Technique

Key Points

• MediaPipe
• OpenCv

Environment

• Python Flask

Version Control

• Git/GitHub

Contact

Sand, Yang

Email: sand050965@gmail.com