What if you don’t have knowledge of AI/ML but still want ML solutions,live streaming and all ? Here we will understand the concept of MediaPipe also we will take an example of finger gesture recognition to make it more simple to understand.
It’s not necessary to have knowledge of AI/ML, to build and use some ML solutions, live streamings and certain other functionalities as we have got MediaPipe.
Let’s understand the concept of MediaPipe along with an example of finger gesture recognition to make it more simple to understand.
Introduction To Mediapipe
Python provides us with an amazing MediaPipe library and the best thing is that it is free open source, cross-platform and ready to use AI/ML solution.
We can use MediaPipe in Android and IOS easily.
Now the question is, where can we use and implement MediaPipe ?
The list is quite long but we have listed some trending use cases here,
- Face Detection
- Face Mesh
- Hands/Fingers detection and classification
- HolisticObjectron and many more
Use Of Mediapipe With Python
Refer to the following URL to install Mediapipe in Python:
Without further detailed discussion, let’s explore finger gesture recognition as an example to understand Mediapipe.
Finger gesture recognition using Hand model of MediaPipe
What Is Finger Gesture Recognition ?
Finger gesture recognition is the process of understanding the movements of the fingers and translating the same in understandable language.
In order to recognize the symbols, we must first follow the movement of the fingers.
Above figure is an example of hand gesture recognition.
In this example we are detecting the hand landmarks and on the basis of that we will recognize the gesture.
How Hand Landmark Detection Works ?
With the help of the Hand model of MediaPipe, we can track the hands and finger landmarks through the code.
Multi_handness : This helped us to detect the hand side
i.e either it is left hand or right hand.
After successful detection of the hand, the next step is to detect which finger is showing up the movement.
Detect The Fingers
The concept is very simple by comparing the y-coordinate of “finger-tip” and “finger-pip” landmarks. If the finger is UP then y-coordinate of
finger-tip < ( less than ) finger-pip .
For the”thumb” game to change, it depends on the side of the hands .We have to compare the x coordinate of thumb-tip and thumb-mcp.
From the above content, I hope you have understood the use of MediaPipe. This was just one example which we implemented, there are many others. For more information you can visit Github. Happy coding!