← All work
Case study · 04/08
Virtual Try-On
Mobile app using computer vision to virtually try glasses and recommend frames based on face shape.
(01) Problem
Users needed a way to virtually try glasses frames via their mobile camera. The system required accurate placement of frames and recommendations based on facial characteristics.
(02) Approach
- 01Implemented OpenCV and dlib 68-point landmarks to detect facial features for frame placement.
- 02Used a CNN and KMeans method to classify face shape for frame recommendations.
- 03Integrated MiVOLO with YOLOv8 to estimate user age and gender.
- 04Built a React Native mobile interface for camera interaction and results display.
(03) Architecture
React Native AppFlask BackendOpenCV/dlib LandmarksFace Shape CNNMiVOLO/YOLOv8Recommendation Output
(04) The hard part
Aligning frames accurately to the eyes on both still and live images proved difficult. Combining several machine learning models within a single request required careful orchestration.
(05) Result
The system successfully classified face shape and estimated age and gender on test photos.