Glimmer AR Studio
A high-fidelity, real-time AR jewellery try-on engine running entirely in the browser, featuring a custom-trained YOLOv8 model, MediaPipe Pose fusion, and dynamic physics simulations.
YOLOv8 (Ultralytics)
MediaPipe
ONNX Runtime Web
PyTorch
JavaScript
PythonOverview
Glimmer AR bridges the gap between complex machine learning models and seamless web experiences. By combining state-of-the-art object detection with specialized web rendering techniques, users can virtually try on earrings and necklaces in real-time with stunning accuracy.
Key Technical Achievements
- Custom AI Pipeline: Trained a specialized YOLOv8 model from scratch to detect specific anatomical landmarks (earlobes, nose, eyes) required for precise jewellery anchoring.
- Edge Inference: Utilized ONNX Runtime Web to execute the neural network directly in the browser via WebAssembly (WASM), eliminating server round-trips and ensuring complete user privacy.
- Multi-Model Fusion: Developed an intelligent fusion algorithm that synchronizes the high-frequency YOLO facial detections with MediaPipe's robust body pose estimations to create stable 2D spatial anchors.
- Dynamic Physics Engine: Wrote a custom physics module implementing spring and damping mechanics. This calculates inertia and momentum so digital assets swing naturally as the user moves.
- Mesh Deformation: Engineered a vertex-based image warper that manipulates 2D necklace assets along quadratic Bezier curves, ensuring realistic draping across the chest rather than flat, static overlays.