Zhian (Aria) Ruan
Back to projects

Software · Computer vision · Simulation

Vision Pro Simulator

Real-time gesture-controlled photo manipulation with custom MLP and LSTM models.

Vision Pro Simulator gesture demo

Stack: Python, TensorFlow

Highlights

  • Developed a lightweight Vision Pro simulator with real-time hand gesture and motion recognition achieving >90% accuracy using custom MLP and LSTM neural networks, trained in under 90 seconds on CPU
  • Implemented interactive photo manipulation features including drag-and-drop, scaling, rotation, and drawing tools using computer vision techniques and bilinear interpolation
  • Devised a robust hand tracking system maintaining accuracy across varied lighting conditions and multi-user scenarios, with a modular pipeline for detection, tracking, and gesture recognition

Video demo

Source code

View on GitHub · Full video on YouTube