WPI - Computer Science Department, PhD Proposal Defense , Yiqin Zhao " Dynamic Lighting Estimation for Augmented Reality Systems ""
2:00 pm to 3:00 pm
Dynamic Lighting Estimation for Augmented Reality Systems
Yiqin Zhao
PhD Student
WPI – Computer Science Department
Tuesday, January 28, 2025
Time: 2:00 PM to 3:00 PM
Zoom: https://wpi.zoom.us/j/2207940527
Committee Members :
Prof. Tian Guo, Dissertation Advisor - WPI - Computer Science
Prof. Robert Walls - WPI - Computer Science
Prof. Erin Solovey - WPI - Computer Science
Prof. Sheng Wei (external committee member) - Rutgers - Electrical and Computer Engineering
Abstract :
High-quality environment lighting is the foundation of creating immersive user experiences in mobile augmented reality (AR) applications. However, achieving visually coherent environment lighting estimation for Mobile AR is challenging due to several key limitations associated with AR device sensing capabilities, including limitations in device camera FoV and pixel dynamic ranges. Recent advancements in 3D vision and generative AI present many new opportunities for high-quality lighting estimation. While these are promising solutions, challenges remain in addressing the system latency, privacy concerns, and test-time quality of lighting estimation systems. These challenges are critical for dynamic and interactive AR applications and profoundly impact the real-world performance of AR systems.
This dissertation addresses these key challenges through four primary research tasks: (1) Mobility-aware lighting estimation. (2) Privacy-preserving reflection rendering. (3) Context-guided generative lighting estimation. And (4) foundational model-assisted perception services. Recognizing the potential of foundational models in AR perception, we propose the plan worked FARSight, a cooperative perception framework that rethinks the edge-assisted AR perception systems in the era of foundational models. This planed work aim to design novel edge-offloading system architectures to effectively integrate foundational models with real-time AR perception workflow and address the long latency of foundational model inference. We also seek to explore efficient ways of incorporating perception prior knowledge of foundational models into the AR perception workflow.