Recent article in Science Direct.
Doesn't mention us but obviously related to neuromorphic cameras and pose estimation in gaming.
I recall Nviso were doing pose as well and probs emotion3D?
Anyway, I just found it interesting at the end where some of their financial support comes from, that they will look to test properly on Loihi 2 in the future (obvious research chip access) and where one of the co-authors works.
28 August 2023
Neuromorphic high-frequency 3D dancing pose estimation in dynamic environment
Abstract
Technology-mediated dance experiences, as a medium of entertainment, are a key element in both traditional and virtual reality-based gaming platforms. These platforms predominantly depend on unobtrusive and continuous human pose estimation as a means of capturing input. Current solutions primarily employ RGB or RGB-Depth cameras for dance gaming applications; however, the former is hindered by low-light conditions due to motion blur and reduced sensitivity, while the latter exhibits excessive power consumption, diminished frame rates, and restricted operational distance. Boasting ultra-low latency, energy efficiency, and a wide dynamic range, neuromorphic cameras present a viable solution to surmount these limitations. Here, we introduce
YeLan, a neuromorphic camera-driven, three-dimensional, high-frequency human pose estimation (HPE) system capable of withstanding low-light environments and dynamic backgrounds. We have compiled the first-ever neuromorphic camera dance HPE dataset and devised a fully adaptable motion-to-event, physics-conscious simulator. YeLan surpasses
baseline models under strenuous conditions and exhibits resilience against varying clothing types, background motion, viewing angles, occlusions, and lighting fluctuations.
9. Limitation and Future Works
......Additionally, we aim to investigate energy-efficient implementations on neuromorphic computing platforms such as Intel Loihi 2.
Declaration of Competing Interest
The authors declare the following financial interests/personal relationships which may be considered as potential competing interests: Hava Siegelmann reports financial support was provided by
Defense Advanced Research Projects Agency. Tauhidur Rahman reports financial support was provided by
Defense Advanced Research Projects Agency. Tauhidur Rahman reports financial support was provided by
National Science Foundation. Coauthor employed by
Qualcomm Technologies, Inc. - Upal Mahbub..