Towards gesture-based multi-user interactions in collaborative virtual environments
This project deals with the creation of a virtual reality (VR) setup that enables multiple users to participate in collaborative virtual environments and interact via gestures. A collaborative VR session is established through a network of users that is composed of a server and a set of clients. The server manages the communication amongst clients and is created by one of the users. Each user's VR setup consists of a Head Mounted Display (HMD) for immersive visualisation, a hand tracking system to interact with virtual objects and a single-hand joypad to move in the virtual environment. We use Google Cardboard as a HMD for the VR experience and a Leap Motion for hand tracking, thus making our solution low cost. We evaluate our VR setup though a forensics use case, where real-world objects pertaining to a simulated crime scene are included in a VR environment, acquired using a smartphone-based 3D reconstruction pipeline. Users can interact using virtual gesture-based tools such as pointers and rulers.
The use-case shown in the animated images below involves two users concurrently connected via WiFi in the same local network which were connected through their personal Leap Motion to interact via gestures. We authored the VR environment using the 3D models obtained with the smartphone-based reconstruction pipeline and included a reconstructed couch, a reconstructed lamp and a knife. Users are interacting with their hands, and the hands are visible in each other's views.