Leap Motion

Distributed data exchange with Leap Motion

This project deals with the creation of a distributed communication strategy to handle high-throughput visualization data in collaborative virtual environments (CVEs). CVEs can connect people in social virtual spaces even when they are geographically distant from each other. Hand interactions are fundamental to enable natural collaboration and immersive experiences as they are a visually intuitive means of communication. However, scalability is challenging as numerous participants typically produce a large volume of visualization data that may overload a single node if the management is centralized. In this project we develop a transmission strategy where the high-throughput visualization data (e.g. hand joints) is exchanged amongst participants in a distributed fashion. We use a level-of-detail strategy to further reduce the network traffic accounting for spatial distances amongst participants in the virtual space. We design an experiment where we analyze the network traffic in a virtual environment with up to seven participants whose hands are tracked using Leap Motion. The experiments show that the proposed method can effectively reduce the network traffic of visualization data when compared to a centralized approach.

The flow diagram below shows the proposed distributed mechanism. Each client is connected to the Leap Motion's web service that is hosted on a computer and leap-frames are transmitted using JSON format. The client deserialises the JSON-formatted leap-frames using C# classes of Unity3D. Requests are dealt by the level of detail (LOD) manager. The clients send visualization data at the requested level of detail. The figure shows three conceptual blocks: green for the local Leap Motion connection, blue for the local processing of the hand visualization and red for remote hand visualization.

The figure below shows five clients that are within the same VR space and visualisation data are exchanged using the proposed distributed mechanism with levels of detail. The left-hand figure is a screenshot taken from a Cardboard device, while the right-hand figure is a screenshot taken from a desktop computer. The distance between clients is such that they create two groups of data exchange: one group of two clients (left-hand figure) and one group of three clients (right-hand figure). In each group there is one client waving its hands. With this configuration L0 (max resolution) is applied between clients of each group and L3 (no data) is applied between clients of different groups. The green arrows illustrate whom the visualisation data is shared with. From the Cardboard's view we can see the group of three clients in the background and we can observe that their hands' movements would not be visible even if their visualisation data were transmitted to the clients of the other group.