You are here

Distributed data exchange with Leap Motion

This project deals with the creation of a distributed communication strategy to handle high-throughput visualisation data in collaborative virtual environments (CVEs). CVEs can connect people in social virtual spaces even when they are geographically distant from each other. Hand interactions are fundamental to enable natural collaboration and immersive experiences as they are a visually intuitive means of communication. However, scalability is challenging as numerous participants typically produce a large volume of visualisation data that may overload a single node if the management is centralised. In this project we develop a transmission strategy where the high-throughput visualisation data (e.g. hand joints) is exchanged amongst participants in a distributed fashion. We use a level-of-detail strategy to further reduce the network traffic accounting for spatial distances amongst participants in the virtual space. We design an experiment where we analyse the network traffic in a virtual environment with up to seven participants whose hands are tracked using Leap Motion. The experiments show that the proposed method can effectively reduce the network traffic of visualisation data when compared to a centralised approach.
The flow diagram below shows the proposed distributed mechanism. Each client is connected to the Leap Motion's web service that is hosted on a computer and leap-frames are transmitted using JSON format. The client deserialises the JSON-formatted leap-frames using C# classes of Unity3D. Requests are dealt by the level of detail (LOD) manager. The clients send visualisation data at the requested level of detail. The figure shows three conceptual blocks: green for the local Leap Motion connection, blue for the local processing of the hand visualisation and red for remote hand visualisation.
The figure below shows five clients that are within the same VR space and visualisation data are exchanged using the proposed distributed mechanism with levels of detail. The left-hand figure is a screenshot taken from a Cardboard device, while the right-hand figure is a screenshot taken from a desktop computer. The distance between clients is such that they create two groups of data exchange: one group of two clients (left-hand figure) and one group of three clients (right-hand figure). In each group there is one client waving its hands. With this configuration L0 (max resolution) is applied between clients of each group and L3 (no data) is applied between clients of different groups. The green arrows illustrate whom the visualisation data is shared with. From the Cardboard's view we can see the group of three clients in the background and we can observe that their hands' movements would not be visible even if their visualisation data were transmitted to the clients of the other group.
M. Pani and F. Poiesi
"Distributed data exchange with Leap Motion"
Proc. of International Conference on Augmented Reality, Virtual Reality and Computer Graphics - AVR
Otranto, Lecce, IT, 24-27 June 2018


Wednesday, 15 November, 2017 to Friday, 2 February, 2018
3 months
Mirko Pani ( Dep. of Information Engineering and Comp. Science UniTN
Fabio Poiesi ( FBK