Tracko: Ad-hoc Mobile 3D Tracking Using Bluetooth Low Energy and Inaudible Signals for Cross-Device Interaction

Haojian Jin, Christian Holz and Kasper Hornbæk. UIST 2015.

Video

Figure 1

Tracko

Tracko provides mobile devices with a spatial awareness of surrounding devices in 3D, allowing users to interact across devices in the very space they act in. Tracko readily runs on commodity devices with no added components and requires no calibration or external infrastructure. (a) Here, Tracko enables file sharing across devices simply by swiping items towards the recipient. (b) Tracko detects the presence of other devices using Bluetooth low energy, (c) determines their distance and 3D direction from the difference in arrival times of exchanged inaudible stereo signals, and detects quick interactions using inertial sensors.

Abstract

While current mobile devices detect the presence of surrounding devices, they lack a truly spatial awareness to bring them into the user's natural 3D space. We present Tracko, a 3D tracking system between two or more commodity devices without added components or device synchronization. Tracko achieves this by fusing three signal types. 1) Tracko infers the presence of and rough distance to other devices from the strength of Bluetooth low energy signals. 2) Tracko exchanges a series of inaudible stereo sounds and derives a set of accurate distances between devices from the difference in their arrival times. A Kalman filter integrates both signal cues to place collocated devices in a shared 3D space, combining the robustness of Bluetooth with the accuracy of audio signals for relative 3D tracking. 3) Tracko incorporates inertial sensors to refine 3D estimates and support quick interactions. Tracko robustly tracks devices in 3D with a mean error of 6.5 cm within 0.5 m and a 15.3 cm error within 1 m, which validates Tracko’s suitability for cross-device interactions.

Publication

Haojian Jin, Christian Holz and Kasper Hornbæk. 2015. Tracko: Ad-hoc Mobile 3D Tracking Using Bluetooth Low Energy and Acoustic Signals For Cross-Device Interaction. In Proceedings of the 28th annual ACM symposium on User interface software and technology (UIST '15). Charlotte, NC (November 8-11, 2015). ACM, New York, NY, USA, 147–156.

Other figures

Tracko's acoustic ranging between devices

Acoustic ranging between two devices. Device DA emits a signal at t0 and records it at t1. Device DB receives this signal at r0. t2, t3, r1 indicate the corresponding time stamps when Device B emits an acoustic signal in response.

Tracko's inaudible signals

Tracko’s inaudible signals comprise an initial beacon and a payload that encodes sender ID, speaker ID, and a counter using biorthogonal chirps for stable audio decoding.

Tracko's biorthogonal signals

Biorthogonal chirps encode 2 bits as a series of 4 up or down chirps, such that each is orthogonal to each other.

Tracko's residual model

Visualization of the results produced by Tracko's residual model. (left) The residual distribution over the whole 3D space. (right) The 1000-point cloud with smallest residuals. The two solutions with lowest residuals are highlighted in red.

Errors in Tracko's audio-based distances

Distance errors for Tracko’s predictions of roundtrip distances. Bars present the percentage of error distribution.

Errors in Tracko's 3D tracking

Accuracy of the 3D coordinate estimates calculated by our residual model for the remote device in 3D space.

Tracko's on-the-fly BLE-to-distance calibration

Errors of BLE-based distance estimations. Tracko’s on-the-fly calibration yields more accurate distances than existing apps or the typical RSSI-to-distance mapping.

Pouring images from one mobile device into another using Tracko

Tracko enables devices to support new interaction techniques, here pouring objects from one device into another.

Presentation deck

Tracko presentation Slide 1
Tracko presentation Slide 2
Tracko presentation Slide 3
Tracko presentation Slide 4
Tracko presentation Slide 5
Tracko presentation Slide 6
Tracko presentation Slide 7
Tracko presentation Slide 8
Tracko presentation Slide 9
Tracko presentation Slide 10
Tracko presentation Slide 11
Tracko presentation Slide 12
Tracko presentation Slide 14
Tracko presentation Slide 15
Tracko presentation Slide 16
Tracko presentation Slide 17
Tracko presentation Slide 18
Tracko presentation Slide 19
Tracko presentation Slide 20
Tracko presentation Slide 21
Tracko presentation Slide 22
Tracko presentation Slide 23
Tracko presentation Slide 24
Tracko presentation Slide 25
Tracko presentation Slide 26
Tracko presentation Slide 27
Tracko presentation Slide 28
Tracko presentation Slide 29
Tracko presentation Slide 30
Tracko presentation Slide 31
Tracko presentation Slide 32
Tracko presentation Slide 33
Tracko presentation Slide 34
Tracko presentation Slide 35
Tracko presentation Slide 36
Tracko presentation Slide 37
Tracko presentation Slide 38
Tracko presentation Slide 39
Tracko presentation Slide 40
Tracko presentation Slide 41
Tracko presentation Slide 42
Tracko presentation Slide 43
Tracko presentation Slide 44
Tracko presentation Slide 45
Tracko presentation Slide 46
Tracko presentation Slide 47
Tracko presentation Slide 48
Tracko presentation Slide 49
Tracko presentation Slide 50
Tracko presentation Slide 51
Tracko presentation Slide 52
Tracko presentation Slide 53
Tracko presentation Slide 54
Tracko presentation Slide 55
Tracko presentation Slide 56
Tracko presentation Slide 57
Tracko presentation Slide 58
Tracko presentation Slide 59
Tracko presentation Slide 60
Tracko presentation Slide 61
Tracko presentation Slide 62
Tracko presentation Slide 63
Tracko presentation Slide 64
Tracko presentation Slide 65
Tracko presentation Slide 66
Tracko presentation Slide 67
Tracko presentation Slide 68
Tracko presentation Slide 69
Tracko presentation Slide 70
Tracko presentation Slide 71
Tracko presentation Slide 72
Tracko presentation Slide 73
Tracko presentation Slide 74
Tracko presentation Slide 75
Tracko presentation Slide 76
Tracko presentation Slide 77
Tracko presentation Slide 78
Tracko presentation Slide 79
Tracko presentation Slide 80
Tracko presentation Slide 81
Tracko presentation Slide 82
Tracko presentation Slide 83
Tracko presentation Slide 84
Tracko presentation Slide 85
Tracko presentation Slide 86
Tracko presentation Slide 87
Tracko presentation Slide 88
Tracko presentation Slide 89