Reconstructing what you said: Text Inference using Smartphone Motion

Duncan Hodges, Oliver Buckley

Research output: Contribution to journalArticle

4 Citations (Scopus)
36 Downloads (Pure)

Abstract

Smartphones and tablets are becoming ubiquitous within our connected lives and as a result these devices are increasingly being used for more and more sensitive applications, such as banking. The security of the information within these sensitive applications is managed through a variety of different processes, all of which minimise the exposure of this sensitive information to other potentially malicious applications. This paper documents experiments with the 'zero-permission' motion sensors on the device as a side-channel for inferring the text typed into a sensitive application. These sensors are freely accessible without the phone user having to give permission. The research was able to, on average, identify nearly 30 percent of typed bigrams from unseen words, using a very small volume of training data, which was less than the size of a tweet. Given the natural redundancy in language this performance is often enough to understand the phrase being typed. We found that large devices were typically more vulnerable, as were users who held the device in one hand whilst typing with fingers. Of those bigrams which were not correctly identified over 60 percent of the errors involved the space bar and nearly half of the errors are within two keys on the keyboard.

Original languageEnglish
Article number8401706
Pages (from-to)947-959
Number of pages13
JournalIEEE Transactions on Mobile Computing
Volume18
Issue number4
Early online date2 Jul 2018
DOIs
Publication statusPublished - 1 Apr 2019

Keywords

  • Computer security
  • computer hacking
  • keyboards
  • sensors

Cite this