Today, the Intelligent Personal Assistant (IPA) industry is a growing one. They can be found embedded in home appliances, cars, phones, smart-watches and more, with new devices being added to that list every day. This spread though has met both positive and negative reactions because as much as they do give provide a boost to the usability of technology, their effective operation requires them to have access to a lot of user data, sometimes including private data. There are concerns that this private data could be exploited or fall into the hands of malicious actors. This article sheds a light on the operation of these IPAs focusing on the most popular –Amazon’s Alexa–, the privacy concerns surrounding them and users’ attitude towards the concerns in an attempt to give an answer to the question ‘Do IPAs mean the end for privacy?’.

Thanks to the rapid advancements in technology as well as the continuous human desire for convenience, voice-controlled intelligent personal assistants (IPAs) are ubiquitous today. We have them in our phones, cars, computers and even our homes. The functions of voice assistants are various, most popular today in smartphones and standalone devices known as smart speakers for homes. As the Internet of Things (IoT) keeps growing, more and more devices and appliances are being made ‘smart’ and added to IoT networks. This along with other factors such as cost and ease of management has led to more users opting for smart speakers rather than traditional home automation systems in their smart homes. This is a no-brainer as smart speakers provide arguably the same level of convenience, without the complex programming needed with home automation systems. There are different providers of IPA services but Amazon currently controls 70% of the market with their Alexa Voice Service (AVS), which is available both in their proprietary firmware –Echo-, as well as third-party hardware. In this article I will be focused on the AVS and will sometimes use the terms to refer to IPAs as a whole.

Although the increasing spread of the IPAs has been welcome by many due to the convenience they provide, there have also been many security and privacy concerns due to the way the devices operate, from their continuous listening microphones, to the way the information is processed. These concerns are along the lines of what personal or private information is being passed on to the service provider or even third parties and how they are misusing or could misuse this information to exploit users, as well as this data being potentially at risk to malicious hackers. I will be discussing in this article, the operation of the AVS in a bid to highlight privacy attack landscape and the possible points where a user’s private information can be leaked either to Amazon, third-parties or hackers. I will also delve into these concerns, Amazon’s response to them, and users’ attitude towards them in greater detail as I attempt to answer the question ‘Are Intelligent personal assistants the end for privacy?’.

IPAs can offer hands-free voice control for improved convenience. To do this efficiently however, the device on which they operate must have an always-listening microphone. This of course has is a course for concern but Amazon has maintained that the device is only listening for a ‘wake word’ and that information is only recorded or transmitted after this wake word is heard. For the devices to interact with users, the AVS uses Alexa Skills –voice-enabled apps that developers can create for the Alexa service-. Amazon has provided the Alexa Skills Kit interface within the Amazon Web Services (AWS) to aid developers to create these apps, which are built either as basic response apps or web services that interact with third party API and hosted on a cloud-based VM service such as the AWS Lambda services. When a user speaks into the device –Echo or third party– to interact with Alexa, the device records the voice data sends the speech input to the AVS. The AVS parses it and processes it into a JSON format and then sends it to the Alexa Skill Kit interface where some pattern matching is done in order to decide which function the user wants to interact with. The data is then sent to the corresponding HTTP endpoint that will execute the required function. There might be need to interact with third party APIs at this stage either to retrieve or process necessary information. When the endpoint server has performed the request, it produces JSON output data which is returned to the Alexa Web Services. The AVS then processes the data back into speech and delivers it to the speaker device as Alexa’s response to the user. The diagram below illustrates this process.

Now that we understand how the AVS operates we can better discuss the points at which security and privacy may be compromised in using these devices. I have grouped these points of privacy leakage into three to help in the discussion namely; the Hardware and Firmware, AVS operation and communication channel, and the Service Provider and Third Parties.
Hardware and Firmware: The Amazon Echo firmware is built on the Amazon Fire OS –a stripped-down version of the android OS–, and like other firmware is inherently vulnerable to attacks. There are even cheaper alternatives to the Echo that are also AVS enabled which might be even more vulnerable. Skilled hackers can exploit these vulnerabilities to wiretap devices, gain access to stored conversations, and even make unauthorized transactions.
AVS operation and communication channels: The operation of the AVS involves accurately interpreting voice commands by taking account of different languages, accents, tones, contexts and degrees of ambient clutter. This requires a lot more computational power than a single device can contain and thus most of the work is performed in the cloud. Although best practice implementations reduce the chance of exploitation, the mechanism of transmission still presents an opportunity for attackers to intercept or redirect data.
Service Provider and Third Parties: The handling of user data by service providers and third parties presents another point of data leakage. A. Alhadlaq, in his report “Privacy in the Amazon Alexa Skills Ecosystem”, wrote that 75% of Alexa skills lacked a privacy policy and that a significant percentage of those that had one had invalid ones or ones not customized specifically for Alexa. Even if Amazon handles user private data properly, it can still be vulnerable when handled over to one the third party providers for these skills.

As mentioned earlier, the spread of IPAs has been met with mixed reactions. There have been concerns, mainly from ’privacy-watchdog’ individuals and organizations that recognize the various points of privacy leakage while using the devices and know the significance of the issue. Dr. Michael Veale, a lecturer in digital rights and regulation at the UCL Faculty of Law, describes IPAs as “an infrastructure that can be later co-opted in undesirable ways by large multinationals and state surveillance apparatus, and compromised by malicious hackers”, and there have already been occurrences that suggest we are at this later point. It is no news that Amazon and Google have a business model that thrives on micro-target advertising. Personally I have had a few occurrences in which I have received advertisements on my phone for items I had previously looked up on another device and I have friends who have had the same experience, only they did not look-up the items but had earlier had conversations about them in the presence of their IPAs. Amazon has said that the AVS does not record or transmit any data until the wake word is heard and while they have been studies that corroborate this claim, there have also been situations in which Alexa has woken up accidentally. One study found that this was the case in one out of ten transcripts analyzed. To show how impactful this can be, take the example of the couple that had their private conversation recorded by their Echo and sent to a contact without permission.

In the past, all audio data processed by Alexa was permanently stored, this of course faced a lot of scrutiny so Amazon provided the option for users to personally review and delete their recordings, even with voice commands. Then there is the issue of the review of these recordings by third parties. Amazon admits to reviewing an ‘extremely small sample’ of requests in order to improve on the efficiency of the AVS. They also have provided users with the chance the to opt out of this review process but some will argue they do this knowing that most users of technology will never change defaults and that if they really were concerned about users’ privacy they will make the process opt-in rather than opt-out. In the Alexa privacy terms, they instead warn that deleting recording may degrade the user experience, twice. Their cleverly worded policies obscure the nature and extent of the data collection and the review process in a way that takes advantage of users’ attitude towards privacy. Most users are seeking convenience and simply do not know enough about the privacy threat to think it such big a deal. Other concerns that have arisen include the privacy of secondary and incidental users –housemates, family, friends, and visitors of users-. They did not agree to the terms of use that the AVS user agreed to but can still have their private data leaked. Also, there is the possibility of the always-listening microphone feature still being deployed to do some passive tracking even while it might not be transmitting. There are no technical obstacles to this, and all Amazon probably has to do is mask it as a convenient new feature or an improvement on a service such as ‘know when someone is at your house’ or ‘know what your kids are watching on TV’. This is technically constant surveillance and it might not be long until governments join in on the act for ‘the greater good of national security’. There have already been cases of Alexa recordings showing up in court as evidence although the FBI has refused to confirm or deny that it was using AVS for surveillance purposes.

The perception that Amazon, like Facebook and Google –the other data collection giants– like to give out to the media and anyone who is asking questions is that they are trying to realize the peak of convenience that technology can offer and that privacy breaches are an unplanned occurrence that might be necessary for this goal to be achieved. They frame data collection as a means to improve services and often downplay the immense commercial benefits. There are many who do not buy this though and believe it is a calculated business decision. Furthermore, there are concerns that with the amount of data being collected, companies like Amazon can go even go beyond knowing and predicting people’s actions in order to monetize them, to actually controlling the actions. Take the example of the smartphone game Pokémon Go which took the world by storm for a short period after it debuted in 2016. Its creators were able to obtain profitable deals with companies like McDonald’s and Starbucks by selling attractive virtual locations to the highest bidder. You go out thinking you are going to collect a rare Pokémon and you might do so, only you also collect a Big Mac or a medium Cappuccino along with it. Then there are malicious actors. IPA devices, as well as their operation mechanisms and channels present avenues in which users’ privacy can be violated by malicious hackers or crackers. The AVS is virtually a bank of private information and depending not only on the security mechanisms put in place by Amazon, but also third party service providers and the way in which they are utilized, the information can be accessed by these hackers.

The increasing adoption of IPAs is greatly due to the attitude of their users towards privacy. Most users are seeking convenience and are either unaware of, or seriously underestimate the risks involved in using them, with a few completely lacking privacy concerns. One study that included IPA users found that most of them either exhibited some trust in their service providers or government, feeling that they were not personally targeted or that the existing strategies for mitigation like privacy controls were sufficient. Lau et al.’s study performed to assess the awareness and usability of these controls and understand users’ privacy seeking behaviors found that while most users actually do fall into this category, a few never even considered privacy as an issue they should be concerned about, with some seemingly already resigned to the loss of control over their privacy. They also found that although many users were aware of the privacy controls, they were rarely used for privacy reasons. This was due to some users understanding that using them would negate the functionality of the device, some not really understanding how they work and some not thinking them enough to meet their privacy control needs, expressing a desire for proactive controls modeled like private browser windows that automatically clear your history at the end of a session, rather the retroactive ones currently available such as audio recording reviews.

Today, even with all these privacy concerns, the IPA industry is a growing one. Alexa is getting embedded into so many devices that in the near future she will be in every room of most houses. This means massive commercial gain for companies like Amazon who have found effective methods of monetizing this growth. Although they maintain that they value their users’ privacy and are doing their best towards protecting it, it is quite obvious much more can be done. Some viable suggestions include; firstly, on the basis of data collection, that they follow Apple’s example and make it an opt-in situation rather than opt-out, and also better integrate privacy controls into smart-speaker interaction. Also, the fact that there could be review of recordings by humans should be more advertised so all users are notified what they are getting into. Then, as technology continues to develop and processing power increases, more tasks could be performed on the devices directly rather than by remote servers. Then to handle the threat of malicious actors, manufacturers should ensure their hardware is built with better security measures against exploits by malicious actors such as signing their hardware with authenticated security certificates that would prevent unauthorized access to the firmware.
Another issue to be looked into sooner or later is the privacy tensions arise between primary, secondary, and incidental users as the growth of the IPA industry means that very soon it will be difficult for anyone not to fall into one of these categories even if they tried. With time hopefully, users will be better informed of the privacy risks involved but for now even when there is a privacy leak, the fact that most seem not to think of the convenience IPAs provide as something not worth exchanging their privacy for means there is the likelihood for them to take settlements or ‘hush money’.

In all, I would say the answer to the question, ‘Do IPAs mean the end for privacy?’ is one that is complex. I would say it is as ‘simple’ as knowing what the users will decide, if you can read minds that is.