Amazon’s Human Helpers Are Quietly Listening in on Some Alexa Recordings

The process is necessary because Alexa has limits to its ability to train itself, especially when it comes to garbled phrasing, accents, slang, regional words, other languages, and the like. Last year, Wired reported that “active learning” techniques in which the system identifies areas where it could improve via human assistance had “helped substantially cut down on Alexa’s error rates.” Wired wrote that adding in support for “transfer learning,” where Alexa tries to apply previously learned skills to new ones, has helped developers “ cut down on the grunt work they’d otherwise face.”

Newer is “self learning,” in which Alexa tries to pick up on context clues to understand commands which aren’t issued in a hyper-specific way (i.e., “Alexa, play 102.5 FM The Bone” vs “Alexa, play The Bone”). According to Wired, Amazon plans to eventually have Alexa recognize emotions of users, which critics have suggested could lead to manipulative marketing tactics. In an article in Scientific American last month, Amazon director of applied science Ruhi Sarikaya argued that such massive amounts of data will soon need analysis that voice recognition systems will have to switch from a “supervised” learning model “toward semi-supervised, weakly supervised and unsupervised learning. Our systems need to learn how to improve themselves.”

Bloomberg interviewed seven separate sources about the program, some of whom said Amazon’s workers are expected to analyze approximately 1,000 audio clips per nine-hour shift. Most of the time the work is “mundane,” Bloomberg wrote:

One worker in Boston said he mined accumulated voice data for specific utterances such as “Taylor Swift” and annotated them to indicate the searcher meant the musical artist. Occasionally the listeners pick up things Echo owners likely would rather stay private: a woman singing badly off key in the shower, say, or a child screaming for help. The teams use internal chat rooms to share files when they need help parsing a muddled word—or come across an amusing recording.