Big Tech climate plans need to tackle business travel, flights
This ultimately resulted in the Amazon Glow, a unique device for kids that combines an 8-inch LCD screen designed for a traditional video calling experience with an additional 19.2″ projected touch screen that lets children manipulate objects, incorporate physical play pieces and even “scan” their own toys. The Glow allows kids to read books, play games and create art together with grandparents and other trusted adults participating from afar via mobile devices.
The Amazon Glow combines an LCD screen with an interactive projector.Image: Amazon.
The Amazon Glow launched as an invite-only early access product last year and became widely available in the U.S. at the end of March. It’s the first interactive projection device sold by Amazon, and it could be a stepping stone for the company to use the technology in other areas, including next-generation smart home devices, or even the enterprise.
It’s also a fascinating story of trial-and-error development, and small, scrappy teams ideating inside of giant corporations. “We knew what we wanted to do, but we didn’t really have an idea how to do it,” said Amazon Senior Hardware Engineer Martin Aalund, a founding member of the Glow team.
Kids are terrible camera operators
The Amazon Glow was first announced at the company’s hardware event last fall. Amazon also used the annual showcase to announce countless other new and upgraded devices, but the Glow stood out, both for its technology as well as its intended purpose: Amazon built the Glow to give grandparents and other remote family members and caregivers a chance to be more involved in the lives of children ages 3 to 9. It’s a need that became very obvious during the pandemic — a time that also demonstrated how traditional video calling solutions weren’t up to the task in a more remote world.
“They’re designed for conversation between adults, not for playing with children,” said Behrang Assadi, head of Marketing for Amazon Glow. “Kids are terrible cameramen. They tend to not sit still. They get easily distracted by other features of phones, and sometimes they just put it down and walk away.” Aalund agreed. “You get good pictures of the floor and the ceiling and up the child’s nose, but very rarely of what you’re interested in,” he said.
Sketch of a first prototype: a robot that would follow the child around during video calls.Image: Amazon
Aalund’s team wanted to overcome those challenges and allow remote callers to play a more active role, despite potentially being thousands of miles apart. “We wanted the adult to be able to reach into that child’s space and have a meaningful interaction [as if] they’re in the same room,” Aalund said.
After formulating that end goal in a white paper, the Glow team began with weekly design sprints, toying with a range of ideas and building prototypes to test them out. “We looked at using cranes and robotic arms to grab objects and move them around,” Aalund said. Then, the team had a first of many lightbulb moments: Why build robot arms if you already have a pair of human arms at your disposal? “We got some early feedback [telling us that] we can use the child as a manipulator,” he said.
Still, the team wasn’t ready to give up on the idea of building a robot altogether. The thinking was: If kids weren’t able to stay in one place, then why not follow them around? “I went out and bought a bunch of VEX Robotics kits, and we built up little prototypes and drove them around,” Aalund said. “We created a little mobile robot platform and drove that around with a camera on it.”
From lasers to flashlights to projectors
The team soon discovered, however, that a robot, especially one that followed a child around, came with its own set of issues. “We realized that there’s challenges with a remote person navigating [the robot], especially in a real home that may have stairs, obstacles and clutter on the floor,” Aalund said.
Of course, that’s not a new problem to anyone looking to build consumer robots — Amazon included. “We later discovered there was another team working to solve a lot of those problems,” Aalund said. “But that’s the Astro story.”
With robots out of the question, the team began looking at alternatives. One idea involved adding a camera that could follow the child around the room with a pan and tilt mechanism while keeping the device itself in place. Add a laser pointer, and you could even play I-Spy with the child. “We thought that was a pretty cool concept, but we were concerned with the laser and eye safety,” Aalund recalled.
Another Glow prototype: a giant projector.Image: Amazon
The team briefly considered a flashlight as an eye-safe alternative, but struggled to get it to work with its existing prototype hardware. “The little handheld mechanism we had wasn’t really strong enough to move a flashlight around,” Aalund said. So he decided to fake it. “We can simulate this,” he recalled thinking. “Let’s just grab a projector, shining down on the floor where we’re playing games, and let’s just create a flashlight and move it around the playspace to highlight things.”
The projector was initially just meant to be a quick hack to save a prescheduled demo — but the results were so compelling that the approach quickly became a centerpiece of the Glow hardware. “It set off a lightbulb,” Aalund said. “We could read books, we could actually play games where the remote person has virtual pieces and the child has physical pieces.”
Testing it with kids, getting approval from parents
Developing the Glow included a lot of tests with children and their caregivers, which helped refine the projection technology. At first, the Glow team was thinking about just using a traditional projector. “Initially, it was primarily for that remote person to reach in and augment the child’s environment, to move light around, to move virtual pieces,” Aalund said. “[When] we put it in front of some kids, the first thing they did was try [to] pinch and zoom and touch and interact.”
That’s how the team realized it had to make the projector interactive, and effectively turn the projected space into a touch screen. “This was actually fairly far along in our development process,” Aalund said. The team ultimately settled on using an IR camera for hand tracking that could be used in a variety of lighting conditions, including the light emitted by the projector.
A lot of the testing of early Glow prototypes involved make-believe, according to Aalund. “We did a lot of ‘Wizard of Oz,’ man-behind-the-curtain things,” he said. “Rather than actually detecting what the child was doing, we would sit in another room. When we saw the kid touch a spot, we’d click a key and have it change what was displayed.”
Early on, the Glow team was playing with the idea to make the device look like a stuffed animal. Parents were not amused.Image: Amazon
Testing the device with kids required ingenuity. Getting buy-in from parents was just as challenging, albeit for very different reasons. Early on, the Glow team was playing with the idea of making the device look like a toy itself. “[For] our very first prototype, we actually took an Amazon [Echo] Show and put it inside a stuffed animal,” Aalund said.
There were some thermal issues with that approach, and having a stuffie that doubles as a projector also didn’t seem the best idea for good sleep habits. However, the biggest pushback came from parents. Anything that would look like a toy would have to live in the kid’s room because it clashed with their living room design, some of them argued. Yet others objected to that approach, because they didn’t like the idea of getting judged by their relatives if their kid’s room was messy. In the end, the Glow team settled on a more neutral hardware design.
Next up: Music, coding and … elder care?
While a lot of effort went into building the physical product, the team also worked hard on the Glow’s apps and games, which had to be optimized for both the projector and remote interaction. “We weren’t able to just take apps and games that live on existing mobile app stores,” Assadi said. “We ended up building most of it from scratch.”
The result includes games like Go Fish, chess and whack-a-mole, some of which can only be played together with a remote participant. That focus on making the games part of a call was intentional, according to Aalund. “We didn’t want this to be another game console or content consumption device,” he said. The Glow also includes access to picture books and chapter books, some of which offer some interactivity.
One of the more unique experiences gives children the ability to scan real-life objects, and then incorporate them into their digital art. Amazon also sells Tangram puzzle pieces that can be used to solve puzzles, and the company has plans to launch additional add-on packs that further allow users to merge physical with digital play, starting with coding bits. Also on the horizon: music games. And eventually, developers may even be able to build their own apps for the platform.
The Amazon Glow is optimized for children interacting with remote caregivers, but its technology could also be used for many other purposes.Image: Amazon.
This also opens up the opportunity to use the Glow for other target audiences. “There are some really interesting applications,” Assadi said. “My mind goes to senior assisted living homes, where senior adults want to interact with a caregiver, whether it’s local or remote, and play games like Bridge.”
Ultimately, Amazon may even decide to take some of the lessons it learned from developing the Glow and incorporate projection into devices with other form factors. “We’ve had people within the company ask us about our hardware and software stack for all sorts of different things,” said Aalund. “From office environments to video conferencing, where this could make a great whiteboard, to tutoring. All sorts of different things.”
Amazon did explore using projection mapping for AR-like experiences in the past, and may be getting ready to revive some of those ideas, if recent job listings are any indication. Chances are that those ideas never amount to anything. Or perhaps they’ll take on a life of their own, and turn into something completely different.
That’s perhaps the biggest lesson from the development of the Amazon Glow: Even in a gigantic company like Amazon, there are small, startup-like teams that just throw ideas against the wall and see what sticks. Experiment with DIY robotics kits, wing a few demos, until finally a lightbulb goes off and something compelling emerges.
“We were very scrappy,” Aalund said.
