What should we do with our hands?
Mark Kubinec has taught intro chemistry to tens of thousands of students with his online videos for the University of California, Berkeley. Now he can add at least one computer algorithm to his list of trainees, too.
Kubinec (shown) joined a star-studded cast in helping a machine-learning algorithm study—and then predict—how people gesture while they speak (arXiv 2019, arXiv: 1906.04160). Comedians John Oliver, Conan O’Brien, and Ellen DeGeneres also lent a hand. Or, more accurately, visual data points corresponding to their hands, necks, wrists, elbows, and shoulders.
The key similarity between these subjects is that they all have hours of online video footage showing them mostly still, save for their gestures. This allowed the algorithm to study those gestures relative to each person’s audio. Then, when given a new audio clip from a subject, the algorithm could predict how that person might move. Now, it wasn’t a perfect match all the time, but it did outperform other prediction models, such as selecting a random pose from a person’s gesticular repertoire.
Kubinec started making his online general chemistry videos in 1999, when such a thing was unheard of. “Twenty years ago, we had to teach students how to use the internet,” he recalls. Now, he’s imagining using an algorithm that could automatically create an animated version of himself for future educational videos. Students still prefer having something speaking to them in a video rather than a disembodied voice, Kubinec says. Recording and editing audio and video for an actual person is time consuming, so using audio only could make things quicker and easier.
And Kubinec might get his wish sooner than he anticipated. Shiry Ginosar, the first author of the machine-learning study, is also at UC Berkeley. She tells Newscripts that her group is looking for subjects to work with using their 3-D scanner for making personalized avatars.
Dancing is for the birds
If you thought that was all the good news you would read coming from the marriage of science and online videos this week, think again. Neuroscientists studying Snowball the cockatoo and internet sensation have evidence that some parrots are born to dance.
About a decade ago, researchers led by Aniruddh D. Patel of Tufts University concluded that Snowball was the first animal other than humans to keep a beat to music. The list has grown since then. Sea lions, for instance, can step (well, gyrate) in time, too.
Now Patel and his colleagues—including Snowball’s owner, Irena Schulz—report that the cockatoo was trying out new moves that he hadn’t seen anyone demonstrate before (Curr. Biol. 2019, DOI: 10.1016/j.cub.2019.05.035). This suggests that, like humans, cockatoos possess an innate creativity and spontaneity, Patel tells Newscripts.
If you’re like us, you’ve spent countless hours watching animals dance online and are probably wondering what makes Snowball special. Well, when, say, a golden retriever dons a dress and dances with its owner, it’s doing so because it has been trained, probably using food, Patel explains. It’s a learned behavior with a physical reward.
Snowball, on the other hand, isn’t trained, and his rewards are social, coming from attention, encouragement, and perhaps the pleasure of moving to music. It’s kind of like a child first starting to dance. There’s no instruction, just some life-affirming bobbing and stomping along with the oohs and claps of an adoring audience.
Which does bring us to a word of caution. Although it’s undeniably fun to watch cockatoos dance, they are incredibly demanding. Schulz says they’re like 3- or 4-year-olds who live for 50 years, according to Patel. “Enjoy watching the videos and thinking about the science,” Patel says. “But think twice before getting a parrot as a pet.”
Matt Davenport wrote this week’s column. Please send comments and suggestions to firstname.lastname@example.org