We sat down with Rhizomatiks chief Manabe Daito to talk about his work with drones and augmented reality and how he is using them to push the boundaries of artistic expression forward.
Manabe Daito
Media artist, programmer, DJ, and executive at Rhizomatiks. Born in Tokyo in 1976. Attended the Tokyo University of Science before entering the Institute of Advanced Media Arts and Sciences. Cofounded Rhizomatiks after graduating in 2006. Since 2015, has also been running Rhizomatiks Research, the company’s creation and technology team. Creates works that utilize the human body in conjunction with programming, securing him various awards both in Japan and abroad. His personal website is www.daito.ws and the Rhizomatiks website is rhizomatiks.com.
Read in other languages
English
日本語
简体字
繁體字
Français
Español
العربية
Русский
During the closing ceremony of the Rio De Janeiro Olympics in August 2016, Japanese Prime Minister Abe Shinzō, dressed as Mario from the popular Nintendo series of video games, made a surprise appearance at the flag handover. This was followed by a live broadcast of performers in the Rio stadium overlaid with computer-generated images of the 33 events planned to be held at the 2020 Tokyo Olympics, making for a performance the likes of which the world had never seen.
A photo of the AR performance from the Rio de Janeiro closing ceremony, where Rhizomatiks combined elements of live performance with CG images of the 33 events to be held at the next Summer Games in Tokyo. (Photo courtesy of Rhizomatiks Research)
The team behind the AR production was Rhizomatiks and Manabe Daito, one of its cofounders. The company, launched in 2006, tackles everything from software programming to hardware architecture, user interface design, and the artistic aspects that productions like these call for. Rhizomatiks Research, which he helms, is the research and development branch of the organization.
Rhizomatiks has made itself known throughout the entertainment industry through its contributions to projects including the world’s first 360º virtual reality concert, streamed in real time for the Icelandic singer Björk, and performances by the Japanese techno-pop band Perfume, which relied on Rhizomatics for the technical support necessary to coordinate lasers and drones. As part of his dizzyingly varied career, Manabe also produces artwork on his own and has won awards at several international media art and advertising festivals.
“People sometimes ask how I keep my artistic endeavors and entertainment industry work walled off from each other. Sure, there are certain projects that I’m only be able to achieve working solo as an artist. But there are also projects that wouldn’t be feasible without the backing of corporations or public institutions. And there are some things initially conceived as pure art projects that, through advancements in technology, eventually come into common use all over the world five to ten years down the line. One example of this would be games played on touch screens. But media art that relies on little more than technological novelty is destined to be consumed quickly and to fall into obsolescence after that novelty has worn off.”
From Experimental Art to Entertainment
Technological leaps like big data, the Internet of things, and artificial intelligence are all drastically changing the way society functions, and Manabe is eagerly applying these concepts to his own work. One project that helped to propel him into the public eye was a series of works in which he affixed medical-use electrodes to his own face, and then to his friends, and electrically stimulated the facial muscles. The videos of this experimental work that he uploaded to YouTube proved to be a turning point in his career.
Manabe Daito’s 2008 Electric Stimulus to Face—Test 4. (Video courtesy of Rhizomatiks Research)
“While I was at the Institute of Advanced Media Arts and Sciences, I started experimenting with expressing myself through programming. Still, I wasn’t ever able to show off my work on any platforms that garnered significant attention from the public. There is, after all, a limit to the audience you can reach by holding exhibitions and distributing DVDs—so for me, YouTube was a big step up. As soon as I posted a video, people all over the world were talking about it in what seemed like minutes.”
And with that, all of a sudden Manabe found himself getting offers for jobs and collaborative projects from all around the world.
“Still, developing new business fields really isn’t my goal here. The reason I produce art is to shine a spotlight onto the possibilities and the dangers that new technology brings with it and to fuel discussions on those issues at a societal level.”
What, then, brought Manabe’s experimental efforts into their close juxtaposition with mainstream entertainment today?
“From around 2004 to 2009, I helped out with a production that involved Fujimoto Takayuki of the artist collective Dumb Type, a group known for performances melding video, sound, and contemporary dance. It was then that I invited the director and choreographer Mikiko, with whom I’d wanted to collaborate for a while, to a Dumb Type show. We started working together after that, when Mikiko came to us and said: ‘For Perfume’s next gig, I want to do something that uses technology.’”
That idea eventually coalesced into Perfume’s 2010 Tokyo Dome performance, which included balloons floating around the stage being shot down with lasers, members of the group being scanned in 3D using industrial-grade cameras and projected during the show, and numerous other technological displays that took fans completely by surprise. News of the production quickly made its way to the ears of other cutting-edge creators, causing quite a stir despite this community being one that typically showed little interest in the mainstream entertainment world.
“Since we were using new technology that had no precedent of being used in a live performance setting, we had to run tests over and over to make sure it would function. We couldn’t get this sort of work just by pitching our ideas to concert organizers. But once we got a successful run under our belt, then other outfits opened up to the idea of letting us do a show for them. It’s our accumulation of small successes that has led us to these big productions.”
On the World Stage at the Olympic Games
It was in this fashion that Manabe’s works, combining facets of media art with experimentation and the artist’s message, sought to achieve a greater physicality, one that needed to be physically experienced. Manabe and company moved on to their next experiment, again collaborating with Mikiko—this time working not with Perfume, but with her female-only dance company ElevenPlay.
“One thing I tried was using sensors to pick up on dancers’ movements and then reproduce those movements with a flock of drones,” explains Manabe. One experiment involved putting audience members in personal mobility units with VR headsets, transporting them in between the realms of the real and the virtual—an experiment in combining visuals and sound, hardware and software, and the human body itself. “The experience I gained through that project helped me when it came time to put on the live performance for the Rio De Janeiro Olympics.”
Given the international prestige of a global performance platform like the Olympics, failure is simply not an option. To try and bring any kind of live performance elements onto that stage carries with it an enormous risk. But that is just what Manabe and his team did, creating a performance that used augmented reality by sensing motions in real time, combining that data with the choreography of dancers and flickering three-dimensional frames. To synchronize all of these elements, he would need a system architecture allowing extreme precision and detailed simulation.
“The performance for the Olympic closing ceremony was unprecedented,” says Manabe. “After Prime Minister Abe took the stage, we used AR visuals to show the thirty-three events that will be held at the Tokyo Olympics. In order to really create the sense of atmosphere we needed, it had to be done in real time. Just pressing play on some CG videos that had already been created didn’t interest me. This was an event that people all over the world were watching live, and if the visuals themselves weren’t live, it seemed pointless. It wasn’t simply about using new technology—I wanted to bring people the context of a live performance, something that would really move their hearts and minds.”
In the end, the Olympic presentation was a success, giving both Manabe and Rhizomatiks a jolt of global fame. His works are also increasingly featured on television, with each showing inspiring a wave of social media reactions that sometimes border on the bewildered: “That was amazing, but I have no idea what it was that I just watched.”
“I’m not surprised when I hear that kind of reaction,” laughs Manabe. “For the past several years, I’ve been pursuing the idea of making people unsure of where the line lies between reality and virtual reality. What is really interesting to me is how people’s posts on social media outlets have shown their growing awareness of cutting-edge technology. During NHK’s 2017 New Year’s Eve Kōhaku Uta Gassen music program, they had lights in the windows of buildings in Shibuya, dancing up and down like an equalizer to the tune of Perfume’s music. People were out there wondering, ‘Is this really happening? Is this VR?’ They were taking photos and investigating it and posting about it all over social media. It’s these kinds of things that I find interesting.”
To Move People’s Hearts and Minds
Though his work is steadily earning him public recognition, Manabe has recently started to feel that he wants to get back to his guerrilla art roots again.
“In Japan, there are always budgetary restrictions on what you can do, and there’s a trend away from truly adventurous pursuit of artistic expression. China is the scene for true guerrilla-style artistic expression these days. Chinese people aren’t shy when it comes to imitating past artwork. So, in that vein, I sometimes get offers from people in China stating, ‘We tried to imitate your work but were unable to reproduce it, which is why we would like to collaborate with you.’” [Laughs] “Still, in terms of the amount of energy poured into new artistic expressions and the speed at which art gets created, I think no one in Japan can rival the Chinese when it comes to this.
“On the other hand,” he notes, “there are European and American companies that are working to try and produce artistic expressions like ours—but with teams of several hundred people, which gives me cause for concern.”
New technologies are being born one after the other, fading just as quickly into obsolescence. This rapid pace of development is accompanied by a rise in the population of “digital artists” putting these latest trends to work in their creations. What drives Manabe to keep innovating in the face of this increasingly fierce global competition?
“It’s probably my desire to create things that move people. Still, that’s a difficult thing to do. The kind of artistic expression that my team and I are after differs fundamentally from a technological demonstration. When you’re just demonstrating your research results, all you need to do is show off the specs of what your tech can do. When it comes to art and entertainment, though, you need to think about how it can become a form of expression.”
And it isn’t only about the latest technology, notes Manabe. “You don’t necessarily need to do something that’s difficult from a technical standpoint. For instance, our 2010 piece Fade Out was created using lasers purchased from an online auction site for relatively low cost, and the idea behind it was really simple. Really, the most difficult part of the process is coming up with things that nobody would’ve thought of and then realizing them.”
The 2010 work 405nm Laser Fade Out Test 2 (Manabe Daito and Ishibashi Motoi). Lasers are used to illuminate a screen coated in luminescent paint. By carefully controlling the order in which the lasers are fired at each part of the screen and making use of the paint’s property of dimming over time, the artists create an image that gradually grows in complexity and detail. (Video courtesy of Rhizomatiks Research)
On the polar opposite end of the spectrum, Manabe is also interested in the human body and has been taking dance lessons to learn more about the physical potential of humans. He is also working on an experiment one might call “the ultimate guerrilla art piece.”
“I’m excited to say that we’ve finally started experimenting with a concept that I’ve been interested in for a while now: running electricity through my brain,” explains the artist. “With the help of physicians, I hope to use TMS, or transcranial magnetic stimulation, to halt activity in the language center of the neocortex and see what kind of effect that would have on speech. At some point I think I’d like to install electronic microchips in my brain, but I’ll never be able to find out what sorts of interesting possibilities that holds until I try it myself,” he laughs.
“My goal is to keep at my research and experimentation research so I can continue creating works that won’t lose their freshness, no matter how much technology progresses.”
(Originally published in Japanese on March 15, 2018. Interview and text by Fukasawa Keita. Photographs by Ōkōchi Tadashi.)
A photo of the AR performance from the Rio de Janeiro closing ceremony, where Rhizomatiks combined elements of live performance with CG images of the 33 events to be held at the next Summer Games in Tokyo. (Photo courtesy of Rhizomatiks Research)
A photo of Perfume’s March 17, 2015, performance at SXSW in Austin, Texas. (© Amuse Inc., Universal Music LLC, Rhizomatiks, and Dentsu Inc.)
