iPhone | iPad
Android
Huawei
Another four countries have also been identified as deployment grounds for new projects
KT APPDOWNLOAD
UAE: Chinese scientist bands with robots to create music
Human, artificial intelligence collaboration sees bots ideate and deliver new forms of music
A Chinese computer scientist and professional musician is redefining the level of human-computer interaction in the creation of music.
Prof. Gus G. Xia, a visiting professor at Mohamed Bin Zayed University of Artificial Intelligence (MBZUAI), is using machine learning to make robots improvise and contribute ideas to deliver new musical work. If a human drummer or a pianist is unwell to take part in a performance, Prof. Xia will save the day with his AI robot.
“I use interactive intelligent systems and technologies to extend human musical expression and capabilities,” said Prof. Xia, who is currently an assistant professor in computer science at NYU Shanghai. He received his PhD in the machine learning department at Carnegie Mellon University, US, where he studied machine learning and computer music.
Prof. Xia’s research work equips the automatic accompaniment system with machine learning, robotic embodiment and improvisation on folk, pop or popular songs or melodies. The existing system of automatic accompaniment is where the computer is fed with a predefined score and once a person plays a piece, the system delivers a performance complementing the dynamics and tempo.
Robot AI musicians to improvise
Prof. Xia has been doing research on human-computer interaction for the past 10 years or so.
“There are several levels of application. In the first level, the computer knows what I am going to play. In real time, I may speed up or slow down, the algorithm tracks my tempo, and delivers the complement. In the second level, when I am doing my expressive performance, the machine can learn from the rehearsal to better anticipate my next movement. In the third level, there is robot embodiment. The robot can have its own body gestures and facial expressions according to my musical expression and timings,” said Prof. Xia, director, the Music X Lab.
Prof Gus Xia interacts with gestural expressions
Currently, Prof. Xia is taking it to the next level by enabling the system to improvise, merge its ideas with the musical played by a human to produce a new composition.
“In the fourth level, that I am doing now, is improvisation. In this case, the robot doesn’t fully know what I am going to play. It’s not a predefined score. It’s a predefined lead sheet (a form of musical notation) only with a raw sketch of what I am going to do. I may improvise and play an existing piece or something new, and based on that material, the robot will improvise. It will merge some of its own ideas to my music and deliver a new thing. There are different layers, and I am building the fifth layer.”
System to assist, not replace, humans
Prof. Xia’s research work involves trying out the system on piano, saxophone and flute. He uses algorithms, which are rooted in the field of computer vision and computer graphics.
“We can use algorithms to analyse music, to say what genre it is, where the piece, the chords, from very low level to high level.”
By creating such a robotic system, Prof. Xia aims to assist humans and to perform with them.
“I don’t intend to substitute humans with my system. But the system could be part of a band.”
He is optimistic that the AI robot system can take a request from the audience and play a tune.
Students at MBZUAI
Couldn’t afford a pianist, created a robot
Prof. Xia started learning music when he was 3 years old, started playing the saxophone at 10 years and the Chinese flute when he was 12. He was the prime soloist of the Chinese Music Institute of Peking University, China, and played a solo concert in 2010. His inventions were part of the exploits and experiments done during his PhD days.
“One of the reasons why I created this robot to do a complement for me is because during my PhD studies I couldn’t afford a professional for my performance and decided to invent my own. I am planning to hold a theme concert in Dubai next month. The system hopefully will be part of it.”
In the coming months, Prof. Xia will be interacting with students and doing research work at MBZUAI in Abu Dhabi.
ALSO READ:
More news from
The luxurious residence is attracting interest from affluent buyers worldwide
The sound first originated from a skull-shaped whistle found with a skeleton during a late 1990s excavation in Mexico
US pop culture for decades has dominated much of the globe although it has mostly grown without government support
Around 18,000 Gazans have permits from Israeli authorities to work outside the blockaded enclave
The Sharjah resident took home a whopping prize of Dh1 million after participating in Mahzooz for over a year
At least 100 killed and 150 suffered burns, smoke inhalation and crush wounds sustained in the stampede when nearly 900 panicked guests tried to flee the reception centre