News & Events

Inventing for the simulation world

Professor Taku Komura, Professor of Department of Computer Science

Developing state-of-the-art animation/simulation techniques has been the dedicated pursuit of Professor Taku Komura whose research career took him to both the UK and Hong Kong.

He had worked at the University of Edinburgh for 15 years before he joined the University of Hong Kong (HKU) as a Professor at Department of Computer Science in 2020. He greatly appreciates the strong tradition of research in both places, and is happy to continue his work on character animation and 3D modelling back in Asia. Born in Japan, he received his BSc, MSc and PhD in Information Science from the University of Tokyo.

“Life in the UK was great, and I received a lot of support from my department and colleagues there, resulting in a lot of fruitful outputs.  But the food and culture in Hong Kong were what I was always missing,” said Professor Komura, who had worked at the City University of Hong Kong and RIKEN in the early 2000s. In Edinburgh, his main research interests were on physically-based animation and the application of machine learning techniques for animation synthesis. The generous support for research distributed through various channels such as special grants to universities, hardware funding, PhD studentships, was very encouraging to him. In 2014 and 2017, he received the Royal Society Industry Fellowship and the Google AR/VR Research Award respectively.

There were also lots of collaboration and discussions with researchers in robotics, computer vision, computational linguistics and computer architecture at the University of Edinburgh’s School of Informatics. The open, friendly discussions among fellow colleagues and students helped nurture new research directions in him, he noted.

Different collaborations

In Hong Kong, he has found new opportunities for collaborations with the industry, through projects with giant firms such as Meta, Huawei, Tencent and Softbank. He is also involved in international collaborations here such as the Centre for Transformative Garment Production (TransGP), which is jointly formed by HKU and Tohoku University to tackle real-world technical problems facing the garment production industry. The Centre has been admitted to the AIR@InnoHK Research Cluster with funding support from the Government’s Innovation and Technology Commission.

TransGP strives to leverage proprietary AI and robotics technologies to enhance the operations and output of the garment sector, including automated sewing and handling capabilities, shorter product development cycles, and improved efficiency.

Project collaboration with Tencent

Professor Komura is researching into robots that can help improve work safety and spare human labour from laborious or dangerous procedures. Issues concerning workers’ welfare are given much attention in Europe, he says. Ultimately, the operating costs of companies would rise if no improvement is made in the work environment.

The same concern for workers’ protection during the manufacturing process deserves equal if not more attention in Asia, he said, adding: “Robots provide a good alternative to the traditional reliance on intensive human labour in the garment industry.”

Drawn to the virtual world as a teen

Since he was in secondary school, he has been interested in alternatives, or more precisely, simulation characters, albeit not in the workplace. Like many teens, he was drawn to playing computer games, especially Namco games like Xevious, Gaplus, PacLand, Dragon Buster. Writing game programmes with his ATARI computer was one of his favourite hobbies.

That was the reason he decided to major in computer science at university. His other hobby back then was boxing – he was ranked 10th once in the Japanese National Ranking. Also a fan of martial arts, his participation in a university lab inspired his later pursuits.

Article about Professor Komura’s boxing achievement when he was young

“I was fascinated by a long-term project at the lab to analyse the movements of Shorinji Kempo fighters (a Japanese version of martial arts based on the Shaolin Kung Fu). We captured the motions of fighters using motion capture devices which were rare at the time.  This has led me into the area of character motion synthesis for animation purpose.”

Some of his character animation techniques have been adopted by the computer game industry for real-time character control. He achieved a  breakthrough through the invention of deep learning techniques to learn from motion capture data. His team was actually among the first in the world to use the deep learning approach.

 “Such data-driven character animation is one of the main streams in computer graphics nowadays. We have been developing techniques for full-body motion synthesis for biped (e.g., humans) and quadruped characters (e.g., dogs), collective sports like basketball and football, body-environment interaction, hand-object manipulation, facial animation, etc.,” he said.

Techniques for full-body motion synthesis for collective sports

Last year, his technique for character motion synthesis received a Best Paper Award from SIGGRAPH, the annual event launched in 1974 to foster and celebrate innovations in computer graphics and interactive techniques.

His other area of expertise is physical simulation needed for analysing various natural phenomena, product design, and computer animation. He regards it as his mission to educate and train more students in the important discipline, with the goal to “grow a good team for physical simulation and to do interdisciplinary research with engineers, doctors and dentists.”

Professor Komura received a Best Paper Award from SIGGRAPH

Already he and his team have developed techniques that are fastest in the world for finite element simulation and collision detection. Such techniques are useful for designing clothes, or enhancing the effectiveness  of virtual try-ons at clothing stores or factories, VR experience or real-time computer games.

There is room for much more research to be done. The present quality of virtual try-on of clothes is not very good, he said. “You stand in front of the camera and want to see how you look in a certain outfit. Companies want to make the process look more realistic as if you have really put on the clothes.”

Virtual try-on of clothes

Diverse applications

Researchers worldwide are seeking to achieve precise, high quality simulations, from simulations of deformation of the face to muscle, bodily movements, for applications in diverse areas. “Once you have a model, you can apply it,” said the professor.

For orthodontics asked to carry out a teeth alignment, for example, a successful model will enable them to better plan the treatment beforehand by scanning patients’ teeth, facial expressions, and be able to visualise to  them how they will look after receiving the treatment. “Patients can see how their look will improve as a result of the procedures.”

There are wide applications in the medical field. Simulations of a certain part of the body can be used to help show to patients how their body function or movement will improve upon receiving treatment.

“Say if there is an operation on the muscles or bones, a doctor can project what kind of result the treatment can give,” he explained.

Virtual surgery is another beneficiary of the top-notch technique, which can be adopted for the training of medical students.

In Hong Kong and elsewhere, one focus of the latest research is on reproducing variations of human interactions, as opposed to mere generalisations of human movements as what has already been achieved, and diverse scenarios such as a human being falling off from somewhere or responding after getting injured. “It’s about trying to make the experience novel to users; that's what the industry wants.”

The highly lucrative computer game industry is certainly eager to tap into existing expertise to make the games more enticing. The industry is popular with computer science graduates and has already hired many for its growth and development, said Professor Komura.

Ample room for research

He sees no end to attempts either within the industry or on campuses to create simulations appearing exactly like real-life objects or capturing an increasing array of human functions. All it takes could be time and endless amount of work. “We have these devices to capture the human motion where you put these markers on the body and you can digitise the movement, then based on this you can collect lots of data, be it facial animation or facial motion or full body motion and then you can learn how the humans move. Once you have learned that you can create the animation of the humans in movies or computer games.”

Simulation of breaking a statue using extended finite element method (XFEM)

Now supervising six doctoral students, he hopes to continue with his research till he retires. “It is an exciting time now; you can learn lots of things just from the data. And there are lots of things to do for simulation, for example, trying to improve the algorithm so that it is faster and more accurate. Further improvement can also drive down cost.”

Scientists have used the now popular concept of neural networks to help computers learn about different phenomena. One end goal could be to construct a realistic virtual human that can speak and move like real humans. But to Professor Komura, the process of constantly learning, making new discoveries through collaboration with students, colleagues and teaching is already satisfying.  “I wish to continue developing state-of-the-art animation/simulation techniques that can produce impressive results.”