• About Us
  • Contact Us
  • Today Headline
  • Write for us
Today Headline
No Result
View All Result
  • breaking news today
    • Politics news
    • Sports
    • Science News & Society
  • Entertainment News
    • Movie
    • Gaming
  • Technology News
    • Automotive
    • Software & IT
  • Health News
    • Lifestyle
    • Insurance
  • Finance News
    • Money
  • Enterprise
  • Contact Us
  • breaking news today
    • Politics news
    • Sports
    • Science News & Society
  • Entertainment News
    • Movie
    • Gaming
  • Technology News
    • Automotive
    • Software & IT
  • Health News
    • Lifestyle
    • Insurance
  • Finance News
    • Money
  • Enterprise
  • Contact Us
No Result
View All Result
TodayHeadline
No Result
View All Result

A technique to teach bimanual robots stir-fry cooking

June 20, 2022
in Technology News
Reading Time: 5 mins read


A technique to teach bimanual robots stir-fry cooking

The proposed decoupled coordination framework comprises two parts: offline learning from demonstration and online task execution. In the offline process, the dual arms are decoupled into different roles and treated separately by classical and neural network methods. Here we adopted a Dynamical Movement Primitive (DMP) representation and proposed a spatio-temporal neural network — Structured-Transformer for learning the coordination. In the online process, the left-arm motion is adjusted according to the visual feedback, and the corresponding right-arm motion is generated by the pre-trained Structured-Transformer model based on the left-arm motion. Credit: Liu et al.

As robots make their way into a variety of real-world environments, roboticists are trying to ensure that they can efficiently complete a growing number of tasks. For robots that are designed to assist humans in their homes, this includes household chores, such as cleaning, tidying up and cooking.

Researchers at the Idiap Research Institute in Switzerland, the Chinese University of Hong Kong (CUHK) and Wuhan University (WHU) have recently developed a machine learning-based method to specifically teach robots to master stir-fry, the Chinese culinary cooking technique. Their method, presented in a paper published in IEEE Robotics and Automation Letters, combines the use of a transformer-based model and a graph neural network (GNN).

“Our recent work is the joint effort of three labs: the Robot Learning & Interaction group led by Dr. Sylvain Calinon at the Idiap Research Institute and the Collaborative and Versatile Robots laboratory led by Prof. Fei Chen Cuhk and the lab led by Prof. Miao Li at WHU,” Junjia Liu, one of the researchers who carried out the study, told TechXplore. “Our three labs have been studying and working together for about ten years. We have a particular interest in making intelligent robots that can prepare food for people.”

Dr. Calinon, Prof. Chen and Prof. Miao have been trying to enhance the cooking skills of robots for several years now. In their recent study, they decided to focus on the Chinese culinary arts, specifically stir-fry, a cooking technique that entails frying ingredients over high heat while stirring them, generally using a Wok pan.

“While domestic service robots have been developed considerably in recent years, creating a robot chef in the semi-structured kitchen environment remains a grand challenge,” Liu said.

“Food preparation and cooking are two crucial activities in the household, and a robot chef that can follow arbitrary recipes and cook automatically would be practical and bring a new interactive entertainment experience.”

A technique to teach bimanual robots stir-fry cooking

Structured-Transformer is a bimanual coordination mechanism that regards coordination as a sequence transduction problem and learns coordination by embedding a Graph Convolutional Layer halfway through the Transformer decoder to explicitly consider the robot hardware structure. Credit: Liu et al.

Stir-fry, the cooking style that the team focused on in their recent paper, involves complex bimanual skills that are difficult to teach to robots. To effectively do this, Liu and his colleagues first tried to train a bimanual coordination model known as a “structured-transformer” using human demonstrations.

“This mechanism regards coordination as a sequence transduction problem between the movements of both arms and adopts a combined model of transformer and GNN to achieve this,” Liu explained. “Thus, in the online process, the left-arm movement is adjusted according to the visual feedback, and the corresponding right-arm movement is generated by the pre-trained structured-transformer model based on the left-arm movement.”

The researchers assessed their model’s performance both in simulations and on a physical two-handed robotic platform, known as the Panda robot. In these tests, their model allowed the robot to successfully and realistically reproduce the motions involved in stir-fry.

“The main contribution of this paper is to consider the coordination mechanism of bimanual robots explicitly in the form of sequence transduction,” Liu said. “Compared with classical learning from demonstration methods and deep learning/reinforcement learning based methods, our decoupled framework skillfully combines both these techniques. In fact, it can have both the generalization of the former and the expressivity of the latter.”

In the future, the model introduced by this team of researchers could enable the development of robots that can cook meals both in home environments and at public venues. In addition, the same approach could be used to train robots on other tasks that involve the use of two arms and hands. Meanwhile, Liu and his colleagues plan to continue working on their model, to improve its performance and generalizability.

“We will now introduce higher dimensional information to learn more humanoid motion in kitchen skills, such as visual and electromyography signals,” Liu added. “The estimation of semi-fluid contents in this work was simplified as two-dimensional image segmentation, and we only used the relative displacement as the desired target. Thus, we also plan to propose a more comprehensive framework that consists of both the movements of bimanual manipulators and the state change of the object.”


A beaver-inspired method to guide the movements of a one-legged swimming robot


More information:
Junjia Liu et al, Robot Cooking With Stir-Fry: Bimanual Non-Prehensile Manipulation of Semi-Fluid Objects, IEEE Robotics and Automation Letters (2022). DOI: 10.1109 / LRA.2022.3153728

© 2022 Science X Network

Citation:
A technique to teach bimanual robots stir-fry cooking (2022, June 17)
retrieved 20 June 2022
from https://techxplore.com/news/2022-06-technique-bimanual-robots-stir-fry-cooking.html

This document is subject to copyright. Apart from any fair dealing for the purpose of private study or research, no
part may be reproduced without the written permission. The content is provided for information purposes only.

  • Trending
  • Comments
  • Latest
fe8f40fd 12bf 4847 8e61 6d0dca7136fe gettyimages 899764870 – TodayHeadline

Cleaning A Kid’s Car Seat? Don’t Soak The Straps, Says Safety Advocate

GettyImages 1494618254 – TodayHeadline

Roland Garros live scores 2023: Iga Swiatek updates

treatment creates stee scaled – TodayHeadline

Treatment creates steel alloys with superior strength and plasticity

Sackler familys legacy of pain Hundreds of thousands dead and – TodayHeadline

Sackler family’s legacy of pain: Hundreds of thousands dead and millions addicted to opioids

49023281 11cc 4974 8431 9458d96eba06 – TodayHeadline

Standard Chartered, PwC make case for programmable CBDC in China Greater Bay Area

'My Airbnb host threw me out and accused me of being a sex worker - I'm traumatised'

'My Airbnb host threw me out and accused me of being a sex worker – I'm traumatised'

833287c07ce774e94e3274cd66ee5033 – TodayHeadline

Africa’s week in pictures: 26 May-1 June 2023

e006cbd8 1afd 4a58 8f4a 4515c562c61f – TodayHeadline

CrowdStrike: slowing growth makes for cyber insecurity

PopularStories

49023281 11cc 4974 8431 9458d96eba06 – TodayHeadline
Money

Standard Chartered, PwC make case for programmable CBDC in China Greater Bay Area

'My Airbnb host threw me out and accused me of being a sex worker - I'm traumatised'
Politics news

'My Airbnb host threw me out and accused me of being a sex worker – I'm traumatised'

833287c07ce774e94e3274cd66ee5033 – TodayHeadline
breaking news today

Africa’s week in pictures: 26 May-1 June 2023

e006cbd8 1afd 4a58 8f4a 4515c562c61f – TodayHeadline
Technology News

CrowdStrike: slowing growth makes for cyber insecurity

About Us

Todayheadline the independent news and topics discovery
A home-grown and independent news and topic aggregation . displays breaking news linking to news websites all around the world.

Follow Us

Latest News

49023281 11cc 4974 8431 9458d96eba06 – TodayHeadline

Standard Chartered, PwC make case for programmable CBDC in China Greater Bay Area

'My Airbnb host threw me out and accused me of being a sex worker - I'm traumatised'

'My Airbnb host threw me out and accused me of being a sex worker – I'm traumatised'

833287c07ce774e94e3274cd66ee5033 – TodayHeadline

Africa’s week in pictures: 26 May-1 June 2023

49023281 11cc 4974 8431 9458d96eba06 – TodayHeadline

Standard Chartered, PwC make case for programmable CBDC in China Greater Bay Area

'My Airbnb host threw me out and accused me of being a sex worker - I'm traumatised'

'My Airbnb host threw me out and accused me of being a sex worker – I'm traumatised'

833287c07ce774e94e3274cd66ee5033 – TodayHeadline

Africa’s week in pictures: 26 May-1 June 2023

  • Real Estate
  • Parenting
  • Cooking
  • NFL Games On TV Today
  • Travel and Tourism
  • Home & Garden
  • Pets
  • Privacy & Policy
  • Contact
  • About

© 2023 All rights are reserved Today headline

No Result
View All Result
  • Real Estate
  • Parenting
  • Cooking
  • NFL Games On TV Today
  • Travel and Tourism
  • Home & Garden
  • Pets
  • Privacy & Policy
  • Contact
  • About

© 2023 All rights are reserved Today headline