Ludovic Righetti

This paper was a finalist for the Best Paper Award on Mobile Manipulation at IROS 2023. It is one of 5 finalists out of 2760 submitted papers at one of the largest robotics conferences in the world.

In the field of robotics, the intricate dance of planning how machines touch and maneuver objects is a linchpin for granting them the autonomy to execute intricate tasks. However, this endeavor remains a challenge. The pursuit of identifying dynamically viable sequences of contacts between robotic manipulators and objects often unravels into a web of formidable combinatorial and nonlinear complications.

Consider the ostensibly simple act of reorienting an object nestled on a table using a two-fingered robotic hand. The strategic planning of contacts necessitates a thoughtful consideration of interaction forces. For instance, a cube succumbs to rotation through forces applied from its sides. In stark contrast, when faced with a slender plate, the robotic fingers must exert a downward pressure — initiating a frictional interplay — to achieve a comparable outcome.

The crux of the challenge lies in the nuanced orchestration of interaction forces and judicious contact switches. As the robotic fingers approach their kinematic limits, a process of breaking and re-establishing contacts becomes imperative to propel the object into further rotation. These twin challenges, encapsulated in interaction forces and contact switches, underscore the complexity inherent in planning the manipulation of objects.

Over the past decade, trajectory optimization has emerged as the favored approach for orchestrating multi-contact motion planning. This methodology holds sway due to its capacity to construct efficient formulations for navigating the intricate terrain of interaction forces. Yet, a lingering problem persists—the effective integration of planning for contact modes remains elusive, primarily attributed to its discrete nature, injecting a disruptive discontinuity into the dynamics at the crucial juncture of contact switches.

Now, a team of NYU researchers led by Ludovic Righetti, Associate Professor of Electrical and Computer Engineering Department and Mechanical and Aerospace Engineering, as well as a member of the Center for Urban Science + Progress, have developed a strategy for planning object manipulation, leveraging the power of Monte Carlo Tree Search (MCTS) to discern optimal contact sequences. Complementing this, an adept trajectory optimization algorithm, rooted in Alternating Direction Method of Multipliers (ADMM), evaluates the dynamic feasibility of potential contact sequences. The algorithm was previously developed by Righetti’s group, The Machines in Motion Laboratory.

The team, including Ph.D. students Huaijiang Zhu and Avadesh Meduri, made a key innovation in expediting MCTS involving the development of a goal-conditioned policy-value network, guiding the search toward promising nodes. Additionally, manipulation-specific heuristics prove instrumental in markedly shrinking the search space.

The efficacy of the approach is underscored through a series of meticulous object manipulation experiments conducted both in a physics simulator and on tangible hardware. The methodology exhibits a favorable scalability for protracted manipulation sequences, a testament to the learned policy-value network.

This advancement substantially elevates the planning success rate, marking a significant stride in the realm of object manipulation planning.

Zhu, H., Meduri, A., & Righetti, L. (2023, March 19). Efficient object manipulation planning with Monte Carlo Tree Search. arXiv.org. https://arxiv.org/abs/2206.09023

Huaijiang Zhu, a Ph.D. student in electrical and computer engineering was lead author.

There are many real-world — and, someday, off-world — applications for light-weight, energy-efficient, fully autonomous robots. Yet the more autonomous a robot is, the greater its computational requirements. Onboarding the components to handle this computational function adds weight, cost and reduces potential for applications in hostile environments.

It might thus be desirable to offload intensive computation--not only sensing and planning, but also low-level whole-body control--to remote servers in order to reduce on-board computational needs. Fifth Generation (5G) wireless cellular technology, with its low latency and high bandwidth capabilities, has the potential to unlock cloud-based high performance control of complex robots. However, state-of-the-art control algorithms for legged robots can only tolerate very low control delays, which even ultra-low latency 5G edge computing can sometimes fail to achieve.

In this work, the investigators, led by Ludovic Righetti, associate professor of electrical and computer engineering and mechanical and aerospace engineering, and a member of NYU WIRELESS, investigate the problem of cloud-based whole-body control of legged robots over a 5G link. Their novel approach consists of a standard optimization-based controller on the network edge and a local linear, approximately optimal controller that significantly reduces on-board computational needs while increasing robustness to delay and possible loss of communication. Simulation experiments on humanoid balancing and walking tasks that includes a realistic 5G communication model demonstrate significant improvement of the reliability of robot locomotion under jitter and delays likely to experienced in 5G wireless links.

The team included Kai Pfeiffer, a postdoctoral researcher, and Manali Sharma, a master's student in the Department of Electrical and Computer Engineering; Sundeep Rangan, professor of electrical and computer engineering and associate director of NYU WIRELESS; and Marco Mezzavilla, a research scientist at NYU WIRELESS.

The project, funded in part by the National Science Foundation National Robotics Initiative and OPPO Mobile Telecommunications, will be presented at the IEEE/RSJ International Conference on Intelligent Robots and Systems in October, 2020.