Introduction

This page is a run through of my master’s thesis from which this publication is derived and deals with coordinating multiple robots in online order fulfilment systems.

Visiting Pebbles on Rectangular Grids - The Ideas

The name was inspired by Kornhauser et al’s work entitled Coordinating Pebble Motion On Graphs (PMOG). Abstractly, pebbles represent agents requiring access to shared resources represented by Graph nodes. Specifically, only one pebble can access a node at a time. The goal of PMOG is to determine whether a target configuration is reachable from the initial arrangement of pebbles and provide the sequence of required moves in this case.

Concretely, PMOG is applicable in robotic warehousing systems: the pebbles represent robots that access physical locations. This example is not as far fetched as it sounds: thousands of robots are delivering packages from storage to packing stations in warehouses every day and Kiva Systems is the company that pioneered their use, before being acquired by Amazon in 2012. It’s also not new: Staples was the first company to go live with a Kiva system in 2006! A 2016 report estimated that Amazon was operating ~30k units and by 2022 Amazon reported 520,000 world-wide. The next two images show the layout of a typical robotic distribution center and the robots they employ. Robots pick up and deliver shelves of inventory to pick/pack workers. See the image source for more information.

Distribution Center Layout (image source)
Distribution Center Layout
Kiva Robot Carrying Inventory Pod (image source)
Kiva Robot Carrying Shelf

PMOG generalizes and solves a fundamental problem. However, the constraints could be relaxed for this application: robots only need to visit destination nodes, they don’t need to be in a pre-defined end configuration at the same time. Hence the ‘visiting’ part of the title. This relaxation has implications for the feasibility check of the original PMOG: it’s easy to come up with feasible visiting PMOG cases that are infeasible in PMOG. Since warehouses are rectangular workspaces, my work is restricted to graphs made up of bi-connected rectangular grids. As long as there is at least one open node and the graph is bi-connected, every destination assignment is feasible in Visiting Pebble Motion on Rectangual Grids.

The Implementation - a First Try

Let robots find their way using something like A*, if another robot gets in the way, simply wait until the node is free again. Using the Alphabet Soup multi-robot simulator, I modified the default behaviour to strictly adhere to the underlying navigation graph. Then, I only allowed robots to enter a node if it is empty. The next video shows what it looks like: the robots are pink, replenishing stations are blue, shelves are purple and packing stations are green. In this simulator, the shelves contain letters and the parcels being packed are words from a dictionary.

Alphabet Soup Simulation of Distribution Centre

There is a problem though: eventually the robots get stuck waiting for each other as shown in the image below. This is known as deadlock.

Robots in an infinite wait
Robots in an infinite wait

A Step Back - Dealing with Deadlocks… One at a Time

Let’s focus on a single stuck robot and assume it can order others to move out of its way. A Breadth First Search (BFS) for the first open node reveals the shortest path of moves to clear the blocked node (i.e the ‘swap path’). The process is shown in the following animation. In some cases, robots start in their target positions, or reach them by moving out of the way for others, this is a fortunate accident.

Visiting Pebble Motion Algorithm Demonstration
Visiting Pebble Motion Algorithm Demonstration

The next video shows a simulator I wrote in Java to develop the deadlock resolution strategy. Square brackets represent grid positions and numbers represent robots. Each node is the target of the robot with the same number and the grid is numbered in row major order, meaning if every robot were in its destination position, it would look like this:

Row major target assigment.
Row Major Target Assigment

The previously described process is repeated for every robot. The robot currently in control of all moves is indicated as ‘Moving pebble’ and all unfinished robots are shown in the ‘Pebbles queue’. The pebbles queue determines which robot gains control next and was populated randomly, since the order doesn’t matter. As expected, all robots eventually end up in ‘Completed’.

Visiting Pebble Simulator

A Jump Forward - Dealing with Deadlocks… Many at a Time

Simultaneous motion is achieved by assigning a priority to each robot and defining a round consisting of two phases. In a first phase, robots instruct each other to move along the swap path determined using BFS. A robot sends instructions either to itself or the robot at the tip of the swap path. Robots send instructions to themselves in two scenarios: 1) to move to the next node, and 2) to remain in place while ordering others to move. This ensures that high priority robots decrease their distance from their target monotonically, which in turn ensures that progress is being made. Idle robots don’t send instructions but receive them. At the end of this phase, each robot selects the highest priority instruction and forwards it as a node access request, maintaining the original sender’s priority.

In the second phase, nodes select the highest priority request to notify successful requesters and these robots move. Then, all robots empty their instruction queues. This ends the round and the process repeats until all tasks are complete, at which point no one sends instructions and everything stops. In the next video, the same task assignments are used as before. Lower numbers mean higher priorities, making life hard for robot 10!

Simultaneous Visiting Pebbles

Going All the Way - Eliminating the Synchronous Assumption

So far actions were synchronized into rounds and robots/nodes knew when transitions occur. However, the real world is asynchronous and these timings aren’t known beforehand. This could be fixed by synchronizing these moves into fixed time slices. However, unavoidable variances mean that some robots are faster than others making synchronization wasteful, and without a centralized synchronization signal, robots don’t know when the synchronization barriers occur so that they can replan or grant access.

The answer to reducing this waste is to notify robots when their instructions have failed. Nodes notify all requesters when access is granted, and requesters react accordingly. Failed requests are relayed by the requesters to their originators. Instructed robots don’t know whether their instruction queues already contain the highest priority instruction they will get, so they immediately request what they have, and update accordingly when new requests arrive by withdrawing/resubmitting requests.

Getting the details (available in my thesis) right is really difficult. Instead of slogging through that, let’s see it in action! In the next videos, robots indicate completion by highlighting themselves in red and indicate node visits by marking them with a purple circle. These robots have randomly initialized acceleration and speed limits to simulate real-world variances. The first video has tasks assigned in row-major order, however, any task assignment is allowed. In fact, robots can even all have the same target node as shown in the second video which is an example of an assignment that is infeasible in PMOG.

Row Major Assignment
Central Node Assignment

So far only single tasks were assigned. Multiple tasks can be assigned by setting each robot’s priority equal to their task number. Even if one task is delayed by many others, as overriding tasks complete, the blocked task’s priority effectively raises until it overrides every other robot. The video below shows robots cyclically completing multiple random node visits without marking their destinations.

Cyclic Random Assignments

The Final Test

Using random cyclic task assignments, the next figure shows the number of completed missions versus number of robots for one hour of simulated time, 144 grid nodes and up to 143 robots. The legend indicates experiments for grids with different aspect ratios. The maximum number of completed tasks always occurs for around 30 robots independently of the grid aspect ratio. The simulations were run headless and no deadlocks or collisions occurred, that’s 1001 flawless simulations!

Completed tasks vs number of robots and board
Completed Tasks vs Number of Robots

Conclusion

The relaxations on PMOG allow robots to deliver payloads without collisions or deadlocks in continuous asynchronous time, requiring comparatively little computation. The computation can also easily be distributed among multiple agents without requiring central coordination. These were the extents of my research scope and there is much room for improvement in terms of optimality. However, coordinating n robots with only one open spot to move is no mean feat and I am proud of what I achieved in this work.