We competed in the second edition of the Smart City Robotics Challenge (SciRoc) last September and finished in second in the Coffee Shop episode. Congratulations SocRob@Home!

For this specific research challenge, under this year’s theme “Smart Inclusion”, each team robot was required to recognize and report the status of all tables inside the shop, to take orders from customers and to deliver objects to and from the customers’ table.

What does it take to a robot assist in a coffee shop?

To decide the next steps, the robot needs to navigate around the shop and assess the status of all tables. If a table is clean but with customers present at the table, the robot will accept the order, via spoken language, and report it to the kitchen. This requires various steps such as:

Location and mapping: the first action is to assess the status and location of each table, defining which table needs to be served. The number of people and the presence or absence of objects in each table, defines the status.

Take the order: Then, the robot has to understand the order, dividing it by intention (what does the person want?) and objects (how many and which ones).

Report and collect the order: After placing the order, it has to recognize the objects and if one of the items is missing or incorrect, the robot must identify it and correct the mistake.

The processing time needed for the robot to understand the order was one of the major problems that the team encountered in the competition. A task that takes under a second to process in site, it took up to three minutes remotely. “It was a little weird this waiting time between the client order and the robot response action. The clients and even the organization thought that the robot had crashed, because they weren’t seeing the commands as I was” laughed Rui Bettencourt, PhD student at ISR-Lisboa and ScoRob team member. “Other than that, we think that our experience in previous competitions help us to achieve such a good result”.

What are the achievements needed to accomplish the task and the penalizing behaviors?

The task is considered a success if the robot reports correctly all the tables that need serving, the status of all the tables (including the number of customers at each table and if it is able to reach an unserved table and ask the order. Then, if it correctly understands and communicates the order of the customer to the counter and if it recognizes the wrong or missing item and requests correction of the order. A run terminates when the robot has delivered the order to the table.

If the robot hits any of the furniture of objects or it requires multiple repetition of a speech command, it’s penalized.

SocRob is a long-term project at Instituto de Sistemas e Robótica de Lisboa, encompassing a broad range of efforts towards teams of robots to perform tasks, with a particular focus on the participation in scientific competitions.

Hosted in Bologna, Italy last 6th – 10th September, the event took place at the historic Palazzo Re Enzo, in the main square and heart of Bologna. To overcome Covid-19 travel restrictions, the teams were able to compete remotely or in person.