Industry 4.0: towards an intelligent collaboration with robots

How many times, at the grocery shop, while waiting for our turn to be served we naturally asked ourselves: should I just queue and wait or should I swing by another aisle in the meanwhile? Well… just a simple question that however entails quite a few reasoning: how fast are the attendants in serving other people? how much time do I have without loosing my turn?

Now, let’s virtually move this paradigm to the factory of the future. Our guest star is now a collaborative robot that has to decide when the human fellow co-worker will require its assistance.

This is what we are currently doing in our lab at Politecnico di Milano to allow the robot to answer the following questions:

  • which activity is the human operator more likely to perform next?
  • what is the time when an activity requiring my assistance is expected to be initiated by the human?

But let’s proceed step by step…

The first ingredient that we need is an efficient way to categorise human’s actions. For this goal, we used some Bayesian statistics and some computer vision. The result is shown in this video: the robot completes the operation initiated by the human.

Now we have correctly characterised what the human is doing, but what’s next? Well, based on some machine learning algorithms, and specifically based on pattern recognition, we were also able to predict the next sequence of actions and their duration. The result is shown in the next video: the robot is autonomously responsible for a quality control task, while the operator is involved in some assembly operations. As soon as the human completes his task, the collaborative phase can start and the robot is ready to help. The promptness of the robot is achieved thanks to the observation of previous executions and allows the robot itself to be ready when required by the human.

The algorithm has been compared to a purely reactive approach, during which the robot always starts its own task unless the human has already initiated the collaborative phase. The proactive behavior outperforms the purely reactive one by reducing the cycle time and its variability, hence achieving the perfect production leveling (or heijunka, 平準化, in Japanese).



  1. A.M. Zanchettin, P. Rocco – “Probabilistic inference of human arm reaching target for effective human-robot collaboration”, IROS 2017, Vancouver (Canada), September 24th – 28th, 2017.
  2. A.M. Zanchettin, A. Casalino, L. Piroddi, P. Rocco – “Prediction of human activity patterns for human-robot collaborative assembly tasks”, IEEE Transactions on Industrial Informatics.

Leave a Reply

Your email address will not be published. Required fields are marked *