Rock to Road

Features Heavy Equipment
The autonomous job site: AI takes some control of heavy equipment

August 17, 2021  By Jacob Stoller

As the number of autonomous features available for heavy equipment continues to grow, it’s hard not to think of the automotive industry’s quest to develop self-driving cars.

While both industries utilize the powerful combination of artificial intelligence and advanced vision systems to achieve humanlike control, there’s more to the differences than meets the eye.

“If I’m in a car and want to get to the airport, I go along a fixed route – it’s just about one car, and that environment’s not changing,” says Cameron Clark, business area manager for civil engineering and construction at Trimble.

“But in construction environments you’ve got multiple machines and fleets working together, and the environment is not going to be what it was yesterday.”


In construction work environments, operators frequently execute non-routine tasks and draw on skills refined over 20 or 30 years. Their role, consequently, is much tougher to automate than that of the average driver; and vendor strategies, at least for the short term, are not about replacing operators, but leveraging their skills more widely.

“When you have a piece of equipment like an excavator, for example, that excavator is usually the the choke point on the production of that job site,” says Bryce Satterly, smart construction solutions manager for Komatsu North America in Salt Lake City, Utah. “So, the guy in that seat is the one that needs to be making things as efficient as possible. Because if he slows down, the whole fleet or the whole crew slows down.”

Operators stretched to the limit

As experienced operators retire in larger numbers, there’s a growing dependence on those still in the field. Many are stretched to the limit not just from physical demands, but from the growing mental stress of the additional duties they are being asked to undertake.

“There’s such a lack of experienced personnel out there that these operators are being asked to do more and more every day,” says Michael Salyers, senior product manager at Komatsu America Industries LLC. “So, they’re not just being asked to operate the machine. Now they have to guide other pieces of equipment on the job site. So, they’re not just working the material, but also thinking about how to manage the material and its placement. And they’re doing that all day long.”

Task automation enabled by features like blade control on dozers and excavators has helped ease some of the physical stress, particularly with repetitive tasks such as spreading layers of material.

The real power of the technology, however, is in its ability to automate according to a larger game plan.

“It’s about being able to automate not just the machine, but the site,” says Clark. “I think automation makes sense on a machine-by-machine basis, but it adds so much more value when you tie it into the bigger picture.”

Task automation enabled by features like blade control on dozers and excavators has helped ease some of the physical stress, particularly with repetitive tasks such as spreading layers of material.

According to this larger vision, multiple machines will be working together autonomously.

“In a roadwork project, we’re working towards a whole autonomous system where all the equipment is working together,” says Aaron Kleingartner, product and dealer marketing manager for Doosan Infracore North America.

“For example, the system will know that it needs to move a specific amount of material. So, the ADT will travel where it needs to so that the excavator can load it and take it to a stockpile. Then a wheel loader loads the material into an over-the-road truck.”

Keeping safe

Safety is, of course, an essential priority for the final product, and also, during testing and development.

As with protypical self-driving cars, AI in heavy equipment uses camera and LiDAR images to create 3D terrain maps which serve as virtual test environments.

“We can use AI to create thousands and thousands of variations of that map representing a rainy day, a lot of mud, a lot of snow, ice on the road. etc. ,” says Luca Delgrossi, head of technology for Volvo Automation Solutions. “So, you can basically play the scenario millions of times in different conditions. And artificial intelligence is very useful to create all these scenarios.”

The power of self-learning neural networks allows most testing – Delgrossi estimates 95 per cent – to be conducted in simulations, avoiding the danger to operators of testing with live machines.

Volvo also imposes special rules on AI in order to ensure safety.

“The problem is that we cannot use the artificial intelligence-based system to formally prove the safety concept,” says Delgrossi.

“For this reason, we build boundaries to the system that are not based on AI. And we know that if we operate within these boundaries, we will be safe.”

This constraint means the progress with AI is slower than industries where some degree of error is tolerable.

“Ours is one of the industries where it is more difficult to use AI,” says Delgrossi. “Other industries don’t have that constraint.”

Moving forward

As the technology evolves and new features make automation viable in more job scenarios, one of the biggest challenges will be giving crews the time to absorb the new technology.

“A colleague of mine put it very well when he said that every crew starts out on a job two days behind schedule,” says Satterly. “Construction is so competitive that crews don’t have enough time to plan things out as much as they should before they start work.”

The good news is that the technology will make the industry more attractive to younger people, who are desperately needed to fill the labour gap.

“A lot of my generation grew up playing video games and playing with computers,” says Satterly, “and then we had flip phones and that transitioned to smartphones. And we learned how to use that technology to the fullest extent. So, I think this technology really hones in on that skill set that we’ve all grown up with.”

Print this page


Stories continue below