Mobile Robotics Challenges

I'm continuously fascinated by the evolving landscape of automation and robotics; it's why I work part-time as the Safety Innovation Lead at the Australian Automation and Robotics Precinct . With the rapid advancements in automation and robotics technology, the shift towards highly automated systems is inevitable, particularly in mining, but it also brings forth significant challenges and opportunities in managing health and safety. One of the significant challenges of safely integrating mobile machine automation into high risk industries is the inherent limitation of relying solely on human oversight as a risk control for autonomous systems. The resulting human work contains risks of boredom, confusion, cognitive limitations, loss of situational awareness, and automation bias which all contribute to degradation in human and organisational performance. These psychosocial risk factors highlight the urgent need for machines that can manage safety autonomously. At the Australian Automation & Robotics Precinct, we provide a unique sandbox for testing automation technologies. This environment allows us to push regulatory boundaries and innovate safely, ensuring that our advancements in automation are both effective and aligned with global safety standards. I've spent some time exploring robotics & automation in Europe over the past couple of years and will be visiting automation centres in the UK this week. Europe has consistently been at the forefront of machinery safety regulation. The recent publication of the updated EU Machinery Regulation 2023/1230 which becomes legally binding on January 20, 2027, is designed to ensure safe interaction between humans and machines, adapting continuously to technical developments (especially modern AI technologies). It sets a high standard that greatly influences global safety practices. Meanwhile, in Australia, while we rely on the AS/NZS 4024 series first published in the mid-1990s, there’s a growing need to update our standards to reflect the current technological landscape. If you're interested in learning more about the safety of mobile autonomous systems check out the paper titled "A comprehensive approach to safety for highly automated off-road machinery under Regulation 2023/1230" in the latest issue of Safety Science. And stay tuned for the official opening of the Australian Automation & Robotics Precinct HQ later in the year. #Automation #Robotics #MachineSafety #AI #SafetyInnovation #SafetyTechNews #SafetyTech

7 lessons from AirSim: I ran the autonomous systems and robotics research effort at Microsoft for nearly a decade and here are my biggest learnings. Complete blog: https://sca.fo/AAeoC 1. The “PyTorch moment” for robotics needs to come before the “ChatGPT moment”. While there is anticipation towards Foundation Models for robots, scarcity of technical folks well versed in both deep ML and robotics, and a lack of resources for rapid iterations present significant barriers. We need more experts to work on robot and physical intelligence. 2. Most AI workloads on robots can primarily be solved by deep learning. Building robot intelligence requires simultaneously solving a multitude of AI problems, such as perception, state estimation, mapping, planning, control, etc. We are increasingly seeing successes of deep ML across the entire robotics stack. 3. Existing robotic tools are suboptimal for deep ML. Most of the tools originated before the advent of deep ML and cloud and were not designed to address AI. Legacy tools are hard to parallelize on GPU clusters. Infrastructure that is data first, parallelizable, and integrates cloud deeply throughout the robot’s lifecycle is a must. 4. Robotic foundation mosaics + agentic architectures are more likely to deliver than monolithic robot foundation models. The ability to program robots efficiently is one of the most requested use cases and a research area in itself. It currently takes a technical team weeks to program robot behavior. It is clear that foundation mosaics and agentic architecture can deliver huge value now. 5. Cloud + connectivity trumps compute on edge – Yes, even for robotics! Most operator-based robot enterprises either discard or minimally catalog the data due to a lack of data management pipelines and connectivity. Given that robotics is truly a multitasking domain – a robot needs to solve for multiple tasks at once. Connection to the cloud for data management, model refinement, and the ability to make several inference calls simultaneously would be a game changer. 6. Current approaches to robot AI Safety are inadequate Safety research for robotics is at an interesting crossroads. Neurosymbolic representation and analysis is likely an important technique that will enable the application of safety frameworks to robotics. 7. Open source can add to the overhead As a strong advocate for open-source, much of my work has been shared. While open-source offers many benefits, there are a few challenges, especially for robotics, that are less frequently discussed: Robotics is a fragmented and siloed field, and likely initially there will be more users than contributors. Within large orgs, the scope of open-source initiatives may also face limits. AirSim pushed the boundaries of the technology and provided a deep insight into R&D processes. The future of robotics will be built on the principle of being open. Stay tuned as we continue to build @Scafoai

Many have tried mobile robotic picking before and failed. Remember the Fetch Mobile Manipulator or IAM Robotics’ Swift? None of them succeeded commercially. Why? They all tried to pick items directly from shelves – just like humans. But here’s why that approach doesn’t work: 𝟏. 𝐔𝐧𝐫𝐞𝐥𝐢𝐚𝐛𝐥𝐞 𝐇𝐚𝐧𝐝𝐥𝐢𝐧𝐠: Items on shelves can easily fall, making the process unpredictable and error-prone. 𝟐. 𝐂𝐨𝐦𝐩𝐥𝐞𝐱 𝐌𝐚𝐧𝐢𝐩𝐮𝐥𝐚𝐭𝐢𝐨𝐧: Picking freestanding objects requires advanced 6-axis robots, driving up costs. 𝟑. 𝐈𝐧𝐞𝐟𝐟𝐢𝐜𝐢𝐞𝐧𝐭 𝐑𝐞𝐩𝐥𝐞𝐧𝐢𝐬𝐡𝐦𝐞𝐧𝐭: Inventory placement becomes a logistical nightmare, with items on shelves needing precise positioning for #robots to pick them. That’s why we decided to take a radically different approach at Brightpick. We went back to first principles and designed a mobile robotic picker that picks vertically from totes instead of horizontally from shelves, using proven bin picking technology and AI from our sister company Photoneo. The result? Brightpick Autopicker is today the only commercially-viable mobile manipulator on the market, with almost 100 already deployed with customers on long-term contracts. #technology #innovation

I spent the last year and a half building autonomous systems for orchards at Bonsai Robotics. The biggest surprise? Connectivity is the infrastructure problem nobody talks about. Everyone focuses on the robotics—the perception systems, the path planning, the manipulation. But when you're operating in a 500-acre almond orchard in Australia or the Central Valley, you're dealing with spotty cellular coverage, dust that degrades signal quality, and distances that make WiFi impractical. The robots can see. They can navigate. They can make decisions. But if they can't reliably communicate with fleet management systems or push telemetry data for analysis, you're running blind. This isn't just an ag problem. I've seen similar challenges in all off-road and remote applications, including marine robotics with Wave Gliders operating thousands of miles offshore, army tanks on the frontlines, and rail vehicles and trucks in rural ODDs. The solution isn't just "add more cellular towers." It requires edge computing architectures that let vehicles operate autonomously when connectivity drops, smart data prioritization that pushes critical telemetry first, and mesh networking between vehicles to create resilient communication networks. Connectivity infrastructure is as important as the autonomy stack itself. You can't deploy at scale without solving both. What connectivity challenges have you seen in deploying hardware in remote environments?

𝗥𝗲𝗶𝗻𝗳𝗼𝗿𝗰𝗲𝗺𝗲𝗻𝘁 𝗟𝗲𝗮𝗿𝗻𝗶𝗻𝗴 𝗮𝗻𝗱 𝘁𝗵𝗲 𝗡𝗲𝘅𝘁 𝗘𝗿𝗮 𝗼𝗳 𝗥𝗼𝗯𝗼𝘁𝗶𝗰𝘀 Reinforcement Learning has become the intelligence engine behind the next generation of autonomous machines. It allows robots to learn through experience, adapt to complex environments, and make decisions in real time. Researchers across the world are pushing this field forward, and the progress made between 2023 and 2025 has transformed what we thought robots could do. Modern systems now learn from high-dimensional sensory data like vision, tactile signals, and proprioception. They no longer rely on brittle rules or hand-designed controllers. Instead, they build internal models of the world and use them to plan, predict, and act with remarkable precision. Transformative breakthroughs like Dreamer world models, transformer-driven action policies, diffusion-based decision systems, and hybrid model-based control have allowed robots to move, grasp, manipulate, and navigate with a sophistication that simply didn’t exist a few years ago. Robots today learn faster, require fewer human demonstrations, and succeed in dynamic, contact-rich tasks that were once thought impossible. They can adapt their strategies on the fly when the environment changes. They can infer hidden states, anticipate future outcomes, and recover from failures with very little supervision. High-resolution tactile sensing, latent-space world models, and large-scale datasets of real robot behavior have made this evolution inevitable. Yet even with all this progress, several challenges still define the frontier. Robots must close the gap between simulation and the real world, learn to operate safely around people, build long-horizon memory, and coordinate with swarms of peers under partial observability. These problems are the heart of the next leap in autonomy. They will define which systems are capable of real mission-scale reasoning instead of short-horizon actions. The coming years will belong to hybrid systems that combine world models, foundation models, and real-time control. They will continuously update their understanding of the world as sensors age, as hardware wears, and as environments become unpredictable. They will rely on new forms of tactile intelligence, more efficient learning pipelines, and architectures that blend imagination with grounded physics. Every major advance in robotics over the past decade has moved toward one goal. Autonomy that is resilient. Autonomy that adapts. Autonomy that learns at the speed of the world itself. Singularity Systems is moving this space.

🎥 GNSS-denied localization using 3D LiDAR with #HiveRobotics I'm thrilled to share a new case study developed in collaboration with #HiveRobotics, who designs autonomous delivery robots operating in highly diverse environments such as dense urban areas, vacation resorts and private sites. These use cases are particularly challenging from a localization standpoint: 🚫 GNSS-denied or GNSS-degraded environments caused by high buildings in urban environments, but also dense tree canopies and narrow paths 🤖 Small ground robots, driving close to the surface with limited sensor height 🌄 Uneven and non-standard terrains, introducing additional noise and dynamics in perception and motion To address these constraints, #HiveRobotics deploys Exwayz’s map-based localization approach, which relies on the creation of high-accuracy 3D #LiDAR maps of the operating environment and embedded real-time localization running fully onboard the robot. By leveraging stable 3D geometric features of the environment, this approach delivers robust and precise positioning. This enables reliable autonomous navigation in complex outdoor settings, including areas where satellite-based localization may be limited or unavailable. The video below shows the robot’s real-time 3D LiDAR #localization overlaid on the map, demonstrating stable positioning in real operating conditions. I personally love the 3D map background, where we can barely see the ground because of the vegetation density and height 🌲 Many thanks to the #HiveRobotics team for their trust 🙏🏻 and congratulations to them for tackling such demanding real-world autonomy use cases, this is where robotics truly gets interesting 🤖 A detailed case study is available on our website 👉🏻 link in the comments for those who want to explore the technical aspects further! #Robotics #AutonomousSystems #LiDAR #3DMapping #Localization #GNSSDenied #OutdoorAutonomy #Exwayz #HiveRobotics Hassan Bouchiba Antoine Plat Romain Bonjean

Most people think that the hardest problem in robotics is getting one robot to do every task. It’s not. Everyone who’s deployed a robot to a real customer knows this. The hardest problem is going the “last mile” from 90% reliability to 99.9%: rapidly adapting a robot to perform a specific task with extremely high performance. Why is this so hard? ⛔ Irreversible failures: In the physical world, critical mistakes like breaking an item or hurting a person are not acceptable. But it’s hard to avoid these failures without specifically designing the physical system to minimize these errors. 👀 Diagnosing mistakes: It’s not as easy as telling the robot what it should be doing in natural language. Observability systems are needed to ensure that when failures occur, they can be root caused. ⁉️ Lack of a fallback plan: When digital models don’t know something, they can search a knowledge base like the internet. When robots can’t figure out how to do something, there may be no automated fallback option. This is why it’s so critical to make seamless human intervention tools and systems. Read more here:  https://lnkd.in/gDYNXTEn

In today's Unsupervised Learning I dive deep into self-driving cars and everything AI x hardware with Vincent Vanhoucke, Distinguished Engineer at Waymo and former Head of Robotics at DeepMind. Vincent has spent years at the intersection of AI and robotics, shaping how machines perceive, plan, and act in the physical world. From self-driving cars navigating complex cityscapes to the future of generalist robots, Vincent breaks down the real challenges — and unexpected breakthroughs — in bringing AI out of the cloud and onto the streets. Some highlights: 1️⃣ The milestones that matter in AI x robotics In self-driving the challenge has shifted from getting cars to drive autonomously to handling the long tail of rare, unpredictable edge cases that emerge over millions of miles driven. Vincent highlighted that the development of physically realistic world models, enabling robots and autonomous vehicles to simulate and train for countless real-world scenarios with high fidelity would be game-changing. Ultimately, scaling and real-world deployment, rather than isolated lab successes, are the true markers of progress in AI-driven robotics. 2️⃣ The Impact of LLMs on robotics Vincent shared how LLMs and VLMs have had a transformative impact on robotics by introducing world knowledge into AI systems, significantly enhancing their perception and reasoning capabilities. Unlike traditional models that rely solely on sensor data from specific environments, LLMs can provide contextual understanding, allowing robots to recognize objects or situations they've never directly encountered — like identifying unfamiliar police cars in a new city or recognizing rare accident scenarios. This semantic awareness helps self-driving cars and robots better interpret complex, real-world environments. By scaling up and leveraging LLMs, robotics can now bridge the gap between raw data perception and higher-level reasoning, pushing machines closer to human-like understanding. 3️⃣ How Waymo enters new cities When Waymo enters a new city, their focus is on ensuring the system can handle local nuances. The core models are designed to be highly portable across different environments, but specific elements—like recognizing unique emergency vehicle designs or adapting to new traffic patterns—require validation to maintain safety and reliability. A significant part of the process involves extensive evaluation and testing, often using simulations to explore edge cases rather than simply gathering more real-world data. Additionally, Waymo works closely with regulators and local communities to ensure compliance and public trust. Vincent emphasizes that the biggest hurdle isn’t always technical but about gaining social acceptance. A truly fascinating conversation on topics I've wanted to cover for awhile check out the full discussion below: YouTube: https://lnkd.in/gEFNDDR6 Spotify: https://bit.ly/4gXP8gK Apple: https://bit.ly/4gU2HNX

🤷♂️ The Harsh Truth About Mobile Robot Implementation 👉 Everyone loves the idea of automation. Fewer injuries. Higher efficiency. Future-proofing your operations. Sounds amazing, right? 😩 Then why do so many companies struggle—or worse, fail—to implement mobile robots? Because reality hits hard. 🚧 Brownfield Environments = A Nightmare Your facility wasn’t built for robots. Floors need modification. Network infrastructure needs upgrades. Processes need rethinking. What seemed like a plug-and-play solution now looks like a full-blown construction project. 💰 Integration = Hidden Costs Everywhere You don’t just buy a robot—you purchase software, connectivity, and IT security headaches. Suddenly, your "cost-saving" robot is burning through your budget. 🔄 One-Size-Fits-All? Not Really. Off-the-shelf robots follow strict rules. But real-world environments don’t. Custom solutions? Expensive and time-consuming. You’re stuck between "it doesn’t fit" and "it costs too much." 🤷 Non-Technical Users = Left Behind If your team needs an engineering degree just to operate a robot, guess what? They won’t use it. And if they don’t use it, you’ve just invested in a very expensive paperweight. 💡 The Fix? Simplicity + Flexibility What if automation didn’t require overhauling your facility? What if setup took minutes, not months? What if non-tech users could deploy robots without relying on IT? 🐕 Enter THOUZER—a collaborative transport robot that works with your existing setup, not against it. No crazy infrastructure changes. No unnecessary complexity. Just real automation for real operations. 🚀 Automation should empower people, not replace them. If your robots complicate things, maybe you’re doing it wrong. What’s been your biggest challenge with mobile robots? 👇

🤖 Many people agree that automation will transform job sites, but few can define how. In this session, Steven Uecke (SE, PE, P.Eng) from SuperDroid Robots shares hard-earned lessons from developing Groundhog, an autonomous reality capture robot designed to navigate dynamic environments without getting stuck. From overcoming technical barriers to proving real-world value, this talk dives into what it takes to bring robotics into construction. Key Takeaways 💡 -Autonomous robotics is here, but adoption in construction comes with unique challenges. -Overcoming technical barriers is key, from localization to power efficiency, making robots job-site ready isn’t simple. -The value must be clear, how do we prove ROI for automation in an industry built on tradition? -Where do we go from here? Robotics is advancing, but how do we scale deployment across the AEC industry? The full video is LIVE! Watch here ———> https://lnkd.in/g6W7y_Z7 Reality Capture Network