Spotlight: Using Web-Based Technologies to Make Robotics Accessible to All
Ruffin White ·
Esther Weon ·
Ruffin White is a PhD researcher focused on robotics cybersecurity and an advocate for repeatable, reproducible robotics research. As a multidisciplinary roboticist and research fellow at the Università Ca' Foscari Venezia in Venice, Ruffin works to break down the technical transfer barriers between research and industry by improving the usability, accessibility, and security of open source robotics software. In his spare time, Ruffin enjoys pioneering, experimenting with new technologies to address big picture issues in robotics development and education, and maintaining the Docker tooling and CI infrastructure for ROS and Nav2.
Tell us a little bit about how you first became interested in robotics.
As an undergraduate student, I was studying electrical engineering at the Rose-Hulman Institute of Technology when I decided to join a robotics competition team.
I was initially attracted by the prospect of working in a multidisciplinary space, but became completely enthralled by the experience of bringing my ideas to life. I was creating something with both an autonomous and physical manifest to interact with the real world. Robotics built on the same excitement I'd felt when I first started experimenting with embedded devices, and broadened my notions of what assistive technologies like voice dictation and screen readers could evolve into (e.g. robot wheelchairs and navigation assists for the visually impaired).
Our team competed in IGVC (Intelligent Ground Vehicle Competition), and that incredible experience inspired me to dive into related course offerings.
Ruffin collaborating with his IGVC teammates throughout the years – both in the lab and out on the competition field.
I eventually decided I wanted to pursue robotics as a career, but I saw that the most interesting opportunities required graduate experience. This led me to apply to Carnegie Mellon University’s Robotics Institute Summer Scholars (RISS) program, where I worked on my first robotics research project developing an aerial mapping platform for inspecting civil infrastructures. Shortly afterwards, I joined Georgia Tech’s Robotics PhD program, where I eventually met my doctoral advisor Henrik Christensen and joined his Cognitive Robotics Laboratory.
This led to my first internship at the Open Robotics Foundation, where I started looking at a large barrier to robotics adoption: security. At the time, everything in ROS 1 was unencrypted, over clear text, with no authentication or access control. This posed a dilemma for industrial robotics applications, which often had to follow specific security specifications or certifications. All this experience evolved into my thesis for my doctoral studies – making robotics software development more repeatable, reproducible, and secure.
Eventually, I moved with the Cognitive Robotics Laboratory to University of California, San Diego as a graduate research assistant to help spearhead their robotics initiative and earn my PhD. One of our larger aspirations was to help robots achieve lifelong autonomy – i.e. be deployed for long scales of time and automatically accommodate for changing environments. As part of this work, I focused heavily on traditional and semantic SLAM and perception-based object tracking.
What challenges are you most interested in tackling in your research?
Before ROS, robotics research labs and universities were bumping up against this issue of constantly reinventing the wheel. Robotics is inherently multidisciplinary, so a monolithic architecture can’t solve all your challenges – roboticists need a way to interoperate and expand on core functionalities. To solve this problem, folks were building their own middleware frameworks and developing their own in-house versions of various robotics capabilities in parallel, duplicating a lot of valuable work.
ROS was largely designed to address these issues – it provided a standardized software ecosystem for roboticists to collaborate in, as well as a wide range of plug-and-play libraries and algorithms to share. But there are still a lot of friction points that need to be ironed out. We have a replication crisis that web-based tools like IPython Notebooks and Jupyter Labs have just begun to address, but it’s far from “solved”. In fact, the Institute of Electrical and Electronics Engineers (IEEE) published an entire volume on the importance of repeatable and reproducible results when benchmarking robotics algorithms.
On a visit to Willow Garage in 2010, Jorge Cham of PhdComics learned from the team that “a typical grad student spends about 70% of their time reinventing the wheel.” He drew a fairly poignant comic of the state of robotics research before Open Robotics and ROS existed.
During my research at UC San Diego, I got especially disgruntled with how annoying it was to recreate an environment for a repeated task. I found myself running into the same confusing roadblocks over and over when trying to get a new algorithm release working – like figuring out the appropriate compile settings or tweaking other parameters.
The issues I experienced inspired me to address repeatability and reproducibility head-on in a ROSCon talk (“Enabling Docker for Robotics Applications”). Using Linux containers was popular in web development at the time, and so my research partners and I had borrowed that concept and applied it to robotics.
In short, I think it should be extremely easy for robotics researchers to codify their contributions, so that other folks in the community can build on that body of work. In addition to publishing the literature and the code behind these findings, roboticists should be able to provide the entire environment used to conduct the experiments – so that fellow researchers can easily compile, run, and reevaluate your exact results from ground zero.
What robotics projects are you working on now?
Partway through my current research fellowship at Università Ca' Foscari Venezia, I had the idea to start a robotics course here in Venice. My advisor had mentioned that he’d tried to teach an introductory robotics workshop once and found it very challenging. They’d made surprisingly little progress, because the TA spent most of the time debugging individual computing environments. We identified this as part of the perils of teaching a multidisciplinary technical subject – it requires a baseline of concrete digital literacy for students to even start making sense of the theory.
But in creating this robotics course, we ran into a lot of bureaucracy – getting funding, purchasing hardware, formulating a curriculum. All this red tape made me realize how fortunate I’d been at large state-of-the-art institutions like Georgia Tech and UCSD. Our small computer science department here in Italy just didn’t have access to the same financial and technical resources I’d taken for granted. This got me thinking about all the other institutions in the world that want to teach cutting-edge robotics research, but lack the resources to do that locally.
Before creating a curriculum, I decided it was more important to first help students set up a working development environment. I wanted to create the technical infrastructure needed to help students spend more time learning, versus fighting their setup.
I'm a long-time collaborator with the Nav2 project, and they were kind enough to act as my guinea pigs in this quest for beginner-friendly robotics environments. As a proof of concept, I started experimenting on streamlining their continuous integration infrastructure – we wanted to lower the barrier for open source folks to contribute to the Nav2 repo. This meant making it easier than ever to set up a ROS environment, clone the repo, install the necessarily development tools, and start compiling code.
Meanwhile, I’d been looking at GitHub's Codespaces project – they basically provide fully configured development environments in the cloud. We decided to piggyback on this technology to provide Nav2 contributors with a ready-to-go environment. They could compile the project and immediately start opening pull requests and testing in the same environment as our continuous integration. They would also be able to do everything from the browser without relying on any expensive hardware, dramatically reducing developer friction and encouraging more contributors to get involved.
An Android tablet connecting to a prebuilt Codespace on Nav2's GitHub repository. It demonstrates launching the patrolling security demo in the remote Dev Container and installing/opening several self-hosted PWAs (Progressive Web Apps) to visualize and debug ROS code.
With a scalable template for teaching robotics, any interested institution – even those on a shoestring budget and with bootstrapped resources – could provide this for their students.
What does the ideal long-term application of your research look like?
I’m hoping that my repeatability, reproducibility, and accessibility research can help make robotics development both more approachable for newcomers and more seamless for veterans. I’m also hoping that we can set new standards with modern web tools like development containers to make robotics education more accessible.
Our Codespace for Nav2 – providing people with limited web-based tools to contribute to that repository easily – was a proof of concept. From there, we’re trying to make this broadly applicable for a wider audience. Historically, efforts to teach coding have reached a select few – but more people are acknowledging the disparities in computing access and trying to combat the digital divide. Robotics students could use a setup similar to the Nav2 Codespace to dive into robotics – even if all they have is a school-issued Chromebook or a family-owned iPad. High schools and universities could also leverage similar technology to streamline how they teach robotics development.
Accessibility to robotics knowledge has been approached multiple times. The Construct published a massive open online course (MOOC) using cloud-based VMs and a virtual desktop accessible from any browser. AWS’s remote RoboMaker – while more catered towards continuous development and deployment – provided a cloud-based reference environment for robotics programming.
I wanted my solution to provide a polished, responsive, and native development experience, so I started looking into web-based tools. Foxglove Studio was the first example I found. I also remembered a project called Gzweb from my time at Open Robotics. It made everything accessible via a browser window by acting as a WebGL client for Gazebo Classic. I remember starting up an AWS instance, running Gazebo, launching Gzweb, and then sharing a link with my team. At one point, the entire office was on one simulation, playing with robots and messing with each other's kinematics and spawning new objects. It was all so seamless – with just a link, everyone could get involved. No one had to install a custom QT client on their computer, figure out the correct port, or initiate a connection. This experience was exactly what I wanted to emulate – helping anyone with a very modest amount of digital literacy get up and running quickly.
What’s been your biggest challenge and greatest reward working on this project?
I've spent the last decade in robotics, so web development is a fairly new domain to me. But exploring unfamiliar territory (e.g. web technologies like WebSockets, WebTransport, and WebRTC) has been a fun challenge and interesting deep dive. The fact that web tends to give you immediate feedback – analogous to programming a controller and instantly seeing the lights flash – has also been especially gratifying.
I've always enjoyed addressing the bigger picture challenges – whether it’s repeatability and reproducibility, security, or accessibility. One of my ultimate goals is to lower the barrier of entry to robotics development so that more people can get involved. Knowing that my work can serve such a wide range of people and have such an outsized impact is very rewarding.
When I talk about my research in improving accessibility to robotics education, everyone can understand and get behind the cause. Previously, I’d worked on disclosing security vulnerabilities for DDS vendors. After spending weeks getting the paper together and filing the reports, we got a rejection saying that our research wasn’t novel or impactful enough for distribution. With my current work, it’s really nice to have the community understand what you’re working on and profusely thank you for that effort. That positive feedback has been incredibly motivating.
What are the next steps for you and the project?
While we have a rough proof of concept with the Nav2 repo, we definitely have future work. We want to migrate our infrastructure from Gazebo Classic to the latest version. To use Codespaces for the same use cases, we'll also have to adapt to Gzweb2, which the Open Robotics folks have been working on in the background.
There are some technical challenges we want to explore as well. Visualizing dense data with high sample rates – e.g. navigation stack with local path trajectories – can saturate low-bandwidth connections. If I do anything heavy, my internet connection here in Venice starts buckling under the pressure of TCP and WebSockets connections. I'd like to explore something more responsive or with more powerful QoS options – like Foxglove’s WebSocket connection, WebTransport, the Gzweb2 bridge, or even a combination of the above. It’d be cool to get a single view for your simulation and robot sensor suite.
The final push would be to package this template and make it easily distributable. I could build the features out as much as I like, but if it's not easy to install and integrate, people won’t see its value or adopt it. We want a small footprint that still provides all the functionality you need, to be easily incorporated into developers’ workflows and professors’ course materials.
Read more:
Fetch remote meshes, use additional URDF sources, and visualize collision geometries
Hans-Joachim KrauchHans-Joachim KrauchEsther WeonEsther Weon
Dynamically rename topics to better organize and navigate complex datasets
Esther WeonEsther Weon
Miles EganMiles EganGet blog posts sent directly to your inbox.