RealWear: Assisted reality - saving time and lives on the frontline

“A lot of people think about RealWear as giving superpowers to frontline workers.”

Technology has become integrated into every aspect of the way we work. But what if you work in a job where your hands are tied doing other things, like operating machinery or climbing to the top of a radio tower? Checking an iPad, or dialling into Zoom becomes a little more challenging!

And that’s where Assisted Reality and RealWear’s incredible suite of head wearables comes in. Realwear devices are easy to clip on to hard hats, baseball caps or PPE and provide an easy-to-use interface similar to a smartphone which is powered entirely by voice. This provides a range of workers with tools to access additional information and enhance their work without them having to down tools. For example, construction workers are able to see the temperatures of electric cables in thermal mode, and paramedics have the ability to link up with specialist doctors to advise them on the scene, all hands-free.

To find out about this tech our reporter Joel Shupack visited RealWear’s HQ in Vancouver, Washington, to meet chairman and CEO Andrew Crosstowski (Chrostowski). We also hear from Brian Thompson, who heads up PTC’s CAD division. He explains why PTC’s CAD software CREO has been vital to RealWear.

Find out more about RealWear here.

Find out more about CREO here.

Your host is Paul Haimes from industrial software company PTC. 

Episodes are released bi-weekly. Follow us on LinkedIn and Twitter for updates.

Third Angle is an 18Sixty production for PTC. Executive producer is Jacqui Cook. Sound design and editing by Ollie Guillou. Location recording by Hannah Dean. And music by Rowan Bishop.

Transcript

Welcome to Third Angle, where assisted reality is saving time and lives on the frontline.

I’m your host, Paul Haimes from industrial software company PTC. In this podcast, we share the moments where digital transforms physical and meet the brilliant minds behind some of the most innovative products from around the world, each powered by PTC technology. 

Technology has become integrated into every aspect of the way we work, especially for people who spend a big chunk of time sitting at a desk for their job. But whether you’re updating a document on your iPad or dialling into Zoom from your laptop, this tech is not hands-free. So frontline workers like paramedics, engineers and mechanics, people who have their hands tied doing their job often miss out on the edge and convenience of computers. And that’s where assisted reality and RealWear’s incredible suite of hands-free voice-enabled head wearables comes in. And to find out about this tech, and in particular their flagship product the Navigator 520, we visited RealWear’s HQ in Vancouver, Washington to meet Chairman and CEO Andrew Chrostowski.

We are unique, I think, in positioning ourselves as a wearable company that is truly focused on the frontline professional. We care about productivity and safety, and we’ve designed our products around the idea that a connected worker is a safer worker and a more productive worker. 

Hello, and welcome to historic Fort RealWear. We are the headquarters of the RealWear company here in Vancouver, Washington, on the site of the army barracks here in Fort Vancouver. So this is the original Vancouver that was created when the British were here before they went north and created what is now British Columbia’s Vancouver. It really makes for a unique corporate environment here. We can do great walks outside, with the beautiful trees and the walkways. And yet we have this great historic facility that we’ve outfitted with some of our latest technology to develop the world’s premier wearable computer. 

Let’s go inside and take a look at the facility. So this is a rack of just a sample of the dozens of hard hats that are worn by frontline workers around the world. Typically the broad brim MSA style for mining, or hats that are used in climbing – so think about the guys who are climbing up on those 200-foot wind towers, generation towers – and then you have the traditional hard hat that might be used on a construction site. What we’ve done is our device fits all of these different methods by having adapters that we have produced that allow you to clip your AR device to any of these hardhat configurations around the world. So it makes it super easy to fit the way the workers are already functioning. So we like to adapt to how they work, not try to force them to adapt to ours. And just like these hard hats are designed for abuse, so is our product. And so it’s designed IP66 that you can drop it six feet onto concrete, you can wear it out in the rain, you can throw it in the back of your pickup truck, you can drop it in the mud, and it’s just going to work just like these hard hats. It’s designed to work for that frontline worker who needs it to be rugged. 

So let me give you an example of just how rugged we are. You wouldn’t think twice about dropping this hard hat onto the floor. That’s what I’m doing now. And we’ve built our Navigator 520 to be just as rugged. So I’m going to drop it the same way. And I don’t worry about what’s happening. 

Of course the best way to experience the Navigator platform is to wear it. We like to talk about the RealWear smile, which takes about 30 seconds from when you put this device on and you give it a command for the first time, and you see it respond seamlessly to your direction. But for those who can’t see it, we start with a horseshoe-shaped device. It fits easily on your head, it clips to hard hats, baseball caps, or a headband strap like you see here. It’s designed to be worn whether PPE is required or not. When you wear it, this device is super lightweight. It’s designed to be worn all day. 

So when a worker puts on a Navigator 500 or 520, they’re presented with a monocular display, meaning it can be seen on either eye, left or right. It appears as a 10-inch tablet held at arm’s length, full-colour display, HD, which allows them to see what you would see if you’re staring at your tablet computer. So a home screen with icons. The difference is instead of touch and gesture to control your navigation between those icons, you’re using voice. Every part of that screen is keyed to be a command. If I were to train someone to use a RealWear platform, all I have to tell them is, “Say what you see.” And so if you see a command on the screen, and you want to start my camera, you say, “My camera,” and it does it. It will do it in 17 languages. You can identify a workflow, a document, any app that’s there, and you can navigate between long menus, which is moving your head slightly to scroll through long lists, etc.

Augmented reality is an important part of understanding how we interface with a three-dimensional world, this is something that PTC has focused on a lot in terms of 3D. The challenge that you have with a frontline worker is that these very immersive ‘data first, reality second’ environments, take away your focus from what could be a dangerous environment, that six-inch step in front of you, that co-bot that’s moving next to you. And so RealWear has focused on assisted reality, which is a step down from that, that gives you that screen just like we were talking about where airline pilot can look down at their instrument panel, the driver can glance down at their dashboard, and then back out and keep their situational awareness going. So that safety portion of having full situational awareness, and the ability to get the information that you need, when you need it, was built in from the beginning with safety as a mindset. 

So I’m now wearing the Navigator 520. And I’m going to issue a command. And you see that while I’m talking here, there’s no wake-up word, it’s constantly listening. And it’s going to take what I say in context to what’s on the screen. So for instance, I’ll say, “My camera” and you get that chirp that indicates that the device has registered the command and is moving forward. It’s now loading the infrared camera, and I’m looking around the room at all of us in thermal imagery. So I’m actually seeing all the actual temperatures of everyone as we’re going through that. So I like to say this is giving predator vision to frontline workers. As I look at you, I can see that your eyes are the hottest part of your body. Right now your nose is colder, you can see what you’re wearing is insulating you from some of the temperature here, you can see that the device on your hip is the hottest thing. So you’ve got your microphone and recorder there are putting out heat. So you’re able to see not only what those temperatures are, but because I’m seeing this in a fused video, I’m seeing the optical overlaid with the thermal. I also have spatial relationships. So I can actually read, if you had a nameplate on your shirt, I’d be able to read that, where if I was looking only in pure thermal you would not. So this gives you the ability, if you think about a frontline worker going to an electrical box and opening it, you have to be able to read the numbers on the breakers as well as seeing what the breaker temperatures are, if you’re looking for a change there. So this fuse vision allows you to see both of those things, whereas pure thermal wouldn’t. 

We’re entering our specified engineering design space that we utilise here. We’re going to do a quick demo of why, if you’re going to have a voice controlled system, RealWear needs to be able to operate in a high-noise environment where a lot of frontline workers work. So as we begin to create some of these sounds here, and we measure these quite carefully, it’ll still respond to my voice. So once you run that power drill again for a moment. It’s about 85-90 decibels. And now while that’s running again, I’ll simply give a command “my camera”, and it responded even with a 94-decibel noise level, it still listened to my voice and actuated the command. That’s the kind of value in a frontline situation, you can imagine the frustration if you’re a frontline worker, working hands-free, giving voice commands, and you weren’t getting the response you want from the system. We’ve put a lot of energy, IP and technology into that noise reduction and voice recognition. 

Let’s give a great example of how you would use this in a real-life situation. So we’ve set up a Zoom call, we can demonstrate here in the building. When we want to connect a frontline professional to a meeting, it has to be easy for them to want to use it. In this case, we’ve done we’ve set up our demo with Zoom. So I’ve launched the hands-free app for Zoom in my navigator, and now all I need to do is scan a QR code, so I’m going to look and give the command “scan QR code”. It’s seeing the code, has now registered it, and I’m now in the meeting, in the waiting room waiting to be let in. It’s that easy. And I never had to take my hands off my tools or what I was doing to make that connection. 

We have a number of partners who have developed products that work directly with our RealWear system. Zoom is one of our key partners, and we worked with them closely to develop Hands-free for Zoom, which we demonstrated the ease of which it can be used with just a QR code to enter a room. We can do the same thing with PTC Chalk, we can do it with Microsoft Teams, we can work with TeamViewer, and a number of our partners who have optimised their software for our system. And that ecosystem really is what makes our system so valuable to our frontline workers and customers. 

Let’s go ahead and add some ambient noise. I’m holding a sound-level meter in my hand here. So let’s get the decibels. Currently, we’re about 52 decibels, we’re adding some noise, up to about 75 decibels, 77. So we’re working in a 70-decibel environment, it’s still hearing me clearly. If we go into the remote room while I stay here in the noisy environment, we can have a conversation over the Zoom call to see how the noise affects the other end. So now this is what I sound like in a quiet 50-decibel kind of environment, sort of a quiet background room, normal office. But now let’s go out on the job site. And now, how’s my voice sound at that end? Still clear. We can hear the drill in the background, but you’re coming in loud and clear. And I can hear you perfectly too. So that’s the advantage of being connected with the noise reduction that powers our voice system and the Navigator 500 and 520. 

We now have several generations of digital natives coming into the workforce. And today, it’s absolute cognitive dissonance to hand someone who lives their life completely in their smartphone and their tablet. They come to work, they have to put that down, and someone gives them a manual and a clipboard and a pen. Those days of accepting that’s the way we do it are just no longer making sense. So industries that adapt these technologies to the way people work in their personal lives and outside, and bringing those advantages to the workplace, are going to find workers more interested in doing some of these more physically challenging jobs, because they’re going to feel they have the right tools to get it done. So I think it not only addresses the companies' need to get things done better, safer, and faster, but it addresses the way people want to work today, which is to work smart. 

I think there’s two views of the future: one that’s human-centric, and one that sort of automation-centric. At RealWear, we really focus on the value that the person brings to these frontline roles. And if you think about a voice assistant like we have in our homes – many of us work with our smart tablets or smartphones – voice is going to become the interface to these AI systems that are going to support the systems of record. So when you think about the user, that remote intelligence is going to provide them with context and information as they need it in the workplace. So it’s going to make it even faster and easier for frontline professionals to get the job done, and do what they do best. Have that human-centric approach to things. 

And that’s not limited to industry think of medicine. As doctors are working through medical cases, they can now access that additional information. And we’ve demonstrated that actually at SWMG, which is the Special Warfare Medical Group, where they train all of the Special Forces medics, and they’ve actually demonstrated a medic in a moving vehicle, treating a patient connected to a satellite link to another doctor, and they’re walking them through trauma cases that they might not otherwise be able to save. And so really, you can apply this idea of doing more with what you know by accessing resources outside of your frontline environment to virtually every industry. It’s going to change lives. 

What gets us excited here at RealWear is we focus on the future of work and what it means to be able to have people work more effectively, more safely, to literally feel more empowered, more engaged in everything they do. And I think that, that seeing that future work being bright for these 2bn frontline workers that today are not connected, and having everyone recognised in the not too distant future, that if you have a frontline worker or professional who is not connected, you’re going to be at a competitive disadvantage to your competitors who do – and that’s what gets us up every morning to go tackle the challenges of the future of work.

PTC already knows RealWear in terms of our AR solution, Vuforia Chalk, and the ability for us to provide augmented content through the eyepiece of the RealWear device. But here, we’re actually looking into the design of the eyewear devices and how RealWear uses Creo when they start the engineering process. So it’s time to meet our expert, Brian Thompson, who heads up PTC’s CAD division. Now, Brian, we’ve spoken about Creo previously, but to recap, can you give the listeners a high-level overview, a refresher of Creo, and how it’s helping innovative companies like RealWear?

Sure, I’d be happy to. Thanks for having me on again. RealWear is a great company to talk about. I love their use of Creo. So Creo is PTC’s a flagship 3D-CAD application. 3D-CAD technology enables customers to build digital prototypes that are fully detailed in all respects in 3D, and make sure that all the components of the design will come together correctly as they intend, and also test the design for functional capabilities, work on designs, manufacturability processes, all associated to that core 3D design model. So it has huge time savings, and frankly it helps design engineers reduce a lot of risk in designs as they transfer them over to manufacturing by doing all of the testing and development and checking all in 3D before they make a single part. 

So in the case of RealWear, what we have found and taking a look at their use of Creo, they are really taking advantage of the fact that a powerful 3D design environment like Creo gives design engineers great tools for exploring multiple design concepts. It also gives them tools for being able to edit existing designs really flexibly, maybe in ways they didn’t anticipate. And not only that, to handle the complexity of say, multiple different types of users that have to work at the same time that have really different goals and their use of the technology, but need to contribute to the design in their own way.

And I think in specific terms, Brian, what we’re talking about here is their use of one of the Creo modules called flexible modelling, and another one on the advanced assembly capabilities that exist within Creo. So could you perhaps give the listeners a little bit more of a deeper understanding as to how that’s relevant for RealWear, and how they’re using those capabilities in Creo?

I want to start with advanced assembly because AX is typically associated with very, very large designs. It’s not necessarily only useful, though, for those types of scenarios. What it’s also really helpful with is defining and organising interfaces between parts of the design that need wildly different skillsets from a design point of view, and significantly different areas of expertise from a design point of view. And in a RealWear device, where you have a combination of highly ergonomic designs coupled with very sophisticated electronics, the places where different parts of the design come together and the types of contributors to the design that have to work together in a single environment to make sure that they’re all accommodating one another’s needs, that is also a very, very powerful concept that AX can help customers with.

Now, we’ve also touched on flexible modelling extension, perhaps you could just give a little bit of an insight into how they’re using that capability and what that means for the particular challenges that they face.

So flexible modelling assembly is a set of tools that allow design engineers to make changes to the design in a way that were not necessarily anticipated by the underlying parametric feature definitions. Of course, it also works for geometry-based designs that don’t have any underlying parametric features. But in this case, with RealWear, all we have is a situation where we have a very highly stylized product, because it’s going around a human head, that has to fit oftentimes inside an environment that has constraints that are really unknown to the design engineers from the very beginning. What we’re referring to is the integration of RealWear devices on someone’s head inside a helmet. And there is a tremendous variety of helmets that a design engineer at RealWear might have to anticipate fitting their device into. And I would submit that it might be very difficult to develop the parametric definition of the models of the components that make up the RealWear device in a way that could potentially anticipate all the ways that different design helmets might really just get in the way. And this is where flexible modelling extension can come into play because we can tweak, update, vary geometry in ways that were not originally anticipated. And what’s really powerful about it is you don’t lose the information that you added to the model by making the geometric change to accommodate, say, a particular helmet that just came out from a new vendor that RealWear might be partnering with, you actually capture that information in a feature. It’s just a very unique type of feature that makes a modification to the geometry. And so, RealWear gets the opportunity to catalogue how they’ve had to tweak designs according to various helmet configurations that they have accommodated as part of their product portfolio strategy in a way that is still very powerful, still maintainable and still can be catalogued as part of their design process.