In 1978, a nuclear-materials facility worker in Siberia made a mistake that cost him both of his arms. It was the result of a series of actions—a combination of intentional and unintentional deviations from procedure—and he was lucky not to have been killed. If only there had been a way for him to know beforehand that despite thinking he was about to do the correct thing, or at worst a benign thing, he was actually about to do the exact wrong thing.
The safety of workers in nuclear facilities is paramount, so protocols and regulations are prepared with tremendous forethought and detail. This painstaking planning is good for safety, but it can also bring down efficiency. Now, scientists at Los Alamos are using augmented reality and other technology to create a smart nuclear infrastructure that can streamline safety procedures while advancing the efficiency of materials handling in special nuclear facilities.
Augmented reality (AR) is one of the latest technologies to come along and disrupt just about every industry from automotive manufacturing to fashion design, from video games to nuclear safety. A user wears a special headset that has built-in cameras, speakers, and a transparent projection screen. Unlike virtual reality (VR), which replaces the real world with a fully simulated artificial environment, AR inserts simulated elements into the user’s actual physical environment. For example, a user might see, hear, and be able to read about the natural history of a life-sized 3D holographic rhinoceros, as it stands panting in the middle of her living room. Because AR technology dovetails digital content with the real world, it is a powerful tool for on-the-fly information storage and retrieval—with a swipe of the finger, a flick of the eye, or a spoken command, the user can see, hear, and interact with a whole new framework of information.
As Laboratory workers retire, younger workers naturally take their places. Many new workers are being trained to work with fissionable materials like uranium and plutonium—atoms with large atomic nuclei that can release energy by splitting into smaller nuclei. These materials are perhaps best known for their use in nuclear weapons, but they are also used in other technological applications, such as nuclear energy for electricity as well as heat and power sources for spacecraft.
The handling of fissionable materials—whether for the development of new technology or for meeting nuclear nonproliferation commitments, which necessitates chemical conversion or disposal of the fissionable materials—requires specific procedures to ensure containment. The danger is the radiation continually emitted by these materials, which in moderate doses can damage DNA and lead to certain types of cancer, and in higher doses (as might result from extreme improper handling) can kill human cells outright.
In order to protect fissionable-materials handlers, containment has to be multilayered and redundant. The material itself is typically contained inside a canister, which is contained inside a glovebox, which is contained inside a room, which is contained inside a facility. Each layer from the canister to the facility is specifically engineered for the safety of the workers and the security of the material.
Whether preparing samples for analysis, managing waste streams, or conducting chemical conversion, fissionable-materials handlers do most of their work inside gloveboxes. Gloveboxes are fully sealed steel chambers that have leaded glass windows and thick, protective gloves affixed to ports in the front panel. The worker interacts with the contents of the glovebox through the built-in gloves so that the protective barrier between worker and material always remains intact. Gloveboxes are connected in series to facilitate moving material between workstations, and special containers are also sometimes used.
With a swipe of a finger or a spoken command, a user can interact with a whole new framework of information.
In addition to maintaining complete containment, workers have to adhere to a number of other technical specifications that affect criticality safety. For example, there are strict limits on the quantity and type of fissionable material that can be present in any one location. This restriction is to avoid achieving a critical mass—the tipping point when a fission chain reaction goes from unsustainable to sustainable—which occurs when the quantity of material is too high. The limits are set conservatively, well shy of critical mass. While all procedural deviations are taken seriously, and violations by workers are rare, most are unintentional limit violations, not criticality violations, and thus have minimal impact on the safety margin.
To move a quantity of material from one room to another, the worker has to strategize a multistep route via connected gloveboxes and overhead trolleys. It’s an elaborate process that can take hours. The worker has to first plan a route by computer, then physically walk the route, visiting each location to check current material inventory, confirm limits, and determine feasibility, then return to the computer and submit the proposed move for approval. This process relies heavily on printed pages, so if the information on one page needs to change midway through the move, the worker has to leave the glovebox, monitor her hands to check for contamination, walk to a computer terminal, sign in, make the change, reprint the page, sign out, walk back to the glovebox, and resume work.
A glovebox worker can retrieve the information she needs without removing her hands from the glovebox.
“Operators have told me that the hours spent planning material moves are a big chunk of their overall effort and that time would be better spent on actual material processing,” says Los Alamos engineer Troy Harden. “We’ve streamlined the process as much as possible with the technology we currently have, but with new technology like AR we can improve efficiency more without adversely impacting worker safety or material security.”
As is often the case with infrastructure, the need to modernize was apparent before the technology to do so was ready. But now the technology is here.
Summon wizard
In an otherwise nondescript Los Alamos office suite, people carefully roam the halls, abruptly stopping to reach out and tap at…nothing. Muttering voices can be heard summoning invisible wizards, while other voices dismiss wizards. These are the scientists and engineers who are developing AR for nuclear criticality safety, and the fantastical voice commands “summon wizard” and “dismiss wizard” are how they initiate and terminate AR sessions.
For the past few years, Los Alamos engineer David Mascareñas and his team have been developing an AR system that will help track fissionable materials, update inventory databases in real-time, and provide on-the-fly information to fissionable-materials handlers. The commercial AR hardware they rely on is a headset called a HoloLens that was released by Microsoft in 2016. The HoloLens has been described as the first fully untethered holographic computer and is equipped with depth cameras, red-blue-green cameras, microphones, Bluetooth, WiFi, spatial sound, spatial mapping, and inertial measurement capabilities. Despite its unsurprising form—a ring of black plastic hardware worn about the head—the HoloLens is just the thing for Mascareñas’s team to make its AR work a reality.
Long interested in human-machine interfaces, Mascareñas began developing VR tools for nuclear criticality safety several years ago. But for industrial operations, he knew that AR was going to be the better approach once it came of age. Unlike VR, AR allows the user to maintain a view of the real world, which is important for safety.
“People have long imagined this sort of thing being possible,” he says, “it’s exciting that it’s actually happening now.”
The HoloLens has a built-in spatial-mapping feature, so as the user looks around, the computer creates a 3D map of the space and its elements—walls, furniture, other people—that continually adjusts itself as the user or the elements move around the space. The headset then projects, either floating in midair or overlaid on a nearby surface like a wall, holographic images typical of computer displays: word-processing documents, websites, maps, photo libraries, videos, etc. With small movements of her head, the user can position a holographic cursor onto whichever element she wants to click, then make a pinching gesture within the camera’s field of view, and the HoloLens opens the chosen file, folder, or window.
A glovebox worker can read updated instructions, log her actions, watch a video of a container being sealed, confirm the history of a sample, or instantaneously retrieve just about any information she might need without touching anything or leaving her workstation. With a detailed 3D map of the facility and real-time material tracking, she could even plan her material move right from her glovebox.
In addition to the standard sensors of the HoloLens, Mascareñas and his team have retrofitted several other sensors of specific import to nuclear criticality safety. They added a thermal imager, so a user can see that an object is hot. They also interfaced a higher-resolution camera, which, when combined with data from other sensors, can reveal internal structural differences that aren’t apparent by superficial inspection. Most recently, the team has demonstrated AR manual remote control: when the user moves her arm, a holographic arm copies the movement, and elsewhere, perhaps in a place unfit for a human arm, a robotic arm makes the same movement.
The potential of AR for nuclear criticality safety and other national security applications is hard to overstate. The untethering of a worker from a traditional computer workstation will allow unprecedented efficiency, especially when combined with other technological updates.
Failsafe framework
While AR is a flashy new technology, it’s not the only technology being tapped for the smart nuclear infrastructure. As the broad term “infrastructure” implies, the goal is a system overhaul, a full upgrade that will automate the more cumbersome operations of a special nuclear facility.
The worker was conducting measurements, an operation that included moving plutonium ingots into and out of heavily shielded containment canisters inside of a glovebox. The administrative limit for this glovebox was set at one ingot per canister, and it was difficult to see inside the canisters. Operators were supposed to complete their assigned operations themselves: Operator A was to remove six ingots from his canisters, one at a time and in a prescribed order, and then put new ingots into each of the empty canisters.
Instead, he removed and replaced two ingots, then asked Operator B to step in for a few minutes—a violation of procedure—and gave him only verbal instructions.
A less flashy, more foundational element of the smart nuclear infrastructure is the data-management system that operates behind the scenes, which is being reconfigured to work with AR. Tracking the immediate location of fissionable materials as well as their ownership—who or what is in control at the moment—is the main data-management task. After a material move, the same lengthy process it took to plan is used to manually update the material inventory database. This means that during the move and for some period of time after the move, the information in the database is wrong. Certainly there are mechanisms in place to handle this lag, but they are workarounds, and it would be better if it were just faster.
Right now the whole process is paper driven. There are “use every time” instructions that a worker has to carry with her at all times during an operation, and there are reference documents that must be readily available. Both categories would be accessible via AR.
“This is a tool for improving our conduct of operations across the board,” says Julio Trujillo of the Laboratory’s Plutonium Strategy Infrastructure Division. “Material moves are one example: a worker conducting a move, if asked by a manager, has to be able to access the latest revision of her reference document quickly. The improved managing of documents will help workers perform their work more efficiently.”
If the database is the backbone, and the AR capabilities are the sensory organs, then the connective tissue of the smart nuclear infrastructure, the thing that ties it all together, is the smart-cart system for automatic updating. This is a material-transportation system whereby push carts essentially “know” what they are carrying, where it is coming from, where it is going, and who is driving. The carts, as well as the users, canisters, rooms, hallways, gloveboxes, and safe boxes, all have unique near-field-communication (NFC) identification tags. These get scanned by NFC readers as they go by or get used, enabling the database to keep track of the activity in the facility in real time. Some elements, like canisters, gloveboxes, and safe boxes also have quick response (QR) codes that the camera in an AR headset can read; then the headset displays up-to-date need-to-know information in holographic form. A QR code on a canister could, for example, confirm the canister’s contents to a handler before the handler opens it. Although NFC and QR technologies aren’t new, putting them together, along with AR and the reconfigured database for a smart nuclear infrastructure, is a novel approach to nuclear criticality safety.
Intelligent infrastructure could prevent an accident like this at several points in the process. First, operators would not be able to leave their stations or step in for coworkers without the system recording the personnel change and alerting a supervisor. Second, written instructions and videos of each worker’s actions would be accessible via AR. Third, the canister would alert both operators to its contents, so they could not unintentionally violate administrative limits, much less achieve criticality.
The present location and ownership tracking system, while accurate on a daily or hourly time scale, still suffers from time lags due to slow material moves and manual information updates. The smart nuclear infrastructure, on the other hand, has everything codified by NFC tags, enabling changes in location and ownership to be logged as they occur. While updates will be instantaneous and automatic, material moves will still need to be meticulously planned beforehand. But with up-to-the-moment accurate information and integrated 3D facility mapping, even that task will be expedited, thereby achieving the goal of fewer hours spent planning and more spent on mission-based operations.
Vision for the future
The ideal scenario for Harden, Mascareñas, Trujillo, and the rest of the smart nuclear infrastructure team is full implementation within the next ten years. The longish rollout is due to a fly in the ointment: Cyber security in special nuclear facilities categorically outlaws wireless networking, and Bluetooth-enabled devices are not allowed at the Laboratory at all. HoloLenses use both of these technologies, so the proof-of-principle work has been done in a mock facility. It’s not going to be easy to reconcile this problem—it may require adapting to new AR hardware, or it may entail developing new security technology—so the team is setting a realistic time frame.
The next generation of facilities workers are already fluent in these kinds of technologies.
Even without the HoloLens part in place, the real-time database and smart cart system will enable pseudo-real-time updates and ensure that information is correct at all times. The material tracking and ownership updates will occur in real time, but retrieval of that information will still suffer a time lag because it will rely on desktop computer stations. But that’s enough of an improvement for the project to keep moving forward while the cyber security issues get worked out.
There are intermediate possibilities too. One option is to put a Bluetooth-disabled digital tablet in every room for workers to plan and visualize material moves. This isn’t as seamless as AR, but nor is it as cumbersome as a desktop computer terminal.
Despite the wireless-communication speed bump, the AR portion is moving forward. In the next year or two, a mock smart nuclear infrastructure tool will be implemented for training new personnel. The next generation of nuclear-facilities workers will consist largely of people who are already fluent in these kinds of technologies, so it makes sense to train them using a familiar platform.
PHOTO CREDIT: Michael Pierce
The technology is also highly transferable, and there are several nonnuclear-facility offshoots in the works. Conventional infrastructure—roads, bridges, and buildings—is generally built for a 50-year life span. When a crack in concrete appears, AR can help with safe inspection and documentation. A user can visually map and measure the crack from a safe proximity and store that information for future users. Additionally, Mascareñas recently collaborated with the city of Los Alamos and assistant professor Fernando Moreu from the University of New Mexico to use AR to measure non-rectangular areas of concrete in various public spaces. It proved to be less resource intensive than a survey crew and had comparable accuracy.
New technologies like AR present an opportunity to improve how infrastructure of any kind is built and maintained. People whose job it is to handle dangerous material or go into dangerous places will see their work experiences transformed. No longer will they be fettered by desktop computers or bound by paper printouts. The possibilities for intelligent infrastructure, especially when it comes to national security, do indeed seem to be endless.