Introduction
Advanced (scanning) transmission electron microscopes, (S)TEMs, can resolve individual atoms and measure atomic displacements with near picometer precision. Their capabilities are enabling groundbreaking discoveries in fields ranging from materials science to medicine. Scanning electron microscopes (SEMs) and dual-beam focused-ion-beam (FIB) instruments are multi-technique platforms that are being used for research ranging from in situ studies of mechanical properties to 3D reconstruction of biomaterials. Coupling these powerful instruments with high-speed digital communications promises to revolutionize the way scientists collaborate with each other and train the next generation of researchers. By duplicating the control interface and connecting to the microscope through a high-speed network, users outside of the microscope control room can operate the instrument as if they were sitting in front of it. This enables researchers in different locations to work together, sharing observations in real time, and enables trainees to gain the hands-on experience they require to become proficient in the use of these sophisticated techniques. The creation of a digital lecture theater that provides this capability at every seat results in an electron microscopy collaboratory that revolutionizes teaching, learning, and research in electron microscopy and characterization.
The Center for Electron Microscopy and Analysis at OSU
The Center for Electron Microscopy and Analysis (CEMAS) at The Ohio State University (OSU) is designed to provide the ideal environment for electron microscopy—designed by microscopists for microscopy (cemas.osu.edu). The innovative building design and quality infrastructure ensure that every instrument in the facility meets or exceeds manufacturer performance specifications (Figure 1). The physical infrastructure is further supported by an impressive intellectual foundation of nineteen permanent staff, including eight faculty members, nine research officers, two administrative officers, and two resident ThermoFisherScientific (FEI Company) specialists—a research applications engineer and a product marketing engineer. The expertise of this cohort spans all aspects of electron and ion beam instrumentation making CEMAS a “one-stop shop” that attracts users from OSU, as well as from universities, government laboratories, and industries across the globe.
Figure 1 The innovative building design and infrastructure at CEMAS ensures that instruments such as this monochromated aberration-corrected Titan 80-300 instrument is performing beyond the manufacturer specifications.
Remote microscopy
To meet the objective of providing wide-ranging access to top-of-the-line instrumentation, CEMAS has implemented a microscopy program that is having far-reaching ramifications in the way advanced microscopy capabilities are made available to a broader range of university and industrial clients, in both efficiency and economics. Furthermore, remote microscopy is having a major impact on the way microscopy and other characterization techniques are taught and the way both industry and academia collaborate. CEMAS continues to expand the program throughout Ohio and across the nation, and it is exploring the possibility of international links. The first phase of the program was instituted with funding from the Ohio Third Frontier Program and FEI Company, now ThermoFisherScientific, in partnership with the University of Dayton and the Air Force Research Laboratory at Wright Patterson Air Force Base [1].
Remote Collaboration
Having created a world-class facility, CEMAS wanted to extend its use to the maximum number of potential users, both on campus and off. The goal was to make the advanced instrumentation and skilled technical support available to users in all disciplines within the University and, equally important, to industrial, academic, and government across the nation.
In this age of high-speed communication, one must question the need to travel long distances in order to use even the most advanced microscopes. Scientists and researchers routinely hold video conferences and are quite used to participating in remote collaborations. So why not do the same with the operation of the microscopes at CEMAS? With this in mind, researchers and staff tackled the problem of creating a practical remote capability for CEMAS’s extensive collection of advanced electron microscopes.
Earlier remote operations
The idea of remote operation of electron microscopes is not new. There have been many “demonstrations” at microscopy conferences of data collection on a vendor’s exhibition booth from a microscope that is hundreds or even thousands of miles away. However, these are often a long way from being “live and real-time” operation; rather, they demonstrate that “over the shoulder” or “remote desktop” viewing is possible, but “live interactive control” of the microscope is much more challenging. Earlier efforts to develop “telemicroscopy” in universities and research centers were often limited either by the network bandwidth/latency or could only give limited control of the microscope functions [Reference Voelkl2–Reference Leyva-Porrasa9]. For example, while at Imperial College London, the lead author demonstrated the use of a microscope at Oakridge National Laboratory in Tennessee to the UK science minister during a visit to the Materials Department in London but with only limited microscope functionality [Reference Perkins10]. To our knowledge, the first implementation of a classroom environment with “live and interactive” microscope control was at Carnegie-Mellon University where De Graef et al. designed and implemented a remote laboratory where microscopes were “hardwired” to multiple stations to allow groups of students to be trained on SEM techniques [Reference De Graef11]. While this approach was effective locally, the approach was not scalable beyond the host institution. The topic has remained of interest to the community, and Sinclair and others have organized series of international meetings to discuss this topic and identify best practice [Reference Sinclair12].
Time delay
At CEMAS the first step was to identify the technical challenges that needed to be overcome. It quickly became apparent that reliable, high-speed communication between CEMAS and the remote station was an absolute requirement. Many researchers have experienced an international call where there is a time delay in the connection. Such a delay, especially in the midst of detailed technical conversation, can be very frustrating. Now imagine that same delay introduced into the control of a microscope. When the operator adjusts the focus or makes an alignment correction while looking at the output of the detector on the computer screen, he or she expects the response on the screen to be instantaneous. If the operator is sitting in a room 200 miles away and there is any delay in the response, operating the instrument quickly becomes inefficient and execution of a complex experiment becomes almost impossible. In this scenario, most CEMAS users will eventually conclude, “Forget it, I’m going to jump in the car and drive to Columbus.”
Network latency
The network communication required for remote microscopy must have two characteristics. First, it needs relatively high bandwidth because each image may contain many megabytes of data. But it needs more than just simple bandwidth. Networks make connections through switches, routers, and firewalls. The network path length and each device introduce delay. This delay is referred to as the network’s switching latency. Other factors such as congestion can cause jitter, where the latency is dynamic or, in the worst case, causes packet loss. The key to delivering the speed and reliability needed for remote microscopy is a high-quality connection, with low latency and minimal jitter and packet loss. If the latency is consistently low enough, then the user can sit in a remote site anywhere in the world and feel a suitable response to operator commands.
High-speed connections
The Ohio Academic Resources Network (OARnet, www.oar.net) and Internet2 (www.internet2.edu) provided the solution to this challenge (Figure 2). CEMAS has a direct connection to the OARnet, a 100-gigabit-per-second backbone that runs throughout the state of Ohio. OARnet was designed to move big data quickly and efficiently with minimal latency. It provides high-speed/low-latency connections to every major city in Ohio. Beyond Ohio, OARnet connects to Internet2, a nationwide high-speed network.
Figure 2 By utilizing the bandwidth and low latency of the OARnet and Internet2 networks, CEMAS has been able to install -control stations in locations hundreds of miles from Columbus, Ohio, that allow users to operate the microscopes without any perceptible delay.
First remote access partners
Having identified an appropriate network solution, the next step in developing a remote microscopy capability was to prove the concept through a practical demonstration with a remote partner willing to work through the development process. The University of Dayton (about 90 miles from Columbus) and the Air Force Research Laboratory at Wright Patterson Air Force Base, near Dayton, agreed to participate.
Working with FEI Company (ThermoFisherScientific), the development team designed a remote-control station that replicates a set of microscope controls, without the microscope. There are a variety of video encoders and decoders and other hardware and software that maximize the speed of the signal and handle all the different detectors. After a thorough process of debugging and optimization, the first remote-control station was installed at the Air Force Research Laboratory (Figure 3), and a few weeks later the second went to the University of Dayton [1]. Both remote-control stations are currently in use. Users send their specimens by courier to the microscopy facility in Columbus. When the sample is in the microscope, the user receives an indication on the network that the system is ready to go, and then they can use the microscope in the same manner as if they were sitting in front of it.
Figure 3 Image of the operation station installed in the Air Force Research Laboratory at Wright-Patterson Air Force Base in Dayton, Ohio.
Expanding the program
Having successfully proven the concept of functional remote microscopy, researchers at CEMAS are now discussing expansion of the project with federal funding agencies. The National Science Foundation (NSF) is very familiar with the challenges facing programs such as the Major Research Instrumentation scheme and is keen to see how the capabilities at CEMAS can be developed and extended. Ultimately, this approach could help NSF deliver higher research impact with a limited capital budget. For example, with NSF support a remote system was successfully installed at North Carolina Agricultural and Technical University (N.C. A&T) in June of 2018. This system, which is the first to be installed outside of the OARnet network, is able to control the instruments at CEMAS with the same high-quality operational standards as the direct OARnet connected systems at the University of Dayton and the Air Force Research Laboratory.
Ohio State University and N.C. A&T, an HBCU institution with an outstanding track record of addressing diversity in STEM education, have signed a strategic research partnership agreement that covers many common areas of activity, including electron and ion beam characterization. N.C. A&T, the first university partner outside Ohio, is home to the NSF Engineering Research Center for Revolutionizing Metallic Biomaterials (ERC-RMB). The link with CEMAS will allow ERC-RMB students to connect to our facility remotely, take part in interactive courses in advanced microscopy, and use CEMAS instruments in their research. The partnership with N.C.A&T is an excellent example of remote microscopy’s potential to bring together two world-leading research organizations to maximize the impact of research activities and federal funding.
Going Worldwide
“If this works in Ohio is there any reason why it can’t work in any state?” This is a question that has been asked by program managers in several funding agencies. The follow-on question is: “If this works for electron microscopy can it work for other expensive techniques those agencies are frequently asked to support, such as nuclear magnetic resonance (NMR) or beam lines for X-ray diffraction?” Though microscopy is at the heart of CEMAS’s efforts, the concept clearly goes way beyond that.
Teaching microscopy
This is not only about providing researchers with access to advanced instrumentation for scientific research, this is about changing the way electron microscopy is taught and, ultimately, how the operation of any complex system is taught. Most electron microscopy teaching labs feature an electron microscope in a confined space with an “expert” operator sitting at the controls in front of the microscope, and four or five students crowded together peering over the operator’s shoulder. The expert demonstrates something, and then the students take turns at the controls duplicating the exercise. That’s how microscopists are trained today, and that’s how they have been trained for the past 50 years. It is tremendously inefficient. It limits the number of students to those that can clearly see over the instructor’s shoulder, and the presence of so many bodies close to the instrument can actually interfere with the instrument’s performance. Advanced microscopes are typically controlled remotely from a neighboring room because humans perturb the environment enough to interfere with the microscope’s ability to perform at the highest level. If a TEM can be operated remotely from the room next door, why not make that room larger and set it up specifically to enhance the training experience?
The OSU-FEI Electron Microscopy Collaboratory
Classroom-laboratory
A digital lecture theater at CEMAS, completed in 2016, seats 33 students and has a video wall at the front that is 6 feet tall and 20 feet wide (Figure 4). An instructor can deliver a lecture and present examples from the microscope on the screen at the same time. Also visible are all the control panels of microscope, and the outputs of the detectors and the imaging system (Figure 5). The instructor has a set of digital microscope controls at the lectern, and all of the students in the theater have access to full microscope controls, in some instances from their own laptop or tablet. So rather than describing the theory and practice of operating the microscope to the students in the lecture hall and moving to the microscope in another location to demonstrate, the instructor can take control of the microscope and show the students the use of the instrument in real time. In turn the instructor can “flip” from the classroom to the laboratory environment, for example, inviting Francesca in seat 14: “You now have control. Will you please repeat what I have just shown you?” This not only overcomes the space limitations of microscope rooms, by creating an environment that is conducive to engaged learning, it changes the way in which students learn advanced characterization techniques. Trainees are sharing in the peer-group experience, not just relying on the one-to-one interaction with the microscope expert. The opportunities for education research on how students learn most effectively in this hybrid environment are currently being explored at CEMAS in collaboration with colleagues in the Department of Engineering Education at OSU.
Figure 4 Image of the OSU-FEI Electron Microscopy Collaboratory space at CEMAS. Multiple operation stations allow every microscope—TEM, SEM, FIB—to be controlled from this hybrid laboratory.
Figure 5 Operation of the aberration-corrected microscope in the OSU-FEI collaboratory space allows all members of the audience to learn microscopy and interact with the microscope in an environment that is conducive to active and engaged learning.
Remote students
The concept of the OSU-FEI Electron Microscopy Collaboratory goes beyond this. The lecture space has world-class audio and video conferencing capabilities, and with the OARnet connection instruction doesn’t need to be restricted to students in that room. The students could just as easily be at the University of Akron or in Cincinnati at some industrial or government facility. They can all have the same connection. All are able to see and interact, not only with the instructor, but also with the other users in the lecture theatre or anywhere in the world. Each student can see exactly what’s going on with the microscope. Most importantly, the students get an intimate feel for the operation of the microscope that cannot be gained in any way other than by turning the knobs and watching the result. This has the potential to revolutionize the way these kinds of skills are taught at the university level and provides new opportunities for highly effective workforce development and lifelong learning for engineers and scientists in technologically driven industrial sectors such as materials, manufacturing, pharmaceuticals, and medicine.
The entry level cost of the equipment necessary to create and install a remote operation station is relatively modest (<$100K) compared with the cost of advanced electron microscopes. This provides an opportunity for educators and researchers at universities and colleges with limited resources to gain access to over $40M of advanced instrumentation at CEMAS to support their teaching and advance their research programs. This is the democratization of electron microscopy, which leads the way to the democratization of science.
A new model for collaboration
The OSU-FEI electron microscopy collaboratory and our developments in remote microscopy provide a new and powerful platform for collaboration. We are transforming teleconferences to “telecollaborations.” The real-time operation of the microscope allows researchers to connect from different locations, not just to learn and work on their own, but to collaborate. Users can be in one location or in multiple locations; they can all talk to each other and see the screen, and anyone can take control.
Multidisciplinary research collaboration is essential in order to progress in many fields of research and to tackle the grand challenges facing the world in the 21st century. Whether it is to address drug development, clean water, sustainable manufacturing, or climate change, researchers from a wide range of disciplines and from an even greater number of organizations are trying to find efficient ways to work together. Our facility presented its vision at the 2015 Materials Research Society fall meeting as part of a symposium entitled “Engaged Learning of Science and Engineering in the 21st Century.” Many researchers are embracing the use of technologies that we take for granted in our regular lives—cell phones and tablet computers, for example—in teaching, learning, and research. Taking advantage of these technologies for collaboration and education is the vision behind the development of remote microscopy at CEMAS, and remote microscopy is only the beginning.
Conclusion
Learning advanced electron microscopy techniques traditionally has been a one-on-one activity for a student sitting in front of an instrument with an expert microscopist. The OSU-FEI electron microscopy collaboratory allows students in a full classroom to participate in this learning process—each person operating a set of microscope controls identical to those at the actual instrument. This innovation multiples the number of individuals who can experience advanced electron microscopy first hand. Moreover, students and collaborators may be miles away in another city and still operate the microscope as if they were sitting in front of it. This is transforming teaching, learning, and research in electron microscopy and is helping democratize scientific research and STEM education by allowing all institutions, large and small, to access a world-class microscopy facility to advance their educational and research programs.
Acknowledgements
CEMAS would like to acknowledge financial support from The Ohio State University, the Department of Materials Science and Engineering at OSU, the Ohio Third Frontier Program, FEI Company (ThermoFisherScientific), and the Air Force Research Laboratory (AFRL) to help create the collaboratory and install remote operation systems at AFRL and University of Dayton. We acknowledge support from NSF (Award Number: 0812348 “Remote Microscopy Station to Provide Direct Access to Image and Probe-Corrected Transmission Electron Microscopes for Transformative Atomically-Resolved Analysis of Structural and Functional Materials for Future Materials Innovation and Manufacturing”) to create the remote operation system at North Carolina Agricultural and Technical University.
