Stephen J. Young - US grants

technology /technique development; microscopy; building /facility design /renovation; biotechnology; biomedical resource; biomedical equipment development; bioengineering /biomedical engineering;

We are developing a system to control the NCMIR Intermediate Voltage Electron Microscope (IVEM) remotely over the Internet and to easily distribute computationally intensive tasks such as those required for electron microscope tomography to high performance computers. A preliminary system was designed to test the feasibility of remote control and demonstrated at the SIGGRAPH and the Supercomputing conferences in 1992. As a result of the success of this demonstration and the interest and positive response from the NCRR we have been developing a more refined system. Our ultimate goal is to turn the IVEM and associated image processing and analysis software into a network resource available to researchers nationwide as part of the National Information Infrastructure. We expect that remote operation will increase access and use of the resource, facilitate more extensive data acquisition and analyses for collaborative projects, and allow more efficient use of the resource staff. To carry out this extensive project, we were encouraged by the NCRR to seek additional funding to support development of those aspects of the telemicroscopy project primarily related to remote operation and going beyond the development of enhancements for the local use of the microscope and associated computation. Accordingly, we submitted a proposal for a "Collaboratory for Microscopic Digital Anatomy (CMDA)" that was funded by the NSF as a National Challenge grant in September 1994. Mark Ellisman is PI and Sid Karin, Director of the San Diego Supercomputer Center (SDSC) and Don Greenberg, Director of Cornell University Program of Computer Graphics are Co-PIs. NCMIR and SDSC are primarily responsible for the overall system design of the CMDA including task management, network communication, enhancement of functions controlling the microscope, and the development of an improved user interface. Cornell is focusing on the development of advanced rendering methods that will in corporate error metrics. This NSF grant funds the major portion of the work on the telemicroscopy project. This funding provides additional benefits to the NCMIR beyond the develop of a remote control system since many of the CMDA activities, in particular those associated with microscope automation, are synergistic with the goals of the resource. In addition, the project supports and enhances a mutually beneficial interaction with the SDSC (and now the NPACI). As the result of a positive reviews by NSF site visit teams we were awarded an NSF Creativity Extension Award, adding two more years of support to the CMDA project which will carry it to September of 1999. An early version of the CMDA was demonstrated at the Supercomputing 95 conference in San Diego in December 1995 and is described in Young et al. (Int. J. Supercomputer Applications and High Performance Computing, 10: 170-181, 1996). The present CMDA (CMDA 1.0) incorporates routines developed by NCMIR to automate electron microscope focus and registration as well as the sequential operations required for acquisition of mosaics and tomographic tilt series. Initial testing with researchers at UC Berkeley, and the first intensive use began in March 1997 with collaborative researchers at the University of Oregon. Results of these tests have already indicated the value of remote interaction in conducting collaborative research. In addition, they have provided important feedback on the system leading to changes which are being incorporated into the next version, CMDA 2.0. The CMDA project has been recently recognized as a state-of-the-art next generation technology. NCMIR was i nvited to demonstrate the system at the Internet 2 conference held in October 1997 in Washington DC. The CMDA project has been recently selected as a finalist in the Next Generation category of the Global Information Infrastrucure competition, an industry-wide competition. The winner of this competition will be announced on April 20, 1998. After a second extensive design period focused on extending the functionality of the current system, the next version, CMDA II, is now undergoing testing and final development. As described in the CMDA II design document, the new CMDA II software architecture will incorporate numerous improvements into the current CMDA I system. These include the capability of collaboration involving an unlimited number of users, and enhanced system and data security. The CMDA II also incorporates image data compression for more rapid transmission of data acquired from the microsocpe during remote sessions. The new design incorporates a CORBA interface. This language and platform independent distributed processing interface will facilitate remote access to high performance computing for tomographic reconstruction, analysis, and visualization. In a related research project, partly supported by another NCRR-sponsored Resource grant to Sid Karin, the "National Biological Computing Resource" (NBCR), we have implemented both iterative and simple R-weighted tomographic reconstruction on the massively parallel 400 node Intel Paragon computer at the SDSC and have recently ported this program to the Cray T3E. We have also developed a remote interface these reconstruction platforms. A major emphasis in the current period is to bring on an increasing number of remote sites using the CMDA. To accomplish this, we have extended the initial design to incorporate web-based technology, including initial work with a web-based user interface using video images from the microscope to provide enhanced interactive control of the microscope to augment automated functions. The platform-independent web interface will enable use of the CMDA by a large number of researchers. As part of this design, we have written and are testing a data organization system, Gridset, based on the HDF file format. Tests of the web-based interactive microscope controller are currently underway with NCMIR collaborators in Oregon, Montana, and New York. Finally, during this year we have submitted a proposal to the NIH/NCRR call for supplementary grant to on collaboratory test-beds. This proposal will expand of the teleinstrumentation / collaboratory activities to include other NCRR resources and will benefit from resources provided by the newly established National Partnership for Advanced Computing Infrastructure.

The major goals for these software projects are to design, develop and implement programs for semiautomated segmentation and classification of 3D volumes, and to develop software for the visualization of three-dimensional structure. In the previous years of this project we have developed the program suites, Synu and Ducky, for performing sophisticated three-dimensional visualizations. These programs have been found to be extremely useful by biologists and have been distributed to many investigators throughout the US and internationally. During this year we have focused on improvements in software for segmentation and classification as well as programs for combining volume datasets for use in performing serial tomography and for merging volume correlated datasets acquired from the light and microscope (Perkins et al., 1997). Manual segmentation-tracing programs: In many light and electron microscopic studies, the objects of interest for three-dimensional reconstruction or analysis are delineated by membranes. These membranes are often extremely difficult to define using image processing techniques. Generally it has been found necessary to specify the location of the membranes using an interactive manual tracing method. The program XVOXTRACE developed at NCMIR is specifically designed for manually contouring features of structures in tomographic volumes by tracing with a mouse on planes of the volume along any one of the three major axes. Additional guides are provided to achieve accurate tracing. Thus, as the contours are traced, they can be viewed simultaneously superimposed on selected images from the original tilt series. Pairs of tilt-series images with the contours superimposed may also be displayed as anaglyphic or stereoscopic views. In addition, the accuracy of tracing can be evaluated by examining the contours superimposed on renderings of the volume viewed from an arbitrarily selected orientation. We have developing an improved version of this program which will include stereoscopic volume rendering as well as enhanced contour editing facilities such as automatic contour resplining. Capabilities for panning and zooming on a view of a selected region of the specimen in the volume slice, volume rendering, and tilt displays will also be provided. Semiautomated particle detection: Several on-going NCMIR research projects are interested in defining the three-dimensional distribution of particle-like structures such as synaptic vesicles. In collaboration with Dr. Ioana Martin (Dept. Computer Science, Univ. Indiana, South Bend) we have been developing semiautomated tool for detecting and defining particles. An initial tool was developed based on methods used by Dr. Martin to detect virus particles in two-dimensional electron micrographs. The tool was evaluated to detection of synaptic vesicles in tomographic reconstructions. Although partly successful, we expect that algorithms, currently under development, using three-dimensional information will be considerably more accurate. Measurements based on contours: XDEND is a graphics-based analysis tool developed at NCMIR to obtain measurements of the length, surface area and volume of objects defined as sets of contours such as those produced with XVOXTRACE. In addition, contours can also be derived directly from data volumes in those cases in which the structure can be segmented using simple built-in thresholding. Components of the structure may be resectioned into contours in the transverse, sagittal, or coronal axes to obtain the best orientation for length measurements. The length measurements are based on linear segments interconnecting the centroids of the contours. The program automatically detects branching segments. Objects may be viewed in perspective as contour outlines, centroids or skeletons. XDEND can also be used to reclassify and edit objects after recontouring. We are currently improving this program by providing resectioning along an arbitrary axis, and more accurate skeletonization methods. Software for serial tomography and volume merging We previously developed and implemented a procedure for combining serial section techniques with tomographic reconstruction (Soto et al., 1994). For a fixed tilt range and tilt increment, the resolution of the volume reconstructed by tomography decreases as the thickness of the object under reconstruction is increased. Thus, the resolution obtainable in reconstructions of very thick specimens is relatively poor. To overcome this limitation, we combined tomography with the serial section reconstruction technique to reconstruct large structures with adequate resolution. A series of consecutive sections is cut with a thickness sufficient to achieve the resolution desired in the reconstruction. Each section is reconstructed using tomography. Subsequently, the resulting volumes in the series are aligned and merged into a single volume. We recently developed two programs, MOG and TRANSMOG that are useful in the alignment of serial volumes. Fiducial marks are assigned to features in the raw or segmented sections from the consecutive volumes using the NCMIR fiducial marking tool, FIDO. MOG computes the polynomial function coefficients necessary to align the two sections, including a correction for in-plane warpage that may have occurred between the two volumes. The translations computed by MOG are then applied by TRANSMOG to the remaining sections in the volume. These have programs are also useful for aligning and scaling correlated three-dimensional light and electron microscope data and have been used for this purpose on a collaborative study examining alterations in cardiac muscle in a model of heart failure. Further, this software has also been found useful for assessing changes in structures due to shrinkage. The NCMIR program MOG can be used to obtain numeric estimates of changes in single sections resulting from beam exposure or specimen preparation.