- Open Access
Active microscope stabilization in three dimensions using image correlation
Optical Nanoscopy volume 2, Article number: 3 (2013)
Super-resolution microscopy techniques are often extremely susceptible to sample drift due to their high spatial resolution and the long time needed for data acquisition. While several techniques for stabilizing against drift exist, many require complicated additional hardware or intrusive sample preparations. We introduce a method that requires no additional sample preparation, is simple to implement and simultaneously corrects for x, y and z drift.
We use bright-field images of the specimen itself to calculate drift in all three dimensions: x, y and z. Bright-field images are acquired on an inexpensive CCD. By correlating each acquired bright-field image with an in-focus and two out-of-focus reference images we determine and actively correct for drift at rates of a few Hertz. This method can maintain stability to within 10 nm for x and y and 20 nm for z over several minutes.
Our active drift stabilization system is capable of simultaneously compensating x, y and z drift through an image-based correlation method that requires no special sample treatment or extensive microscope modifications. While other techniques may provide better stability, especially for higher frequency drift, our method is easy to implement and widely applicable in terms of both sample type and microscopy technique.
In recent years a number of optical microscopy techniques have pushed image resolution to values smaller than 100 nm. Such super-resolution methods have already contributed to and promise to advance further biological sciences as they provide details at the molecular level without the invasiveness of electron microscopy (Hell 2009; Huang et al. 2010).
One approach of super-resolution microscopy, initially named as stochastic optical reconstruction microscopy (STORM) (Rust et al. 2006), photoactivated localization microscopy (PALM) (Betzig et al. 2006), or fluorescence photoactivation localization microscopy (FPALM) (Hess et al. 2006), relies on stochastic activation of sparse sets of individual fluorophores and subsequent determination of their positions. This approach has been under active development (Egner et al. 2007; Heilemann et al. 2008; Shroff et al. 2008; Lew et al. 2011) and has routinely demonstrated precisions at the 20–30 nm scale in localizing single fluorophores. Three -dimensional single-molecule localization further adds in the capability to resolve 3D structures in a cell (Huang et al. 2008b; Juette et al. 2008; Shtengel et al. 2009). On the other hand, super-resolution image reconstruction typically needs thousands to tens of thousands of camera frames to accumulate a sufficient number of fluorophore positions. Consequently, the total acquisition time can run to many minutes. Commercial microscope frames, however, have not been designed to maintain nanometer-scale stability during such a long acquisition time. Therefore, sample drift due to thermal fluctuation, mechanical relaxation, or vibrations may easily overwhelm the localization precision. Hence, a critical component in the practice of super-resolution microscopy is stabilizing the microscope during acquisition and/or removing residual drift during image analysis.
Many methods exist to address this stage drift issue. They differ in either how the drift is measured (using fiducial markers or inherent sample properties) or how it is corrected (software-based post-acquisition correction versus hardware-based active stabilization during acquisition). Drift tracking using fluorescent or scattering fiducial markers has been widely used in either software correction or hardware stabilization (Carter et al. 2007; Pertsinidis et al. 2010; Lee et al. 2012), demonstrating extremely high stability. Nevertheless, adding fiducial markers to the same focal plane as the structure of interest is sometimes impractical, especially when it is inside a large cell or thick tissue. In this case, one must use intrinsic sample contrast. For example, the reflection at the coverglass surface can tell the focal drift in the z direction as long as the refractive index of the mounting medium differs substantially from that of the coverglass. This scheme has been implemented in a number of commercial microscopy systems. In the xy direction, software-based sample drift correction has been enabled by correlation analysis of wide-field fluorescence images (Bates et al. 2007), bright-field images (Mennella et al. 2012), or the molecule coordinates (Huang et al. 2008b; Geisler et al. 2012; Mlodzianoski et al. 2011). Despite the advantage of requiring no additional sample treatment, these methods have limitations: they usually have a correction time-step substantially slower than the camera frame rate. More importantly, software methods will fail if drift in z exceeds the focal depth, which is just a few hundred nanometers. Therefore, an image-correlation-based method for hardware stabilization in all three dimensions would be highly desired.
Here we demonstrate a simple system to accomplish this goal. By continuously collecting infrared (IR) transmitted-light images, our method actively stabilizes the sample in x, y and z, without added complications to either sample preparation or optical setup. In combination with high-refractive-index media to reduce the spherical aberration caused by glass-media refractive index mismatching (Huang et al. 2008a), we have demonstrated 3D super-resolution imaging inside a Drosophila embryo.
3D drift tracking by image correlation
As with many other techniques to measure image drift, we employ cross-correlations using fast Fourier transforms. Cross-correlating two images yields an equally-sized image with a maximum whose position depends on the relative shift between the two images and whose intensity depends on their likeness. By tracking the peak position of the correlation function, we can determine the xy drift in the image plane.
Here, we extend the practice of image correlation to also monitor out-of-plane drift. Defocusing caused by z drift perpendicular to the image plane results in a decrease in the peak height of the correlation function. Nevertheless, this decrease by itself does not indicate the direction of the drift (positive or negative z). Therefore, we correlate the image with an image taken above the focal plane and another image taken below the focal plane. An upward sample drift will lead to an increase in the peak value of the first correlation and a corresponding decrease of the second, whereas a downward drift will have the opposite effect. The difference of these two peak values then uniquely determines the amount and direction of z drift.
In practice, before data acquisition begins, we record three reference bright-field images at, below and above the focal plane (denoted as I 0, I -, and I +, respectively, Figure 1). The in-focus and out-of-focus images are separated by a z distance of d. During data acquisition, bright-field images are simultaneously captured, with the n th frame denoted as I n . For convenience, we represent the correlation function of two images I a and I b as
For each new bright-field image acquired, I n , three correlation functions are calculated: with the in-focus reference I 0 and with each out-of-focus reference I - and I +. The resulted correlation functions, C -,n , C 0,n and C +,n , are then each fit to a 2D Gaussian function in a 20-by-20 pixel window to find both the peak value, PV, and peak position, PX and PY. The x and y drift can be derived from the in-focus correlation:
When the z drift is small, it can be approximated by a linear combination of the two out-of-focus correlations:
The coefficient α can be measured from the three reference images:
Our super-resolution microscope, as described previously (Mennella et al. 2012), is custom-built from a Nikon Eclipse Ti-E inverted microscope. The microscope is placed on a vibration isolation table (ST-UT2, Newport) in an optics room without special temperature control and located on the 3rd floor of a building. We equipped the microscope with a piezo z stage (Nano-Z100, Mad City Labs) mounted on a motorized xy stage with a step resolution of 10 nm (SCAN IM 120×100, Märzhauser). Four lasers (405 nm, 488 nm and 642 nm, Vortran Lasers, and 561 nm, Sapphire 561–200, Coherent) were combined and focused to the back focal plane of the objective (Nikon 100× Plan Apo VC NA 1.4) for activation and imaging. A dichroic mirror (zt405/488/561/640rpc, Chroma or T660lpxr, Chroma) and an appropriate band-pass filter (ET702/75 m, Chroma, for fluorescence emission of Alexa Fluor 647) ensure the separation of the excitation light and the fluorescence emission. A cylindrical lens (700 mm focal length) is inserted between the microscope body and the image plane at the side port to create astigmatic aberration that allows 3D single-particle localization and 3D super-resolution microscopy (Huang et al. 2008b). Super-resolution images are recorded with an electron multiplying CCD camera (EMCCD, iXon+ DU897E-C20-BV, Andor) using a custom-written software in Python.
To incorporate our new drift correction scheme, we have made the following modifications (Figure 2): an IR LED (850 nm; M850L2, Thorlabs) to replace the trans-illumination halogen lamp for bright-field imaging, a short-pass filter in front of the EMCCD to block the IR light (ET750SP-2p, Chroma), a dichroic mirror (765dcspxr, Chroma) to direct the IR light collected by the objective through a band-pass filter (HQ850/90 ×, Chroma), and a 75 mm lens to image the IR light on an inexpensive CCD camera (DMK 31AU03, The Imaging Source). The IR wavelength was chosen so that it is long enough to avoid overlap with photoactivation or fluorescence emission wavelengths, and short enough for ordinary silicon-based CCD or CMOS cameras to have sufficient sensitivity. We used a custom-written program in Python to collect and analyze the IR images in real time. The program controls the piezo z stage and the motorized xy stage to actively compensate the drift of the sample.
Sample preparation and imaging
We have used our microscope to stabilize two different samples: fixed Drosophila S2 cells grown on a coverglass-bottom petri dish, and dissected, fixed Drosophila embryo sandwiched between a glass slide and a piece of coverglass. To evaluate the performance of our focus stabilization system, we added 100-nm-diameter TetraSpec fluorescent beads (Invitrogen) to the sample and allowed the beads to adsorb to the coverglass surface. Facilitated by the cylindrical lens, we tracked the 3D positions of these beads using the EMCCD camera, and measured the stage drift by averaging the movement of multiple beads.
Using STORM and our drift correction method we imaged Drosophila embryos expressing GFP:: Cdc42 in the aCC/RP2 motoneurons stained as described previously (Kamiyama and Chiba 2009). Briefly, the embryos were fillet-dissected, fixed with 4% paraformaldehyde for 10 min, and stained by anti-GFP antibody (Invitrogen) on a glass slide. Following incubation with the anti-rabbit secondary antibodies conjugated with Alexa Fluor 405 and Alexa Fluor 647 (Invitrogen) for 2 hours, they were post-fixed with 4% paraformaldehyde for 5 min. To increase the refractive index of the mounting media we used a solution of 80% 2-2' Thiodiethanol (TDE) along with 100 mM mercaptoethylamine at pH 8.0, 5% glucose (wt/vol) and oxygen scavenging enzymes (0.5 mg/ml glucose oxidase (G2133, Sigma-Aldrich) and 40 μg/ml catalase (C30, Sigma-Aldrich).
Results and discussion
Tracking z drift using image correlation
First, we demonstrate how out-of-focus image correlation can reflect the stage drift in the z direction. Figure 3a shows how the peak value of the correlation function changes when we scan the piezo stage in ±1 μm range. By choosing the two out-of-focus reference images at ± 700 nm from the focal plane, Figure 3b demonstrates that small z changes (< 200 nm) leads to almost linear changes in ζ. Therefore, Eq. (4) can be efficiently used to follow stage drift in the z direction.
The ζ - Δz plot in Figure 3b provides a way to precisely calibrate the response factor α. In practice, we choose to use Equation 6 for simplicity. We note that the exact value of α does not need to be precisely known. In fact, the relationship between Δz and ζ may not even be linear over the range of z-positions about which the sample may drift. While overestimating α will result in stage oscillations, underestimating it will limit the speed in which correction takes place. We find that this method most appropriately compensates for random, low-frequency drift. Therefore, we chose to err on the side of slightly underestimating the relationship between displacement in z and the peak value change of the image correlations.
Active stage stabilization in all three dimensions
By feeding back the measured Δx and Δy to the motorized xy stage and Δz to the piezo z stage, we were able to continuously correct for the stage drift. We typically select a 128 × 128 pixel (50 × 50 μm) high contrast region from the acquired 1024 × 768 bright-field image to calculate the correlation functions. For thin biological samples that are mostly transparent, the contrast of bright-field images actually arise from the refractive index inhomogeneity in the defocused region in a similar mechanism as phase-contrast microscopy. Because this contrast is usually very low, to reduce the error of drift tracking, we use a weighted, rolling average of 10 to 20 correlation time steps to compute the response to drift. While this limits our ability to correct for higher frequency drift, it enables use of samples where contrast is minimal, e.g., in specimens mounted in high refractive index media. Our low-cost CCD camera has a frame rate limited to 30 Hz. With additional time needed to perform the image correlation, Gaussian fitting, and taking into consideration the response time of our mechanical xy stage, our actual feedback rate is between 5 and 10 Hz. Obviously, a higher feedback rate can be easily achieved with a faster IR camera and a fast-responding piezo xyz stage.
To test the performance of our active drift correction system, we tracked the 3D positions of fluorescent beads added to a Drosophila embryo sample using the fluorescence images collected by the EMCCD camera. With the feedback disengaged, Figure 4a shows that over a time period of 500 seconds, the average coordinates of 4 beads drifted more than 100 nm. This scale of drift will undoubtedly impair the performance of super-resolution microscopy experiments. On the other hand, it is evident that the drift mainly occurred at the time scale of seconds or longer. Therefore, our 5 to 10 Hz feedback rate should be sufficient to stabilize the stage against the drift. Indeed, by engaging the feedback (Figure 4b), the stage was locked within a root-mean-square position error of about 10 nm in the xy plane and about 20 nm in the z direction over a time period of 300 seconds.
3D STORM in Drosophila embryo
To demonstrate the application of our active stabilization system, we performed 3D STORM of the small GTPase Cdc42 in the aCC and RP2 motoneuron in Drosophila embryo. Even after dissection, these neurons are about 10 μm deep into the embryo. Imaging at such a depth in aqueous buffer suffers greatly from the spherical aberration caused by the refractive index mismatch between the coverglass (n = 1.52) and the medium (n ≈ 1.34) (Huang et al. 2008a). Therefore, we used 80% TDE in water (n = 1.47) as the base of our imaging medium (Staudt et al. 2007) to get rid of the spherical aberration problem. We have verified that the photoswitching characteristics of Alexa Fluor 647 are largely unaffected as long as the pH of the medium is carefully adjusted to 8.0 (data not shown).
Our high-refractive-index medium substantially reduces the reflection at the coverglass-medium interface. Existing focus stabilization systems thus do not function reliably because they rely on this reflection. For example, the Perfect Focus System on our Nikon microscope was unable to engage. In addition, because the structure of interest lies inside the sample, adding fiducial markers to the same focal plane as the neurons is impractical. On the contrary, our imaging-based stage stabilization allowed us to image at several μm past the coverglass. The 850 nm IR light intensity at the sample was measured to be 4.9 mW/cm2. This low intensity, together with its long wavelength, ensures that fluorophores will not be activated by constant IR illumination. For comparison, our 642 nm excitation laser power density was about 0.5 kW/cm2 at the sample, and the 405 nm activation laser power density was about 50 mW/cm2. Overlaying z slices acquired at multiple focal planes further produced a super-resolution image covering a 2 μm z range (Huang et al. 2008a). In Figure 5 we show the resulting image which required no post-processing drift correction. By analyzing the correlation of localized molecule coordinates (as described by Huang et al. 2008b), we have found that the residual drift is less than ±10 nm during the 6 min acquisition for one focal plane (Figure 5b).
In summary, we have shown a simple approach to actively stabilize the microscope sample against drift in all three dimensions during super-resolution data acquisition. In our current implementation using bright-field imaging and a low cost CCD camera, we can reduce the drift in xy to about 10 nm and in z to about 20 nm over the duration of several minutes. The principal advantages of our method are its ease of implementation and wide applicability: we were able to measure drift in all three dimensions simultaneously without adding much complexity or cost to our microscope and with no special sample preparations.
It is clear that the performance of our current stabilization system design is limited by the low contrast of bright-field images. Although bright-field images can contain a large number of photons, most of them contribute to a homogeneous background that does not provide any information about the stage position. On the other hand, our design is flexible enough to use any imaging modality that is sensitive to sample translation and defocusing. For example, it can be easily adapted to contrast enhancement methods such as differential interference contrast (DIC) microscopy (by adding in the polarizers and the Wollaston prisms in the IR path) or dark-field microscopy (by switching to a dark-field condenser). These imaging modalities, combined with a faster camera and a piezo xyz stage which has faster response and higher resolution in the xy direction compared to our current motorized stage, should achieve much better stabilities and enable faster feedback rates for more demanding applications.
Finally, although our sample stabilization method is intended for fixed sample imaging, it is worth commenting on its applicability to live microscopy. The challenge here is that morphological change of a living sample could be misinterpreted as a drift in the z direction. Nevertheless, our method could be useful in short-term imaging of thick tissue sections whose overall morphological change can be slow. In addition, the same algorithm can be used for drift tracking and correction with fluorescent or scattering fiducial markers in a similar way as in Lee et al. (2011).
Stochastic optical reconstruction microscopy
Photoactivated localization microscopy
Fluorescence photoactivation microscopy
Differential interference contrast.
Bates M, Huang B, Dempsey GT, Zhuang X: Multicolor super-resolution imaging with photo-switchable fluorescent probes. Science 2007,317(5845):1749–1753. 10.1126/science.1146598
Betzig E, Patterson GH, Sougrat R, Lindwasser OW, Olenych S, Bonifacino JS, Davidson MW, Lippincott-Schwartz J, Hess HF: Imaging intracellular fluorescent proteins at nanometer resolution. Science 2006,313(5793):1642–1645. 10.1126/science.1127344
Carter AR, King GM, Ulrich TA, Halsey W, Alchenberger D, Perkins TT: Stabilization of an optical microscope to 0.1 nm in three dimensions. Appl Opt 2007,46(3):421–427. 10.1364/AO.46.000421
Egner A, Geisler C, von Middendorff C, Bock H, Wenzel D, Medda R, Andresen M, Stiel AC, Jakobs S, Eggeling C, Schonle A, Hell SW: Fluorescence nanoscopy in whole cells by asynchronous localization of photoswitching emitters. Biophys J 2007,93(9):3285–3290. 10.1529/biophysj.107.112201
Geisler C, Hotz T, Schonle A, Hell SW, Munk A, Egner A: Drift estimation for single marker switching based imaging schemes. Opt Express 2012,20(7):7274–7289. 10.1364/OE.20.007274
Heilemann M, van de Linde S, Schuttpelz M, Kasper R, Seefeldt B, Mukherjee A, Tinnefeld P, Sauer M: Subdiffraction-resolution fluorescence imaging with conventional fluorescent probes. Angew Chem Int Ed Engl 2008,47(33):6172–6176. 10.1002/anie.200802376
Hell SW: Microscopy and its focal switch. Nat Methods 2009,6(1):24–32. 10.1038/nmeth.1291
Hess ST, Girirajan TP, Mason MD: Ultra-high resolution imaging by fluorescence photoactivation localization microscopy. Biophys J 2006,91(11):4258–4272. 10.1529/biophysj.106.091116
Huang B, Jones SA, Brandenburg B, Zhuang X: Whole-cell 3D STORM reveals interactions between cellular structures with nanometer-scale resolution. Nat Methods 2008,5(12):1047–1052. 10.1038/nmeth.1274
Huang B, Wang W, Bates M, Zhuang X: Three-dimensional super-resolution imaging by stochastic optical reconstruction microscopy. Science 2008,319(5864):810–813. 10.1126/science.1153529
Huang B, Babcock H, Zhuang X: Breaking the diffraction barrier: super-resolution imaging of cells. Cell 2010,143(7):1047–1058. 10.1016/j.cell.2010.12.002
Juette MF, Gould TJ, Lessard MD, Mlodzianoski MJ, Nagpure BS, Bennett BT, Hess ST, Bewersdorf J: Three-dimensional sub-100 nm resolution fluorescence microscopy of thick samples. Nat Methods 2008,5(6):527–529. 10.1038/nmeth.1211
Kamiyama D, Chiba A: Endogenous activation patterns of Cdc42 GTPase within Drosophila embryos. Science 2009,324(5932):1338–1340. 10.1126/science.1170615
Lee SH, Baday M, Tjioe M, Simonson PD, Zhang R, Cai E, Selvin PR: Using fixed fiduciary markers for stage drift correction. Opt Express 2012,20(11):12177–12183. 10.1364/OE.20.012177
Lew MD, Lee SF, Ptacin JL, Lee MK, Twieg RJ, Shapiro L, Moerner WE: Three-dimensional superresolution colocalization of intracellular protein superstructures and the cell surface in live Caulobacter crescentus. Proc Natl Acad Sci U S A 2011,108(46):E1102–1110. 10.1073/pnas.1114444108
Mennella V, Keszthelyi B, McDonald KL, Chhun B, Kan F, Rogers GC, Huang B, Agard DA: Subdiffraction-resolution fluorescence microscopy reveals a domain of the centrosome critical for pericentriolar material organization. Nat Cell Biol 2012,14(11):1159–1168. 10.1038/ncb2597
Mlodzianoski MJ, Schreiner JM, Callahan SP, Smolkova K, Dlaskova A, Santorova J, Jezek P, Bewersdorf J: Sample drift correction in 3D fluorescence photoactivation localization microscopy. Opt Express 2011,19(16):15009–15019. 10.1364/OE.19.015009
Pertsinidis A, Zhang Y, Chu S: Subnanometre single-molecule localization, registration and distance measurements. Nature 2010,466(7306):647–651. 10.1038/nature09163
Rust MJ, Bates M, Zhuang X: Sub-diffraction-limit imaging by stochastic optical reconstruction microscopy (STORM). Nat Methods 2006,3(10):793–795. 10.1038/nmeth929
Shroff H, Galbraith CG, Galbraith JA, Betzig E: Live-cell photoactivated localization microscopy of nanoscale adhesion dynamics. Nat Methods 2008,5(5):417–423. 10.1038/nmeth.1202
Shtengel G, Galbraith JA, Galbraith CG, Lippincott-Schwartz J, Gillette JM, Manley S, Sougrat R, Waterman CM, Kanchanawong P, Davidson MW, Fetter RD, Hess HF: Interferometric fluorescent super-resolution microscopy resolves 3D cellular ultrastructure. Proc Natl Acad Sci U S A 2009,106(9):3125–3130. 10.1073/pnas.0813131106
Staudt T, Lang MC, Medda R, Engelhardt J, Hell SW: 2,2'-thiodiethanol: a new water soluble mounting medium for high resolution optical microscopy. Microsc Res Tech 2007,70(1):1–9. 10.1002/jemt.20396
The work is supported by the NIH Director’s New Innovator Award 1DP2OD008479-02. B.H. receives the Searle Scholarship and the Packard Fellowship for Science and Engineering.
The authors declare no completing interests.
B.H. conceived the idea, R.M. implemented the system, performed the experiments and analyzed the data, D.K. performed the Drosophila experiments, R.M. and B.H. wrote the manuscript. All authors read and approved the final manuscript.
About this article
Cite this article
McGorty, R., Kamiyama, D. & Huang, B. Active microscope stabilization in three dimensions using image correlation. Opt Nano 2, 3 (2013). https://doi.org/10.1186/2192-2853-2-3
- Super-resolution microscopy
- Image correlation
- Stage stabilization