Optical photon reassignment microscopy (OPRA)
© Roth et al.; licensee Springer. 2013
Received: 21 June 2013
Accepted: 4 October 2013
Published: 18 October 2013
To enhance the resolution of a confocal laser scanning microscope the additional information of a pinhole plane image taken at every excitation scan position can be used (Sheppard 1988). This photon reassignment principle is based on the fact that the most probable position of an emitter is at half way between the nominal focus of the excitation laser and the position corresponding to the (off centre) detection position. Therefore, by reassigning the detected photons to this place, an image with enhanced detection efficiency and resolution is obtained. Here we present optical photon reassignment microscopy (OPRA) which realizes this concept in an all-optical way obviating the need for image-processing. With the help of an additional intermediate optical beam expansion between descanning and a further rescanning of the detected light, an image with the advantages of photon reassignment can be acquired. However, just as in computational photon reassignment, a loss in confocal sectioning performance is caused by working with relatively open pinholes. The OPRA system shares properties such as flexibility and ease of use with a confocal laser scanning microscope, and is therefore expected to be of use for future biomedical routine research.
KeywordsPhoton reassignment Image scanning microscopy Confocal laser scanning microscopy
Confocal laser scanning microscopy (CLSM) is an established tool in fluorescence microscopy and well-known for its optical sectioning ability and high contrast (Pawley 2006; Minsky 1961). These characteristics are achieved by using detectors with a high dynamic range and collecting the emitted light through a pinhole, which is usually aligned to the position of the excitation focus (thus the name “confocal”). The resulting image is constructed by assigning the detected intensity to the corresponding excitation scan position. In 1982 it was shown that it is possible to achieve enhanced resolution by applying an off-axis pinhole (Cox et al. 1982). In 1988 pinhole plane image detection and computational reassignment to a position half way between nominal excitation and detection position was proposed (Sheppard 1988), to improve detection efficiency and resolution. Note that for identical excitation and emission point spread functions (PSF) this reassigned position corresponds to the most probable position of an emitter in the sample. Recent work applied this principle in single (Müller & Enderlein 2010) and multispot excitation (York et al. 2012) to the imaging of biological samples.
Here we present optical photon reassignment microscopy (OPRA). It is an optical realization of these computer based methods which avoids the need for data processing. Furthermore at a different scaling ratio, our method is applicable to the direct visualisation of high-resolution imaging methods like STED.
In normal CLSM the detected intensity values of every scanning position are recorded with an integrating detector such as a photo-multiplier-tube (PMT) or an avalanche photo-diode. If a whole image of the pinhole plane is recorded at each scan position the acquired data-set of a single focal slice can be viewed as a 4 dimensional set of data (intensity values in dependency of xy scan and xy pinhole-plane image-position). Such data can be processed in several ways, for example allowing for a retroactive choice of the pinhole diameter and/or applying multi-view deconvolution (Brakenhoff & Visscher 1992 Heintzmann et al. 2003). Photon reassignment microscopy (Sheppard 1988; Müller & Enderlein 2010) is based on the insight that the most probable origin of the detected photons is at maximum of the joint probability function (i.e. the product of the individual probability functions) of excitation and (off-centre) detection. This is contrary to a CLSM where all detected photons are assigned to the nominal excitation position s.
Additional file 1: Simulation of the OPRA principle. Top row (left to right): object and illumination intensity; pinhole plane; rescanned (m=1); rescanned sum (m=1). Bottom row (left to right): sum confocal (0.3 AU pinhole); pinhole plane (m=0.5); rescanned (m=0.5); rescanned sum (m=0.5). (AVI 578 KB)
We now aim to find an analytical description of the PSF of the overall system. The emission PSF has also undergone the intermediate beam expansion m and can therefore be written as h(x/m), positioned at s(1-m). Thus, the current image generated by the point source at the point of origin is , where s is the nominal scan position and x the image coordinate (measured in sample coordinates).
where and the symmetrical excitation PSF are used.
Thus the additional expansion m should be adjusted to the different width of the excitation and emission PSFs. This difference can be induced by the Stokes shift of the used fluorophores or is a feature of the microscopy technique itself, to which OPRA is applied (e.g. STED microscopy, where σex is significantly smaller than σem).
If no Stokes shift is considered (σ ex = σ em ) we obtain a resolution improvement of over what we would expect for confocal detection with a closed pinhole. This shows that reassignment microscopy realizes high resolution at the theoretical overall detection efficiency of a widefield microscope. OPRA attains the same characteristics as computational reassignment without the need for high-speed pinhole cameras and without the increased read-noise of multiple fast readouts. This raises the acquisition speed as the whole image is acquired in only one exposure frame. An additional pinhole can also be integrated in OPRA (before rescanning) to achieve confocal sectioning. Note that all emitted light of a scan from a planar fluorescent sample would otherwise reach the detector and thus prevent optical sectioning. For a detailed discussion of the sectioning ability see (Sheppard et al. 2013).
To compare theory with measurements, point spread functions were calculated using vectorial theory (here assuming random polarisation) for the respective experimental wavelengths. The excitation PSF was then convolved with the emission PSF and the scale changed by a factor of 2 according to eqn. (2) with m=0.5. To finally account for the size of the beads, this resulting PSF was then convolved in 3D with a three-dimensional spherical volume of 200 nm diameter and the width of the resulting function was fitted with a Gaussian and measured.
To demonstrate the OPRA principle, fluorescent coated beads (FluoSpheres® Carboxylate-Modified Microspheres, 0.2 μm, Yellow-Green Fluorescent (505/515)) with a diameter of 200 nm were imaged. For comparison a widefield excitation lamp (EXFO photonics solutions Inc., X-Cite series 120 Q) was coupled into the setup with an optional dichromatic beam splitter (Carl Zeiss Microscopy GmbH, FT 510), such that the same sample position could be imaged with both methods as shown in Figure 3. For comparison 8 bead-images were analysed and the full-widths at half maxima (FWHM) of fitted 2D Gaussian functions were determined. The mean FWHM of measured 200 nm beads in the widefield image (Figure 3b) is determined to be (473± 19)nm and in the OPRA-image (327± 4)nm (Figure 3a), without using a pinhole. Accounting for the 200nm diameter of the beads, equation (2) predicts a FWHM of 297 nm (256 nm as Gaussian fit without accounting for the bead size) for the bead images acquired with OPRA, and 428nm for the widefield case, with the PSFs generated for the parameters as given above using vectorial diffraction theory. Thus theory and experiment agree to within about 10%. For the OPRA mode and 11% for the widefield detection. As the OPRA image theoretically contains the same number of photons as the widefield image but distributed to a sharper image, it is expected that the OPRA image also looks significantly brighter that the corresponding widefield (or confocal) image. However, since we used a separate illumination source to generate the widefield image, we could not do an appropriate comparison in this study.
Compared to computational photon reassignment no post-processing is required as the summation and photon reassignment is a system inherent property of OPRA. This prevents artefacts (e.g. pixilation artefacts, additional read noise) and is even insensitive to small variations in the scanning process, as those will mostly lead to small brightness changes in the resulting image. This makes it useful for very fast imaging with the scanning-speed and the camera frame-rate as the only limiting factors. We showed that the principle improves the resolution in comparison to classical widefield microscopy and we derived the basic theory for OPRA performance. It should be noted that the required OPRA properties are also achievable with realization methods other than intermediate magnification, such as the use of separate scan-units for illumination and detection light running at different speeds. OPRA can be adapted to various ratios of the sizes of excitation and emission PSF. Therefore the OPRA principle can also be used to optically realize versions of super-resolution modes such as STED, GSD and RESOLFT (Hell 2003). At large transition saturation factors, these methods would profit only marginally from an additional resolution gain but such a setup would, however, enable these modes for the first time to directly generate a highly resolved optical image without even the need for any data acquisition. Even a multi-spot STED, GSD or RESOLFT microscope should be feasible with the OPRA approach, where the super-resolved image is built up during the integration time of a single frame. When using a periodic multi-spot excitation array, a single scan mirror can suffice. This was demonstrated in (York et al. 2013) which appeared after our manuscript was accepted for publication. Rescanning can also be performed with an electronically synchronised second scan mirror (or system of mirrors) or especially in the case of multi-spot illumination the rear side of the scan mirror can be used for rescanning. The camera can also be replaced by the human eye, realizing direct-view versions of STED or RESOLFT microscopy. Due to the simplicity and flexibility of the realizations, OPRA can enhance the performance of nearly every laser scanning microscope.
Currently our system does not use an appropriate tube lens to be free of chromatic aberrations. Future refinements of the imaging and scan optics are expected to push the performance of the system in both modes closer to the theoretical limit, especially for larger numerical apertures and low magnification.
Especially noteworthy is that OPRA achieves a theoretical image brightness superseding the performance of a widefield microscope under the same illumination dose. More photons are concentrated onto the same image pixel area. In this respect it differs from many alternative high-resolution methods which often only “shave” the PSF.
Even though OPRA improves photon reassignment in its all-optical realization, it should be noted that a full pinhole plane scan dataset of images with full dependency of scan and image coordinates is richer, and allows for better ways of image processing. These range from the ability retroactively to select the pinhole size, to optimization strategies such as weighted averaging in Fourier space and combined deconvolution (Heintzmann et al. 2003). As pinhole plane array data does not require a physical pinhole it can avoid the compromise between lateral resolution and optical sectioning performance of OPRA. Nevertheless, OPRA avoids generating large amount of data, along with the additional associated readout noise, with the additional benefit (even over a classical confocal microscope) of an inherent stability against scan imprecisions even when caused by mechanical vibrations influencing the scanners.
Confocal laser scanning microscope
Full width at half maximum
Ground state depletion
Images scanning microscopy
Laser scanning microscope
Optical photon reassignment
Optical transfer function
Point spread function
Reversible saturable optical fluorescence transitions
Scientific complementary metal–oxide–semiconductor
Stimulated emission depletion.
- Brakenhoff GJ, Visscher K: Confocal imaging with bilateral scanning and array detectors. J Microsc 1992, 165: 139–146. 10.1111/j.1365-2818.1992.tb04311.xView ArticleGoogle Scholar
- Cox IJ, Sheppard CJR, Wilson T: Improvement in resolution by nearly confocal microscopy. Appl Opt 1982,21(5):778–781. 10.1364/AO.21.000778PubMedView ArticleGoogle Scholar
- Heintzmann R, Sarafis V, Munroe P, Nailon J, Hanley QS, Jovin TM: Resolution enhancement by subtraction of confocal signals taken at different pinhole sizes. Micron 2003,34(6–7):293–300.PubMedView ArticleGoogle Scholar
- Hell S: Toward fluorescence nanoscopy. Nat Biotechnol 2003, 21: 1347–1355. 10.1038/nbt895PubMedView ArticleGoogle Scholar
- Minsky M: Microscopy apparatus. 1961. U.S. patent 3,013,467. http://www.freepatentsonline.com/3013467.html Google Scholar
- Müller CB, Enderlein J: Image scanning microscopy. Phys Rev Lett 2010,104(198101):1–4.Google Scholar
- Pawley JB (Ed): Handbook of Biological Confocal Microscopy. 3rd edition. New York: Springer; 2006.Google Scholar
- Sheppard CJR: Super-resolution in confocal imaging. Optik (Stuttgart) 1988, 80: 53–54.Google Scholar
- Sheppard CJR, Mehta S, Heintzmann R: Superresolution by image scanning microscopy using pixel reassignment. Optics Lett 2013, 38: 2889–2892. 10.1364/OL.38.002889View ArticleGoogle Scholar
- York AG, Parekh SH, Nogare DD, Fischer RS, Temprine K, Mione M, Chitnis AB, Combs CA, Shroff H: Resolution doubling in live, multicellular organisms via multifocal structured illumination microscopy. Nat Methods 2012,9(7):749–754. 10.1038/nmeth.2025PubMed CentralPubMedView ArticleGoogle Scholar
- York AG, Chandris P, Nogare DD, Head J, Wawrzusin P, Fischer RS, Chitnis A, Shroff H: Instant super-resolution imaging in live cells and embryos via analog image processing. Nat Methods 2013. doi:10.1038/nmeth.2687Google Scholar
This article is published under license to BioMed Central Ltd. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.