Blind assessment of localisation microscope image resolution
© Rees et al.; licensee Springer. 2012
Received: 3 September 2012
Accepted: 13 November 2012
Published: 10 December 2012
Skip to main content
© Rees et al.; licensee Springer. 2012
Received: 3 September 2012
Accepted: 13 November 2012
Published: 10 December 2012
This paper analyses the resolution achieved in localisation microscopy experiments. The resolution is an essential metric for the correct interpretation of super-resolution images, but it varies between specimens due to different localisation precisions and densities.
By analysing localisation microscopy as a statistical method of Density Estimation, we present a method that produces a blind estimate of the resolution in a super-resolved image. This estimate is derived directly from the raw image data without the need for comparisons with known calibration specimens. It is corroborated with simulated and experimental data.
Localisation microscopy has a resolution limit equal to 2σ, where σ is the r.m.s. localisation precision, evaluated as an average Thompson precision, Cramer Rao bound, or otherwise. Further, for a limited-sampling case in which there is only one localisation per fluorophore, the expected resolution of an optimised super-resolution image is worsened to approximately 3σ, due to smoothing processes that are necessarily involved in visualising the specimen with limited data. This 2σ or 3σ resolution can be estimated for any localisation microscopy specimen, and this metric can corroborate or replace empirical estimates of resolution. Other quantifiable resolution losses arise from sparse labelling, fluorescent label size, and motion blur.
Localisation microscopes can image biological samples, under physiological conditions or even in vivo, with an effective resolution better than 20 nm (Betzig et al. 2006, Huang et al. 2008, Schroff et al. 2008). This capability is providing striking new insights into cell biology on this previously inaccessible scale, such as the intracellular layout of the actin cytoskeleton, morphology of membrane complexes, virus particle assembly, and the progression of protein aggregation in neurodegenerative diseases (Hell 2007, Heilemann et al. 2008, Kaminski et al. 2011). In fact the method has the scope to study all manner of intracellular pathology and protein behaviour in unprecedented detail.
Briefly, localisation microscopy techniques such as PALM, GSDIM and (d)STORM, work as follows. A traditional fluorescence microscope is used to take video images of the specimen in question. The specimen is labelled with photoswitchable fluorophores, which highlight the protein or tissue of interest. The user applies controlled illumination and solution chemistry conditions so that these fluorophores photoswitch stochastically, which produces a “blinking” fluorescence. This blinking fluorescence, combined with a suitable camera exposure time, enables the measurement of random, bright spots of light which correspond to sparse subsets of the fluorescent labels. The fluorescence images of individual fluorophores are thus separated, and the position of each one can be precisely determined computationally from its image(s), once a sufficient quantity of data has been captured. Commonly the analysis is done by a “Segmentation and Sparse Gaussian Fitting” algorithm (Wolter et al. 2010). Finally, the fluorophore positions are typically visualized as a fluorophore density map (Baddeley et al. 2010). The resulting super-resolution “image” strongly resembles a conventional fluorescence image, because it is likewise an estimate of the specimen fluorescence density. However the fluorophore density map is typically an order of magnitude better resolved, because it is generated from fluorophore positions determined with high (~10-30 nm) precision.
To correctly interpret images of unknown or little-understood specimens – which is precisely the task for which super resolution imaging is most important – it is crucial to know the resolution of the system. When we refer to the localisation microscopy system, this includes the behaviour of the data fitting and visualisation algorithms that are applied to measurements of a particular specimen, as well as the optical instrument itself. Unfortunately, the resolution achieved by localisation microscopy can vary greatly between samples, because the accuracy of data fitting and visualisation depends on the quality of fluorescence images and background noise that are collected from the specimen. Therefore it is not sufficient to evaluate the instrument resolution using a calibration sample, and then state this as the resolution when studying other objects. Instead, it is important to estimate the particular resolution of each super-resolution image. This can be done empirically when measuring well-known samples such as actin filaments or sparse dye molecules on a coverslip – and this is often done for system calibration (Xu et al. 2012) – because the reconstructed image can be compared with the known structure. For unknown specimens it is necessary to make a blind assessment of how each localisation microscopy image is limited by its own source data. This paper discusses localisation microscopy as a Density Estimation technique (Parzen 1962, Silverman 1986), and thus identifies a resolution limit based on localisation precision – a quantity for which we can obtain a blind estimate from raw image data. Density Estimation theory also addresses the effect of localisation frequency: the best super-resolution image obtained with only one localisation per fluorophore tends to be less well resolved than one constructed with several detections, because successful visualisation of limited data requires broader smoothing. Other issues that can worsen the effective resolution include too-sparse labelling, fluorescent label width, and motion blur, and these effects can also be quantified.
Pre-formed rabbit skeletal muscle actin filaments (Cytoskeleton Inc) were diluted in general actin buffer (Cytoskeleton Inc) to a concentration of 2 μM and incubated with a 1 μl in 200 μl dilution of 6.6 μM phalloidin-Alexa 647 solution (Invitrogen). These were allowed to adhere to LabTek glass chambers for 2 hours prior to being imaged. Before addition of actin filaments, LabTek glass chambers were treated with 2 M glycine for 30 min at 37°C and then coated with 0.05% poly L-lysine solution for 30 min at room temperature.
Hela cells (ATCC, CCL-2) were grown to 80% confluency in LabTek chambers in DMEM (Fisher, VX31966021) containing FBS (Fisher VX10500064). Cells were then starved in DMEM without FBS for 60 min followed by incubation with 1 μg/ml EGF-Alexa 647 (Fisher, VXE35351) in complete culture medium for 30 min at 37°C. Finally cells were fixed with 4% formaldehyde for 10 min and then washed with PBS.
Fluorescence images were taken on an Olympus IX71 inverted widefield microscope, with a 100x, 1.49 NA TIRF objective lens. To induce photoswitching, the LabTek chambers were filled with a “switching buffer” solution: 100 mM mercaptoethylamine (MEA) in phosphate buffered saline (PBS, pH 7.4), together with a glucose-enzyme oxygen scavenger (40 mg/ml glucose, 50 μg/ml glucose oxidase, 1 μg/ml catalase). Samples were illuminated with 642 nm laser light at 2 kW/cm2. Stacks of 104 images were collected at 65 Hz, with 10 ms exposure times.
Simulated images were generated in MATLAB, after (Thompson et al. 2002). For the single fluorophore simulation, the expected number of photons arriving on each camera pixel was calculated by integrating a Gaussian PSF over the pixel, and allocating the corresponding proportion of N detected photons. To simulate quantum photon counting, this number was replaced with a random number generated from a Poisson distribution with the same mean. Independent, Gaussian random noise of standard deviation b was then added to each pixel. To simulate a structured object (lines of fluorophores at 5 nm spacing, crossing at 45º), a subset of the fluorophores was randomly “activated” for each camera frame, and the cumulative image of all the active fluorophores was generated as above. Gaussian noise was added at the end of this process.
Super-resolution images were reconstructed using a “Segmentation and Sparse Gaussian Fitting” algorithm, similar to (Wolter et al. 2010), implemented in MATLAB. In each camera frame, local maxima brighter than a threshold value were fit to a Gaussian PSF, by an iterative least squares method. For each spot, a localised position was obtained, as well as an estimate for the number of photons of fluorescence, N. The camera noise b was estimated on a framewise basis as the standard deviation of camera pixel brightness in regions without any bright spots. Average localisation precision was evaluated from N and b using Equation 1. Localised fluorophore density was visualised as a “histogram raster,” with simulations visualised in greyscale, and experimental results converted from grey to a colormap for clarity of viewing.
The super-resolution “image” typically generated by a localisation microscope is a computational reconstruction of the localised fluorophore density. This reconstruction should be better resolved than a fluorescence image. A conventional fluorescence image suffers directly from the diffraction limit, which causes the specimen fluorescence to be blurred by convolution with the optical point spread function (PSF) of the microscope. By comparison, the reconstructed super-resolution image of fluorophore density is also imperfect – in this case the specimen fluorescence density is degraded by an analogous Localisation Error Function (LEF), which describes the uncertainty of determining fluorophore positions by localisation. Usually the LEF is much smaller than the PSF and the reconstructed images are much better resolved than standard fluorescence images. The difference arises because the PSF represents the spread of a single photon position, whereas the LEF represents the smaller spread in determining the centre position of N photons from the same point source.
Here, <Δx2> estimates the random variance of a localised fluorophore position, when the optical point spread function is approximately Gaussian with standard deviation s; a is the side-length of the detector pixels; N is the number of photons detected from the fluorophore; and b represents the number of photons of random camera noise in each pixel.
Here, p(x) is the LEF, which is defined to have a true variance of σ2. The <Δx2> in Equation 1 provides an estimate of σ2. In what follows we denote estimates of σ as . The p(x) stated here is two dimensional with circular symmetry. If necessary, p(x) could be made elliptical or three dimensional by replacing the term in the exponential with a more general x ′ Ax, and adjusting the normalisation term.
For simple cases with only a few localisations, ρ s can be directly visualised as a scatter plot. More typically, however, this set of fluorophore positions is used to generate a fluorophore density map of the specimen, by one of several possible methods (Baddeley et al. 2010). One effective visualisation is to construct a cumulative super-resolution image by plotting each localisation as a small blur (typically a Gaussian) of fluorescence intensity (Bates et al. 2007). This method is known in statistics as Kernel Density Estimation (Parzen 1962), but for simplicity and accessibility we may refer to its use in localisation microscopy as “Gaussian Visualisation.” In this method, the point-like localisations are smoothed, by convolution with a suitable kernel function K, to estimate the fluorophore density of the specimen. Kernel Density Estimation obviates the hard-edge artefacts seen in histograms; however it requires the user to identify an optimal kernel width. Using a too-wide kernel for the visualisation can throw away resolution by oversmoothing regions of high sampling density. Adaptive-width methods of kernel density estimation can prevent this problem (Breiman et al. 1977), by plotting narrower kernels in densely sampled regions, and Baddeley’s jittered histogram visualisation achieves the same effect.
In the model scenario of “frequent” localisations, n → ∞ and ρ s converges in distribution towards ρ m, i.e. the localisations perfectly map out the measurement probability. The reconstructed super-resolution image, ρ v, will then also converge to ρ m, provided that we “draw” it using a suitably narrow kernel K – tending towards a delta function in the limit of infinite localisations. In this model, with “infinite” localisations, the localisation microscope has generated an image simply by convolving the true fluorophore density with the localisation error function p(x).
Some final remarks on precision-limited resolution are important. This blind estimate of the resolution limit, using Equations 1 and 8, is similar to the empirical resolution evaluated by localising a static point source several times, and reporting the FWHM (~2.4 ) of the spread of localised positions (Xu et al. 2012). Whereas the empirical method is a direct measure of resolution for calibration samples, the blind estimate is an indirect method that does not require a known specimen. Finally, the principle of precision-limited resolution is not restricted to estimates of localisation error obtained by Equation 1. In Bayesian localisation of sparse fluorophores, for example, the Cramer-Rao bound can provide a valid estimate of Δx2 for each fluorophore position (Kay 1993, Ober et al. 2004, Shaevitz 2009).
Crucially, the Precision Limit is the smallest possible distance that can be resolved by localisation microscopy. This resolution is achieved only if the number of localisations is infinite. Sampling limitation worsens the resolution in practice, when the number of localisations is finite, and the extent of this resolution loss can be studied by treating localisation microscopy as a “Density Estimation” task (Silverman 1986) within the model provided by Equations 17.
Here h j is the kernel width which should be applied to the j th localisation, where d jk is the k th nearest neighbour distance of the j th localisation, and α is a scaling constant. Optimisation of α and k is discussed in (Breiman et al. 1977). The key message is that the optimum kernel width depends on both the local sampling density (distance between localisations) and the underlying structure.
Having evaluated the amount of smoothing required to produce the optimal Gaussian Visualisation of a specimen, the length scale of this visualisation can be used to estimate the best image resolution that can be achieved when the number of localisations is finite. An interesting case is the scenario in which each fluorophore is localised exactly once. This could happen as a consequence of measuring a short data series (e.g. due to rapid bleaching), or intentionally (e.g. in a molecule counting experiment designed to capture a single bright image of each fluorophore, before bleaching it). With only a single localisation per fluorophore, the sampling density will probably not tend towards the “infinite” value needed to achieve the Precision Limit of resolution. The “best” visualisation method indicated by Equation 9 is to smooth the localisations with a Gaussian of standard deviation . Let us assume that a sample-dependent treatment with (Breiman et al. 1977), or another optimisation, does not significantly alter this value. In this case the average super-resolution image, <ρ v >, is blurred by convolving the localised positions with a Gaussian visualisation kernel of standard deviation . The localisation microscopy image is therefore degraded twice: first by localisation error, and second by the smoothing required for visualisation. Both are convolutions with a Gaussian of standard deviation ~ , and this double convolution is equivalent to a single convolution with a Gaussian of standard deviation (Hirschman and Widder 1955).
This analysis leads to the remarkably simple result that, whereas the Precision Limit of resolution achieved with an infinite number of localisations is 2 , an image produced with a single localisation per fluorophore has its expected resolution coarsened (assuming optimal visualisation) to about 2 , or approximately 3 . Importantly, this is an average or expected resolution: with a limited number of localisations there are many possible permutations of the super-resolution image, as shown in Figure 1c, but 2 is the average resolution. Furthermore, the optimal amount of smoothing required in Gaussian Visualisation is subjective and cannot be determined exactly without knowing the structure of the specimen beforehand. However, a Precision Limit of 2 , and a limited sampling resolution of 3 , can provide a useful blind estimate of the resolution achieved with a real specimen.
These resolution estimates are derived for localisation microscopy using a Gaussian Visualisation. How do alternative visualisation algorithms compare with this? Histogram visualisation (Wolter et al. 2010), for example, involves “binning” the localisations onto a grid of super-resolution pixels. The width of the histogram pixels effectively smoothes or blurs the localisation data, analogous to the smoothing involved in visualisation by convolving the localisations with a Gaussian. Based on this similarity, it seems plausible that a skilful histogram visualisation can achieve a resolution that is similar to the limit obtained by smoothing with a Gaussian, although the histogram is likely to be slightly worse due to hard-edge features. Certainly the histogram method asymptotes to the same Precision Limit, since in the case of infinite localisations a histogram can be plotted using infinitesimal bin widths and hence no loss of resolution due to plotting. The jittered histogram visualisation is based on a principle similar to an adaptive-width kernel, and should achieve the same resolution (Baddeley et al. 2010). A simple scatter plot, however, is a discontinuous visualisation and its resolution therefore cannot be directly compared with the continuous image visualisations considered above.
Nyquist-limited resolution relates to the fluorescent labelling of a specimen. It is well known that discrete labelling constrains the useful resolution of any imaging method (Schroff et al. 2008). The essential idea is that features smaller than twice the typical separation of fluorescent labels cannot be reliably measured, due to aliasing, even if the precision-limited resolution of the microscope would be sufficient to do so, as shown in Figure 2c. Note that this limit arises from the distance between fluorophores on the sample. This is generally different from the distance between localisations, since a fluorophore might be localised several times; however if the fluorophore is static on the specimen then simply obtaining multiple localisations will not increase the underlying label density, and it is the spacing of the fluorophore labels that determines whether features are measurable. The fluorophore spacing can be estimated based on the known size of antibody labels, or by extrapolating from the density of localisations – in some cases this latter approach is simple, for example if there is precisely one localisation per labelled position (Shim et al. 2012).
Since the Nyquist limit is a concept from Fourier analysis, it strictly refers to subjects that are sampled at regularly spaced intervals. This is not true for localisation microscopy, where fluorophores are usually attached at random positions on their substrate. Further, specimens generally have very dense-labelling in some areas, and very sparse labelling in others. This means that the question of what resolution limit is imposed by label density is more complex than finding the average distance between fluorophores. There is a strong variation in the label density on different parts of a specimen, so fluorophore spacing and hence the Nyquist limit of resolution must be evaluated for particular features. This is a sample-dependent issue, and the best way to evaluate the Nyquist limit of resolution should be considered on a case-by-case basis.
Nyquist-limited resolution can also arise in print or on-screen images. When super-resolution images are displayed as a square raster, the pixel width must be less than half of the resolution that needs to be illustrated. Therefore the value of , from Equation 8, provides an indication of the best pixel size to use when plotting a super-resolution image.
The range of 2-3 then provides a corrected estimate of microscope resolution for the sample in question. Ideally the fluorescent label size and any inexact correction of motion blur should have a negligible effect, compared with fluorophore localisation precision.
The principle outlined so far is that the resolution of localisation microscopy depends on two factors, (a) localisation precision and (b) smoothing required for visualisation of finite data, and that a blind estimate of this resolution can be evaluated from the same source data that is processed to obtain a super-resolution image. This principle extends to three-dimensions; however the maths needed to evaluate the z-resolution varies between the different implementations of 3D localisation microscopy. Here we outline the general principles involved in assessing z-resolution. The task is simply to process the image data to estimate the standard error of each localised position in the z-direction, as well as in the lateral x and y directions. We also need to specify what it means to visualise a structure in three dimensions. In some methods of 3D localisation microscopy the visualisation may be a z-stack of super-resolution images, and in other methods the mean z-position of localisations may be indicated with a colour code (which is unsuitable for showing features at multiple z-positions). Neither of these methods is perfect. To evaluate the theoretical limit of z-resolution, it is possible to extend Equation 7 into three dimensions, so that the three-dimensional scatter plot of localisations (ρ s3) is smoothed into a three dimensional scalar field of localisation density (ρ v3) – probably with an ellipsoidal smoothing function. This representation can then be sliced or projected onto various planes for visualisation in 2D.
The method required to estimate the error of z-localisation depends on the technique used for 3D localisation. In the astigmatism-based method of 3D localisation, the vertical or horizontal ellipticity of a fluorophore image is used to evaluate its z-position. In this method, a Bayesian method of z-localisation is ideally suited to estimating the z-localisation error at the same time as the z-position (Shaevitz 2009). Alternatively the uncertainty in z-position could be evaluated by algebraic error propagation of the uncertainties in the x- and y-widths fitted to each PSF. In either case, once the uncertainties of the z-positions have been evaluated, the z-resolution follows from the method derived above for 2-dimensions. The Precision Limit of z-resolution is simply twice the estimated localisation error, and the limited sampling case is three times. Unfortunately, this resolution has to be evaluated on a regional basis, rather than being uniform over the image. This is because of an interesting drawback of astigmatism-based 3D localisation microscopy: the optical PSF widths, and hence the Precision Limits of resolution, are dependent on z-position and they are different in the x and y directions. The z-resolution also varies with z-position, and is asymmetric. This makes it harder to express the resolution of a super-resolution image, even for a single z-slice. This drawback does not afflict the double-helix method of 3D localisation microscopy, since in that system the lateral and axial localisation precisions are translationally invariant over the depth of field (Badieirostami et al. 2010). In the double helix method, the uncertainty of z-positions could be estimated via a Fisher information approach, or by comparison with tabulated calibration data. Another 3D localisation microscopy technique is the biplane method: in this method, the ratio of image intensities measured from different focal planes within the sample is used to determine the z-position of a fluorophore (Juette et al. 2008). This technique also benefits from translationally invariant lateral resolution, and fairly homogenous axial resolution, however the uncertainty of z-position may have to be estimated via a lookup table of calibration data. The lateral localisation precisions could be estimated using the Thompson formula.
A final comment on z-resolution is that the image of a three-dimensional specimen may contain a much higher density of fluorophores than the image of a flat, two-dimensional specimen. In order to achieve sparse fluorescence images of individual fluorophores, subject to the constraint of limited photoswitching ratios, the spacing of fluorescent labels will need to be greater. This means that the Nyquist limit of resolution in 3D localisation microscopy may pose a greater problem than in 2D.
Localisation microscopy is predicated upon the ability to identify individual fluorophore positions, based on sparse fluorescence image data. Quality control methods are applied, either implicitly or explicitly, to exclude imprecise or spurious localisations, but some mislocalisations may persist which are not from single, photoswitching fluorophores. When the localised fluorescence density is visualized, these features may result in various artefacts. Some examples are: persistent fluorescent background features may be visualized as spots of concentrated fluorophore density; overlapping fluorescence signals from close, but separate, features may be visualized as a spurious blur of fluorophore density in between the features; and mobile fluorescent molecules that wander through the focal plane of the microscope may result in “salt and pepper noise” or “ant tracks.” Uncorrected mechanical drifts of the specimen can generate “motion blur” in the reconstructed image – although this motion blur is in fact a correct visualization of a moving sample, it is unhelpful and worsens the resolution of the system. Lateral drift may be corrected using a method such as fiducial markers. Drift in the z-direction may also occur. As well as leading to a vertical streaking of the visualisation – which resembles a worsened z-resolution – z-drift also causes features to move relative to the focal plane of the microscope. Features that lose focus have a broader optical PSF, and hence a worse localisation precision (after Equation 1), and therefore the Precision Limit of the (lateral) resolution is also worsened for specimens that experience z-drift. Fiducial markers may be able to correct the motion blur caused by z-drift, but can do nothing to prevent the loss of resolution that arises from broader PSFs and the consequent loss of (lateral) localisation precision.
Because mislocalisations arise from a failure of the assumption that underlies localisation microscopy – they occur when it is not possible to localise precise positions exclusively from individual fluorescent labels, and so Equations 2–7 do not properly describe the process – their impact on instrument resolution is difficult to quantify.
The resolution of localisation microscopy depends on the fluorescence properties of the specimen, specifically the number and precision of fluorophore localisations. Therefore it is important to assess the resolution achieved with unknown samples by analyzing their particular datasets, since it can differ significantly from the resolution evaluated for a calibration sample. Fluorophore positions determined by a localisation microscope have a finite error, and the spread of these errors is analogous to the optical point spread function of a traditional light microscope. Since the error of each fluorophore localisation can be estimated using Thompson’s Equation 1, or as a Cramer-Rao bound, the average localisation precision (σ) of a fluorophore density reconstruction leads to a practical estimate of the resolution limit due to instrument precision of 2σ. This resolution limit is readily calculable from raw image data, and provides a useful guideline for the resolution achieved in a localisation microscopy image. Furthermore, in a case of only one localisation per fluorophore, the average resolution is worsened to about 3σ, because of the quantity of smoothing required to produce a realistic visualisation of the specimen using limited data.
As well as the precision and sampling limits of resolution, a separate constraint on the effective resolution of a localisation microscope can arise as Nyquist limitation due to fluorescent label density. Also, the effective localisation precision may be worsened by fluorophore width or motion blur correction. Spurious localisation data should be excluded by stringent quality control criteria, as it can otherwise generate artefacts that are hard to quantify.
Dulbecco's modified eagle medium
Epidermal growth factor
Foetal bovine serum
Localisation error function
Phosphate buffered saline
Point spread function
Total internal reflection fluorescence.
CFK, EJR, and MJE acknowledge support from ARUK (ARUK-EG2012A-1), the EPSRC, BBSRC, MRC and the Wellcome Trust. MJE also acknowledges the Hungarian National Office for Research and Technology, OTKA foundation. DP is supported by a fellowship from the Swiss National Science Foundation. AEK and DJM acknowledge funding from the Chemical and Biological Metrology programme of the UK’s National Measurement Office. We thank Prof Markus Sauer for inspiring discussions and advice on experimental implementations of d STORM.
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.