Precision limit
The super-resolution “image” typically generated by a localisation microscope is a computational reconstruction of the localised fluorophore density. This reconstruction should be better resolved than a fluorescence image. A conventional fluorescence image suffers directly from the diffraction limit, which causes the specimen fluorescence to be blurred by convolution with the optical point spread function (PSF) of the microscope. By comparison, the reconstructed super-resolution image of fluorophore density is also imperfect – in this case the specimen fluorescence density is degraded by an analogous Localisation Error Function (LEF), which describes the uncertainty of determining fluorophore positions by localisation. Usually the LEF is much smaller than the PSF and the reconstructed images are much better resolved than standard fluorescence images. The difference arises because the PSF represents the spread of a single photon position, whereas the LEF represents the smaller spread in determining the centre position of N photons from the same point source.
Fluorescent molecules can be localised as precisely as 10 nm or better in practice, by measuring a few thousand photons, and resolutions of 20 nm can be achieved (Huang et al. 2008). However, the localisation precision Δx depends on properties that vary significantly between samples: primarily the number of photons collected from each fluorophore, and the background noise level. This dependence was derived by Thompson as Equation 1 (Thompson et al. 2002):
(1)
Here, <Δx2> estimates the random variance of a localised fluorophore position, when the optical point spread function is approximately Gaussian with standard deviation s; a is the side-length of the detector pixels; N is the number of photons detected from the fluorophore; and b represents the number of photons of random camera noise in each pixel.
What image resolution is achievable with this localisation precision? Localisation microscope resolution is fundamentally limited by two steps, illustrated in Figure 1: (1) by imperfect determination of fluorophore positions; and (2) by visualising the specimen based on a finite number of fluorophore position samples. Localisation precision concerns the first step, and to quantify its effect on resolution we consider the model scenario of “frequent, unbiased localisation with Gaussian errors.” Here, “frequent” measurement means that each fluorophore tends to be localised many times. “Unbiased” measurement means that localisations are detected from each individual fluorophore at the same average rate, and no spurious localisations are made. The positional errors are taken to be independently normally-distributed (assumed experimentally valid), so the probability density for a fluorophore with position x
0
to be localised at position x is p(x):
(2)
Here, p(x) is the LEF, which is defined to have a true variance of σ2. The <Δx2> in Equation 1 provides an estimate of σ2. In what follows we denote estimates of σ as . The p(x) stated here is two dimensional with circular symmetry. If necessary, p(x) could be made elliptical or three dimensional by replacing the term in the exponential with a more general x′
Ax, and adjusting the normalisation term.
Let a model specimen be defined by a static 2-dimensional fluorophore density, ρ
0(x), comprised of Dirac delta functions representing M distinct fluorophores.
(3)
In the model described above, the localisation microscope identifies fluorophore positions drawn from the probability distribution ρ
m.
(4)
In this model, each fluorophore contributes the same probability weight to the measurement. The * denotes convolution, and c is a normalisation constant.
The localisation microscope samples a set of fluorophore localisations, which can be thought of as a collection of n delta functions, having the same form used to model the specimen in Equation 3 (but generally different positions due to finite localisation errors).
(6)
For simple cases with only a few localisations, ρ
s can be directly visualised as a scatter plot. More typically, however, this set of fluorophore positions is used to generate a fluorophore density map of the specimen, by one of several possible methods (Baddeley et al. 2010). One effective visualisation is to construct a cumulative super-resolution image by plotting each localisation as a small blur (typically a Gaussian) of fluorescence intensity (Bates et al. 2007). This method is known in statistics as Kernel Density Estimation (Parzen 1962), but for simplicity and accessibility we may refer to its use in localisation microscopy as “Gaussian Visualisation.” In this method, the point-like localisations are smoothed, by convolution with a suitable kernel function K, to estimate the fluorophore density of the specimen. Kernel Density Estimation obviates the hard-edge artefacts seen in histograms; however it requires the user to identify an optimal kernel width. Using a too-wide kernel for the visualisation can throw away resolution by oversmoothing regions of high sampling density. Adaptive-width methods of kernel density estimation can prevent this problem (Breiman et al. 1977), by plotting narrower kernels in densely sampled regions, and Baddeley’s jittered histogram visualisation achieves the same effect.
The Gaussian Visualisation method is tractable for resolution analysis, because it involves a convolution analogous to a point spread function. In Equation 7 we therefore consider the super-resolution image described by ρ
v, which is obtained by smoothing the localisations ρ
s with a kernel K. In Gaussian Visualisation K is a 2-dimensional Gaussian, h
j
scales its width to suit the local sampling density, and h
j
-2 normalises the weight of each localisation in the reconstructed image.
(7)
In the model scenario of “frequent” localisations, n → ∞ and ρ
s converges in distribution towards ρ
m, i.e. the localisations perfectly map out the measurement probability. The reconstructed super-resolution image, ρ
v, will then also converge to ρ
m, provided that we “draw” it using a suitably narrow kernel K – tending towards a delta function in the limit of infinite localisations. In this model, with “infinite” localisations, the localisation microscope has generated an image simply by convolving the true fluorophore density with the localisation error function p(x).
Clearly the LEF blurs this ideal super-resolution image in the same manner as a PSF acts on a conventional image. Hence the image resolution can be determined analytically by studying the LEF with an optical resolution criterion. The Sparrow criterion is suitable because it can be evaluated for a Gaussian LEF, whereas the Rayleigh limit is only defined for Airy functions (Sparrow 1916). The Sparrow limit states that point sources are resolvable if they are sufficiently separated that their overlapping images (PSFs) produce a pattern with an intensity minimum lying between their positions. For a Gaussian PSF (or LEF) the Sparrow limit of resolution is satisfied for separations greater than 2σ. Crucially, we can estimate σ for each fluorophore localisation, by applying Equation 1 to the photon statistics (N and b) of the raw image data. This blind estimate () of σ indicates the resolution limit of any super-resolution image. Of course, the estimated localisation precisions typically encompass a spread of values; however since we are assuming independent, normally-distributed errors, we can obtain the estimate from the r.m.s. error (note (Holtzman 1950)).
(8)
If the fluorophore properties are homogenous (in terms of Signal-to-Noise, N and b), then we can take this average over the whole image; or if not then we could evaluate it on a regional basis. We can then state 2 as the Precision Limit of resolution for this particular super-resolution image. This resolution limit can be corroborated by simulations, as illustrated in Figure 2. It also provides a good indication of resolution that is achieved with real experimental specimens (Figure 3), although real samples tend to be slightly less well resolved due to the finite number of localisations, which is discussed in the next section.
Some final remarks on precision-limited resolution are important. This blind estimate of the resolution limit, using Equations 1 and 8, is similar to the empirical resolution evaluated by localising a static point source several times, and reporting the FWHM (~2.4 ) of the spread of localised positions (Xu et al. 2012). Whereas the empirical method is a direct measure of resolution for calibration samples, the blind estimate is an indirect method that does not require a known specimen. Finally, the principle of precision-limited resolution is not restricted to estimates of localisation error obtained by Equation 1. In Bayesian localisation of sparse fluorophores, for example, the Cramer-Rao bound can provide a valid estimate of Δx2 for each fluorophore position (Kay 1993, Ober et al. 2004, Shaevitz 2009).
Sampling limit
Crucially, the Precision Limit is the smallest possible distance that can be resolved by localisation microscopy. This resolution is achieved only if the number of localisations is infinite. Sampling limitation worsens the resolution in practice, when the number of localisations is finite, and the extent of this resolution loss can be studied by treating localisation microscopy as a “Density Estimation” task (Silverman 1986) within the model provided by Equations 17.
The perfect result of super-resolution would be to determine and visualise the exact fluorescence distribution of the specimen, ρ
0. However the finite localisation error means that the best visualisation achievable without deconvolution is ρ
m, analogous to the way a PSF blurs a conventional image. Given only a finite number of localisations, some smoothing must be applied to ρ
s to obtain the best visualisation, ρ
v. The Gaussian Visualisation method blurs the observations and further reduces the image resolution – so why is it necessary? The answer is given by the theory of Density Estimation, in which the objective is to minimise the difference between the visualisation, ρ
v, and the measured quantity ρ
m (Breiman et al. 1977). If a small sample of localisations is blurred too narrowly, then the visualisation will be spiky, so the difference between ρ
v and ρ
m may be drastic, and the super-resolution image could be misleadingly punctate. On the other hand, too coarse a smoothing process will worsen the resemblance of ρ
v to ρ
m, and also waste resolution. Determining the best width of a kernel smoothing filter is a central task in Density Estimation. The best kernel width will optimise some figure of merit, such as the expected mean square error between the true distribution (ρ
m) and the visualisation ρ
v. The optimum kernel can be identified if the underlying structure (ρ
m) is already known, but this is not usually possible in practice. Some cases are well studied, however, and provide guidelines for the best kernel width. If ρ
m is known to be a single Gaussian, then an optimum smoothing kernel is itself a Gaussian of standard deviation h, (Silverman 1986).
In this case m is the underlying standard deviation (estimated by ), and n is the number of localisations. The n-1/5 term means this smoothing filter tends to a delta function as the number of localisations tends to infinity, which means the Precision Limit of resolution will be achieved in this case. For the case of a single localisation, the optimum smoothing filter is a Gaussian of width (standard deviation) ~ . This visualisation is optimised for a specimen described by a single Gaussian ρ
m: it would provide a good visualisation when the object is a single fluorophore on a coverslip, or for well-separated fluorophores. Real microscopy specimens are likely to contain multiple structures with various length scales, and for this type of distribution an adaptive kernel width with superior properties was proposed by (Breiman et al. 1977).
Here h
j
is the kernel width which should be applied to the j th localisation, where d
jk
is the k th nearest neighbour distance of the j th localisation, and α is a scaling constant. Optimisation of α and k is discussed in (Breiman et al. 1977). The key message is that the optimum kernel width depends on both the local sampling density (distance between localisations) and the underlying structure.
Having evaluated the amount of smoothing required to produce the optimal Gaussian Visualisation of a specimen, the length scale of this visualisation can be used to estimate the best image resolution that can be achieved when the number of localisations is finite. An interesting case is the scenario in which each fluorophore is localised exactly once. This could happen as a consequence of measuring a short data series (e.g. due to rapid bleaching), or intentionally (e.g. in a molecule counting experiment designed to capture a single bright image of each fluorophore, before bleaching it). With only a single localisation per fluorophore, the sampling density will probably not tend towards the “infinite” value needed to achieve the Precision Limit of resolution. The “best” visualisation method indicated by Equation 9 is to smooth the localisations with a Gaussian of standard deviation . Let us assume that a sample-dependent treatment with (Breiman et al. 1977), or another optimisation, does not significantly alter this value. In this case the average super-resolution image, <ρ
v >, is blurred by convolving the localised positions with a Gaussian visualisation kernel of standard deviation . The localisation microscopy image is therefore degraded twice: first by localisation error, and second by the smoothing required for visualisation. Both are convolutions with a Gaussian of standard deviation ~ , and this double convolution is equivalent to a single convolution with a Gaussian of standard deviation (Hirschman and Widder 1955).
This analysis leads to the remarkably simple result that, whereas the Precision Limit of resolution achieved with an infinite number of localisations is 2 , an image produced with a single localisation per fluorophore has its expected resolution coarsened (assuming optimal visualisation) to about 2 , or approximately 3 . Importantly, this is an average or expected resolution: with a limited number of localisations there are many possible permutations of the super-resolution image, as shown in Figure 1c, but 2 is the average resolution. Furthermore, the optimal amount of smoothing required in Gaussian Visualisation is subjective and cannot be determined exactly without knowing the structure of the specimen beforehand. However, a Precision Limit of 2 , and a limited sampling resolution of 3 , can provide a useful blind estimate of the resolution achieved with a real specimen.
These resolution estimates are derived for localisation microscopy using a Gaussian Visualisation. How do alternative visualisation algorithms compare with this? Histogram visualisation (Wolter et al. 2010), for example, involves “binning” the localisations onto a grid of super-resolution pixels. The width of the histogram pixels effectively smoothes or blurs the localisation data, analogous to the smoothing involved in visualisation by convolving the localisations with a Gaussian. Based on this similarity, it seems plausible that a skilful histogram visualisation can achieve a resolution that is similar to the limit obtained by smoothing with a Gaussian, although the histogram is likely to be slightly worse due to hard-edge features. Certainly the histogram method asymptotes to the same Precision Limit, since in the case of infinite localisations a histogram can be plotted using infinitesimal bin widths and hence no loss of resolution due to plotting. The jittered histogram visualisation is based on a principle similar to an adaptive-width kernel, and should achieve the same resolution (Baddeley et al. 2010). A simple scatter plot, however, is a discontinuous visualisation and its resolution therefore cannot be directly compared with the continuous image visualisations considered above.
Nyquist (labelling) limit
Nyquist-limited resolution relates to the fluorescent labelling of a specimen. It is well known that discrete labelling constrains the useful resolution of any imaging method (Schroff et al. 2008). The essential idea is that features smaller than twice the typical separation of fluorescent labels cannot be reliably measured, due to aliasing, even if the precision-limited resolution of the microscope would be sufficient to do so, as shown in Figure 2c. Note that this limit arises from the distance between fluorophores on the sample. This is generally different from the distance between localisations, since a fluorophore might be localised several times; however if the fluorophore is static on the specimen then simply obtaining multiple localisations will not increase the underlying label density, and it is the spacing of the fluorophore labels that determines whether features are measurable. The fluorophore spacing can be estimated based on the known size of antibody labels, or by extrapolating from the density of localisations – in some cases this latter approach is simple, for example if there is precisely one localisation per labelled position (Shim et al. 2012).
Since the Nyquist limit is a concept from Fourier analysis, it strictly refers to subjects that are sampled at regularly spaced intervals. This is not true for localisation microscopy, where fluorophores are usually attached at random positions on their substrate. Further, specimens generally have very dense-labelling in some areas, and very sparse labelling in others. This means that the question of what resolution limit is imposed by label density is more complex than finding the average distance between fluorophores. There is a strong variation in the label density on different parts of a specimen, so fluorophore spacing and hence the Nyquist limit of resolution must be evaluated for particular features. This is a sample-dependent issue, and the best way to evaluate the Nyquist limit of resolution should be considered on a case-by-case basis.
Nyquist-limited resolution can also arise in print or on-screen images. When super-resolution images are displayed as a square raster, the pixel width must be less than half of the resolution that needs to be illustrated. Therefore the value of , from Equation 8, provides an indication of the best pixel size to use when plotting a super-resolution image.
Fluorophore width and corrected motion blur
So far the localisation microscope resolution has been evaluated with respect to the actual fluorescence of the sample. However, the purpose of super-resolution is usually to visualise some fluorescently-labelled structure, and this structure is in fact less precisely determined than the fluorescence due to the finite size of the fluorescent labels. If each fluorophore is localised with a precision σ, but the size of an antibody label causes the fluorescence to be displaced from the structure of interest by a distance d, then the effective localisation precision of the instrument is worsened. If the displacement of each localisation is in a random direction then the loss of precision can be modelled as a random walk. (A random displacement of the localised positions may occur for flexibly-attached fluorophores, or as an approximation in the case of rigidly-attached fluorophores – this approximation would be most valid for fluorophores that are more closely spaced than the localisation precision, σ, so that a localisation drawn from a particular area of the sample may come one of various fluorophores, each with static but random orientations, leading to a random offset in the localisation.) A similar loss of resolution can occur when drift-correction is applied to localisation microscopy data. In such cases, a fiducial marker is often used to track the localisation microscopy specimen, and any translational movement of the marker is subtracted from the localised fluorophore positions to mitigate motion blur. If the fiducial marker is itself localised with a finite random error, σfid, then its positional error worsens the effective precision of the corrected data. The instrument precision, fluorescent label size and uncertain drift correction can be combined as a random walk to estimate the effective localisation precision of the instrument, σeff.
(11)
The range of 2-3 then provides a corrected estimate of microscope resolution for the sample in question. Ideally the fluorescent label size and any inexact correction of motion blur should have a negligible effect, compared with fluorophore localisation precision.
Z-resolution in 3D localisation microscopy
The principle outlined so far is that the resolution of localisation microscopy depends on two factors, (a) localisation precision and (b) smoothing required for visualisation of finite data, and that a blind estimate of this resolution can be evaluated from the same source data that is processed to obtain a super-resolution image. This principle extends to three-dimensions; however the maths needed to evaluate the z-resolution varies between the different implementations of 3D localisation microscopy. Here we outline the general principles involved in assessing z-resolution. The task is simply to process the image data to estimate the standard error of each localised position in the z-direction, as well as in the lateral x and y directions. We also need to specify what it means to visualise a structure in three dimensions. In some methods of 3D localisation microscopy the visualisation may be a z-stack of super-resolution images, and in other methods the mean z-position of localisations may be indicated with a colour code (which is unsuitable for showing features at multiple z-positions). Neither of these methods is perfect. To evaluate the theoretical limit of z-resolution, it is possible to extend Equation 7 into three dimensions, so that the three-dimensional scatter plot of localisations (ρ
s3) is smoothed into a three dimensional scalar field of localisation density (ρ
v3) – probably with an ellipsoidal smoothing function. This representation can then be sliced or projected onto various planes for visualisation in 2D.
The method required to estimate the error of z-localisation depends on the technique used for 3D localisation. In the astigmatism-based method of 3D localisation, the vertical or horizontal ellipticity of a fluorophore image is used to evaluate its z-position. In this method, a Bayesian method of z-localisation is ideally suited to estimating the z-localisation error at the same time as the z-position (Shaevitz 2009). Alternatively the uncertainty in z-position could be evaluated by algebraic error propagation of the uncertainties in the x- and y-widths fitted to each PSF. In either case, once the uncertainties of the z-positions have been evaluated, the z-resolution follows from the method derived above for 2-dimensions. The Precision Limit of z-resolution is simply twice the estimated localisation error, and the limited sampling case is three times. Unfortunately, this resolution has to be evaluated on a regional basis, rather than being uniform over the image. This is because of an interesting drawback of astigmatism-based 3D localisation microscopy: the optical PSF widths, and hence the Precision Limits of resolution, are dependent on z-position and they are different in the x and y directions. The z-resolution also varies with z-position, and is asymmetric. This makes it harder to express the resolution of a super-resolution image, even for a single z-slice. This drawback does not afflict the double-helix method of 3D localisation microscopy, since in that system the lateral and axial localisation precisions are translationally invariant over the depth of field (Badieirostami et al. 2010). In the double helix method, the uncertainty of z-positions could be estimated via a Fisher information approach, or by comparison with tabulated calibration data. Another 3D localisation microscopy technique is the biplane method: in this method, the ratio of image intensities measured from different focal planes within the sample is used to determine the z-position of a fluorophore (Juette et al. 2008). This technique also benefits from translationally invariant lateral resolution, and fairly homogenous axial resolution, however the uncertainty of z-position may have to be estimated via a lookup table of calibration data. The lateral localisation precisions could be estimated using the Thompson formula.
A final comment on z-resolution is that the image of a three-dimensional specimen may contain a much higher density of fluorophores than the image of a flat, two-dimensional specimen. In order to achieve sparse fluorescence images of individual fluorophores, subject to the constraint of limited photoswitching ratios, the spacing of fluorescent labels will need to be greater. This means that the Nyquist limit of resolution in 3D localisation microscopy may pose a greater problem than in 2D.
Mislocalisations and uncorrected motion blur
Localisation microscopy is predicated upon the ability to identify individual fluorophore positions, based on sparse fluorescence image data. Quality control methods are applied, either implicitly or explicitly, to exclude imprecise or spurious localisations, but some mislocalisations may persist which are not from single, photoswitching fluorophores. When the localised fluorescence density is visualized, these features may result in various artefacts. Some examples are: persistent fluorescent background features may be visualized as spots of concentrated fluorophore density; overlapping fluorescence signals from close, but separate, features may be visualized as a spurious blur of fluorophore density in between the features; and mobile fluorescent molecules that wander through the focal plane of the microscope may result in “salt and pepper noise” or “ant tracks.” Uncorrected mechanical drifts of the specimen can generate “motion blur” in the reconstructed image – although this motion blur is in fact a correct visualization of a moving sample, it is unhelpful and worsens the resolution of the system. Lateral drift may be corrected using a method such as fiducial markers. Drift in the z-direction may also occur. As well as leading to a vertical streaking of the visualisation – which resembles a worsened z-resolution – z-drift also causes features to move relative to the focal plane of the microscope. Features that lose focus have a broader optical PSF, and hence a worse localisation precision (after Equation 1), and therefore the Precision Limit of the (lateral) resolution is also worsened for specimens that experience z-drift. Fiducial markers may be able to correct the motion blur caused by z-drift, but can do nothing to prevent the loss of resolution that arises from broader PSFs and the consequent loss of (lateral) localisation precision.
Because mislocalisations arise from a failure of the assumption that underlies localisation microscopy – they occur when it is not possible to localise precise positions exclusively from individual fluorescent labels, and so Equations 2–7 do not properly describe the process – their impact on instrument resolution is difficult to quantify.