DeepSTORM3D: Deep Learning for Dense 3D Single Molecule Localization Microscopy

KEYWORDS: 3D single molecule localization microscopy, psf engineering, deep learning.

Localization microscopy is an imaging technique in which the positions of individual nanoscale point emitters (e.g. fluorescent molecules) are determined at high precision from their images. This is the key ingredient in single/multiple-particle-tracking and several super-resolution microscopy approaches like (f)PALM and STORM. Localization in three-dimensions (3D) can be performed by modifying the image that a point-source creates on the camera, namely, the point-spread function (PSF), using additional optical elements (e.g. a DOE/SLM). However, localizing multiple adjacent emitters in 3D poses a significant algorithmic challenge, due to the lateral overlap of their PSFs.

Here we present two fundamental contributions to tackle the problem of high-density overlapping PSFs in 3D localization [1]. First, we employ a convolutional neural network (CNN) for 3D localization from dense fields of overlapping emitters with engineered PSFs, and demonstrate it using the Tetrapod PSF. Second, we design an optimal PSF for high-density 3D microscopic particle localization for a large axial range of 4 µm. This is done by incorporating a physical-simulation layer in the CNN with an adjustable phase modulation, thus jointly learning the reconstruction (decoding) and the optimal PSF (encoding).

Our approach is highly flexible and can be easily adapted to any 3D localization data set. We demonstrate our approach numerically as well as experimentally in two different scenarios: 3D STORM imaging of mitochondria over a large axial range of 4 microns, and single snapshot imaging of dozens of fluorescently labeled telomeres occupying a mammalian nucleus.

[1] Nehme, E., Freedman, D., Gordon, R., Ferdman, B., Michaeli, T., & Shechtman, Y. (2019). Dense three-dimensional localization microscopy by deep learning. arXiv preprint arXiv:1906.09957.‏