OPTIMAL MULTI-CHANNEL WAVEFRONT SHAPING FOR HIGH DENSITY 3D LOCALIZATION MICROSCOPY
2Biomedical Engineering Department and the Lorry I. Lokey Center for Life Sciences and Engineering, Technion - Israel Institute of Technology, Haifa, Israel
3Viterbi Faculty of Electrical Engineering, Technion - Israel Institute of Technology, Haifa, Israel
4Google Research, Google, Haifa, Israel
5Engineering Physics, Polytechnique Montréal, Montréal, Québec, Canada
Snapshot 3D Localization of fluorescent emitters is crucial for many biological applications that require tracking of single particles with super-resolution. Point-Spread Function (PSF) engineering is a common method to achieve instantaneous depth sensing. This is realized by modulating the optical microscope to encode the target molecule’s 3D position under low signal conditions. In single-molecule applications, the common approach is to use a phase only modulation in a single channel to maintain a high pixel-wise signal to noise ratio. These methods substantially extend the PSF laterally (compared to the Airy disc pattern), trading lateral resolution for axial encoding. Here [1] we show that under high labelling density conditions, where a single channel PSF creates overlapping patterns, multi-channel wavefront coding performs considerably better in terms of detectability and precision, even in low-light applications. Our method is implemented via a bifurcated optical system which extends the emission path of an inverted optical microscope with a high NA objective (1.49). An overview of the method is presented in figure 1.
.png)
Figure 1: End-to-end learned phase masks and localization network enable precise sub-cellular 3D tracking in live cells, at extremely high labelling densities. The spatiotemporal trajectories are recovered from a time lapse of snapshot image-pairs encoding 3D information jointly in their PSFs.
We provide with three different approaches to design multi-channel imaging: (1) combining extended depth of field imaging with single channel axial localization, (2) dual channel optimization based on information theory, and (3) end-to-end learning using a neural net. At extreme densities, our results imply that a split-signal system, with end-to-end learned phase masks, doubles the detection rate and reaches improved precision compared to the current single-channel state-of-the-art. We experimentally validate our method by 3D tracking densely labelled telomers in live U2OS cells (figure 2)
.png)
Figure 2: Dense-particle tracking of labelled telomeres in live U2OS cells with the optimized PSFs. (a) A single time point showing the two PSF-modulated images. (b)-(c) 3D spatiotemporal trajectories for telomeres (b) and (c), exhibiting drastically different diffusion behaviors, in different regions of the nucleus. (d) 3D rendered cell with all the accumulated tracks showing the motion tracking of telomeres in 3D. Most telomeres were localized in all frames ( missing localizations). (e) Ensemble mean squared displacement (MSD) of all the estimated tracks, obscures the dynamics of individual particles, such as tracks (b) and (c).
[1] Nehme, E., Ferdman, B., Weiss, L. E., Naor, T., Freedman, D., Michaeli, T., & Shechtman, Y. (2021). Learning optimal wavefront shaping for multi-channel imaging. IEEE Transactions on Pattern Analysis and Machine Intelligence, 43(7), 2179-2192.