Real-time localization and classification for digital microscopy using single-shot convolutional neural networks

Martin Fränzl; Frank Cichos

doi:10.1117/12.2568368

20 August 2020 Real-time localization and classification for digital microscopy using single-shot convolutional neural networks

Martin Fränzl, Frank Cichos

Author Affiliations +

Proceedings Volume 11469, Emerging Topics in Artificial Intelligence 2020; 114691G (2020) https://doi.org/10.1117/12.2568368
Event: SPIE Nanoscience + Engineering, 2020, Online Only

Abstract

We present an adapted single shot neural network architecture (YOLO) for the real-time localization and classification of particles in optical microscopy. Our work is aimed at the manipulation of microscopic objects in real-time by a feedback loop. The network is implemented in Python/Keras using the TensorFlow backend. The trained model is then exported to a GPU supported C library for real-time inference readily integrable in other programming languages such as C++ and LabVIEW. It is capable of localizing and classifying several hundred of microscopic objects even at very low signal-to-noise ratios running for images as large as 416 x 416 pixels with an inference time of about 10 ms. We demonstrate real-time detection in tracking and manipulating active particles of different types. Symmetric active particles, as well as Janus particles propelled by self-thermophoretic laser-induced processes, are identified and controlled via a Photon-Nudging procedure developed in the group.

Conference Presentation

Citation Download Citation

Martin Fränzl and Frank Cichos "Real-time localization and classification for digital microscopy using single-shot convolutional neural networks", Proc. SPIE 11469, Emerging Topics in Artificial Intelligence 2020, 114691G (20 August 2020); https://doi.org/10.1117/12.2568368

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available