Synthetic data-driven deep learning for label-free autonomous atomic force microscopy

1. Binnig, G., Quate, C. F. & Gerber, C. Atomic force microscope. Phys. Rev. Lett. 56, 930 (1986).

   Google Scholar 

2. Loos, J. The art of SPM: scanning probe microscopy in materials science. Adv. Mater. 17, 1821–1833 (2005).

   Google Scholar 

3. Checa, M., Neumayer, S. M., Tsai, W. Y. & Collins, L. Advanced modes of electrostatic and kelvin probe force microscopy for energy applications. In Atomic Force Microscopy for Energy Research 45–104 (CRC Press, 2022).

4. Dufrêne, Y. F. et al. Imaging modes of atomic force microscopy for application in molecular and cell biology. Nat. Nanotechnol. 12, 295–307 (2017).

   Google Scholar 

5. Muzio, M. D., Millan-Solsona, R., Borrell, J. H., Fumagalli, L. & Gomila, G. Cholesterol effect on the specific capacitance of submicrometric DOPC bilayer patches measured by in-liquid scanning dielectric microscopy. Langmuir 36, 12963–12972 (2020).

   Google Scholar 

6. Lozano, H. et al. Electrical properties of outer membrane extensions from Shewanella oneidensis MR-1. Nanoscale 13, 18754–18762 (2021).

   Google Scholar 

7. Ahmad Khalili, A. & Ahmad, M. R. A review of cell adhesion studies for biomedical and biological applications. Int. J. Mol. Sci. 16, 18149–18184 (2015).

   Google Scholar 

8. Checa, M., Millan-Solsona, R., Glinkowska Mares, A., Pujals, S. & Gomila, G. Dielectric imaging of fixed HeLa cells by in-liquid scanning dielectric force volume microscopy. Nanomaterials 11, 1402 (2021).

   Google Scholar 

9. Collins, L. et al. Perspectives in scanning probe microscopy from the 2021 joint international scanning probe microscopy and scanning probe microscopy on soft and polymeric materials conference. Microsc. Anal. 57, 933709 (2021).

10. Millan-Solsona, R. et al. Analysis of biofilm assembly by large area automated AFM. npj Biofilms Microbiomes 11, 75 (2025).

    Google Scholar 

11. Kalinin, S. V. et al. Automated and autonomous experiments in electron and scanning probe microscopy. ACS Nano 15, 12604–12627 (2021).

    Google Scholar 

12. Sadeghian, H. et al. Automated cantilever exchange and optical alignment for high-throughput parallel atomic force microscopy. IEEE/ASME Trans. Mechatron. 22, 2654–2661 (2017).

    Google Scholar 

13. Yoo, R. Y. The story behind the first automatic atomic force microscope. Microsc. Today 30, 40–45 (2022).

    Google Scholar 

14. Casuso, I. & Scheuring, S. Automated setpoint adjustment for biological contact mode atomic force microscopyimaging. Nanotechnology 21, 035104 (2009).

    Google Scholar 

15. Liu, H. et al. Intelligent tuning method of PID parameters based on iterative learning control for atomic force microscopy. Micron 104, 26–36 (2018).

    Google Scholar 

16. Abramovitch, D. Y., Hoen, S. & Workman, R. Semi-automatic tuning of PID gains for atomic force microscopes. In Proceedings of the 2008 American Control Conference 2684–2689 (IEEE, 2008).

17. Zhou, X., Dong, X., Zhang, Y. & Fang, Y. Automatic tuning of PI controller for atomic force microscope based on relay with hysteresis. In Proceedings of the 2009 IEEE International Conference on Control Applications (CCA) & Intelligent Control (ISIC) 1271–1275 (IEEE, 2009).

18. Liu, Y. et al. AEcroscopy: a software–hardware framework empowering microscopy toward automated and autonomous experimentation. Small Methods 8, 2301740 (2024).

    Google Scholar 

19. Checa, M. et al. High-speed mapping of surface charge dynamics using sparse scanning Kelvin probe force microscopy. Nat. Commun. 14, 7196 (2023).

    Google Scholar 

20. Degenhardt, J., Bounaim, M. W., Deng, N., Tutsch, R. & Dai, G. A new kind of atomic force microscopy scan control enabled by artificial intelligence: concept for achieving tip and sample safety through asymmetric control. Nanomanuf. Metrol. 7, 11 (2024).

    Google Scholar 

21. Wei, Z. et al. A Novel Deep Reinforcement Learning Approach for Dynamic Proportional-Integral Control in Scanning Probe Microscopy. Small 21, e03921 (2025).

22. Liu, Y. et al. Machine learning-based reward-driven tuning of scanning probe microscopy: toward fully automated microscopy. ACS Nano 19, 19659–19669 (2025).

    Google Scholar 

23. Liu, Y. & Kalinin, S. V. Pareto-optimal experimentation: human-guided multi-objective bayesian optimization in scanning probe microscopy. Nano Lett. 26, 441–447 (2026).

24. Paruchuri, A., Wang, Y., Gu, X. & Jayaraman, A. Machine learning for analyzing atomic force microscopy (AFM) images generated from polymer blends. Digital Discov. 3, 2533–2550 (2024).

    Google Scholar 

25. Yablon, D. et al. Deep learning to establish structure property relationships of impact copolymers from AFM phase images. Mrs Commun. 11, 962–968 (2021).

    Google Scholar 

26. Checa, M., Millan-Solsona, R., Mares, A. G., Pujals, S. & Gomila, G. Fast label-free nanoscale composition mapping of eukaryotic cells via scanning dielectric force volume microscopy and machine learning. Small Methods 5, 2100279 (2021).

    Google Scholar 

27. Vasudevan, R. K. et al. Autonomous experiments in scanning probe microscopy and spectroscopy: choosing where to explore polarization dynamics in ferroelectrics. ACS Nano 15, 11253–11262 (2021).

    Google Scholar 

28. Rade, J. et al. Deep learning for live cell shape detection and automated afm navigation. Bioengineering 9, 522 (2022).

    Google Scholar 

29. Sotres, J., Boyd, H. & Gonzalez-Martinez, J. F. Enabling autonomous scanning probe microscopy imaging of single molecules with deep learning. Nanoscale 13, 9193–9203 (2021).

    Google Scholar 

30. Huang, B., Li, Z. & Li, J. An artificial intelligence atomic force microscope enabled by machine learning. Nanoscale 10, 21320–21326 (2018).

    Google Scholar 

31. Deng, J., Dong, W., Socher, R., Li, L. J., Li, K. & Fei-Fei, L. ImageNet: A large-scale hierarchical image database. In Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 248–255 (IEEE, 2009).

32. Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z. & Guo, B. Swin Transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) 10012–10022 (IEEE, 2021).

33. Shorten, C. & Khoshgoftaar, T. M. A survey on image data augmentation for deep learning. J. Big Data 6, 1–48 (2019).

    Google Scholar 

34. Nečas, D. & Klapetek, P. Synthetic data in quantitative scanning probe microscopy. Nanomaterials 11, 1746 (2021).

    Google Scholar 

35. Klapetek, P. & Ohlı́dal, I. Theoretical analysis of the atomic force microscopy characterization of columnar thin films. Ultramicroscopy 94, 19–29 (2003).

36. Klapetek, P., Ohlídal, I. & Bílek, J. Influence of the atomic force microscope tip on the multifractal analysis of rough surfaces. Ultramicroscopy 102, 51–59 (2004).

    Google Scholar 

37. Klapetek, P., Valtr, M., Nečas, D., Salyk, O. & Dzik, P. Atomic force microscopy analysis of nanoparticles in non-ideal conditions. Nanoscale Res. Lett. 6, 514 (2011).

    Google Scholar 

38. Chen, Y. Improving dimensional measurement from noisy atomic force microscopy images by non-local means filtering. Scanning 38, 113–120 (2016).

    Google Scholar 

39. Wang, Y.-F. et al. Double-tip artifact removal from atomic force microscopy images. IEEE Trans. Image Process. 25, 2774–2788 (2016).

    Google Scholar 

40. Argento, C. & French, R. Parametric tip model and force–distance relation for Hamaker constant determination from atomic force microscopy. J. Appl. Phys. 80, 6081–6090 (1996).

    Google Scholar 

41. Niina, T., Fuchigami, S. & Takada, S. Flexible fitting of biomolecular structures to atomic force microscopy images via biased molecular simulations. J. Chem. Theory Comput. 16, 1349–1358 (2020).

    Google Scholar 

42. Fuchigami, S., Niina, T. & Takada, S. Case report: Bayesian statistical inference of experimental parameters via biomolecular simulations: atomic force microscopy. Front. Mol. Biosci. 8, 636940 (2021).

    Google Scholar 

43. Alldritt, B. et al. Automated structure discovery in atomic force microscopy. Sci. Adv. 6, eaay6913 (2020).

    Google Scholar 

44. González Lastre, M. et al. Molecular identification via molecular fingerprint extraction from atomic force microscopy images. J. Cheminformatics 16, 130 (2024).

    Google Scholar 

45. Beton, J. G. et al. TopoStats–A program for automated tracing of biomolecules from AFM images. Methods 193, 68–79 (2021).

    Google Scholar 

46. Navarro, P. P. et al. Cell wall synthesis and remodelling dynamics determine division site architecture and cell shape in Escherichia coli. Nat. Microbiol. 7, 1621–1634 (2022).

    Google Scholar 

47. Van Der Hofstadt, M. et al. Internal hydration properties of single bacterial endospores probed by electrostatic force microscopy. ACS Nano 10, 11327–11336 (2016).

    Google Scholar 

48. Lithgow, T., Stubenrauch, C. J. & Stumpf, M. P. Surveying membrane landscapes: a new look at the bacterial cell surface. Nat. Rev. Microbiol. 21, 502–518 (2023).

    Google Scholar 

49. Ultralytics YOLOv8, https://github.com/ultralytics/ultralytics (2023).

50. Ronneberger, O., Fischer, P. & Brox, T. U-net: Convolutional networks for biomedical image segmentation. In Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015. 234–241 (Springer, 2015).

51. He, K., Gkioxari, G., Dollár, P. & Girshick, R. Mask R-CNN. In Proceedings of the IEEE International Conference on Computer Vision (ICCV) 2961–2969 (IEEE, 2017).

52. Pyne, A. L. et al. Base-pair resolution analysis of the effect of supercoiling on DNA flexibility and major groove recognition by triplex-forming oligonucleotides. Nat. Commun. 12, 1053 (2021).

    Google Scholar 

53. Millan-Solsona, R. SimuScan-AFMtools (Version 1.0). Zenodo. https://doi.org/10.5281/zenodo.18665434 (2026).

Download references