FridayAFM - Neural networks and Gwyddion

Héctor here, your AFM expert at Nanosurf calling out for people to share their Friday afternoon experiments. Today I'll introduce you to the world of machine learning and how it can improve your throughput.

In yet another previous life, I spent quite some time working on machine learning. More precisely, I worked on how to use neural networks to approximate unknown functions based on data examples. The idea is that in some cases, figuring out the physical equations that connect a number of inputs to a number of outputs could be tricky (and the "everything is a parabola if you zoom in enough" doesn't help either), and thus having a black box that learns the behaviour of the system and helps making predictions is something good.

The example that I'm going to use today is how to use a neural network in Gwyddion to correct lines and random noise in an image. In other words (see image below), how to generate and train a neural network in Gwyddion that acts at every pixel of an image, to go from a noisy input set of pixels to a clean pixel. 

1 Using a neural network to de-noise AFM data. Main definitions and network architecture.

Many of you might say, but you can do that already with regular Gwyddion tools, why bother?

1. Automatization of processes. It could be tedious to repeat the same steps over and over.
2. Unusual cases. There are situations where is not possible to correct an image or take more data. For those cases having access to another tool could be handy.
3. A learning stepping stone to go further. We need to learn how to crawl before learning how to walk.

So, let's get on with it.

Generate synthetic data.

One of the most useful tools that Gwyddion has is the capability to generate synthetic data and to add noise to it. This capability, combined with the tool to model AFM tips and calculate the tip-sample convolution is very useful. The main application is to decide whether your image has tip artefacts, or if the shapes you see are due to the way the image is being capture or if there is something else to it (maybe the things you were imaging are not the shape you thought they were). Today we will use the synthetic data capability to generate an image with beads of different sizes.

Gwyddion>Data Process>Synthetic>Deposition>Particles

This will open a menu to customize the image generation. In my case, I will use 256 pixels and a 10 μm image.

The coverage, particle radius, and all the other options, are not relevant, feel free to play around to try generating images that match your samples better. Once you have the new image, clone it with the copy button on the Data Browser (see below), we will keep one image as ideal and the other one will be the one with the noise (to rename double click on the name).

Next we will add some random noise to the noise image (make sure you clicked on the noise image to have it selected).

Gwyddion>Data Process>Synthetic>Noise

This will open a menu that lets you fine tune the noise parameters.

Again, it is down to you how much you play with these parameters. Try to make it look like the images you regularly obtain with your AFM, and have in mind that if the noise is larger than the signal, the neural network might not be able to distinguish one from the other. 

Almost forgot, to get the noise based on your current image, you need to choose your current image in the Dimensions tab.

Now, with the new image, it is time to add line noise.

Gwyddion>Data Process>Synthetic>Line Noise

Here, I had better results with non-cumulative noise. This is kind of obvious, a cumulative line noise adds a tilt to the image, a local function like the neural network operating at small scale cannot act on an image-scale feature like tilt. I'm afraid the tilt still needs to be corrected with the traditional functions or you need to figure out a better approach than mine.

With the ideal and the noisy images ready, it is time to train the neural network.

Train the neural network.

The concept of the neural network is what was presented in the first figure. It takes one input as a model, and for each pixel it takes a window of pixels, these pixels are weighted and input into the hidden nodes to obtain the signal.

Gwyddion>Data Process>Multidata>Neural Network Training

The model is then the noisy image we generated, and the signal is the ideal image we had from the beginning.

Now, before hitting training, which parameters should we choose? 

The answer is we don't know a priori, we should train several times with different parameters and see which ones produce the best results.

Things to look for:

• If the error goes up with the steps of the training, probably the parameters are way off and the network cannot obtain the desired result, or the gradient optimization is stuck in a local minima, the best to do is to reinitialize with random weights and train again, or try different parameters.
• If the error drops very quickly and remains low. Probably there is overfitting and the network is going to be very good at converting this image but bad when trying with new data.

What worked best for me for this image is 27 by 27 pixel window, 6 hidden nodes, and 75 training steps. Its performance is not only based on the training process, but also how it behaved with test data (i.e. other noisy images).

Now, one important step. To be able to use this network, you need to save it. You do this after training, by inputting a name e.g. "de-noise", and hitting the store button.

With the neural network trained, it is time to use it with other images and see how well it performs. For that, I used the process described above to generate more testing images.

Use the neural network.

I first used the same process as before to generate a few test images. Then I apply the neural network on them by:

Gwyddion>Data Process>Multidata>Apply Neural Network

This opens the menu with the saved networks to choose which one to apply.

For the results... judge by yourselves.

1 Synthetic images: ideal, with noise, and de-noised using a neural network.

Not bad, but the real test is with images with other shapes. Will it work?

1 Synthetic images: ideal, with noise, and de-noised using a neural network.

It struggles more with the stars, but it does a decent job with the donuts, which is quite surprising, given that there wasn't too much optimization.

Final remarks: Explore, test, practice. This is probably not going to replace your standard practices to process data, but it could be useful in some situations (for instance, when the data looks bad, nothing works to clean it, and there is no possibility to take more images), so if you get a feeling for what it can and cannot do, it could be useful.

I hope you find it useful, and it helps you making your first steps onto machine learning in AFM. Please let me know if you use it, and as usual, if you have suggestions or requests, don't hesitate to contact me.

Further reading:

1 Hoichan Jung, Giwoong Han, Seong Jun Jung, Sung Won Han, Comparative study of deep learning algorithms for atomic force microscopy image denoising,
Micron, Volume 161, (2022),103332. https://doi.org/10.1016/j.micron.2022.103332 

2 Nečas D, Klapetek P. Synthetic Data in Quantitative Scanning Probe Microscopy. Nanomaterials (Basel). 2021 Jul 2;11(7):1746. https://doi.org/10.3390/nano11071746