Nvidia uses deep learning to fix photo noise

Works without ever being shown what a noise-free image looks like

AI expert Nvidia has managed to develop a way to fix grainy or pixelated photos using deep learning.

Most deep learning work in recent years has focused on training a neural network to restore images by showing example pairs of noisy and clean images. The AI then learns how to make up the difference.

However, by simply looking at examples of corrupted photos only, and without ever being shown what a noise-free image looks like, this new AI tool is able to make photos taken in in low light look instantly more clean by automatically removing artefacts, noise, and grain.

The work was developed by Nvidia's researchers alongside Aalto University, and MIT, and will be presented at the International Conference on Machine Learning in Stockholm later this week.

"It is possible to learn to restore signals without ever observing clean ones, at performance sometimes exceeding training using clean exemplars," the scientists said in the research paper.

"[The neural network] is on par with state-of-the-art methods that make use of clean examples - using precisely the same training methodology, and often without appreciable drawbacks in training time or performance."

To make it happen, the researchers used Nvidia Tesla P100 GPUs with the cuDNN-accelerated TensorFlow deep learning framework.

The method could also be used for more altruistic measures, to enhance MRI images, perhaps paving the way to improving medical imaging. It could also be used in other real-world situations where obtaining clean training data is difficult, such as low-light photography, physical-based rendering and magnetic resonance imaging.

"Our proof-of-concept demonstrations point the way to significant potential benefits in these applications by removing the need for potentially strenuous collection of clean data," the research team added.

"Of course, there is no free lunch - we cannot learn to pick up features that are not there in the input data - but this applies equally to training with clean targets."