Researchers at FIT are able to make low quality photos and text more focused

Taking a picture of quite a low quality where the details cannot be made out occasionally happens to everybody. Various computer programs can fix a lot. However, where classic software reaches its limits, one needs more complex operations. That is why the experts at the Faculty of Information Technology work with photography editing using convolutional neural networks. That may allow for making various historical documents at the Moravian Library in Brno become more focused and readable.

Nowadays, convolutional neural networks are mostly used in the field of artificial intelligence. Michal Hradiš and his team decided to use them in their research of automated photography focusing and other corrections of picture imperfections. For two years, they taught the networks to recognise which photos are of good quality and how to process the imperfect ones to be as close to reality as possible. "We used several hundred thousand pairs of pictures, where one was of good quality, and the second one was artificially blurred. The convolutional networks are able to use these examples to learn to change the pictures so that they are as close to the original as possible. If we present them enough training examples, in time, the networks will be able to fix real photographs that they have not seen yet," explained Michal Hradiš of the Department of Computer Graphics and Multimedia. This method is very successful, for example, when photographing text documents with a mobile phone, where even unreadable photos nearly rival the quality of desktop scanners after processing. Not only are the focused documents more readable, they also work much better with OCR, i.e. the automatic text recognition.

The researchers also focused on videos, in particular those from traffic cameras. They experimented with licence plate numbers in pictures from motorway cameras and toll gates. The program is able to determine the licence plate number even from a recording that is not very focused. So far, the method is not being used in this area, but could be applied in the future, for example in security.

At the moment, the scientists are improving the method for texts. "We are preparing a project where we will try to reconstruct old prints and manuscripts. The Moravian Library has a digital archive, making some old documents, such as old newspaper, available to the public. Some of the scans are readable, but with a lot of difficulties. These are the ones we are now using in experiments with the convolutional neural networks," Hradiš said, explaining their plans for the future. The limitation is that a damaged document must retain at least some information on the original contents. For example, it is possible to fill in small torn out parts and the result will look plausible, but the text will not make any sense. Similarly, the letters and font might be changed if the text is too blurred.

Researchers at the Faculty of Information Technology have been interested in convolutional neural networks for years and keep looking for their new applications in practice. They started with recognition and search of the contents of photographs, but this area is now being explored by research teams of big companies such as Google and Facebook. That is why at FIT, the researchers focus on the areas where further use of the networks is yet to be found.