Evaluation of Artificial Intelligence Image Upscaling and its Application in the Identification of Human Remains with Craniofacial Superimposition

Ivaylo Zhekov^*
*Correspondence: Ivaylo Zhekov, Department of General and Clinical Pathology, Forensic Medicine and Deontology, “Prof. Dr. Paraskev Stoyanov” Medical University, Varna, Bulgaria, Email:

Keywords

Artificial intelligence • Image upscaling • Forensic medicine • Craniofacial superimposition • Identification • Forensic anthropology

Introduction

Modern computer technologies play an important role in the development of all branches of the medical science. These new resources can be and are a valuable asset in field of forensic medicine as well, especially when the identification of human remains is concerned. DNA-comparative analysis is an invaluable method of establishing identity, however it is expensive, time consuming and relies on comparative material. This makes it inapplicable in certain cases and we must turn to different methods. Craniofacial superimposition is a method that relies on the anatomical link between the location of the soft tissue surfaces relative to the underlying bone [1,2]. This technique in its classic form requires a relatively intact skull and a photograph of the suspected individual. This image of the individual could be taken from a variety of sources and mediums, which are of differing quality and resolution. This can be problematic in cases where the quality of the images is so poor as to obstruct the accurate identification of the cephalometric landmarks. The same problem could arise if the skull has to be photographed in field conditions with low quality equipment. That can result in an image that has a low resolution that might need enhancement or resizing. A solution to this problem could lie in the emerging new developments in the field of artificial intelligence and deep learning. Different software applications using these technologies were developed with the aim of image enhancement and resolution upscaling that show promising results [3]. However, their application can be tied with number of other complications resulting from the way the image upscaling is achieved. In traditional upscaling, pixels from the lower resolution image are copied and repeated to fill out all the pixels of the higher resolution display [4]. New pixels are conducted based on the nearby range of known data. In this way you gain an image with larger resolution but with same or slightly worse quality. This is not the case with the AI upscaling. This new method involves producing new pixels of picture information to add detail where there wasn't any before, filling in the gaps to build a higher-resolution image and applying machine learning to improve the final product. The deep learning model predicts a high-resolution image that would downscale to look like the original, low-resolution image. A neural network model (which allows programs to recognize patterns and solve common problems in AI) is trained on innumerable images to predict photos with great accuracy. Since this technology creates new information on top of the original image, this introduces the problem of whether or not these enhanced images can be used in craniofacial superimposition. The aim of this study is to establish the strengths and weakness of the afore mentioned technology, by comparing images that have been upscaled using the traditional algorithms to those with AI upscaling.

Methods

The image used for the experiment had to be of excellent quality and the subject had to contain a lot of fine details as well as a physical marker for scale reference. For test subject, a human skull was used as it has a lot of fine detail that can be easily obscured by a photo with poor quality. The skull was placed on a wooden stand and a blue cloth was used as a backdrop for a good contrast. The scale marker, used was a circle divided into four equal quarters. The radius of the circle is exactly 1 centimeter. The marker was placed on the frontal bone of the skull for easy measurements. The camera used the Sony SLT-A58 with SAM II DT 18-55mm, lens [5]. The camera has a 20.4 megapixel cropped sensor which allows for fairly big format and detailed photos. For the shot the “manual” mode was used for optimal picture quality. The camera settings used were as follows: ISO – 100; Shutter speed –1/50 s; f/ number or aperture width –f10.0; lens focal length –55 mm.

To downsize the photos the Microsoft “Paint” app was used as it comes free with the Windows 11 operating system and it is easy to use. It has multiple functionalities one of which allows for rescaling images using traditional algorithms [6]. The chosen software that utilizes AI upscaling was Gigapixel AI v5.5.0 by Topaz labs [7]. It was used as it is easy to use and relatively inexpensive. It also has a free version which allows for testing the product prior to purchase (Figure 1). The software can AI upscale up to 600%. After that it resorts to the traditional upscaling methods. The final images were compared both with a software application and by visual inspection to determine how the images were changed and which came closer to the original. For the purposes of comparison of two images on “pixel by pixel” basis different software had to be used [8]. An open-source application, called “DiffImg” was used, although it had some stability issues [9]. It is easy to use and it can display difference as a coloured overlay and display some error statistics. (Figure 2) The downside is that the compared images have to have the exact same number of pixels.

To aid the visual inspection another similar software application was also used called “FastStone Image Viewer” [10]. This program has the capability of simultaneous assessing of up to four images of the same or of a different pixel count, which allowed the side-by-side comparison between the original image and the upscaled ones. After the original photo was taken, it was transferred onto a PC capable of running the aforementioned software. Then by using “Microsoft Paint” app the image was downsized to 50%, 25% and 20% of the original size. This creates three new images which then are imported into “Gigapixel AI”. The 50% image is upscaled by a scale factor of 2X with a preset AI model set to “standard”. The 25% and 20% image are upscaled by 4X and 5X scale factor with the other available settings remaining the same. The new upscaled images can now be compared to the original image using “DiffImg” and “FastStone Image Viewer”. The downsized images are also upscaled to the original image size using the traditional algorithms again via the “Paint” app that comes with the Windows operating system. The process is strait forward and easy.

Results

The image processing resulted in 10 images whose characteristics are shown in Table 1. For the purpose of the study each image was assigned a shorter designation, shown in Table 1. With downsizing, the file size of the images is also lowered without following any apparent pattern. When the original image was downsized 5 times to 20% of the initial size there is pixel loss. This is because the original pixel count –3632 × 5456 is divided by 5 the result is –726.4 × 1091.2 and since pixels are represented with whole numbers, the pixel count is rounded up. This unfortunately means that in the upscaling process the original pixel count cannot be achieved and thus the image cannot be compared in the “DiffImg” application and only a visual inspection can be carried out.

In the “DiffImg” application, six comparisons were carried out, between image- O, Image-A1, Image-A2, Image-B1 and Image-B2. The data that the application outputs is displayed in Table 2. As a whole the errors, meaning the different pixels in the AI upscaled images are more numerous than in the traditionally upscaled images. The standard deviation in AI upscaled images holds higher values than in traditionally upscaled images, which would signify that the pixels in the new image are much different than the original. With the larger factor of enlargement, the Ai upscaled images show an increase in errors percentage wise and numerically. This is also the case with the images upscaled with the traditional algorithms in the “Paint” app.

The visual comparison shows that the Ai upscaled images appear sharper and with a finer detail than the traditionally upscaled ones. The differences between the images become more apparent at extreme magnification. In Image A2, B2 and C2 the textures are flatter and not as clearly defined (Figures 3-5). There is no clear contrast between the different features. In images A1, B1 and C1 the differences are clearly noticeable even with no magnification. The images appear sharper than the original with a lot of texture and clearly defined lines. When the images are magnified to 300% individual differences in the features of the skull become noticeable (Figures 6-8). Some details that are present in the original image are slightly warped or outright changed. These changes become more frequent the higher the factor of Ai upscaling. At magnification of a 1000% (Figure 9) the higher contrast between the individual pixels becomes more apparent as well as the different color tones that the AI has introduced to the image.

Discussion

One of the earliest methods for craniofacial superimposition relied on overlapping the photo of the skull whose identity is investigated and the photo of the face of the person taken antemortem. The method consisted of obtaining the negative of a person’s face photograph and marking the cephalometric landmarks on it and then repeating the same task on the skull photograph [2,11,12]. Then, both negatives were overlapped and the positive was developed. With the emerging new technologies craniofacial superimposition has also undergone new developments and it has moved away from the original method to a point. It still relies on a 2D image of a face. This image has to be enhanced before the superimposition so that the cephalometric landmarks are visible and discoverable [1].

When an image is processed for visual interpretation, the viewer is the final judge of how well a particular method works [13]. This is even more the case when the viewer has to then apply markers on specific landmarks on the photo. Upscaling a low-resolution image would make this task easier and more accurate at least in theory. In upscaling the whole image is enlarged proportionately so the distance ratios between the different landmarks on the photo’s object should remain the same. It would seem that there is no risk of warping the image. The comparative data gained from the “DiffImg” application would suggest that there is a lot of completely new information introduced to the upscaled image regardless of the method used to achieve it. With traditional upscaling the errors seem to be a lot less but the lowest percentage of errors is still 94.4%. This data makes it hard to determine if there is any benefit to upscaling at all.

The visual differences however are more significant. The AI upscaled images tend to be sharper with more pronounced details than the traditionally upscaled ones. There are some artifacts (changed details or warped features) but those are visible at larger magnifications and they do not change the overall appearance of the image much less its proportions. The frequency of such artifacts increases the higher the upscale factor is, which could be expected since the amount of new information that has to be created also increases. This demonstrates the strength and weaknesses of the technology. There are diminishing return in using higher upscaling factors. Therein lies the main limitation of the technology. However, this could be subject to change as the AI algorithms become more accurate. Since AI upscaling technology is relatively new there is limited research into its application in the field of forensic medicine or medicine in general. As the technology matures and evolves the benefits from it will no doubt increase. Now it finds application in the entertainment industries, namely in video games [14].

Conclusion

In conclusion there is an application for artificial intelligence image upscaling in the identification of human remains with craniofacial superimposition, if the technology is used in moderation, ensuring the maximum quality gains at the cost of minimum artefacts. The point of diminishing returns of using this technology is yet to be determined with further research into the topic, but the results of this study are promising. This technology can be used in the field of forensic anthropology in craniofacial superimposition since it does not change the positions of the cephalometric and craniometric landmarks. When used the AI algorithms sharpen the image and make the textures appear more defined, making it easier to find the landmarks.

Acknowledgement

This research was funded by the Bulgarian Ministry of Education and Science with the national program “Young scientists and postdoctoral students -2”

Conflict of Interest

The author declares that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

References

1. Damas, Sergio, Oscar Cordón and Oscar Ibáñez.“Handbook on craniofacial superimposition: The MEPROCS project.” Springer Nature (2020).

   Google Scholar, Crossref, Indexed at

2. Damas, Sergio, Oscar Cordon, Oscar Ibanez and Jose Santamaria, et al. “Forensic identification by computer-aided craniofacial superimposition: A survey.”ACM Computing Surveys (CSUR)43 (2011): 1-27.

   Google Scholar, Crossref, Indexed at

3. NVIDIA Blog. “What is AI upscaling?.” Accessed April 8 (2023).
4. TensorPix. “AI upscaling vs. standard upscaling methods.” Published July 14 (2023).
5. Sony UK. “SLT-A58 specifications” Accessed August 28 (2023)
6. Davison, Patrick. "Because of the pixels: On the history, form and influence of MS Paint."J Vis Cult 13 (2014): 275-297.

   Google Scholar, Crossref, Indexed at

7. Gigapixel AI. Accessed August 25 (2023).
8. Katukam, Ravi and P. Sindhoora. "Image comparison methods & tools: A review."Image35 (2015): 42.

   Google Scholar

9. DiffImg download. SourceForge.net. Accessed August 25, 2023. https://sourceforge.net/projects/diffimg/
10. “Faststone image viewer - powerful and intuitive photo viewer, editor and batch converter.” Accessed August 28 (2023).
11. Rudolf Martin. “Textbook of anthropology - in a systematic presentation.” - PhilPapers Accessed September 2 (2023).
12. Caple, Jodi and Carl N. Stephan. "A standardized nomenclature for craniofacial and facial anthropometry."Int J Legal Med130 (2016): 863-879.

    Google Scholar, Crossref, Indexed at

13. Gonzalez, Rafael C.“Digital image processing.” Pearson education india (2009).
14. Watson, Alexander. “Deep learning techniques for super-resolution in video games.”arXiv preprint arXiv (2020): 2012.09810.

    Google Scholar, Crossref