Combining Texton Broadcasting with Noise Injection in Stylegan-2 to Create a More Universal Method of Texture Synthesis

Philip Riepe^*
*Correspondence: Philip Riepe, Department of Communication, Islamic Azad University, Tehran, Iran, Email:

Keywords

Broadcasting • Image analysis • Texton

Introduction

Because it provides crucial information for material appearance, comprehension, and characterization, texture is an important visual attribute for human perception and image analysis. A number of applications, including computer graphics, virtual reality, image analysis and compression, and human-computer interaction, rely on texture understanding, particularly texture analysis and synthesis. The stochastic nature of texture and human perception, which is the ultimate judge of texture quality, must be taken into account when studying texture analysis and synthesis. Since multiscale frequency decompositions have been used to model early visual processing in the brain, a number of authors have proposed algorithms for texture analysis and synthesis. In contrast, a statistical method for texture analysis is required due to the stochastic nature of texture. Portilla and Simon celli, who created a statistical model based on a steerable filter decomposition for synthesizing a wide range of textures, have proposed the most comprehensive parametric method for texture analysis and synthesis. Their objective was to offer a universal statistical model that parametrizes the space of visual textures, but it fails to model all textures successfully. As a result, the entire mathematical and perceptual characterization of texture remains a challenging issue [1-4].

The resurgence of neural networks has piqued a lot of interest in both academia and industry, and it holds the promise of pushing the boundaries in a lot of different areas of research. The generative adversarial network (GAN) is one of the most successful models. It has been used in a variety of applications, including inpainting, image-to-image translation, image superresolution, and the generation of human faces, anime characters, objects, and scenes. The issue of texture [modeling] synthesis, on the other hand, has not received as much attention. Despite being a subset of texture modeling, work has been done on texture recognition and classification, which is a different problem. Utilizing the GAN framework for texture synthesis and, ultimately, a more comprehensive mathematical and perceptual description of the visual texture space are the primary focuses of this work. A multiscale texton broadcasting module is added to the StyleGAN-2 framework as part of our novel approach to universal texture synthesis. This makes it possible to create a wide range of regular and stochastic textures [5].

Literature Review

An image that is spatially homogeneous and typically contains repeated elements with erratic variations in position, orientation, and color is generally accepted as a definition of visual texture. One of the pioneers of texture analysis and perception, Bela Julesz, coined the term "textons" to describe the repeated elements that make up a texture. Texture appearance typically consists of both periodic structure and stochastic variations, and it can range from completely random variations to completely regular periodic structure. However, our experiments revealed two ways in which the StyleGAN-2 framework fails to capture the periodic aspect of textures: First, when the latent space is randomly sampled, the trained model produces significantly fewer periodic textures; Second, once a latent vector for a periodic texture is found, injecting the model with various samples of multi-scale noise cannot produce distinct texture crops; this means that the synthesized periodic structure is frequently "anchored" in a specific location. We will examine these two flaws and demonstrate that the introduction of the texton broadcasting module can mitigate them [6].

Discussion

Subband decompositions have long been the foundation of conventional parametric approaches to texture analysis and synthesis. Impressive texture synthesis results were achieved by matching the histograms of a steerable filter decomposition by Heeger and Bergen; Their strategy, on the other hand, is restricted to stochastic textures. To synthesize a much broader range of textures, Portilla and Simon celli developed a more complex model that makes use of a wide range of subband statistics. A universal statistical model is used to parametrize the space of visual textures. Additionally, Markov Random Field (MRF)-based parametric approaches have demonstrated significant potential for texture synthesis. However, there is no texture modeling involved in exemplar-based texture synthesis [7].

For texture analysis and synthesis, a wide range of deep learningbased methods have also been proposed in recent years. Convolutional neural networks, which have been widely successful in image processing and computer vision tasks, are the foundation of the majority of these strategies. Gatys and co. Utilize a pretrained VGG-19 network to extract feature vectors from a given texture at various spatial scales in the network hierarchy. Next, synthesize another texture that matches the Gram-matrix of feature vector inner products at each scale, beginning with a random image. Ulyanov and others similar performance can be achieved by employing a quick feed-forward generative network. Li and co. further develop a feedforward generative network for the purpose of generating a variety of distinct textures. Rodriguez-Pardo and others extract a template of periodic textures using content loss [8].

Image synthesis has utilized implicit neural representation (INR) techniques in addition to conventional convolutional neural architectures. These techniques use a multilayer perceptron to predict image RGB values that correspond to input pixel coordinates rather than relying on successive convolutional layers that progressively increase the image size. Even though these techniques haven't been developed and looked into specifically for texture synthesis, some of them have offered features that might be useful for texture synthesis, like the ability to extrapolate beyond clearly defined image boundaries [9].

Good fellow and others introduced an adversarial formulation for training a generative model. In this formulation, a second discriminator network determines whether a generated image originates from the actual data distribution and provides feedback. The WGAN uses the Wasserstein distance-also known as the earth mover distance-between probability distributions to reduce mode collapse and enhance learning stability. Alternative formulations of Lipschitz continuity for the WGAN, such as gradient penalty and spectral normalization, result in additional enhancements [10].

Conclusion

Pioneered the use of GANs in texture synthesis. Who came up with the spatial GAN to create textures of any size. Nevertheless, like Gatys et al. The spatial GAN's capabilities are limited to producing identical textures, or one model per texture. Bergmann et al.'s periodic spatial GAN (PSGAN). Represents the first attempt to learn a latent space that can inject a periodic pattern with a random phase term at the base of the generator network to generate periodic textures. However, they trained on a very small dataset, and as we'll see below, the textures that came out aren't all that great.

Acknowledgement

None.

Conflict of Interest

None.

References

1. Cano, D and T. Ha Minh. "Texture synthesis using hierarchical linear transforms." Signal Process 15 (1988): 131-148.

Google Scholar, Crossref, Indexed at

2. Portilla, Javier, Rafael Fonolla Navarro, Oscar Nestares and Antonio Tabernero. "Texture synthesis-by-analysis method based on a multiscale early-vision model." Opt Eng 35 (1996): 2403-2417.

Google Scholar, Crossref, Indexed at

3. Karras, Tero, Samuli Laine and Timo Aila. "A style-based generator architecture for generative adversarial networks." (2019): 4401-4410.

Google Scholar, Crossref, Indexed at

4. Cimpoi, Mircea, Subhransu Maji and Andrea Vedaldi. "Deep filter banks for texture recognition and segmentation." (2015): 3828-3836.

Google Scholar, Crossref, Indexed at

5. Julesz, Bela. "Textons, the elements of texture perception, and their interactions." Nature 290 (1981): 91-97.

Google Scholar, Crossref, Indexed at

6. Wang, Zi-Ming, Meng-Han Li and Gui-Song Xia. "Conditional generative convnets for exemplar-based texture synthesis." IEEE Trans Image Process 30 (2021): 2461-2475.

Google Scholar, Crossref, Indexed at

7. Rodriguez-Pardo, Carlos, Sergio Suja, David Pascual and Jorge Lopez-Moreno et al. "Automatic extraction and synthesis of regular repeatable patterns." Comput Graph 83 (2019): 33-41.

Google Scholar, Crossref, Indexed at

8. Chen, Zhiqin and Hao Zhang. "Learning implicit fields for generative shape modeling." (2019): 5939-5948.

Google Scholar, Crossref, Indexed at

9. Mescheder, Lars, Michael Oechsle, Michael Niemeyer and Sebastian Nowozin, et al. "Occupancy networks: Learning 3d reconstruction in function space."

Google Scholar

10. Bayat, Niloofar, Kunal Mahajan, Sam Denton and Vishal Misra et al. "Down for failure: Active power status monitoring." Future Gener Comput Syst 125 (2021): 629-640.

Google Scholar, Crossref, Indexed at