Abstract
Advances in rendering have led to tremendous growth in texture assets, including resolution, complexity, and novel textures components, but this growth in data volume has not been matched by advances in its compression. Meanwhile Neural Image Compression (NIC) has advanced significantly and shown promising results, but the proposed methods cannot be directly adapted to neural texture compression. First, texture compression requires on-demand and real-time decoding with random access during parallel rendering (e.g. block texture decompression on GPUs). Additionally, NIC does not support multi-resolution reconstruction (mip-levels), nor does it have the ability to efficiently jointly compress different sets of texture channels. In this work, we introduce a novel approach to texture set compression that integrates traditional GPU texture representation and NIC techniques, designed to enable random access and support many-channel texture sets. To achieve this goal, we propose an asymmetric auto-encoder framework that employs a convolutional encoder to capture detailed information in a bottleneck-latent space, and at decoder side we utilize a fully connected network, whose inputs are sampled latent features plus positional in texture coordinate and mip level. This latent data is defined to enable simplified access to multi-resolution data by simply changing the scanning strides. Experimental results demonstrate that this approach provides much better results than conventional texture compression, and significant improvement over the latest method using neural networks.
Qualcomm AI Research is an initiative of Qualcomm Technologies, Inc.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
Available at https://github.com/ARM-software/astc-encoder.
References
Agustsson, E., Minnen, D., Johnston, N., Balle, J., Hwang, S.J., Toderici, G.: Scale-space flow for end-to-end optimized video compression. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8503–8512 (2020)
Agustsson, E., Minnen, D., Toderici, G., Mentzer, F.: Multi-realism image compression with a conditional generator. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 22324–22333 (2023)
Agustsson, E., Tschannen, M., Mentzer, F., Timofte, R., Gool, L.V.: Generative adversarial networks for extreme learned image compression. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 221–231 (2019)
Ballé, J., Minnen, D., Singh, S., Hwang, S.J., Johnston, N.: Variational image compression with a scale hyperprior. In: 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, 30 April–3 May 2018, Conference Track Proceedings (2018)
Ballé, J., Laparra, V., Simoncelli, E.P.: End-to-end optimized image compression. In: International Conference on Learning Representations (2017)
Beers, A.C., Agrawala, M., Chaddha, N.: Rendering from compressed textures. In: Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH 1996, pp. 373–378. Association for Computing Machinery, New York, NY, USA (1996)
Bjontegaard, G.: Calculation of average PSNR differences between RD-curves. Technical report, ITU-T SG 16 Q.6 document VCEG-M33, 13th VCEG meeting, Austin, Texas, USA, April 2001
Delp, E., Mitchell, O.: Image compression using block truncation coding. IEEE Trans. Commun. 27(9), 1335–1342 (1979)
Dupont, E., Goliński, A., Alizadeh, M., Teh, Y.W., Doucet, A.: Coin: compression with implicit neural representations. arXiv preprint arXiv:2103.03123 (2021)
Fenney, S.: Texture compression using low-frequency signal modulation. In: Proceedings of the ACM SIGGRAPH/EUROGRAPHICS Conference on Graphics Hardware, HWWS 2003, pp. 84–91. Eurographics Association, Goslar, DEU (2003)
Finn, C., Abbeel, P., Levine, S.: Model-agnostic meta-learning for fast adaptation of deep networks. In: Precup, D., Teh, Y.W. (eds.) Proceedings of the 34th International Conference on Machine Learning. Proceedings of Machine Learning Research, vol. 70, pp. 1126–1135. PMLR (2017)
Ghouse, N.F., Petersen, J., Wiggers, A., Xu, T., Sautière, G.: A residual diffusion model for high perceptual quality codec augmentation. arXiv preprint arXiv:2301.05489 (2023)
Habibian, A., Rozendaal, T.V., Tomczak, J.M., Cohen, T.S.: Video compression with rate-distortion autoencoders. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 7033–7042 (2019)
He, D., Yang, Z., Peng, W., Ma, R., Qin, H., Wang, Y.: ELIC: efficient learned image compression with unevenly grouped space-channel contextual adaptive coding. In: 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2022
Hill, S., et al.: Physically based shading in theory and practice. In: Proceedings of the ACM SIGGRAPH, SIGGRAPH 2020. Association for Computing Machinery, New York, NY, USA (2020)
Hou, Q., Farhadzadeh, F., Said, A., Sautiere, G., Le, H.: Low-latency neural stereo streaming. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 7974–7984, June 2024
Hu, Z., Lu, G., Guo, J., Liu, S., Jiang, W., Xu, D.: Coarse-to-fine deep video coding with hyperprior-guided mode prediction. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5921–5930 (2022)
Hu, Z., Lu, G., Xu, D.: FVC: a new framework towards deep video compression in feature space. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1502–1511 (2021)
Hughes, J.F., et al.: Computer Graphics: Principles and Practice, 3rd edn. Addison-Wesley (2013)
Jacob, B., et al.: Quantization and training of neural networks for efficient integer-arithmetic-only inference. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2018
Kingma, D.P., Ba, J.L.: Adam: a method for stochastic optimization. In: Proceedings 3rd International Conference on Learning Representations, pp. 1–15 (2014)
Kornblith, S., Norouzi, M., Lee, H., Hinton, G.: Similarity of neural network representations revisited. In: Chaudhuri, K., Salakhutdinov, R. (eds.) Proceedings of the 36th International Conference on Machine Learning. Proceedings of Machine Learning Research, vol. 97, pp. 3519–3529. PMLR (2019)
Le, H., Pourreza, R., Said, A., Sautiere, G., Wiggers, A.: GameCodec: neural cloud gaming video codec. In: BMVC, p. 204 (2022)
Le, H., et al.: MobileCodec: neural inter-frame video compression on mobile devices. In: Proceedings of the 13th ACM Multimedia Systems Conference, pp. 324–330 (2022)
Li, J., Li, B., Lu, Y.: Deep contextual video compression. In: Advances in Neural Information Processing Systems, vol. 34 (2021)
Li, J., Li, B., Lu, Y.: Hybrid spatial-temporal entropy modelling for neural video compression. In: Proceedings of the 30th ACM International Conference on Multimedia (2022)
Li, J., Li, B., Lu, Y.: Neural video compression with diverse contexts. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2023, Vancouver, Canada, 18–22 June 2023 (2023)
Lu, G., Ouyang, W., Xu, D., Zhang, X., Cai, C., Gao, Z.: DVC: an end-to-end deep video compression framework. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11006–11015 (2019)
Mentzer, F., Toderici, G., Tschannen, M., Agustsson, E.: High-fidelity generative image compression. In: Advances in Neural Information Processing Systems, vol. 33, pp. 11913–11924 (2020)
Minnen, D., Ballé, J., Toderici, G.D.: Joint autoregressive and hierarchical priors for learned image compression. In: Advances in Neural Information Processing Systems, vol. 31 (2018)
Muckley, M.J., El-Nouby, A., Ullrich, K., Jégou, H., Verbeek, J.: Improving statistical fidelity for neural image compression with implicit local likelihood models. arXiv preprint arXiv:2301.11189 (2023)
Müller, T., Evans, A., Schied, C., Keller, A.: Instant neural graphics primitives with a multiresolution hash encoding. ACM Trans. Graph. 41(4), 102:1–102:15 (2022)
Nystad, J., Lassen, A., Pomianowski, A., Ellis, S., Olson, T.: Adaptive scalable texture compression. In: Proceedings of the Fourth ACM SIGGRAPH/Eurographics Conference on High-Performance Graphics, EGGH-HPG 2012, pp. 105–114. Eurographics Association, Goslar, DEU (2012)
Park, J.J., Florence, P.R., Straub, J., Newcombe, R.A., Lovegrove, S.: DeepSDF: learning continuous signed distance functions for shape representation. In: Proceedings IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 165–174, January 2019
Rippel, O., Anderson, A.G., Tatwawadi, K., Nair, S., Lytle, C., Bourdev, L.: ELF-VC: efficient learned flexible-rate video coding. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 14479–14488 (2021)
Rippel, O., Nair, S., Lew, C., Branson, S., Anderson, A.G., Bourdev, L.: Learned video compression. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), October 2019
van Rozendaal, T., et al.: MobileNVC: real-time 1080p neural video compression on a mobile device. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 4323–4333 (2024)
Sheng, X., Li, J., Li, B., Li, L., Liu, D., Lu, Y.: Temporal context mining for learned video compression. IEEE Trans. Multimedia (2022)
Shirley, P., Ashikhmin, M., Marschner, S.: Fundamentals of Computer Graphics. AK Peters/CRC Press (2009)
Ström, J., Akenine-Möller, T.: iPACKMAN: high-quality, low-complexity texture compression for mobile phones. In: Proceedings of the ACM SIGGRAPH/EUROGRAPHICS Conference on Graphics Hardware, HWWS 2005, pp. 63–70 (2005)
Strümpler, Y., Postels, J., Yang, R., Van Gool, L., Tombari, F.: Implicit neural representations for image compression. In: Avidan, S., Brostow, G., Cissé, M., Farinella, G.M., Hassner, T. (eds.) ECCV 2022. LNCS, vol. 13686, pp. 74–91. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-19809-0_5
Takikawa, T., et al.: Variable bitrate neural fields. In: ACM SIGGRAPH 2022 Conference Proceedings, pp. 1–9. No. Article 41 in SIGGRAPH 2022. Association for Computing Machinery, New York, NY, USA, July 2022
Theis, L., Shi, W., Cunningham, A., Huszár, F.: Lossy image compression with compressive autoencoders. In: 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, 24–26 April 2017, Conference Track Proceedings. OpenReview.net (2017)
Toderici, G., et al.: Full resolution image compression with recurrent neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5306–5314 (2017)
Vaidyanathan, K., Salvi, M., Wronski, B., Akenine-Möller, T., Ebelin, P., Lefohn, A.: Random-access neural compression of material textures. In: Proceedings of SIGGRAPH (2023)
Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)
Wikipedia. 2024: S3 texture compression (2024). Accessed 25 Apr 2024. https://en.wikipedia.org/wiki/S3_Texture_Compression
Wronski, B.: Neural material (de)compression - data-driven nonlinear dimensionality reduction. 1https://bartwronski.com/2021/05/30/neural-material-decompression-data-driven-nonlinear-dimensionality-reduction/
Zhu, Y., Yang, Y., Cohen, T.: Transformer-based transform coding. In: ICLR (2021)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2025 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Farhadzadeh, F. et al. (2025). Neural Graphics Texture Compression Supporting Random Access. In: Leonardis, A., Ricci, E., Roth, S., Russakovsky, O., Sattler, T., Varol, G. (eds) Computer Vision – ECCV 2024. ECCV 2024. Lecture Notes in Computer Science, vol 15095. Springer, Cham. https://doi.org/10.1007/978-3-031-72913-3_23
Download citation
DOI: https://doi.org/10.1007/978-3-031-72913-3_23
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-72912-6
Online ISBN: 978-3-031-72913-3
eBook Packages: Computer ScienceComputer Science (R0)