Received 27 May 2023, accepted 17 July 2023, date of publication 21 July 2023, date of current version 27 July 2023.

Digital Object Identifier 10.1109/ACCESS.2023.3297873

Intelligent Metaverse Scene Content Construction

JUNXIANG WANG 1, SIRU CHEN 1, YUXUAN LIU 1, AND RICHEN LAU1,2
1 School of Computer and Electronic Information/School of Artificial Intelligence, Nanjing Normal University, Nanjing 210023, China
2 State Key Laboratory of Virtual Reality Technology and Systems, Beihang University, Beijing 100191, China

Corresponding authors: Siru Chen (222202026@njnu.edu.cn) and Richen Lau (329789995@qq.com)

This work was supported in part by the National Natural Science Foundation of China under Grant 61702271; and in part by the Open
Project Program of the State Key Laboratory of Virtual Reality Technology and Systems, Beihang University, under Grant
VRLAB2023B05.

ABSTRACT The integration of artificial intelligence (AI) and virtual reality (VR) has revolutionized
research across various scientific fields, with AI-driven VR simulations finding applications in education,
healthcare, and entertainment. However, existing literature lacks a comprehensive investigation that
systematically summarizes the fundamental characteristics and development trajectory of AI-generated
visual content in the metaverse. This survey focuses on intelligent metaverse scene content construction,
aiming to address this gap by exploring the application of AI in content generation. It investigates
scene content generation, simulation biology, personalized content, and intelligent agents. Analyzing the
current state and identifying common features, this survey provides a detailed description of methods for
constructing intelligent metaverse scenes. The primary contribution is a comprehensive analysis of the
current landscape of intelligent visual content production in the metaverse, highlighting emerging trends.
The discussion on methods for constructing intelligent scene content in the metaverse suggests that in the
era of intelligence, it has the potential to become the dominant approach for content creation in metaverse
scenes.

INDEX TERMS Content generation, metaverse, immersive visualization, deep learning.

I. INTRODUCTION text, images, audio, video, and models, our research focuses
The rapid evolution of metaverse technologies, coupled with on the utilization of AI methods to generate content within
exponential growth in computing power, has brought about metaverse scenes. Distinct from AIGC’s video content
a paradigm shift in the production of visual content. The generation [1], our emphasis is on the creation of specific
traditional manual creation of static scene content has been scenes that empower users to engage in free interaction
revolutionized by the emergence of intelligent metaverse within the generated metaverse environment. This essential
technology, which enables the generation of scene content feature sets our content generation approach apart from
with unprecedented capabilities. This intelligent approach simple videos, as it fosters a highly immersive and interactive
leverages existing data conditions as specialized tools for user experience integrated with the interaction concept of
programmatically constructing and enhancing metaverse Metaverse [2]. Based on the aforementioned considerations,
scenarios. The application of intelligent metaverse scene we have compiled and analyzed recent research on the
content generation extends to various scientific domains construction of intelligent metaverse scene content. We have
including education, biology, medicine, and art. Specifically, classified, summarized, and discussed these works in order to
it involves the dynamic construction of multi-variable scenes inspire future endeavors in this field.
through programming. The rapid advancement of AI technologies has led to the
In contrast to conventional artificial intelligence-generated development of various methodological models that integrate
content (AIGC), which predominantly focuses on generating AI intelligence into automated metaverse scene generation.
Three primary strategies have emerged: the development
The associate editor coordinating the review of this manuscript and of intelligent framework systems for coordinating meta-
approving it for publication was Xiaogang Jin . verse content, simulation of intelligent agents that replicate
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License.
76222 For more information, see https://creativecommons.org/licenses/by-nc-nd/4.0/ VOLUME 11, 2023
J. Wang et al.: Intelligent Metaverse Scene Content Construction

FIGURE 1. We choose a few exemplary papers from each category. Different publications are shown on the horizontal axis, while various kinds of
intelligent visual content production strategies are represented on the vertical axis, including scene content generation (V3D: virtual 3D construction,
CDG: classification-based directed generation, DG: data-driven generation, content construction and improvement (AC: adaptive change, EI: effect
improvement)), simulated biology (ILTS: individual life trait simulation, GBS: group behavior simulation, SFL: stress feedback learning), personalized
contents (ECP: emotional content personalization, NCP: non-emotional content personalization), intelligent agents (IATD: intelligent agents trained based
on data, IACK: intelligent agents constructed from external knowledge systems, intelligent agents trained for interaction data adaptation (SRI: simulating
realistic images, IAAC: interaction actions to adapt agent changes, IPTC: interaction processes to adapt task collaboration)). Many of the papers share the
basic characteristics of multiple intelligent visual content generation techniques at the same time. All the papers are categorized (in different colors) and
disaggregated according to the main techniques they use.

biological logic, and utilization of machine learning or deep representative studies [11], [12], [13]. In contrast, our survey
learning methods to generate or enhance scene content. offers a comprehensive analysis of recent advancements from
Understanding the comprehensive integration of intelligent a cutting-edge perspective, incorporating relevant papers
metaverse scene content generation requires a detailed study from the IEEE Xplore and ACM Digital Library based on
of these strategies and an analysis of the employed methods their relevance and research direction.
to reveal valuable insights into the future trends of metaverse
B. TAXONOMY OF THE SURVEY
scene content generation in the era of AI.
The exploration of AI for generating content in metaverse
scenes has led to the emergence of several common areas
A. RELATED SURVEYS
of interest, including scene content generation, simulated
Previous surveys in this field have predominantly focused on biology, personalized content, and intelligent agents, either
AI-based computer vision imaging, with particular emphasis individually or in combination (Figure 1). It is important
on evolutionary computation and biological vision [3], to highlight that many methods employ a combination
or on generating interactive and immersive content [4]. of these intelligent metaverse scene content generation
Some surveys have delved into the impact of content techniques rather than relying solely on a single approach.
generation in AI on the art industry [5] and explored deep While this survey does not encompass all papers related to
learning-based image style transfer [6]. Additionally, the metaverse scene content generation, its primary focus is on
Unreal 3D authoring engine has shown interest in leveraging the utilization of deep learning or other machine learning
agents and decision-tree-based procedural content generation methods and intelligent frameworks to achieve the generation
techniques for immersive game development [7]. of metaverse scene content.
In comparison to studies that delve into the application
of AI technology in machine learning for character pro- II. SCENE CONTENT GENERATION
gramming and calculation in games [8], or compare search- Integrating AI into the design framework of VR presents
based methods and machine learning methods to traditional a logical solution for overcoming the challenges associated
approaches in surveying game program content [9], our with automated metaverse scene content generation, thereby
survey takes a more comprehensive approach by focusing enhancing the realism of virtual environments. Deep-learning
on the application of AI technology and the generation of modules have emerged as effective tools for achieving
intelligent metaverse scene content. While there have been this objective. Notably, Unreal Engine explored AI-based
surveys that explore the inclusion of human emotions in approaches by utilizing agents and decision trees to generate
virtual agents and the advancements of AI in education [10], content in immersive game programs [7].
these surveys either have limited coverage of content This section introduces several common intelligent meth-
generation or primarily focus on detailed descriptions of a few ods used in the generation and improvement of metaverse

VOLUME 11, 2023 76223

 J. Wang et al.: Intelligent Metaverse Scene Content Construction

scene content. These methods can be broadly divided into

four main categories: virtual 3D reconstruction, directed
generation based on classification, data-driven generation,
and content construction and improvement. It is worth men-
tioning that certain approaches may exhibit characteristics
that align with multiple categories simultaneously. For this
study, we classified them based on their primary features.

A. VIRTUAL 3D CONSTRUCTION
Two-dimensional photos are crucial for reconstructing
three-dimensional scenes because of their accessibility and
the availability of comprehensive data. Wang et al. proposed
a method that utilizes the ant colony algorithm to rapidly
segment video scenes and combines the segmentation
process with the similarity of the same modal data and
the correlation of different modal data to construct multiple
panoramic-image arrangements, thus creating a VR scene
space [14]. Similarly, VIRTOOAIR significantly improves
the system’s posture recovery capability by incorporating FIGURE 2. Virtual 3D construction. (a) The processing pipeline for
VR tracking data and low-semantic input from RGB photos VIRTOOAIR. The data-collecting module for VR tracking is the initial
system component. To learn the upper body joint configuration, a deep
into an end-to-end reconstruction process using deep learning neural network is given the VR tracking data (i.e., HMD and hand
algorithms, as shown in Figure 2(a) [15]. In contrast to motions). Inverse kinematic regression of the deep neural network is
performed using the second module. Processing of RGB images is
purely two-dimensional image data, Zhang et al. applied handled by the third module, which also has a pre-processing procedure
effect processing to two-dimensional images, as shown in (i.e. extraction of bounding boxes). To rebuild the lower body joint
rotation, the bounding box and camera pictures are both continually input
Figure 2(c). The 2D input images were transformed using a into the end-to-end recovery framework (i.e., the fourth module). The
visual art language with a traditional Chinese ink and wash final module’s visualization tool is in charge of properly portraying the
style, followed by a 3D virtual technique to generate a 3D virtual body in VR [15]. (b) On a practice floor plan, tests were conducted.
The original floor layout can be seen in the top image, while the 3D
virtual scene in ink and wash style [16]. Freville et al., on the environment can be seen in the bottom image [17]. (c) Engaging AI [16].
other hand, indirectly employed YOLO for object detection
to identify realistic objects and generate corresponding 3D
world assets at their respective locations, as depicted in transformation of food categories while preserving the shape
Figure 2(b) [17]. of the original food item, as illustrated in Figure 3(b). This
These approaches demonstrate the potential of integrating method leverages conditional CycleGAN (cCy-cleGAN) [20]
deep learning techniques and innovative algorithms for to perform image transformations, along with food segmen-
constructing metaverse scenes. However, challenges remain tation results obtained from U-Net [21], a convolutional
in terms of handling complex scenes, ensuring the accuracy network designed for biomedical image segmentation. The
of the reconstruction process, and achieving a balance transformed images were rendered as textures.
between realism and efficiency. Further research is required These classification-based directed generation methods
to address these challenges and advance the field of virtual offer a more targeted approach to scene generation and
3D construction in metaverse environments. demonstrated the potential of deep learning techniques to
improve the realism and efficiency of metaverse content
B. CLASSIFICATION-BASED DIRECTED GENERATION creation. However, challenges remain, such as the need for
Generating different categories of objects often requires comprehensive and diverse training datasets, and the potential
specialized treatment to achieve the desired results. Naoki for bias in classification systems. Further research is needed
Matsuo et al. employed deep learning networks to classify to address these issues and to develop more sophisticated and
objects into ‘‘spatial boundaries’’ and ‘‘common obstacles.’’ reliable classification-based generation methods.
By overlaying virtual objects onto real objects based on their
classifications, they created a more recognizable VR space C. DATA-DRIVEN GENERATION
that remained enjoyable even in cluttered environments filled Data-driven generation relies on diverse collections of
with obstacles [18], as depicted in Figure 3(a). realistic data to achieve accurate and immersive simulations
Similarly, the generation of food textures can be expe- of various scenarios. For example, in the field of football,
dited through the pre-categorization of food items. The a trained PFNN model [22] can be used to make predictions
FoodChangeLens approach [19] uses a cyclic generative and create a fully immersive 3D environment, allowing
adversarial network (GAN) trained on a large-scale collection analysts, coaches, and players to view an entire game.
of food images from Twitter streams. This enabled the By capturing and learning human flow patterns through agent

76224 VOLUME 11, 2023

J. Wang et al.: Intelligent Metaverse Scene Content Construction

trajectories powered by a neural-network-based animation delivery aspects of content. Iglesias et al. proposed feature
system, player motions in the field and their dynamics from extraction using AI algorithms [27], as shown in Figure 4(c).
one event to the next can be accurately represented [23], They employed visualization methods and new UE4 classes
as depicted in Figure 3(c). In the domain of music instruction, to register and superimpose the collected characteristics on
hand motions can be automatically generated for 3D anima- a 3D model, enabling the identification of structural flaws
tions using MIDI data as the input. This is accomplished by using an AI-based approach. This technique improves the
leveraging a pre-trained Hidden Markov Model (HMM) for photorealism and immersive qualities of content.
fingerings detection [24], as shown in Figure 3(d). QM4VR [28], on the other hand, focuses on delivery
Data-driven generation techniques present exciting oppor- improvements rather than 3D model upgrades. By monitoring
tunities for creating realistic simulations and instructional the operation and performance of the sub-streams and
metaverse animations. The integration of diverse data modifying the load balancing/scheduling method to enhance
sources, such as football trajectories and MIDI inputs, the quality of service (QoS) parameters, the VR Content
enables immersive experience and skill development. How- Quality Multipath Delivery (QM4VR) approach manages
ever, addressing the challenges of data collection, dataset MPTCP sub-streams to distribute priority packets based on
quality, and fostering creativity in content generation are content awareness, as shown in Figure 4(d). This enhances
important areas for future research. By addressing these the delivery of VR content and ensures a smoother and more
limitations, researchers can unlock the full potential of reliable user experience.
data-driven generation in metaverse environments and further Furthermore, the rover positioning system [29] offers a
enhance immersive experiences and interactive capabilities. novel approach for improving the overall effect of the VR
experience, as depicted in Figure 5(a). By utilizing a bird’s
D. CONTENT CONSTRUCTION AND IMPROVEMENT eye view of the VR world and employing the MCL algorithm,
The construction and adaptation of content are essential for monocular visual-inertial odometry, and Siamese neural
creating engaging scenarios that provide users with unique networks [30], the system predicts the rover’s position by
experiences. To achieve this, content must be generated adding GAN-enhanced images to the VR image dataset, and
in a manner that fits realistic scenarios and provides the rover’s performance in the VR environment is enhanced.
natural variability in the development of different scenarios. To enhance the details of the scene contents, Feng Gao et al.
Researchers have explored two approaches to achieving this: applied convolutional neural networks in AI technologies
adaptive changes to content and effect improvements. to extract and reconstruct a virtual field point cloud [31],
as shown in Figure 5(b). They established a coordinate
matching relationship between a computer-generated virtual
1) ADAPTIVE CHANGE
scene and actual reality and improved the scene’s details
For an adaptive change, Lugrin et al. used an AI module using techniques such as the Gouraud shading algorithm
to generate event chains from a triggering perspective [25], and texture mapping. This approach reduces distortion and
as shown in Figure 4(a). They simulated the spontaneous enhances user satisfaction by providing a more visually
movements of objects, causing collisions between them, and appealing and realistic virtual environment.
selected modifications closest to a predefined cost threshold To address the challenge of sustainable scene style
based on heuristic values. This approach triggers conceptual transformation in real-time, Lei Yang et al. proposed methods
events related to the user experience and results in content to increase the frame rate of real-time rendering applications
that adapts to the user’s actions and choices. and applied them to the Barracuda style transformation in
Another example of adaptive change is DeepDive [26], VR [32], as shown in Figure 5(c). By optimizing the rendering
which incorporates AI tools to simulate spontaneous changes process, they enabled smooth and consistent scene style
in the environment to recognize the patterns of systems in transformations, improving the overall visual quality and
the resource environment of ancient civilizations, as shown coherence of the VR experience.
in Figure 4(b). This regional growth approach uses pattern These approaches focus on enhancing the visual quality,
recognition in hydrology, AI collection modules on vegeta- delivery efficiency, and overall immersion in metaverse
tion to predict potential vegetation of cells, A*, and cultural content, leading to more engaging and satisfying user
algorithms to generate reindeer herds, providing opportuni- experiences.
ties to generate and test anthropological and archaeological
theories. III. SIMULATED BIOLOGY
These adaptive change approaches offer users a unique and Simulating real-life situations through the generation of
dynamic experience that adapts to their choices and actions, virtual content has emerged as a valuable approach in the bio-
thereby creating more engaging scenarios. logical sciences. It offers researchers the opportunity to study
and analyze complex biological systems, providing insights
2) EFFECT IMPROVEMENT that may be challenging to obtain through direct observation
To improve the effectiveness of generated content, alone. In recent years, there has been a growing focus on
researchers have focused on enhancing the visual and utilizing AI-generated content to simulate the social attributes

VOLUME 11, 2023 76225

 J. Wang et al.: Intelligent Metaverse Scene Content Construction

FIGURE 3. Classification-based directed generation. (a) Real-time method processing pipeline for creating VR environments based on reality [18].
(b) Overview of the FoodChangeLens holographic system [19]. (c) Overview of the system: Motion-captured animations of individual football players are
utilized to create virtual agents (football players), and data taken from recorded football matches is used to create replay events in the VR
simulation [23]. (d) The input MIDI file comprises time sequences for the various keys, as seen in the top left image. Bottom left image: Our technology
creates a 3D performance animation for both hands based on the MIDI file. The figure on the right shows how the performance animation is displayed in
the HMD and is in tune with a real piano, giving the user real-time visual cues and feedback [24].

and habits of various organisms, including fish, flocks, and and individual responses of artificial fish [35], as shown in
birds, within biological ecological environments [33], [34]. Figure 6(b). To simulate diverse variations in the process of
This section discusses the application of AI content fish egg division, Xu and Zhou suggested a fish egg model
generation in simulated biology, covering three crucial based on the artificial life approach, which incorporates
aspects: individual life trait simulation, group behavior life dynamics, artificial life methodology, AI, and genetic
simulation, and stress feedback learning. By exploring these algorithms [36], as shown in Figure 6(a).
areas, researchers aim to gain a deeper understanding of However, purely artificial life approaches have inherent
the behavior and dynamics of biological systems in virtual time limitations. To address this, Liu proposed the integration
environments. of an evolutionary model into the artificial fish reproduction
process, enabling the gradual enrichment of the vital signs
A. INDIVIDUAL LIFE TRAIT SIMULATION of fish over time through an overall evolutionary perfor-
The simulation of individual organisms is crucial for mance [37], as shown in Figure 6(c). This approach has the
simulating living organisms and for enabling the exploration potential to generate more realistic simulations of individual
of ecological dynamics. Simulating the characteristics of organisms by modeling gradual changes in their life traits
different organisms requires the application of fundamental over time.
principles. For instance, the simulation of an artificial fish The limitations of these approaches are not explicitly
captures the essence of simulating individual organisms. mentioned, but the integration of an evolutionary model to
To achieve a more realistic simulation of individual fish lives, overcome time limitations suggests that the time-consuming
researchers have proposed various methods. nature of simulating individual organisms is a challenge.
Xian-Yu et al. developed a multi-sensory system model Additionally, although these approaches may improve the
that incorporates vision, touch, taste, and smell to mimic simulation of individual organisms, there may still be
the routines and behavior of fish in the wild, thereby limitations in accurately simulating complex ecological
enhancing the simulation of the physiological characteristics dynamics. Further research should focus on addressing these

76226 VOLUME 11, 2023

J. Wang et al.: Intelligent Metaverse Scene Content Construction

FIGURE 4. Content construction and improvement. (a) AI-based behavior has been included in the game engine’s event system, as seen in the top side
figure. View of the system architecture in the bottom figure. The system is built on a gaming engine that has been converted to an immersive display
similar to CAV [25]. (b) Screenshot of the in-depth exploring Interface for DeepDive. An A* ambush herd’s course for a specific herd size is shown on a
map in the upper left corner. Based on the existing rule set, the red circles on the map represent potential site locations. The wheels below show the
algorithm’s herd priority as determined by the user (N = nutrition; R = risk; E = movement force; and T = time to target) [26]. (c) The dashboard (a-e) of
the EPI system is immersive, while the sceneries (f-m) depict real-world surroundings [27]. (d) The fundamental design of QM4VR [28].

limitations to enhance the accuracy of ecological simulations For instance, Mozaffari et al. introduced a machine learning
and promote a better understanding of ecological systems. tracking technique that employs a Bèzier curve method
to smooth the movement of fruit flies, resulting in more
B. GROUP BEHAVIOR SIMULATION natural and lifelike behavior. Additionally, logistic regression
Simulating group behavior is a critical aspect in creating classification was utilized to classify the mating state of the
a comprehensive representation of ecological dynamics, fish based on video frames captured before training [39],
and the integration of AI technologies has revolutionized as shown in Figure 7(a). This innovative approach enhances
the vividness of these simulations in VR environments. the realism of group behavior and enables a more accurate
Researchers have proposed various innovative approaches representation of ecological systems.
to tackle the challenges associated with simulating group Although holistic simulations are valuable, they often lack
behavior. ecological fluidity. To address this limitation, researchers
One notable approach is the use of individual-based cel- have proposed integrating external stimulus demands and
lular automata (CA) models [38] to refine the understanding adaptive mechanisms into simulations of artificial life. Lints
of schooling behavior and guide the construction of natural developed a cluster scheduling algorithm that considers the
collective behavior by specifying individual-level rules. This characteristics of both individuals and groups, allowing for
method not only provides a realistic representation of group integrated judgment of artificial life and better adaptation
dynamics but also allows for the exploration of emergent to environmental changes [40], as shown in Figure 7(b).
properties that arise from interactions between individuals. Moreover, the incorporation of artificial evolution in group

VOLUME 11, 2023 76227

 J. Wang et al.: Intelligent Metaverse Scene Content Construction

simulating crowd behavior while considering obstacle avoid-

ance. The walking areas of crowds can be determined using
an A* search algorithm, resulting in more natural movement
patterns [42], as shown in Figure 7(c). Moreover, the
coordination of group movements was achieved through the
application of a multi-intelligent distributed system model,
as proposed by Reynolds et al. [41]. This innovative approach
ensures that group behavior remains cohesive and avoids
chaotic movements, thereby enhancing the overall realism of
the simulation.
Furthermore, the influence of emotions on group behavior
was explored to add a layer of realism. Noronha et al.
introduced the impact of panic emotions, which modifies
the behavioral parameters of agents and influences their
decision-making processes, particularly regarding the search
for an ideal path within a crowd [43], as shown in Figure 7(d).
This innovative integration of emotional dynamics enhances
the believability of group behavior and contributes to a more
immersive simulation experience.
FIGURE 5. Effect improvement. (a) The picture produced by a GAN and a
VR image [29]. (b) Diagram of the Gulo coloring algorithm’s processing of
The application of AI technologies in group behavior
light intensity [31]. (c) Overview of the algorithm: pixels It +α [pt +α ] in simulation within ecological contexts has resulted in notable
frame B are recreated from consecutive frames I using scene-aided and advancements, including enhanced realism, improved adapt-
image-based reprojection. Whereas image-based reprojection estimates
streams by repeatedly exploring nearby frames for depth and 3D flow ability, and incorporation of complex dynamics. These
fields, scene-aided projection generates scene streams during innovations have expanded our understanding of collective
conventional triangle rasterization [32].
behavior in ecological systems and contributed to the
emergence of new insights and the development of intelligent
simulations. However, it is important to acknowledge that
there may still be limitations and challenges in accurately
representing the intricacies of real-world ecosystems. Scal-
ability, computational demands, and the ability to capture
the full complexity of ecological dynamics are potential
areas for further research and improvement. Addressing
these limitations can help refine the application of AI
in group behavior simulations, ultimately advancing our
understanding of ecological phenomena and enhancing the
effectiveness of intelligent simulations.

C. STRESS FEEDBACK LEARNING

Behavior in natural ecosystems is not static or confined
but responds dynamically to various stressors. To accurately
simulate ecological systems, it is crucial to incorporate stress
feedback learning, which enables organisms to adapt and
evolve in response to unknown situations, cooperative inter-
actions, competition, and other biological relationships. This
FIGURE 6. Individual life trait simulation. (a) The process by which fish section explores innovative approaches to stress feedback
eggs divide their cells [36]. (b) The connection between perception, learning in the context of simulated biology.
choice, and behavior in synthetic fish [35]. (c) An artificial fish is depicted
in a schematic [37].
In the realm of fish populations, the incorporation of a
simplified and evolving model of fishing gear can stim-
ulate sportive evolution through predation dynamics [44],
simulation, as exemplified by the adjustment of boid as depicted in Figure 8(a). This approach introduces selective
parameters during evolution [41], facilitates innovation in the pressures that drive the adaptation and survival of fish popu-
field of group intelligence. This innovative approach enables lations, mimicking the real-world dynamics of predator-prey
the exploration of novel group behaviors and the emergence relationships and environmental changes. By incorporating
of collective intelligence in simulated ecological systems. such stress factors, the simulation becomes more realistic
To address the movement challenges and enhance the and allows the observation of evolutionary responses within
realism of VR environments, researchers have proposed a virtual ecosystem.

76228 VOLUME 11, 2023

J. Wang et al.: Intelligent Metaverse Scene Content Construction

FIGURE 7. Group behavior simulation. (a) An image processing-based one-frame system. Fruit fly identification is shown by the green frame, and mating
is indicated by the red frame [39]. (b) A straightforward flock-based defense mechanism to keep ‘enemy’ at bay. Our controlled creature (facing up)
conflicts with two evil entities in the image’s upper right corner (facing downwards). The enormous circle on the left is a virtual space that serves as our
creature’s ‘‘brain’’; in other words, it may be seen as being housed inside the creature’s head rather than next to it as is shown for visualization. The room
is filled with a collection of little triangles. There are two ‘lights’ (circles) that stand in for the adversary on the room’s walls. There are no little bodies
around the lights (large sketch arrows pointing down). The controlled creature will begin to travel backward when the flock reaches the base of the circle
(the four smaller grey arrows indicate the mapping between the position of the flock and the control signal sent to the creature) [40]. (c) Activity theory is
being used to direct the VR/AI game ’For Ewe’ [42]. (d) An illustration of a mock bird [41]. (e) Zuangentes test screenshot [43].

Another approach to stress feedback learning involves Incorporating additional factors such as predators and
the incorporation of predator-prey relationships into self- food sources further enhances the realism of the sim-
organizing adaptive schemes. Using reinforcement learning ulation. By applying reinforcement learning techniques
techniques [45], a population of agents can learn and adapt to a school of fish, a simulation model can train ML
through trial-and-error exploration by adopting different agents using deep reinforcement learning algorithms [47],
behavioral traits when encountering similar agents and as depicted in Figure 8(c). This approach enables fish to
predators to maximize rewards from the environment [46], learn and adapt their behaviors in response to changing
as shown in Figure 8(b). This innovative approach allows for environmental conditions, predators, and food availability.
the emergence of complex behaviors and strategies within the Through stress feedback learning, the simulated fish
simulated population, providing insights into the dynamics population exhibited sophisticated and realistic responses,
of predator-prey interactions and the evolution of adaptive closely mirroring the dynamics observed in natural
responses. ecosystems.

VOLUME 11, 2023 76229

 J. Wang et al.: Intelligent Metaverse Scene Content Construction

These innovative applications of stress feedback learning A. EMOTIONAL CONTENT PERSONALIZATION

in simulated biology offer valuable insights into the adaptive Emotions play a crucial role in subjectively evaluating the
capabilities of organisms and their response to changing quality of immersive content. Incorporating personalized
environments. These simulations capture the dynamics of sentiment as a target of intelligent learning can effectively
biological systems by simulating stressors and employing fine-tune the results of online content generation, ensuring
learning algorithms, thereby enabling the study of emer- that the generated content is highly relevant to the user’s
gent properties and evolutionary processes. This approach emotional state. This approach enhances the overall user
provides researchers with a deeper understanding of the experience by delivering emotionally engaging and tailored
complex interactions and behaviors exhibited by organisms content.
when exposed to different stressors. Nevertheless, it is crucial An example of personalized VR content is VRelax,
to acknowledge the limitations of these methods. Challenges a personalized VR relaxation therapy approach proposed
may include accurately representing the full complexity by Heyse et al. [48], as shown in Figure 9(a). VRelax
of real biological systems, scaling simulations to larger utilizes semantic reasoning and a decision-making approach
and more intricate scenarios, and ensuring the realism and based on multi-armed bandit (MAB) reinforcement learning.
accuracy of stress feedback mechanisms. Overcoming these It leverages static and dynamic personality data as inputs to
limitations will pave the way for further advancements in sim- select environmental decisions adaptively. By personalizing
ulated biology and will contribute to a more comprehensive the virtual environment and therapy experience, VRelax
understanding of the natural world. improves relaxation therapy outcomes in individuals with
mental health issues.
IV. PERSONALIZED CONTENTS The ISAM model [49] is another approach that enhances
The quality of artificially intelligent content generation in emotional content personalization. This model utilizes ten
a VR environment is determined by both the generation practice images from the International Affective Picture Sys-
capabilities of the system and the input it receives. When tem (IAPs) and a collection of images from Google Images
generating intelligent content, it is crucial to ensure that to improve mood prediction based on the pleasure-arousal-
the generated content meets the needs of real users, as this dominance (PAD) model [50], as shown in Figure 9(b).
directly affects the VR experience and user satisfaction. By intelligently selecting pictures based on the PAD reaction
Therefore, it is essential to improve the outcome of generated of the user, the ISAM model tailors the content to the
content based on the specific needs of individual users. emotional state of the user, enhancing the emotional impact
To address this challenge, several solutions for personal- of the VR experience.
ized content generation have been developed. These solutions Dingli and Bondin adopted a different approach by col-
can be broadly categorized into two groups: those that capture lecting data through wearable devices and using adversarial
sentiment data for analytical feedback to make the content generative networks (GAN) to predict a user’s emotional
more personalized, and those that generate content adjusted journey [51], as shown in Figure 9(c). This allows them to
by non-emotional data feedback from within VR to cater to link emotional states to specific events in the VR experience,
the user’s preferences. enabling the modeling of changes and game adaptations
The first category focuses on capturing sentiment data, based on the user’s emotional responses to factors such
such as user emotions and preferences, to provide analytical as color and sound. This personalized approach enhances
feedback. By analyzing this feedback, the system can gain emotional resonance and immersion in a VR environment.
insight into the user’s emotional state and tailor the generated By contrast, the PerAffect-lyVR system proposed by
content accordingly. This approach ensures that the content Gupta focuses on training with response information and
resonates with the user’s emotions, thereby creating a more leveraging situational and physiological cues from the user’s
personalized and engaging VR experience. interaction to detect emotional states [52], as shown in
The second category involves generating content that is Figure 9(d). By adapting to VR environments and difficulty
adjusted based on non-emotional data feedback obtained levels based on the user’s emotional state, PerAffect-lyVR
from within the VR environment. This feedback summarizes provides a highly personalized and emotionally engaging
the user’s tendencies and preferences, allowing the system experience.
to generate content that aligns with the user’s specific By incorporating emotional content personalization tech-
needs. By adapting the content to the user’s preferences, niques, VR systems can offer tailored experiences that
this approach enhances user satisfaction and the overall align with user emotions and preferences. However, it is
enjoyment of the VR experience. important to address the ethical concerns associated with
By incorporating personalized content-generation solu- the use of personal emotional data in virtual environments.
tions, VR environments can provide users with tailored expe- Despite these considerations, these approaches contribute
riences that cater to their individual needs and preferences. to enhanced emotional engagement, immersion, and overall
This not only enhances user satisfaction but also maximizes satisfaction, resulting in more impactful and meaningful
the potential of artificially intelligent content generation in metaverse experiences for users. Advancements in intelligent
delivering impactful and engaging VR experiences. methods for recognizing human emotions [53] further fuel the
76230 VOLUME 11, 2023
J. Wang et al.: Intelligent Metaverse Scene Content Construction

FIGURE 8. Stress feedback learning. (a) The simulation model’s flowchart. DSr represents the separation between the reference and the specified
individuals, and DRr represents the direction in which they are separated. DHr , name of the reference; AR stands for an angular range of interactions.
D1, the avoidance range; D2, the closeness; SD1, the standard deviation of movement in the avoidance and approach directions; SD2, motion in the
parallel direction [44]. (b) Agent trajectory learning in the 0-500 step range without predators N=10, (R1, R2, R3) = (4, 20, 50) [46]. (c) The perceived radius
of the predator (dark blue circle), the line linking the fish agents to one another (light blue line), and the distance to the meal are all displayed by a dense
school of trained models on all behaviors (solid green circle) [47].

potential of combining personalized content technology and audio, video, and image-based media. By leveraging struc-
emotions, unlocking new possibilities in the field. tured external connections, the PVM enhances the con-
textual relevance of the virtual world, delivering dynamic,
B. NON-EMOTIONAL CONTENT PERSONALIZATION enjoyable, and engaging experiences tailored to the user’s
In addition to emotions, objective data generated from needs.
VR content interactions can provide valuable insights into Personalized game experiences can be achieved by adjust-
personalized content generation. These more deterministic ing the difficulty level and utilizing personalized profiles
data, allow for effective profiling of the user experience, generated from each player’s Hyperseed [59]. Additionally,
enabling the generation of personalized content based on customized categorization of feedback and real-time analysis
specific preferences and needs. of game-level performance dynamics contribute to personal-
Kiourt et al. designed a personal virtual museum (PVM) ized gameplay experiences [55], as shown in Figure 10(b).
that incorporates dynamic systems and pan-institutional mod- VCoach [56] takes a similar approach by analyzing
ular learning objects to facilitate collaboration, knowledge objective user response states such as punching speed,
modeling, and management [54], as shown in Figure 10(a). reaction time, and punching posture during exercise. These
Their approach utilizes tree search, reinforcement learning, data were compared to pre-collected data from professional
supervised learning, and a wide range of computational, boxing coaches, enabling the system to generate adaptive and

VOLUME 11, 2023 76231

 J. Wang et al.: Intelligent Metaverse Scene Content Construction

FIGURE 9. Emotional content personalization. (a) Block representation of a custom VR system for relaxing [48]. (b) Logic flow in iSAM [49]. (c) The
elements that make up the game module are described [51]. (d) Design, setup, and tasks for experimental research focusing on the psychophysical
link between cognitive load and VR’s virtual assistants [52].

personalized boxing training routines in real-time, as shown By harnessing objective data from user interactions,
in Figure 10(c). VR systems can deliver highly personalized content that
When using DTBVis [60] technology, experts can better caters to individual preferences and needs. However, factors
understand the similarities and differences between DTB and such as the availability and quality of data, privacy concerns,
the human brain by supporting iterative exploration at various and the challenge of accurately capturing and interpreting
levels and granularities, along with automatic similarity user interactions can affect the effectiveness of personal-
recommendation and high-dimensional exploration. This ized content delivery. Despite these limitations, leveraging
allows them to customize the model and improve its objective data in VR systems can significantly enhance
functionality. user engagement, immersion, and satisfaction by providing
User interaction data in VR supermarkets can be analyzed tailored experiences that align with specific requirements and
using techniques such as Neural Collaborative Filtering objectives.
(NCF) to provide personalized recommendations [57],
as shown in Figure 10(d). By leveraging collaborative filter- V. INTELLIGENT AGENTS
ing neural networks, the system can generate personalized Scientists have developed intelligent agents that focus on
product recommendations based on invisible ratings derived content generation to enhance the central and immersive
from user data. nature of content generation. Intelligent agents enable virtual
Privacy considerations are essential in personalized con- animation to take on the character of a main storyline,
tent generation, and techniques like Convolutional Neural whereas AI integration enhances the simulation, and the
Networks (CNNs) can be employed to ensure long-term adaptive nature of the intelligent agent avoids the frustration
effectiveness while minimizing the amount of user data of mechanical animation. Intelligent agents can be applied
required for training [58], as shown in Figure 10(e). in three main areas: those based on data training, those

76232 VOLUME 11, 2023

J. Wang et al.: Intelligent Metaverse Scene Content Construction

FIGURE 10. Non-emotional content personalization. (a) Ideally suited for dynamic personal virtual worlds based on several intelligences [54]. (b) Use of
AI and VR to structurally visualize a suggested framework for individualized physical therapy rehabilitation [55]. (c) Overview of the virtual customized
boxing training system VCoach, comprising student performance evaluation, boxing exercise production, and interactive boxing exercise. For iterative skill
improvement, trainees can engage in virtual boxing instruction using a consumer-grade wearable VR device, such as the HTC Vive [56]. (d) The VR
supermarket’s suggested layout [57]. (e) The visualization shows the position and rotation of the genuine hand and the computer-generated hand with
vivid skin tones (solid black). The motion being practiced is elbow bending, and the separation between the two hands is determined by varying the
rotational angle of the elbow itself [58].

constructed from external knowledge systems, and those using MLP-based neural networks, enabling them to acquire
trained to adapt to interactive data. the necessary skills to play tennis proficiently, as shown in
Figure 11(b).
A. INTELLIGENT AGENTS TRAINED BASED ON DATA Furthermore, an RBF_LA learning algorithm was pro-
In the realm of content generation, the data generated during posed for tennis matches [63]. This algorithm incorpo-
the development and evolution of VR content serve as a rates different learning strategies under various conditions,
valuable resource for training intelligent agents. Researchers including no learning process, learning after a match, and
have leveraged the empirical data obtained from VR content learning during a match. By adapting their strategies based
to enhance the intelligence of these agents. By allowing the on real-time feedback, intelligent agents can improve their
data to evolve in real time, the performance of intelligent performance in tennis matches, as depicted in Figure 11(c).
agents can progressively improve, resulting in more realistic In the context of virtual-scene navigation, agents are
and sophisticated content generation. trained to learn and navigate diverse paths [64], as shown in
One notable approach is the application of artificial neural Figure 11(d). Through a reward-based training method, real-
networks (ANN) for training intelligent agents. Paladin [61], time navigation systems enable agents to comprehend and
employed an ANN to control the agents’ behavior. The adapt to obstacles, path safety, traffic conditions, and other
predicted positions obtained from the neural networks are relevant data, resulting in efficient and effective pathfinding.
evaluated for their reasonableness, allowing intelligent agents Augmented learning techniques have also been employed
to adjust their actions accordingly. This adaptive behav- to train intelligent agents. For instance, Linqin Cai et al.
ior, combined with pre-programmed behaviors, enhances successfully trained agents to navigate through a maze
the overall intelligence of the agents, as illustrated in using reinforcement learning (RL) and a Pixy camera
Figure 11(a). sensor to detect and interact with different objects
Another instance involves the use of multilayer percep- in the environment [65]. This approach grants the
tron (MLP) neural networks in mastering fundamental tennis- agents the freedom to choose optimal routes, fostering
playing abilities [62]. By collecting essential information adaptability and intelligence in their decision-making
from a simulated environment, intelligent agents are trained processes.

VOLUME 11, 2023 76233

 J. Wang et al.: Intelligent Metaverse Scene Content Construction

FIGURE 11. Intelligent agents trained based on data. (a) Program for the agency [61]. (b) Building data sets on a virtual tennis court [62]. (c) Structure of
the RBF_LA suggested model [63]. (d) Incorporating agents into virtual worlds [64].

Content generation in VR environments can be signifi- (Deterministic Finite State Machine) states while offering
cantly enhanced by utilizing data-driven training method- significant expressiveness.
ologies and intelligent agents. However, the integration In the case of the Intelligent Virtual Human Animation
of real-time data and adaptive mechanisms may introduce System (IVHAS) [67], which is based on the Semantic Web,
complexities in terms of computational resources and system virtual scenes are defined with rich semantic information.
responsiveness. Despite these limitations, the utilization of This enables computers to comprehend the significance of
data-driven training methodologies and intelligent agents virtual scenes and perform automated analytical processing,
holds great potential for improving content generation as depicted in Figure 12(b). Semantic virtual environments
and creating more immersive experiences in metaverse incorporate various aspects of human behavior, perception,
environments. and behavioral planning in the real world, providing rich
semantic information to objects within virtual scenarios.
B. INTELLIGENT AGENTS CONSTRUCTED FROM Similarly, the intelligent teaching module of the Smart
EXTERNAL KNOWLEDGE SYSTEMS Physics Lab [12] presents its domain knowledge externally to
To ensure compatibility between intelligent agents and the the Intelligent VR Teaching System (IVRTS) [12], as shown
logic of reality, external knowledge systems can be leveraged in Figure 12(c). Utilizing external domain knowledge, the
to enhance an agent’s understanding of logic. By integrating a teaching module enhances its capabilities and provides
real-world logic system, VR simulations can be more closely a more comprehensive and intelligent virtual teaching
aligned with real-world principles. This approach can be experience.
realized through the construction of intelligent agents using By integrating external knowledge systems, intelligent
external knowledge systems. agents in VR environments can align their logic and behavior
One such approach is the FuSM (Fuzzy State Machine) with real-world principles, resulting in more realistic and
[66]. By simulating emotions and controlling the behav- contextually relevant interactions. Nevertheless, challenges
ior of characters affected by these emotions, the FuSM may arise in effectively integrating and updating external
approach creates high-performance AI emotions and AI knowledge sources, ensuring the accuracy and relevance of
systems, as illustrated in Figure 12(a). This technol- the information utilized by agents, and addressing potential
ogy allows for the simultaneous of multiple emotions, biases or limitations in the knowledge systems themselves.
adding an element of unpredictability that enhances the Nonetheless, the integration of external knowledge systems
credibility of virtual characters and robots. FuSM effec- is expected to improve the intelligence and authenticity of
tively addresses the combinatorial explosion of DFSM intelligent agents in a metaverse environment.

76234 VOLUME 11, 2023

J. Wang et al.: Intelligent Metaverse Scene Content Construction

FIGURE 12. Intelligent agents constructed from external knowledge systems. (a) The design of artificial emotion and AI systems [66]. (b) An illustration
of the Semantic Virtual Scene Ontology (SVSO) in schematic form [67]. (c) IVRTS interface and a ballistics lab [12].

C. INTELLIGENT AGENTS TRAINED FOR INTERACTION to obtaining pose data without relying on skeletal movement
DATA ADAPTATION tracking. Instead, they trained a posture classifier using
In addition to the data generated during the development Google’s experimental AI API, ’Teachable Machine’ [69].
of VR scene content, real-world data generated by users By leveraging user-recognized poses, they enable the inter-
through their interactions also play a crucial role. Researchers active creation of avatars in a virtual space, as illustrated in
can collect and leverage relevant data to make intelligent Figure 13(b). This methodology provides improved outcomes
agents more contextually relevant to users’ situations, thereby and a more efficient user experience.
enhancing the overall immersive experience. There are Simulating real images using real-world data is an effective
three primary situations in which interaction data can be technique that can enhance immersion in a metaverse
collected to train intelligent agents: simulating realistic environment and meet users’ emotional needs to enhance
images, adapting agent behavior based on interaction actions, realism. However, there are challenges in obtaining and
and adapting agent collaboration in task processes. processing large-scale and diverse real-world data to ensure
its accuracy and representativeness. The fidelity of an analog
image may be constrained by the quality and resolution of
1) SIMULATING REALISTIC IMAGES the input data. Nevertheless, incorporating data into image
To facilitate a more immersive experience, operators can simulation methods has a positive effect.
simulate an image of a real person by leveraging the image
features of an actual individual. In the case of a virtual 2) INTERACTION ACTIONS TO ADAPT AGENT CHANGES
companion dining system designed for the elderly [68], To cater to different operator categories, it is important to
researchers have implemented a technique that involves adapt the agent’s approach based on the user’s interaction
segmenting the RGB images of a real person’s area using actions. Recognizing this, researchers have devised methods
depth information. By utilizing this depth information, they to evaluate users during their interactions and accordingly
can restore the point cloud of a person’s area and rebuild select the most appropriate agent approach.
the grid of virtual images. Furthermore, they mapped the In the context of home rehabilitation games, Elor and
segmented color information onto the virtual body based Kurniawan developed an upper limb exercise component that
on texture-mapping rules, as depicted in Figure 13(a). This assists users in learning and guiding their exercise move-
approach satisfies the emotional need for enhanced realism ments [70]. To personalize the rehabilitation strategy and
in the virtual companion, resulting in a more authentic and adapt to the exercise difficulty and assistance, they employed
engaging user experience. a technique called generative imitation learning (GAIL)
In the context of avatar creation, researchers such as and proximal strategy optimization (PPO) with the use of
Miao-Chi Liu Chang et al. explored an alternative approach virtual butterflies, as depicted in Figure 13(c). By leveraging

VOLUME 11, 2023 76235

 J. Wang et al.: Intelligent Metaverse Scene Content Construction

FIGURE 13. Simulating realistic images. (a) The figure of the system architecture for a senior dining system using virtual avatars [68]. (b) Body skeleton
joints were found in the recognized produced avatar (right side) as well as in frames taken from the built-in camera (left and center portions) [69]. (c) IB
game competition with skilled players. While defending the butterfly, the user competes with the project IB agent to acquire the most crystals. The
user’s job is to safeguard the butterfly, and the agent is positioned to the user’s right. The scene and game views display crystal routes as well as avatars
of people and agents [70]. (d) The ‘Ant-Man Vision’ experience’s method [71].

these approaches, exercise difficulty and assistance can be collaboration, adding unpredictability and interest to the
dynamically adjusted based on user performance, ensuring a VR experience. Various approaches have been employed to
personalized rehabilitation experience. address this challenge and to enhance task collaboration in
In the case of museum anthropomorphic vision [71], virtual environments.
researchers took a different approach to process user inter- For instance, in the case of Paladin, the challenge of work
actions, as shown in Figure 13(d). They trained a behavioral distribution for cooperation was addressed by utilizing ESP
model using Long Short-Term Memory (LSTM) and utilized neural networks [72], [76], as depicted in Figure 14(a). This
cameras and 3D skeleton position acquisition to detect user approach allows efficient and balanced task allocation among
behavior in real space. Based on this information, they cooperative agents, ensuring smooth collaboration.
modified the state changes of the intelligent agent, Ant-Man, In the context of the IVTS [73], a declarative form based
in the museum. This adaptive approach enables the agent to on Petri Nets (PNs) [77] was utilized to describe training
respond to user actions and to create a more immersive and task planning. Researchers have developed an algorithm to
interactive museum experience. create Task Planning Petri Nets (TP-PNets) and established a
By incorporating interaction actions to adapt to agent Hierarchical Coloured Petri Net (HCPN) model to evaluate
changes, researchers can tailor the agent’s behavior and agent task planning behavior, as shown in Figure 14(b).
responses to individual users, creating a more personalized These measures provided dependability and flexibility in
and engaging user-agent interaction. task planning, allowing agents to adapt their collabora-
tion strategies based on the specific requirements of the
3) INTERACTION PROCESSES TO ADAPT TO TASK tasks.
COLLABORATION With the increase in the complexity of VR content,
The interaction processes not only fine-tune the real-time researchers have focused on creating sophisticated virtual
behavior of the agent but also address the issue of task internal agent models to facilitate intelligent and engaging

76236 VOLUME 11, 2023

J. Wang et al.: Intelligent Metaverse Scene Content Construction

FIGURE 14. Interaction processes to adapt task collaboration. (a) Network collaborations. The artificial neural network uses the values that the agents
have received to conduct tasks on the agents by the network’s outputs. The weights of the linkages from input neuron I to hidden neuron j are shown by
Wij1 , while the links from hidden neuron j to output neuron k are indicated by Wij2 [72]. (b) The IVTS suggested architectural [73]. (c) An intelligent virtual
training system’s framework [74]. (d) Using Q-learning, the target location generator architecture [75].

VR systems. A highly competent avatar model was developed This approach has proven to be an effective choice for
in the field of mine safety instruction [74], as illustrated in improving task collaboration and achieving better rehabili-
Figure 14(c). This model incorporates perceptual, behavioral, tation outcomes.
mental state, and cognitive modules, enabling the avatar to While accurately modeling and predicting user intentions
execute training orders, exhibit the desired physical abilities, and preferences, as well as ensuring seamless coordination
and mediate interactions between real-world and virtual and synchronization among multiple agents, can be complex,
contexts. especially in dynamic and unpredictable environments,
In the Flying with Friends motor disorder rehabilitation researchers aim to enhance the collaborative behavior of
system [75], the Deep Q-Networks (DQN) approach was intelligent agents in virtual environments by incorporating
employed to tackle complex tasks, as shown in Figure 14(d). interaction processes. This approach aims to create a

VOLUME 11, 2023 76237

 J. Wang et al.: Intelligent Metaverse Scene Content Construction

dynamic, engaging, and realistic user experience. Despite B. POTENTIAL TREND DISCUSSION: INTEGRATION OF AI
these challenges, there is optimism that the combination WITH REAL-WORLD DATA AND EXTERNAL KNOWLEDGE
of interaction processes and task collaboration can lead SYSTEMS
to significant improvements in the collaborative behavior In the future, a deeper analysis of data relationships [87],
of intelligent agents, offering users a more immersive and more refined data collection and simulation [88], extraction
satisfying metaverse environment. and growth of anthropomorphic knowledge [89], and the
cross-fusion analysis of knowledge data obtained from
VI. DISCUSSION AND CONCLUSION diverse real-world sources [90] will provide new possibilities
This paper introduced innovative aspects in the field of for the intelligent generation of three-dimensional simulation
intelligent visual content generation, providing a com- metaverse scenes. The integration of AI algorithms with
prehensive overview of AI methods in scene content real-world data and external information systems will con-
generation, simulated biology, personalized content, and tinue to improve, enabling the creation of more realistic and
intelligent agents. The integration of AI algorithms with contextually rich virtual scenes. This integration will lead to
real-world data and external knowledge systems is high- the development of smarter and more creative agents, thereby
lighted, enabling the creation of authentic and contex- enhancing the overall immersive experience for users.
tually rich virtual scenarios. This study emphasizes the
C. POTENTIAL TREND DISCUSSION: INTERDISCIPLINARY
importance of interdisciplinary collaboration and considers
COLLABORATION
human cognitive and emotional factors. Key limitations
and challenges were identified, inspiring future research Collaboration between different disciplines, such as psy-
on creative content generation, ethics, and computational chology, neuroscience, and computer science, will play a
efficiency. Overall, this study enriches the knowledge and crucial role in enhancing our understanding of the cognitive
drives the development of immersive visual content in virtual and emotional impact of intelligent metaverse scene content.
environments. The demand for intelligent visual content generation in
Intelligent visual-content generation has significantly scientific fields such as education [91], medicine [92],
impacted content creation and user experience by enabling chemistry [93], physics [94], and geology [95] is expected
the creation of visually stunning scenes, personalized content, to increase. Intelligent content generation can help beginners
and adaptive intelligent agents in virtual environments. In the in various disciplines gain a fast, effective, and interesting
following sections, we summarize and discuss three potential understanding, while providing experts with a more intuitive,
research trends in the construction of intelligent metaverse rich, and immersive experience. In humanities fields, such as
scene content, namely the improvement of personalized education and art, content sensitivity outweighs accuracy. For
methods and the integration of brain-computer interfaces, example, cross-modal metaverse scene generation, including
the integration of AI with real-world data and external direct text generation [96], will find popularity in artistic
knowledge systems, and the importance of interdisciplinary fields like ancient poetry and artistic styles. In the scientific
collaboration. and engineering fields, accuracy takes precedence over recep-
tivity. For instance, intelligent construction of biologically
A. POTENTIAL TREND DISCUSSION: IMPROVEMENT OF variable metaverse models based on medical knowledge
PERSONALIZED METHODS AND INTEGRATION OF services [97] may become a direction of interest.
BRAIN-COMPUTER INTERFACES In conclusion, the generation of intelligent metaverse
The demand for personalized content tailored to individual scene content has transformed content creation and the
user preferences and emotions is increasing. With the con- user experience. Continued advancements in AI algorithms
tinuous improvement AI algorithms for recognizing human and interdisciplinary collaborations hold promise for further
emotions, methods such as speech emotion recognition [78], improvement. This survey aims to stimulate more ideas about
physiological signal emotion recognition [79], facial image scene content construction through the use of intelligent
emotion recognition [80], text mixing analysis [81], cultural metaverse immersive technology.
subdivision and overall analysis [82] have significantly REFERENCES
enhanced the accuracy of emotion recognition. The advance- [1] X. Li, H. Duan, B. Liu, X. Wang, and F.-Y. Wang, ‘‘A novel framework to
ment of formulaic feedback theory [83], [84] has also generate synthetic video for foreground detection in highway surveillance
scenarios,’’ IEEE Trans. Intell. Transp. Syst., vol. 24, no. 6, pp. 5958–5970,
contributed to more perfect adaptive emotion generation in Jun. 2023.
metaverse scenes. Additionally, the rapid development of [2] Y. Zhao, J. Jiang, Y. Chen, R. Liu, Y. Yang, X. Xue, and S. Chen,
brain-computer interface technology [85] provides the possi- ‘‘Metaverse: Perspectives from graphics, interactions and visualization,’’
Vis. Informat., vol. 6, no. 1, pp. 56–67, Mar. 2022.
bility for real-time emotional feedback, ensuring the real-time
[3] X. Li and Y. Shi, ‘‘Computer vision imaging based on artificial
and accuracy of adaptive changes in scenes. Integrating intelligence,’’ in Proc. Int. Conf. Virtual Reality Intell. Syst. (ICVRIS),
intelligent analysis of electroencephalogram (EEG) data [86] Aug. 2018, pp. 22–25.
can further enhance the efficiency and diversity of content [4] R. Liu, M. Gao, L. Wang, X. Wang, Y. Xiang, A. Zhang, J. Xia,
Y. Chen, and S. Chen, ‘‘Interactive extended reality techniques in
generation, resulting in more realistic and personalized information visualization,’’ IEEE Trans. Hum.-Mach. Syst., vol. 52, no. 6,
metaverse content. pp. 1338–1351, Dec. 2022.

76238 VOLUME 11, 2023

J. Wang et al.: Intelligent Metaverse Scene Content Construction

[5] L. Li, ‘‘The impact of artificial intelligence painting on contemporary art [26] T. Palazzolo, A. Lemke, S. Saad, C. Zhang, S. P. Jayanti, J. O’Shea,
from disco diffusion’s painting creation experiment,’’ in Proc. Int. Conf. and R. G. Reynolds, ‘‘Deepdive: Using AI, machine learning, and virtual
Frontiers Artif. Intell. Mach. Learn. (FAIML), Jun. 2022, pp. 52–56. reality to explore ancient submerged civilizations,’’ in Proc. 3rd Int. Conf.
[6] X. Shi and W. Hu, ‘‘A survey of image style transfer research,’’ in Proc. 2nd Artif. Intell. Industries, Sep. 2020, pp. 79–82.
Int. Conf. Comput. Sci., Electron. Inf. Eng. Intell. Control Technol. (CEI), [27] M. I. Iglesias, M. Jenkins, and G. Morison, ‘‘An enhanced photorealistic
Sep. 2022, pp. 133–137. immersive system using augmented situated visualization within virtual
[7] J. P. A. Campos and R. Rieder, ‘‘Procedural content generation using reality,’’ in Proc. IEEE Conf. Virtual Reality 3D User Interfaces Abstr.
artificial intelligence for unique virtual reality game experiences,’’ in Proc. Workshops (VRW), Mar. 2021, pp. 514–515.
21st Symp. Virtual Augmented Reality (SVR), Oct. 2019, pp. 147–151. [28] F. Silva, M. A. Togou, and G.-M. Muntean, ‘‘An innovative algorithm
[8] C. Tang, Z. Wang, X. Sima, and L. Zhang, ‘‘Research on artificial for improved quality multipath delivery of virtual reality content,’’ in
intelligence algorithm and its application in games,’’ in Proc. 2nd Int. Conf. Proc. IEEE Int. Symp. Broadband Multimedia Syst. Broadcast. (BMSB),
Artif. Intell. Adv. Manuf. (AIAM), Oct. 2020, pp. 386–389. Oct. 2020, pp. 1–6.
[9] Y. Zhang, G. Zhang, and X. Huang, ‘‘A survey of procedural content [29] V. Franchi and E. Ntagiou, ‘‘Augmentation of a virtual reality environment
generation for games,’’ in Proc. Int. Conf. Culture-Oriented Sci. Technol. using generative adversarial networks,’’ in Proc. IEEE Int. Conf. Artif.
(CoST), Aug. 2022, pp. 186–190. Intell. Virtual Reality (AIVR), Nov. 2021, pp. 219–223.
[10] F. Lo, F. Su, S. Chen, J. Qiu, and J. Du, ‘‘Artificial intelligence aided [30] G. Koch, R. Zemel, and R. Salakhutdinov, ‘‘Siamese neural networks for
innovation education based on multiple intelligence,’’ in Proc. IEEE Int. one-shot image recognition,’’ in Proc. ICML, 2015, vol. 2, no. 1, pp. 1–30.
Conf. Artif. Intell., Robot., Commun. (ICAIRC), Jun. 2021, pp. 12–15. [31] F. Gao and G. Xu, ‘‘Research on multi-mode interactive virtual reality
[11] K. Kroes, I. Saccardi, and J. Masthoff, ‘‘Empathizing with virtual scene enhancement based on artificial intelligence,’’ in Proc. 12th Int.
agents: The effect of personification and general empathic tendencies,’’ Conf. Measuring Technol. Mechatronics Autom. (ICMTMA), Feb. 2020,
in Proc. IEEE Int. Conf. Artif. Intell. Virtual Reality (AIVR), Dec. 2022, pp. 391–396.
pp. 73–81. [32] L. Yang, Y.-C. Tse, P. V. Sander, J. Lawrence, D. Nehab, H. Hoppe,
[12] F. Dudyrev, O. Maksimenkova, and D. Mikhailenko, ‘‘Intelligent virtual and C. L. Wilkins, ‘‘Image-based bidirectional scene reprojection,’’
reality tutoring systems as a new generation of simulators: Requirements ACM Trans. Graph., vol. 30, no. 6, pp. 1–10, Dec. 2011, doi:
and opportunities,’’ in Proc. IEEE Global Eng. Educ. Conf. (EDUCON), 10.1145/2070781.2024184.
Apr. 2021, pp. 706–718. [33] A. Pilco, X. Luo, A. A. N. Newball, C. Zuniga, and C. Lozano-Garzón,
[13] R. Kaviyaraj and M. Uma, ‘‘A survey on future of augmented reality with ‘‘Procedural animation generation technology of virtual fish flock,’’
AI in education,’’ in Proc. Int. Conf. Artif. Intell. Smart Syst. (ICAIS), in Proc. Int. Conf. Virtual Reality Visualizat. (ICVRV), Nov. 2019,
Mar. 2021, pp. 47–52. pp. 233–237.
[14] R. Wang, L. Jiang, J. Yuan, X. Xu, and W. Wang, ‘‘Virtual reality scene [34] D. Cracknell, M. P. White, S. Pahl, W. J. Nichols, and M. H. Depledge,
construction based on multimodal video scene segmentation algorithm,’’ ‘‘Marine biota and psychological well-being: A preliminary examination
in Proc. IEEE 8th Joint Int. Inf. Technol. Artif. Intell. Conf. (ITAIC), of dose–response effects in an aquarium setting,’’ Environ. Behav., vol. 48,
May 2019, pp. 1817–1820. no. 10, pp. 1242–1269, Dec. 2016, doi: 10.1177/0013916515597512.
[15] A. Becher, C. Axenie, and T. Grauschopf, ‘‘VIRTOOAIR: Virtual reality [35] M. Xian-Yu, B. Xiao-Juan, Z. Shu-Jun, and T. Xu-Yan, ‘‘Design of multi-
TOOlbox for avatar intelligent reconstruction,’’ in Proc. IEEE Int. perception system model for artificial fish in virtual environment,’’ in Proc.
Symp. Mixed Augmented Reality Adjunct (ISMAR-Adjunct), Oct. 2018, IEEE Int. Conf. Ind. Informat., Aug. 2006, pp. 1108–1112.
pp. 275–279. [36] Q. Xu and M. Zhou, ‘‘The modeling of fish roe based on artificial life
[16] Y. Zhang, ‘‘Application of interactive artificial intelligence in data method,’’ in Proc. Int. Conf. Mech. Sci., Electric Eng. Comput. (MEC),
evaluation of ink animation production,’’ in Proc. Int. Conf. Appl. Artif. Aug. 2011, pp. 880–882.
Intell. Comput. (ICAAIC), May 2022, pp. 34–37. [37] Y. Liu, ‘‘Research on the cognitive model artificial fish based on virtual
[17] T. Fréville, C. Hamesse, B. Pairet, R. Lahouli, and R. Haelterman, marine environment,’’ in Proc. 8th Int. Conf. Inf. Technol. Med. Educ.
‘‘From floor plans to virtual reality,’’ in Proc. IEEE Int. Conf. Artif. Intell. (ITME), Dec. 2016, pp. 719–723.
Virtual Reality (AIVR), Nov. 2021, pp. 129–133. [38] J. K. Parrish, S. V. Viscido, and D. Grunbaum, ‘‘Self-organized fish
[18] N. Matsuo and M. Imura, ‘‘Turning a messy room into a fully immersive schools: An examination of emergent properties,’’ Biol. Bull., vol. 202,
VR playground,’’ in Proc. IEEE Conf. Virtual Reality 3D User Interfaces no. 3, pp. 296–305, Jun. 2002.
Abstr. Workshops (VRW), Mar. 2021, pp. 759–760. [39] M. H. Mozaffari, S. Wen, and W.-S. Lee, ‘‘Virtual reality and tracking the
[19] S. Naritomi, R. Tanno, T. Ege, and K. Yanai, ‘‘FoodChangeLens: CNN- mating behavior of fruit flies: A machine learning approach,’’ in Proc.
based food transformation on HoloLens,’’ in Proc. IEEE Int. Conf. Artif. IEEE Int. Conf. Comput. Intell. Virtual Environments Meas. Syst. Appl.
Intell. Virtual Reality (AIVR), Dec. 2018, pp. 197–199. (CIVEMSA), Jun. 2018, pp. 1–5.
[20] J.-Y. Zhu, T. Park, P. Isola, and A. A. Efros, ‘‘Unpaired image-to-image [40] T. Lints, ‘‘FlockHeadz: Virtual flock in a room used as a controller,’’ in
translation using cycle-consistent adversarial networks,’’ in Proc. IEEE Int. Proc. IEEE Swarm Intell. Symp., Sep. 2008, pp. 1–5.
Conf. Comput. Vis. (ICCV), Oct. 2017, pp. 2242–2251. [41] C. W. Reynolds, ‘‘Flocks, herds and schools: A distributed behavioral
[21] O. Ronneberger, P. Fischer, and T. Brox, ‘‘U-Net: Convolutional networks model,’’ ACM SIGGRAPH Comput. Graph., vol. 21, no. 4, pp. 25–34,
for biomedical image segmentation,’’ in Medical Image Computing Aug. 1987.
and Computer-Assisted Intervention—MICCAI, N. Navab, J. Hornegger, [42] H. X. Trejo, ‘‘Creating an immersed sheep and wool VR/AI experience,’’
W. M. Wells, and A. F. Frangi, Eds. Cham, Switzerland: Springer, 2015, in Proc. IEEE Int. Conf. Artif. Intell. Virtual Reality (AIVR), Dec. 2019,
pp. 234–241. pp. 237–2371.
[22] D. Holden, T. Komura, and J. Saito, ‘‘Phase-functioned neural networks [43] G. Noronha, G. Batista, and L. P. Soares, ‘‘Crowd simulation with
for character control,’’ ACM Trans. Graph., vol. 36, no. 4, pp. 1–13, augmented reality,’’ in Proc. XV Symp. Virtual Augmented Reality,
Aug. 2017. May 2013, pp. 228–231.
[23] W. Xu, I. Alarab, C. Lloyd-Buckingham, S. Bowden, B. Noer, F. Charles, [44] I. Aoki, ‘‘A simulation study on the schooling mechanism in fish,’’ Nippon
S. Prakoonwit, A. Callaway, S. Ellis, and C. Jones, ‘‘Re-enacting Suisan Gakkaishi, vol. 48, no. 8, pp. 1081–1088, 1982.
football matches in VR using virtual agents’ realistic behaviours,’’ in [45] L. P. Kaelbling, M. L. Littman, and A. W. Moore, ‘‘Reinforcement
Proc. IEEE Int. Conf. Artif. Intell. Virtual Reality (AIVR), Dec. 2022, learning: A survey,’’ J. Artif. Intell. Res., vol. 4, no. 1, pp. 237–285,
pp. 119–123. Jan. 1996.
[24] R. Guo, J. Cui, W. Zhao, S. Li, and A. Hao, ‘‘Hand-by-hand mentor: [46] K. Morihiro, T. Isokawa, H. Nishimura, and N. Matsui, ‘‘Characteristics
An AR based training system for piano performance,’’ in Proc. IEEE Conf. of flocking behavior model by reinforcement learning scheme,’’ in Proc.
Virtual Reality 3D User Interfaces Abstr. Workshops (VRW), Mar. 2021, SICE-ICASE Int. Joint Conf., 2006, pp. 4551–4556.
pp. 436–437. [47] R. Wangananont, N. Buppodom, S. Chanthanuraks, and V. Kotrajaras,
[25] J. Lugrin, M. Cavazza, M. Palmer, and S. Crooks, ‘‘Artificial intelligence- ‘‘Simulation of homogenous fish schools in the presence of food and
mediated interaction in virtual reality art,’’ IEEE Intell. Syst., vol. 21, no. 5, predators using reinforcement learning,’’ in Proc. 17th Int. Joint Symp.
pp. 54–62, Sep. 2006. Artif. Intell. Natural Lang. Process. (iSAI-NLP), Nov. 2022, pp. 1–6.

VOLUME 11, 2023 76239

 J. Wang et al.: Intelligent Metaverse Scene Content Construction

[48] J. Heyse, T. D. Jonge, M. T. Vega, F. D. Backere, and F. De Turck, [70] A. Elor and S. Kurniawan, ‘‘Deep reinforcement learning in immersive
‘‘A personalized virtual reality experience for relaxation therapy,’’ in Proc. virtual reality exergame for agent movement guidance,’’ in Proc. IEEE 8th
11th Int. Conf. Quality Multimedia Exper. (QoMEX), Jun. 2019, pp. 1–3. Int. Conf. Serious Games Appl. Health (SeGAH), Aug. 2020, pp. 1–7.
[49] A. Elor and A. Song, ‘‘ISAM: Personalizing an artificial intelligence [71] Y.-L. Chen, Y.-X. Cheng, I.-Y. Hung, and S.-W. Sun, ‘‘Ant-man vision in
model for emotion with pleasure-arousal-dominance in immersive virtual the museum with interactive and immersive surreal experience based on
reality,’’ in Proc. 15th IEEE Int. Conf. Autom. Face Gesture Recognit. (FG machine learning,’’ in Proc. IEEE Int. Conf. Artif. Intell. Virtual Reality
), Nov. 2020, pp. 572–576. (AIVR), Dec. 2022, pp. 242–244.
[50] A. Mehrabian, ‘‘Pleasure-arousal-dominance: A general framework for [72] L. Jia and M. Zhenjiang, ‘‘Entertainment oriented intelligent virtual
describing and measuring individual differences in temperament,’’ Current environment with agent and neural networks,’’ in Proc. IEEE Int. Workshop
Psychol., vol. 14, no. 4, pp. 261–292, Dec. 1996. Haptic, Audio Vis. Environments Games, Oct. 2007, pp. 90–95.
[73] L. Cai, T. Mei, Y. Sun, L. Sun, and Z. Ma, ‘‘Modeling and analyzing multi-
[51] A. Dingli and L. Bondin, ‘‘Realtime adaptive virtual reality for pain
agent task plans for intelligent virtual training system using Petri Nets,’’ in
reduction,’’ in Proc. IEEE Conf. Games (CoG), Aug. 2019, pp. 1–4.
Proc. 6th World Congr. Intell. Control Autom., 2006, pp. 4766–4770.
[52] K. Gupta, ‘‘[DC] exploration of context and physiological cues for [74] L. Cai, T. Mei, and Y. Sun, ‘‘Multi-agent based intelligent virtual
personalized emotion-adaptive virtual reality,’’ in Proc. IEEE Conf. training system,’’ in Proc. 7th World Congr. Intell. Control Autom., 2008,
Virtual Reality 3D User Interfaces Abstr. Workshops (VRW), Mar. 2022, pp. 4226–4230.
pp. 960–961. [75] D. Hornak, M. Jascur, N. Ferencik, and M. Bundzel, ‘‘Proof of concept:
[53] P. Dhope and M. B. Neelagar, ‘‘Real-time emotion recognition from facial Using reinforcement learning agent as an adversary in serious games,’’
expressions using artificial intelligence,’’ in Proc. 2nd Int. Conf. Artif. in Proc. IEEE Int. Work Conf. Bioinspired Intell. (IWOBI), Jul. 2019,
Intell. Signal Process. (AISP), Feb. 2022, pp. 1–6. pp. 000111–000116.
[54] C. Kiourt, G. Pavlidis, A. Koutsoudis, and D. Kalles, ‘‘Multi-agents based [76] B. D. Bryant and R. Miikkulainen, ‘‘Neuroevolution for adaotive teams,’’
virtual environments for cultural heritage,’’ in Proc. 26th Int. Conf. Inf., in Proc. Congr. Evol. Comput., 2003, pp. 2194–2201.
Commun. Autom. Technol. (ICAT), Oct. 2017, pp. 1–6. [77] T. Murata, ‘‘Petri Nets: Properties, analysis and applications,’’ Proc. IEEE,
[55] T. Kempitiya, D. De Silva, E. Rio, R. Skarbez, and D. Alahakoon, vol. 77, no. 4, pp. 541–580, Apr. 1989.
‘‘Personalised physiotherapy rehabilitation using artificial intelligence and [78] S.-L. Yeh, Y.-S. Lin, and C.-C. Lee, ‘‘A dialogical emotion decoder for
virtual reality gaming,’’ in Proc. 15th Int. Conf. Human Syst. Interact. speech emotion recognition in spoken dialog,’’ in Proc. IEEE Int. Conf.
(HSI), Jul. 2022, pp. 1–6. Acoust., Speech Signal Process. (ICASSP), May 2020, pp. 6479–6483.
[79] C. Joesph, A. Rajeswari, B. Premalatha, and C. Balapriya, ‘‘Implementa-
[56] H. Chen, Y. Wang, and W. Liang, ‘‘VCoach: Enabling personalized boxing
tion of physiological signal based emotion recognition algorithm,’’ in Proc.
training in virtual reality,’’ in Proc. IEEE Conf. Virtual Reality 3D User
IEEE 36th Int. Conf. Data Eng. (ICDE), Apr. 2020, pp. 2075–2079.
Interfaces Abstr. Workshops (VRW), Mar. 2022, pp. 546–547.
[80] X. Li and R. Lian, ‘‘Correlation modeling of moral emotion based on facial
[57] D. Shravani, Y. R. Prajwal, P. V. Atreyas, and G. Shobha, ‘‘VR image emotion recognition algorithm,’’ in Proc. Int. Conf. Augmented
supermarket: A virtual reality online shopping platform with a dynamic Intell. Sustain. Syst. (ICAISS), Nov. 2022, pp. 560–562.
recommendation system,’’ in Proc. IEEE Int. Conf. Artif. Intell. Virtual [81] M. A. Mahima, N. C. Patel, S. Ravichandran, N. Aishwarya, and
Reality (AIVR), Nov. 2021, pp. 119–123. S. Maradithaya, ‘‘A text-based hybrid approach for multiple emotion
[58] A. M. Soccini and F. Cena, ‘‘The ethics of rehabilitation in virtual reality: detection using contextual and semantic analysis,’’ in Proc. Int. Conf.
The role of self-avatars and deep learning,’’ in Proc. IEEE Int. Conf. Artif. Innov. Comput., Intell. Commun. Smart Electr. Syst. (ICSES), Sep. 2021,
Intell. Virtual Reality (AIVR), Nov. 2021, pp. 324–328. pp. 1–6.
[59] E. Osipov, S. Kahawala, D. Haputhanthri, T. Kempitiya, D. D. Silva, [82] Z. Zhang, L. Xie, and Z. Zhao, ‘‘Temporal integration of emotion
D. Alahakoon, and D. Kleyko, ‘‘Hyperseed: Unsupervised learning with perception for cross-cultural and multi-emotion music,’’ in Proc. Int. Conf.
vector symbolic architectures,’’ IEEE Trans. Neural Netw. Learn. Syst., Culture-Oriented Sci. Technol. (ICCST), Nov. 2021, pp. 198–203.
early access, Nov. 16, 2022, doi: 10.1109/TNNLS.2022.3211274. [83] A. Dixit, M. Vashishtha, and K. Guleri, ‘‘An AI based formulated feedback-
[60] Y. Li, X. Li, S. Shen, L. Zeng, R. Liu, Q. Zheng, J. Feng, and S. Chen, system for interpreting conclusive emotions for a group of people,’’ in Proc.
‘‘DTBVis: An interactive visual comparison system for digital twin brain IEEE Global Conf. Comput., Power Commun. Technol. (GlobConPT),
and human brain,’’ Vis. Informat., vol. 7, no. 2, pp. 41–53, Jun. 2023. Sep. 2022, pp. 1–5.
[84] T. Akahori, K. Dohsaka, M. Ishii, and H. Ito, ‘‘Efficient creation of
[61] J. Li and Z. Miao, ‘‘Agent based intelligent virtual environment and its
Japanese tweet emotion dataset using sentence-final expressions,’’ in
entertainment application,’’ in Proc. 8th Int. Conf. Signal Process., vol. 2,
Proc. IEEE 3rd Global Conf. Life Sci. Technol. (LifeTech), Mar. 2021,
2006.
pp. 501–505.
[62] S. M. A. Moazzen and A. Ahmadi, ‘‘3D simulation of tennis game using [85] A. S. M. S. Reaj, M. Maniruzzaman, and A. A. J. Jim, ‘‘Emotion
learning algorithms by application of virtual reality,’’ in Proc. Iranian Conf. recognition using EEG-based brain computer interface,’’ in Proc. Int. Conf.
Electr. Eng. (ICEE), May 2017, pp. 1527–1531. Electron., Commun. Inf. Technol. (ICECIT), Sep. 2021, pp. 1–4.
[63] E. Rahimi and A. Ahmadi, ‘‘An AI-based tennis game by application of [86] J. Li, S. Qiu, Y.-Y. Shen, C.-L. Liu, and H. He, ‘‘Multisource transfer
virtual reality components,’’ in Proc. Iranian Conf. Electr. Eng. (ICEE), learning for cross-subject EEG emotion recognition,’’ IEEE Trans.
May 2017, pp. 2165–2170. Cybern., vol. 50, no. 7, pp. 3281–3293, Jul. 2020.
[64] S. Dawn, U. Saraogi, and U. S. Thakur, ‘‘Agent-based learning for auto- [87] K. Liu, M. Yang, X. Li, K. Zhang, X. Xia, and H. Yan, ‘‘M-data-fabric:
navigation within the virtual city,’’ in Proc. Int. Conf. Comput. Perform. A data fabric system based on metadata,’’ in Proc. IEEE 5th Int. Conf. Big
Eval. (ComPE), Jul. 2020, pp. 007–012. Data Artif. Intell. (BDAI), Jul. 2022, pp. 57–62.
[65] V. V. R. M. K. Rao, N. Adhikari, and A. D. Ghimire, ‘‘Towards training an [88] X. Zhou, ‘‘Tennis motion precision video monitoring simulation based
agent in augmented reality world with reinforcement learning,’’ in Proc. on artificial intelligence algorithm,’’ in Proc. Int. Conf. Artif. Intell. Auto.
17th Int. Conf. Control, Autom. Syst. (ICCAS), Oct. 2017, pp. 1884–1888. Robot Syst. (AIARS), Jul. 2022, pp. 211–215.
[66] J. Li, Z. Wang, and Y. Zhang, ‘‘An implementation of artificial emotion [89] A. S. Ahmad, ‘‘Brain inspired cognitive artificial intelligence for
based on fuzzy state machine,’’ in Proc. 3rd Int. Conf. Intell. Human-Mach. knowledge extraction and intelligent instrumentation system,’’ in Proc. Int.
Syst. Cybern., 2011, pp. 83–86. Symp. Electron. Smart Devices (ISESD), Oct. 2017, pp. 352–356.
[90] L. M. Alkwai, ‘‘An explainable artificial-intelligence-based CNN model
[67] C.-T. Song and H. Wang, ‘‘Research and implementation of virtual human for knowledge extraction from the social Internet of Things: Proposing
animation based on distributed semantic environment,’’ in Proc. IEEE Int. a new model,’’ IEEE Syst., Man, Cybern. Mag., vol. 8, no. 4, pp. 48–51,
Conf. Electron. Inf. Commun. Technol. (ICEICT), Aug. 2016, pp. 78–81. Oct. 2022.
[68] R. Dai and Z. Pan, ‘‘A virtual companion empty-nest elderly dining system [91] Z. Dong, ‘‘Roman research progress and prospects of ‘artificial intelligence
based on virtual avatars,’’ in Proc. IEEE 7th Int. Conf. Virtual Reality + education’ area based on CITESPACE knowledge spectrum,’’ in Proc.
(ICVR), May 2021, pp. 446–451. 3rd Int. Conf. Educ., Knowl. Inf. Manag., 2022, pp. 331–334.
[69] M. L. Chang, Y.-H. Huang, W.-C. Lin, and S.-W. Sun, ‘‘Digital fabrication: [92] X. Su, Z. You, D. Huang, L. Wang, L. Wong, B. Ji, and B. Zhao,
Machine learning-based immersive experiencing for the virtual space in ‘‘Biomedical knowledge graph embedding with capsule network for multi-
a future museum,’’ in Proc. IEEE Int. Conf. Artif. Intell. Virtual Reality label drug-drug interaction prediction,’’ IEEE Trans. Knowl. Data Eng.,
(AIVR), Nov. 2021, pp. 102–105. vol. 35, no. 6, pp. 5640–5651, Jun. 2023.

76240 VOLUME 11, 2023

J. Wang et al.: Intelligent Metaverse Scene Content Construction

[93] X. Zhao, J. Greenberg, S. McClellan, Y.-J. Hu, S. Lopez, S. K. Saikin, SIRU CHEN is currently pursuing the degree with
X. Hu, and Y. An, ‘‘Knowledge graph-empowered materials discovery,’’ Nanjing Normal University. Her current research
in Proc. IEEE Int. Conf. Big Data, Dec. 2021, pp. 4628–4632. interests include scientific data visualization and
[94] T. Ma, L. Miao, Y. Du, and H. Wang, ‘‘Calibration data prediction of wind immersive visualization. She is the cofourth author
tunnel balance based on regression tree,’’ in Proc. 16th Int. Conf. Intell. of the 2021 IEEE Pacific VIS Best Poster paper.
Syst. Knowl. Eng. (ISKE), Nov. 2021, pp. 362–367. She is also the author of the paper Multi-User
[95] Q. Wang, A. P. Doulgeris, and T. Eltoft, ‘‘Physics-aware training data Collaborative Volume Data Illustration and Visu-
to improve machine learning for sea ice classification from Sentinel-1 alization presented at IEEE Visualization 2020
SAR scenes,’’ in Proc. IEEE Int. Geosci. Remote Sens. Symp., Jul. 2022,
(IEEE VIS).
pp. 4992–4995.
[96] S. Zhang, J. Yuan, M. Liao, and L. Zhang, ‘‘text2video: Text-driven
talking-head video synthesis with personalized phoneme—Pose dictio-
nary,’’ in Proc. IEEE Int. Conf. Acoust., Speech Signal Process. (ICASSP),
May 2022, pp. 2659–2663.
[97] M. Zhang, Z. Yang, C. Liu, and L. Fang, ‘‘Traditional Chinese
medicine knowledge service based on semi-supervised BERT-BiLSTM- YUXUAN LIU is currently pursuing the bachelor’s
CRF model,’’ in Proc. Int. Conf. Service Sci. (ICSS), Aug. 2020, pp. 64–69. degree with Nanjing Normal University. Her
current research interests include visualization and
virtual reality.

RICHEN LAU received the Ph.D. degree from

Peking University, in 2017. He is an Associate
JUNXIANG WANG is currently pursuing the Professor with Nanjing Normal University. He has
bachelor’s degree with Nanjing Normal Univer- published more than 30 peer-reviewed journals
sity. He has been the author of several software and conference papers, including ACM CHI,
copyrights. His current research interests include IEEE VIS, IEEE ISMAR, IEEE PacificVis, IEEE
artificial intelligence, visualization, virtual reality, THMS, and Bioinformatics. He was a PC/TPC
deep learning, and content generation. Member of IEEE VIS (short papers), ACHI,
ChinaVis, and HPC China. He has supervised
students to receive the Best Survey Paper Award
from ChinaVis 2019 and the Best Poster Award from IEEE PacificVis.

VOLUME 11, 2023 76241