AR VR Unit 2 - VR Modelling
AR VR Unit 2 - VR Modelling
VR MODELING
Modeling – Geometric Modeling – Virtual Object Shape – Object Visual
Appearance – KinematicsModeling – Transformation Matrices – Object Position –
Transformation Invariants –Object Hierarchies –Viewing the 3D World – Physical
Modeling – Collision Detection – Surface Deformation – Force Computation –
Force Smoothing and Mapping – Behavior Modeling – Model Management
Definition:
VR modeling is the process of creating, modifying, and interacting with 3D models within a virtual
reality (VR) environment. Unlike traditional 3D modeling, which is done on a 2D screen using a
keyboard and mouse, VR modeling allows users to engage directly with their designs in a fully
immersive, three-dimensional space. This approach enables more intuitive manipulation of objects
through natural gestures and movements, using VR controllers or hand-tracking technology. VR
modeling is commonly used in fields such as architecture, game design, product design, and digital
art, providing a more engaging and spatially aware way to visualize and refine 3D creations.
1. Modeling in 3D Graphics
Modeling in 3D graphics involves creating digital representations of objects that can be manipulated
and visualized within a virtual environment. These models are essential in various fields, including
virtual reality, gaming, computer-aided design (CAD), and animation. The process of modeling
covers various aspects, such as,
Geometric modeling
Kinematics modeling
Physical modeling
Model behavior
Model management
1. VR Engine
Role: The VR engine is the core computational unit of the VR system. It often
includes a multiprocessor system and graphics accelerators to handle the intensive
calculations required for rendering 3D environments and processing real-time
interactions.
Function: The engine processes input from the user, runs simulations, renders the
virtual environment, and updates the scene based on user interactions and other
parameters. It acts as the brain of the VR system.
2. I/O Devices
Role: Input/Output (I/O) devices are the tools through which the user interacts with
the VR environment. These can include devices like:
o Mouse, Joystick: Traditional input devices for navigation and interaction.
o Glove: A specialized input device that captures hand movements and gestures.
o HMD (Head-Mounted Display): The primary output device that displays the
virtual environment to the user, often with head-tracking capabilities.
Function: I/O devices collect user inputs (like movements or commands) and
transmit them to the VR engine. They also deliver outputs (like visual or tactile
feedback) to the user, making the interaction with the virtual environment immersive.
3. User
Role: The user could be anyone interacting with the VR system, such as a
programmer developing VR content, a trainee undergoing simulation-based training,
or an operator controlling a remote system.
Function: The user interacts with the virtual environment through the I/O devices,
receiving sensory feedback and providing input that drives the virtual experience.
4. Task
Role: The task represents the specific activity the user is performing in the VR
environment. This could vary widely, including:
o Training: Simulations for learning and skill development.
o Programming: Developing or modifying VR environments.
o Teleoperation: Controlling a remote system or machine in real-time using
VR.
Function: The task influences the requirements and design of the VR experience,
determining what kind of interactions, simulations, and feedback are needed.
Role: The software includes I/O drivers, VR toolkits, and other applications that
manage the VR system's operations. Databases store the necessary data, such as 3D
models, textures, and other resources.
Function: Software acts as the interface between the hardware (I/O devices, VR
engine) and the user, handling the input/output operations, rendering, and other
processes. Databases provide the content needed to create and interact with the virtual
environment.
Interactions:
VR Engine ↔ I/O Devices: The VR engine receives user inputs from the I/O devices
and sends outputs (like visual displays or haptic feedback) back to these devices.
VR Engine ↔ Software & Databases: The engine uses software and data from
databases to render scenes, process user actions, and manage the VR environment.
User ↔ I/O Devices: The user interacts with the VR system via the I/O devices,
receiving sensory information and providing inputs that influence the virtual
experience.
User ↔ Task: The user's actions in the VR system are determined by the task they
are performing, which also influences how the system is configured and used.
Geometric modeling is the mathematical representation of an object's shape and structure in 3D space.
It involves defining the surfaces, edges, and vertices that make up the object, allowing it to be
rendered, manipulated, and viewed in a virtual environment. Geometric models can be categorized
into several types:
Polygonal Models: These models use polygons, typically triangles, to represent the object's surface.
Each polygon is defined by its vertices, edges, and a normal vector that indicates its orientation in
space. Polygonal models are commonly used due to their simplicity and efficiency in rendering.
NURBS (Non-Uniform Rational B-Splines): NURBS are mathematical representations that define
smooth curves and surfaces using control points. Unlike polygonal models, NURBS can accurately
represent complex shapes with fewer data points, making them ideal for precision modeling in
industries like automotive and aerospace design.
Subdivision Surfaces: This technique involves refining polygonal meshes to create smooth surfaces.
It is commonly used in animation and character modeling, where a balance between smoothness and
detail is required.
1. Surface Shape
Surface shape refers to how the geometry of virtual objects is defined and represented in a
VR environment. This involves creating the structure or framework that forms the shape of
the objects within the virtual world.
Triangle Meshes:
o Triangle meshes are a common method for representing 3D objects in
computer graphics.
o A mesh is composed of numerous triangles, which are the simplest polygons
that can be used to approximate the surfaces of 3D objects.
o Triangle meshes are efficient for rendering complex shapes and are widely
supported in most graphics hardware and software.
Different LODs (Levels of Detail):
o LOD refers to the technique of adjusting the complexity of a 3D model based
on its distance from the viewer or its importance in the scene.
o Closer objects may be rendered with high detail, while distant objects are
simplified to save computational resources.
o This technique helps maintain performance while still providing a visually
acceptable experience in VR.
Spline Curves and Surfaces:
o Splines are mathematical representations used to create smooth and flexible
curves and surfaces in 3D modeling.
o They are particularly useful for designing organic shapes and objects that
require smooth transitions and curves.
o Spline-based modeling allows for the precise control of curves, making it an
essential tool in creating realistic models in VR.
2. Visual Appearance
Visual appearance deals with how the surface of 3D objects looks in a VR environment. It
includes techniques that enhance the realism and visual quality of the objects by simulating
the way light interacts with surfaces.
2. Kinematics Modeling
Kinematics modeling involves the study of motion without considering the forces that cause it. In 3D
graphics, kinematics is used to describe the movement and positioning of objects within a scene.
Transformation matrices are mathematical tools used to perform operations such as translation,
rotation, and scaling on objects in 3D space. These matrices are essential in kinematics modeling, as
they allow objects to be manipulated precisely.
Translation: A translation matrix moves an object from one position to another in 3D space.
It is represented by a 4x4 matrix that shifts the object's coordinates by a specified vector.
Rotation: A rotation matrix rotates an object around an axis in 3D space. Rotation can be
represented using Euler angles, quaternions, or rotation matrices, each offering different
advantages in terms of performance and ease of use.
Scaling: A scaling matrix changes the size of an object by multiplying its coordinates by a
scaling factor. Uniform scaling uses the same factor for all dimensions, while non-uniform
scaling applies different factors to each axis.
World Coordinates: The global coordinate system that defines the absolute position of
objects within the virtual environment.
Local Coordinates: The coordinate system relative to the object's own origin.
Transformations are often applied in local coordinates before being converted to world
coordinates.
Pivot Points: The point around which an object is rotated or scaled. By changing the pivot
point, designers can control how an object moves in relation to other objects or the
environment.
Distance: The distance between two points on an object remains constant after a rotation or
translation.
Angles: The angles between edges or faces of an object remain unchanged during rotation
and translation, preserving the object's shape.
Collinearity: Points that lie on a straight line remain collinear after a transformation, ensuring
the object's geometric structure is maintained.
Object hierarchies define the relationships between objects in a scene, where some objects (children)
are dependent on others (parents). This hierarchical structure is crucial in animation, game
development, and complex scene management.
Parent-Child Relationships: In a hierarchy, the child object's position, rotation, and scale are
relative to its parent. Transforming the parent automatically transforms the child, enabling
coordinated movement and interaction.
Transform Nodes: In a scene graph, transform nodes represent the transformations applied to
objects. These nodes can be organized hierarchically to manage complex scenes efficiently.
Scene Graphs: A data structure that organizes objects in a scene based on their hierarchical
relationships. Scene graphs are used in rendering engines to optimize rendering and manage
object interactions.
The equation represents a series of transformations (T) that convert the position of a fingertip from its
local coordinate system to a global coordinate system. Each T represents a transformation matrix
between different coordinate frames.
Breaking it down:
The (t) notation indicates that these transformations can vary with time.
The dot (•) at the end suggests matrix multiplication, as these transformations would be combined by
multiplying their respective matrices.
This equation is likely used in VR hand tracking systems to calculate the precise position of a user's
fingertip in the global coordinate system of the virtual environment, taking into account all the
intermediate transformations from the tracking hardware to the final display.
Tracking Virtual Hands
TW←hand(t):
o This represents the transformation matrix that describes the position and
orientation of the hand relative to the world coordinate system at time ttt.
o It defines where the hand is located in the VR world and how it is oriented.
TW←source:
o This is the transformation matrix that describes the position and orientation of
the "source" (which could be a larger body part like the arm or torso) relative
to the world coordinate system.
o This matrix defines where the source is placed within the VR world.
Tsource←receiver(t):
o This matrix represents the transformation from the "receiver" (which could be
the wrist or forearm) to the "source," potentially dependent on time ttt.
o It shows how the receiver is positioned and oriented relative to the source.
In this context, the "hand" is a specific part of a user's body, and its position and orientation
within the VR world need to be computed through the following steps:
TW←object(t)
o This represents the transformation matrix that describes the position and
orientation of an object relative to the world coordinate system at time t.
o It tells us where the object is located in the world and how it is oriented.
TW←source:
o This is the transformation matrix that describes the position and orientation of
the "source" relative to the world.
o A "source" could be any reference frame or point in space that you want to use
as a starting point for further transformations.
Tsource←receiver(t):
o This matrix represents the transformation from the "receiver" to the "source,"
possibly dependent on time ttt.
o It shows how the "receiver" (another point or object in space) is positioned
and oriented relative to the "source."
Treceiver←object:
o This matrix describes the position and orientation of the object relative to the
"receiver."
o It tells us how the object is placed relative to another frame of reference,
which in this case is the "receiver."
In VR, objects are often transformed through a series of steps to place them correctly in the
virtual world. The sequence of transformations might include:
Let's say you have a VR scenario where an object (like a virtual tool) is held by a user. The
object needs to be positioned correctly in the VR world:
The tool (object) might be positioned relative to the user's hand (receiver).
The user's hand might be positioned relative to the user's body (source).
The user's body might be positioned within the VR world (world coordinates).
Viewing the 3D world involves rendering a 3D scene onto a 2D screen, simulating depth, perspective,
and camera angles to create the illusion of three-dimensionality.
The virtual camera is responsible for capturing the scene and determining how it is projected onto the
screen. Different camera models offer various ways to visualize the 3D world:
Perspective Camera: Mimics human vision by projecting objects based on their distance
from the camera. Objects further away appear smaller, creating a sense of depth.
Orthographic Camera: Projects objects without perspective, meaning that objects appear the
same size regardless of their distance from the camera. This model is often used in technical
drawings and CAD applications.
Field of View (FOV): The extent of the observable world seen through the camera. A wider
FOV creates a more expansive view but may introduce distortion.
Clipping planes define the boundaries of the visible scene, ensuring that objects outside a certain
range are not rendered. This improves performance and prevents visual artifacts:
Near Clipping Plane: The closest distance from the camera at which objects are rendered.
Objects closer than this plane are not visible.
Far Clipping Plane: The furthest distance from the camera at which objects are rendered.
Objects beyond this plane are not visible.
Depth perception is crucial in creating a realistic 3D scene. Various techniques are used to enhance
depth perception and optimize rendering:
Z-Buffering: A technique that stores depth information for each pixel, ensuring that only the
closest object is rendered at each pixel location.
Frustum Culling: The process of removing objects that lie outside the camera's viewing
frustum, improving rendering efficiency by not processing objects that are not visible.
4. Physical Modeling
Physical modeling simulates the physical properties of objects, such as mass, elasticity, and friction,
allowing for realistic interactions and behaviors in a virtual environment.
Collision detection is the process of determining when two or more objects in a scene intersect or
come into contact. It is fundamental to physical modeling, as it ensures that objects behave
realistically when they interact:
Bounding Volumes: Simplified shapes, such as spheres, boxes, or capsules, that approximate
the object's geometry. Bounding volumes are used to quickly determine potential collisions
before performing more detailed checks.
Spatial Partitioning: Techniques like quadtrees, octrees, and BSP trees divide the scene into
manageable sections, allowing for efficient collision detection by reducing the number of
objects that need to be tested against each other.
Continuous Collision Detection: Prevents objects from "tunneling" through each other by
considering their motion over time, ensuring that collisions are detected even if they occur
between discrete frames.
4.2 Grasping:
Grasping refers to the action of holding or manipulating objects within the virtual environment. The
haptic interface simulates the forces and sensations involved in grasping an object, such as the
pressure felt when holding an item or the feedback when lifting and moving it within the VR space.
Surface deformation involves altering an object's shape in response to external forces or interactions.
This is important for simulating soft bodies, cloth, and other deformable materials:
Compliance is the measure of a surface’s ability to yield under pressure, while Texture
refers to the feel or appearance of a surface (e.g., smooth, rough). The haptic interface
simulates these characteristics, allowing the user to perceive different textures and the
softness or hardness of virtual objects through touch.
Hard Contact involves the interaction with solid or rigid virtual objects. When a user touches
or presses against a virtual object that is modeled to be hard, the haptic system simulates the
resistance and feedback that one would expect when encountering a solid surface in the real
world.
Physical Constraints are the rules or limits that govern how objects behave and interact in
the virtual environment. These constraints could include boundaries that objects cannot pass
through, or restrictions on movement, ensuring that virtual interactions respect the physical
laws of the simulated environment. The haptic interface helps enforce these constraints,
providing physical feedback when a user tries to push beyond these limits.
The rendering pipeline is a crucial concept in computer graphics, describing the sequence of steps that
transform 3D scene data into a 2D image on a display. Here's a concise overview of the typical stages
in a modern rendering pipeline:
1. Application stage
2. Geometry processing
3. Rasterization
4. Pixel processing
5. Output merging
This image illustrates two parallel pipelines in VR modeling: the traversal/force/tactile pipeline for
haptic feedback, and the traditional visual rendering pipeline. Let me break down both pipelines:
1. Force/Tactile pipeline
2. Visual Pipeline
1. Force/Tactile Pipeline:
This pipeline is responsible for providing haptic feedback in VR modeling, simulating the
sense of touch and force interactions.
a) Collision Detection:
This is the initial stage where the system identifies when virtual objects come into
contact with each other or with the user's virtual representation (like a virtual hand).
It involves complex algorithms to detect intersections between 3D geometries in real-
time.
b) Force Calculation:
Once a collision is detected, this stage computes the forces that should be applied
based on the physical properties of the virtual objects (mass, elasticity, etc.).
It considers factors like the velocity of impact, the direction of force, and the material
properties.
c) Force Smoothing:
d) Force Mapping:
Here, the calculated and smoothed forces are mapped to the specific capabilities of the
haptic device being used.
This might involve scaling forces to fit within the device's output range or translating
complex force vectors into simpler feedback mechanisms.
e) Haptic Texturing:
f) Display (Tactile):
The final stage where the processed haptic information is sent to the haptic device,
often represented by a hand in VR systems.
This could involve actuators that provide force feedback or vibrations to simulate
touch sensations.
The 3D scene data is processed, including object positions, camera location, and any
dynamic elements.
This stage may also handle culling to determine which objects are potentially visible.
Objects are transformed from world space to view space, essentially positioning them
relative to the camera (or VR headset's perspective).
Lighting calculations are performed, including ambient, diffuse, and specular lighting.
For VR, this often involves more complex techniques to handle the stereoscopic
nature of VR displays.
The 3D scene is projected onto a 2D plane, typically using perspective projection for
VR.
In VR, this involves creating two slightly different projections for each eye to create
the stereoscopic effect.
Texture mapping is applied to the geometries, adding detail and realism to surfaces.
This stage might also include other pixel-level effects like normal mapping or parallax
mapping.
f) Display (Visual):
The final rendered image is sent to the VR headset's display, typically a high-refresh-
rate screen split for each eye.
5. Behavior Modeling
Behavior modeling involves defining how objects in a virtual environment behave and interact with
each other, based on predefined rules, logic, or artificial intelligence.
Autonomous agents are objects or characters that can act independently, making decisions based on
their environment and internal states:
Finite State Machines (FSMs): A common method for controlling behavior, where the agent
switches between different states (e.g., idle, moving, attacking) based on triggers or
conditions.
Pathfinding and Navigation: Techniques like A* (A-star) and Dijkstra's algorithm enable
agents to find the most efficient path through an environment, avoiding obstacles and
reaching goals.
Crowd simulation models the behavior of large groups of agents, such as pedestrians or vehicles,
interacting in a shared space:
Boids Algorithm: Simulates flocking behavior, where each agent follows simple rules like
alignment, cohesion, and separation, leading to emergent group behavior.
Social Forces: Models how individuals within a crowd exert forces on each other, influencing
movement and avoiding collisions.
Multi-Agent Systems: Coordinate the actions of multiple agents to achieve group objectives,
often used in simulations of traffic, evacuations, and public events.
Types:
1. Reflex Behavior in VR Modeling:
Reflex behavior refers to automatic, rapid responses to stimuli without conscious thought. In VR
modeling, this could involve:
Automatic hand movements when touching virtual objects
Flinching or blinking in response to sudden visual or auditory stimuli
Balance adjustments in response to perceived motion
Implementing reflex behaviors often requires:
Low-latency input processing
Predictive algorithms to anticipate user actions
Integration with the haptic feedback system shown in the image
6. Model Management
Model management involves organizing, storing, and manipulating 3D models and their associated
data within a virtual environment or simulation:
Scene management involves organizing and optimizing the rendering of a complex scene with
multiple objects:
Level of Detail (LOD): Techniques that reduce the complexity of objects based on their
distance from the camera, improving rendering performance without sacrificing visual
quality.
Instancing: A method for rendering multiple copies of the same object efficiently, often used
in scenes with repeating elements like trees or buildings.
Occlusion Culling: The process of not rendering objects that are hidden behind others,
reducing the number of objects that need to be processed and drawn.
Efficient data structures are essential for managing the complexity of 3D models:
Meshes and Vertex Buffers: Store the geometric data of a model, including vertices,
normals, and texture coordinates. These structures are optimized for fast access and
manipulation during rendering.
Textures and Material Libraries: Organize and manage the appearance data associated with
models, including diffuse maps, normal maps, and specular maps.
Scene Graphs: A hierarchical data structure that organizes objects in a scene, managing their
transformations, properties, and relationships.
In complex projects, version control systems and collaborative tools are essential for managing
changes to 3D models and ensuring consistency across a team:
Version Control Systems (VCS): Tools like Git are used to track changes to models,
textures, and code, allowing teams to collaborate efficiently and revert to previous versions if
necessary.
Asset Management Systems: Platforms that store and organize all the assets in a project,
including 3D models, textures, animations, and scripts. These systems often integrate with
VCS and scene management tools.