Scribd
Upload
13 views1 page

Exam Firstpage

The document discusses various types of sensors used in robotics, including proprioceptive and exteroceptive sensors, as well as passive and active sensors. It covers concepts such as RGB vs. depth images, Kalman filters for state estimation, deep learning applications for grasp detection, and kinematics in robot manipulation. Additionally, it explains control strategies like PID and Model Predictive Control, along with the Markov Decision Process and the Bellman Equation for decision-making in dynamic systems.

Uploaded by

Lithika Ramesh
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
13 views1 page

Exam Firstpage

The document discusses various types of sensors used in robotics, including proprioceptive and exteroceptive sensors, as well as passive and active sensors. It covers concepts such as RGB vs. depth images, Kalman filters for state estimation, deep learning applications for grasp detection, and kinematics in robot manipulation. Additionally, it explains control strategies like PID and Model Predictive Control, along with the Markov Decision Process and the Bellman Equation for decision-making in dynamic systems.

Uploaded by

Lithika Ramesh
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 1

SENSOR AND PERCEPTION

Classification of sensors:
• Proprioceptive sensors: Measuring the internal state of the system (or robot). E.g. Heading sensors, IMU, econders.
• Exteroceptive Sensors: Measuring the external environment what’s happening outside the system (or robot). E.g. LIDAR, Sonar, GPS.
Passive sensors: Measure energy coming front the environment, very much influenced by the environment. E.g. compass, gyroscopes, contact switches.
Active sensors:Emit their proper energy and measure the reaction Better performance , but some influence on environment Eg. optical encoders,
magnetic encoders
Difference between RGB and Depth Image: RGB image made up of 3 colour channels, where each pixel contains 3 values representing the intensity
of each colour (e.g. image classification). Depth image is a grayscale image, where each pixel value represents a distance, showing how far an object is at
that pixel from the camera (e.g.3D mapping, obstacle detection).
Kalman Filter: An efficient recursive filter estimating the internal state of a linear dynamic system from a series of noisy measurements.
Dynamic Model: Relationship between the input and output.
System State: The set of parameters which describe the current state of the system at any particular time.
Measurements Noise: The difference between the measured data and the reality. for example the ground being uneven can affect the radars capacity
to measure properly.
Process Noise: the difference between the actual state of the system and the predicted state. it accounts for the unknown or unpredictable changes in
the system like wind.
1
1. INPUT: Initial Measurement from sensor = zn 4. UPDATE: State update (position): x̂n,n = x̂n,n−1 + N (zn −x̂n,n−1 ),
2. PREDICT: Dynamic Model: x̂n+1,n = x̂n,n + ∆tẋ ˆn,n where Kn = N1
ˆn+1,n = ẋ
If velocity is constant: ẋ ˆn,n , where ẋ = v (z −x̂n,n−1 )
State update (velocity): x̂n,n = x̂n,n−1 + β n ∆t
Covariance Extrapolation (To predict uncertainty): pn+1,n = pn,n
p 5. Uncertainty Update: pn,n = (1 − K n )p n,n−1
3. Kalman Gain: Kn = p n,n−1
n,n−1 +r
DEEP LEARNING FOR ROBOT PERCEPTION
Loss Functions: to evaluate to what extent the actual outputs y are correctly predicted by the model outputs How to find size of the output layer
W2 xH2 xK
Input: W1 xH1 xC
• W2 = ( W1 −F S
+2P
)+1 • H2 = ( H2 −F
S
+2P
)+1 • K= number of filters
To find number of params in the layer:
• F 2 CK = Num. of params in each filter • F 2 CK ∗ K = Num. of params in the layer
Grasp Detection: Input: RGB image or 3D Point cloud. Output: Grasp location. Assumptions: Object is static & on a table. Constraints: Real-time,
Accuracy. Two small tasks: 2D Grasp detection (visual) & Motion planning (action)
Grasp Detection with CNN: Given an image of an object we want to find a way to safely pick up and hold that object.
• We define a “grasp” with 5 values (x, y, θ, width, height)
1 Pn
• Loss function: MSE = n i=1 (Yi − Ŷi )
2

• Activation Function: Linear ReLU


• To Evaluate results using Rectangle metric: The grasp angle is within 30 degrees of the ground truth grasp. The Jaccard index of the predicted
grasp and the ground truth is greater than 25%.
Assume we use a point cloud. Design a deep network to adapt to the point cloud input:
Pi T Tj
• Input Process raw point cloud data (x, y, z) coordinates. • Compute similarity matrix F using dot product: Fi,j = ||Pi ||||Tj ||
• Feature Extraction Conv1D → BatchN orm • Apply Softmax to generate per-point affordance probability scores S.
• Point Feature Transformation Output: Point features • Si,j representing the probability of point i having affordance j
P1 , P2 , ..., Pn where each Pi is a d-dimensional feature vector
• Text Encoding Branch Input: Affordance labels L = l1 , l2 , ..., lm
into Text Encoder ftext → T1 , ..
6DOF Pose Estimation: Input: RGB image.Output: 6DOF pose of an object:Quaternion, Object detection, Object segmentation.
KINEMATIC, CONTROL, AND REINFORCEMENT LEARNING
Kinematics: The study of objects (robots) in motion.
Robot Manipulator:A kinematic chain: an assembly of pairs of rigid bodies that can move respect to one another via a mechanical constraint. The
rigid bodies are called links. The mechanical constraints are called joints and represented with a single numeric value called a joint variable qi : joint
variable for joint i.
• Revolute(or rotary): like a hinge, allows relative rotation about a fixed axis between two links, axis of rotation is the z axis by convention.
• Prismatic(or linear): like a piston, allows relative translation along a fixed axis between two links, axis of translation is the z axis by convention,
+ our convention: joint i connects link i - 1 to link i, when joint i is actuated, link i moves.
Forward Kinematics: Determine where the robot’s hand is (if all joint variables are known). x1 = L cos θ, y1 = L sin θ
Inverse Kinematics:Calculate what each joint variable is (If we desire that the hand be located at a particular point). θ = cos−1 ( xl1 )
Denavit-Hartenberg Convention:
• PID: Proportional: Adjusts output based on current error (difference between the desired setpoint and the actual process value) The larger the
error, the stronger the correction. Effect: Provides immediate response to errors, improving speed, but can cause steady-state error.
• Integrate: Accumulates past errors over time and integrates it. Eliminates residual steady-state error by continuing to adjust until the error is
zero. Effect: Improves accuracy over time but can lead to overshoot if too aggressive.
• Derivative: Predicts future error based on its rate of change.Acts on the trend of the error to dampen the system response. Effect: Reduces
overshoot and improves stability but is sensitive to noise.
Model Predictive Control(MPC): An advanced control strategy that uses a mathematical model of the system to predict future behavior and optimise
control actions over a finite time horizon. At each control step, MPC:
• Predicts future outputs based on current state and inputs. Solves an • Applies the first control input from the optimized sequence.
optimization problem to minimize a cost function (e.g., error, energy, • Repeats this process at the next time step using updated measure-
or deviation). ments.
PID vs.MPC: • PID controller handles only a single input and a single output (SISO • MPC controller is a more advanced method of process con
systems). for MIMO systems (Multiple Inputs, multiple Outputs).
Markov Decision Process (MDP): A mathematical representation of a sequential decision-making problem in which:
• A system evolves through time. • Actions incur immediate costs or accrue immediate rewards and af-
• A decision maker controls it by taking actions at pre-specified points fect the subsequent system state.
of time. • P (St+1 = s′ |St = st , At = at , St−1 = st−1 , At−1 , . . . S0 = s0 )
n o = P (St+1 = s′ |St = st , At = at )
P
Bellman Equation: v(s) = maxa∈As r(s, a) + λ j∈S p(j|s, a)v(j)

You might also like

pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy