Embedded Intelligent Systems

Frankfurt University of Applied Sciences

Prof. Dr. Peter Nauth
 Definitions
and
Fundamentals
25.03.2021 Prof. Dr. Peter Nauth 2
 Embedded Intelligent Systems

Consist of
• Embedded Systems and
• Intelligent Systems.

25.03.2021 Prof. Dr. Peter Nauth 3

 Embedded Systems
Are microcontrollers (µC, MCU)
• Embedded in a device (system, machine, ..)
• Performing a dedicated task for this device
(as opposed to general purpose µC in aPC).
Examples are microcontrollers embedded in
elevators or in the ABS (Anti-Blocking-System)
of cars.

25.03.2021 Prof. Dr. Peter Nauth 4

 Intelligent Systems
• Use intelligent algorithms in order to
• Execute (human-like) sophisticated tasks
such as
- Computer Vision
- Speach Recognition
- Machine decision making or
- Autonomous driving.

25.03.2021 Prof. Dr. Peter Nauth 5

 Embedded Intelligent Systems
Are defined as
• Embedded Systems which execute
intelligent algorithms.
Examples are microcontroller embedded in
smart sensors, smart cameras, smart
phones, autonomous vehicles,
autonomous robots etc.

25.03.2021 Prof. Dr. Peter Nauth 6

 Microcontroller (µC, MCU)
ADC Microcontroller DAC

• Program Execution
• Timer/Counter control
RS 232, USB
• PWM control PWM
I²C, CAN
• ext./int. Interrupts
• ADC/ DAC control HDMI
Digital I/O
• Interface controls
µC: Atmega MCUs, 8051 MCUs, Arduino, myRIO, ……..

25.03.2021 Prof. Dr. Peter Nauth 7

 Embedded Intelligent Systems
Sensors Microcontroller Actuators
Cameras
• Data Acquisition
• Data Transformation
Networks Displays
• Pattern Recognition
• Reasoning
Switches
• Decision Making Transducers
• Machine Learning
• Data Output

25.03.2021 Prof. Dr. Peter Nauth 8

 Programming Languages

• Machine Oriented Assembly Code

• Text Based Programming Language
C, Pascal, Java, Python, …
• Graphical Programming Language
LabVIEW: GUI -> Front Panel
Program Code -> Block Diagram

25.03.2021 Prof. Dr. Peter Nauth 9

 Embedded Design

25.03.2021 Prof. Dr. Peter Nauth 10

Embedded Design (HW & SW)
Test Szenarios

URS Acceptance Test

FDS
Integration Test

MDS Modul Test

Specs must be traceable

Development
from one phase to the other!
 Embedded Design (HW & SW)
URS: User Requirement Specifications

FDS: Functional Design Specifications -> Develop a functional model (e.g. block diagram),
group functions into modules (subroutines), define interfaces between modules
such as return parameters or shared variables. Functions which are used by
several modules should be put in a library for common use.

MDS: Modul Design Specifications -> Develop a detailed model of each modul,
i.e. a detailed block diagram or flow chart

Development: Separate development of the modules. The program structure

must mirror the models of the URS, FDS and MDS (trancability)

Modul Test: Separate (!) test of modules with parameters which represent test scenarios

Integration Test: Test of all modules combined

Acceptance Test: Test of the system by the customer or user

 Embedded Design (HW & SW)
• User Requirement Specifications:
Smart Proximity Sensor
• Functional Specifications:
Sensor Control
Machine Decision about presence of obstacles
Output to LCD (error) and Portpin PA0 (decision)
• Modul Design Specifications:
HW-M1: Digital Sensor SRF08
HW-M2: Microcontroller Atmega 8535
SW-M1: Sensor Control via I2C and error output
SW-M2: Machine Decision Making and output

25.03.2021 Prof. Dr. Peter Nauth 13

 Embedded Design (HW & SW)

LCD

I2C SRF08

25.03.2021 Prof. Dr. Peter Nauth 14

 Embedded Design (HW & SW)

25.03.2021 Prof. Dr. Peter Nauth 15

 Embedded Design (HW & SW)

25.03.2021 Prof. Dr. Peter Nauth 16

 Hardware

25.03.2021 Prof. Dr. Peter Nauth 17

 STK 600 Evaluation Board
Microcontrollerboard with
ATmega 2560 Controller

25.03.2021 Prof. Dr. Peter Nauth 18

 ATmega 2560 (Atmel)
Programming
Language (typically):
C, C++, Phyton

[12]
25.03.2021 Prof. Dr. Peter Nauth 19
 ATmega 2560 Specifications

• 8-bit Controller (i.e. Data Bus width is 8 bit)

• 16 MIPS Instruction Rate at 16 MHz CPU-Clock
• 10 8-bit I/O Ports (Port A – F and Port H, J, K)
• 1 6-bit I/O Ports (Port G)
• 2 8-bit Timer (Timer 0 & 2)
• 4 16-bit Timer (Timer 1, 3 - 5)

25.03.2021 Prof. Dr. Peter Nauth 20

 ATmega 2560 Program Skeleton
#ifndef F_CPU
#define F_CPU 8000000UL
#endif
#include <avr/io.h> // IO Headerfile

int main(void) { ……
}
ISR(xxx_vect( { …..
}

25.03.2021 Prof. Dr. Peter Nauth 21

 ATmega 2560 Programming
• MCU functions are controlled by Special
Function Registers (SFR)
• Registers are used in C-code like variables
e.g.: Write 0x2B to pins of Port B
Data output register of Port B is SFR PORTB
-> PORTB = 0x2B;
• Setting or clearing bits requires masking

25.03.2021 Prof. Dr. Peter Nauth 22

 Arduino 2560
Microcontrollerboard with ATmega 2560 Controller

25.03.2021 Prof. Dr. Peter Nauth 23

 Arduino Uno
Microcontrollerboard with ATmega 328P Controller

25.03.2021 Prof. Dr. Peter Nauth 24

 Raspberry Pi
Programming
Language (typically):
C, C++, Phyton

25.03.2021 Prof. Dr. Peter Nauth 25

 Raspberry Pi 3
• CPU: 1.2 GHz quad-core ARM Cortex A53 (64 – Bit µC)
• 40-pin extended GPIO
• 4 USB 2 ports
• 4 Pole stereo output and composite video port
• Full size HDMI
• CSI camera port for connecting a Raspberry Pi camera
• DSI display port for connecting a touchscreen display
• Micro SD for loading operating system and storing data
• Upgraded switched Micro USB power source up to 2.5A
• Network: 10/100 MBPS Ethernet, 802.11n Wireless
LAN, Bluetooth 4.0

25.03.2021 Prof. Dr. Peter Nauth 26

 Raspberry Pi 3

14

25.03.2021 27
 myRIO (National Instruments)
with RTOS

Programming Language:
LabVIEW

[13]

25.03.2021 Prof. Dr. Peter Nauth 28

 myRIO: Hardware Block Diagramm

[13]

25.03.2021 Prof. Dr. Peter Nauth 29

 myRIO: Hardware Specifications

[13]

25.03.2021 Prof. Dr. Peter Nauth 30

 myRIO: Hardware Specifications

[13]

25.03.2021 Prof. Dr. Peter Nauth 31

 myRIO: Hardware Specifications

[13]

25.03.2021 Prof. Dr. Peter Nauth 32

 myRIO: Hardware Specifications

[13]

25.03.2021 Prof. Dr. Peter Nauth 33

 myRIO: Hardware Specifications

[13]

rms: Root Mean Square

25.03.2021 Prof. Dr. Peter Nauth 34

 myRIO: Programming in LabVIEW

25.03.2021 Prof. Dr. Peter Nauth 35

 Interfaces
between MCU
and
Sensors / Actuators
25.03.2021 Prof. Dr. Peter Nauth 36
 Data Interfaces

• Digital Port I/O (parallel Interface)

e.g. data_sensor = PINA;
• Analog Digital Conversion
ADC = VIN * ((2^N)-1) / VREF
• PWM (Pulse Width Modulation)

25.03.2021 Prof. Dr. Peter Nauth 37

 Data Interfaces

• RS 232, USART (Universal Synchronous and

Asynchronous Receiver and Transmitter)
• CAN (Controller Area Network)
• I²C bus (Inter Integrated Circuit),
TWI (Two Wire Interface)

25.03.2021 Prof. Dr. Peter Nauth 38

 Data Interface: I2C Bus

Start / Stop Conditions [12]

Typical Data Transmission [12]

25.03.2021 Prof. Dr. Peter Nauth 39

 ATmega I2C Communication
I2C Registers of ATmega controllers:

• TWDR: Data Register

• TWCR: Command Register
• TWSR: Status Register

25.03.2021 Prof. Dr. Peter Nauth 40

 ATmega I2C Communication
TWDR: Data Register

[12]

Transmission Mode (Write Mode):

Copy Value (Address, Register No, Data) to TWDR
Receiver Mode (Read Mode):
I2C stores value received from slave in TWDR
automatically
25.03.2021 Prof. Dr. Peter Nauth 41
 ATmega I2C Communication
TWCR: Command Register

[12]

TWINT: After SW has set TWINT to 1->

HW starts TWI operation and clears TWINT
After operation has finished ->
HW sets TWINT to 1
TWEN: Enable TWI
25.03.2021 Prof. Dr. Peter Nauth 42
 ATmega I2C Communication
TWSR: Status Register

[12]

TWS7:3 : Acknowledge code about an operation

e.g. Successful transmission of slave address
-> Acknowledge code is 0x18

25.03.2021 Prof. Dr. Peter Nauth 43

 Smart Ultrasonic
Sensors and
I2C Interface

25.03.2021 Prof. Dr. Peter Nauth 44

 Ultrasonic Sensor Physics
Proximity Measurement

v = s / t t: time of
flight
s=vt

d = s/2 = v t / 2
d = (340 m/s * t) / 2

25.03.2021 Prof. Dr. Peter Nauth 45

 SRF08 Ultrasonic Sensor

25.03.2021 Prof. Dr. Peter Nauth 46

 SRF08 Ultrasonic Sensor

[15]

25.03.2021 Prof. Dr. Peter Nauth 47

 SRF08 Ultrasonic Sensor
• Communication via I²C bus
• Default Address: 0xE0
• 36 registers available
• Register 0:
Start sensing in cm by writing 0x51
• Register 1:
Set maximal gain (default: 31 for max gain of 1025)
• Registers 2 – 35:
Read distance of echos of 17 ranges and set range
25.03.2021 Prof. Dr. Peter Nauth 48
 SRF08 Registers

[15]

25.03.2021 Prof. Dr. Peter Nauth 49

 SRF08 Registers

[15]

25.03.2021 Prof. Dr. Peter Nauth 50

 SRF08 Ultrasonic Sensor
• Range = (Range Register + 1) * 43 mm
• Gain = f (Gain Register) –> see table data sheet
Distance of at max gain:
dg = c * t / 2
= c * (Gain Register * 70 µs) /2
= 340 m/s * Gain Register * 70 µs /2
• Distance at absolute maximum gain (=1024)
dg = 340 m/s * 31 * 70 µs /2 = 368 mm

25.03.2021 Prof. Dr. Peter Nauth 51

 SRF08 Ultrasonic Sensor and I2C
Start SRF08 ranging
• Send the start sequence
• Send 0xE0 ( Address of the SRF08, Write Mode)
• Send 0x00 (Address of the command register)
• Send 0x51 (Command to start the SRF08 ranging)
• Send the stop sequence (not necessary, if to be
continued with repeated start)

25.03.2021 Prof. Dr. Peter Nauth 52

 SRF08 Ultrasonic Sensor and I2C
Read distance from SRF08
• Send the start sequence (or repeated start)
• Send 0xE0 (Address of the SRF08, Write Mode)
• Send 0x02 (Address of High Byte 1st echo register)
• Send the start sequence again (repeated start)
• Send 0xE1 (Address of the SRF08, Read Mode)
• Read high byte of 1st echo distance from SRF08
Distance received is stored in TWDR
-> Copy TWDR to a variable
• Send the stop sequence.
25.03.2021 Prof. Dr. Peter Nauth 53
 Smart Ultrasonic Sensor
// Program Smart Ultrasonic Sensor (SW-M1)
#Define SLA_W 0xE0;
#Define SLA_R 0xE1;
#Define MT_SLA_ACK 0x18;
#Define MT_DATA_ACK 0x28;
#Define R_ADR_SLA_ACK 0x40;
#Define R_DATA_SLA_ACK 0x50;
#Define START 0x08;
#Define REP_START 0x10;
uint16_t dist, dist_LB, dist_HB; // Distances
25.03.2021 Prof. Dr. Peter Nauth 54
 Smart Ultrasonic Sensor

int Check(uint8_t Check_Code)

{
// Wait for TWINT Flag set
// -> Operation finished
while (!(TWCR & (1<<TWINT)));
// Check if Operation was sucessful
if ((TWSR & 0xF8) != Check_Code) error();
}
25.03.2021 Prof. Dr. Peter Nauth 55
 Smart Ultrasonic Sensor

int error()
{
// Print Error Message
lcd_clrscr();
lcd_puts(“Transmission failed“);
}

25.03.2021 Prof. Dr. Peter Nauth 56

 Smart Ultrasonic Sensor
int sensor-control_SW-M1(){
// Start SRF08 ranging:
//Send START condition
TWCR = (1<<TWINT)|(1<<TWSTA)|(1<<TWEN);
// Wait TWINT set and Check successful transmission
Check(START);

25.03.2021 Prof. Dr. Peter Nauth 57

 Smart Ultrasonic Sensor
// Load SLA_W to TWDR Register
TWDR = SLA_W;
// Start Transmission of Slave Address in Write Mode
TWCR = (1<<TWINT) |(1<<TWEN);
// Wait TWINT set and Check successful transmission
Check(MT_SLA_ACK);

25.03.2021 Prof. Dr. Peter Nauth 58

 Smart Ultrasonic Sensor
// Load DATA into TWDR Register
TWDR = 0x00;
// Start transmission of DATA (=Register Number)
TWCR = (1<<TWINT) |(1<<TWEN);
// Wait TWINT set and Check successful transmission
Check(MT_DATA_ACK);

25.03.2021 Prof. Dr. Peter Nauth 59

 Smart Ultrasonic Sensor
// Load DATA into TWDR Register
TWDR = 0x51;
// Start transmission of DATA (=Start Ranging in cm)
TWCR = (1<<TWINT) |(1<<TWEN);
// Wait TWINT set and Check successful transmission
Check(MT_DATA_ACK);

25.03.2021 Prof. Dr. Peter Nauth 60

 Smart Ultrasonic Sensor
// Read distance from SRF08 register 0x02:
//Send Repeated START condition
TWCR = (1<<TWINT)|(1<<TWSTA)|(1<<TWEN);
// Wait TWINT set and Check successful transmission
Check(REP_START);

25.03.2021 Prof. Dr. Peter Nauth 61

 Smart Ultrasonic Sensor
// Load SLA_W to TWDR Register
TWDR = SLA_W;
// Start Transmission of Slave Address in Write Mode
TWCR = (1<<TWINT) |(1<<TWEN);
// Wait TWINT set and Check successful transmission
Check(MT_SLA_ACK);

25.03.2021 Prof. Dr. Peter Nauth 62

 Smart Ultrasonic Sensor
// Load DATA into TWDR Register
TWDR = 0x02;
// Start transmission of DATA (=Register Number)
TWCR = (1<<TWINT) |(1<<TWEN);
// Wait TWINT set and Check successful transmission
Check(MT_DATA_ACK);

25.03.2021 Prof. Dr. Peter Nauth 63

 Smart Ultrasonic Sensor
//Send Repeated START condition
TWCR = (1<<TWINT)|(1<<TWSTA)|(1<<TWEN);
// Wait TWINT set and Check successful transmission
Check(REP_START);

25.03.2021 Prof. Dr. Peter Nauth 64

 Smart Ultrasonic Sensor
// Load SLA_R to TWDR Register
TWDR = SLA_R;
// Start Transmission of Slave Address in Read Mode
TWCR = (1<<TWINT) |(1<<TWEN);
// Wait TWINT set and Check successful receive
Check(R_ADR_SLA_ACK);
// Save distance (high byte) to variable dist
dist_HB = TWDR;

25.03.2021 Prof. Dr. Peter Nauth 65

 Smart Ultrasonic Sensor
// Read distance from SRF08 register 0x03
…… //Repeat p.61 to p.65 with register 3
…… // Save distance (Low byte) in variable dist_LB
// Calc complete distance and store in variable dist
dist = dist_HB * 256 + dist_LB;

// Send STOP condition

TWCR = (1<<TWINT)|(1<<TWSTO)|(1<<TWEN);
// End SW module sensor control (SW-M1)
}
25.03.2021 Prof. Dr. Peter Nauth 66
 Smart Ultrasonic Sensor
int decision-making_SW-M2()
{
// Decide whether object is nearer than 100 cm:
PORTA &= ~ (1<<PA0);
if (dist < 100) //dist holds complete distance
PORTA |= (1<<PA0);
// End SW module decision making (SW-M2)
}
25.03.2021 Prof. Dr. Peter Nauth 67
 myRIO I2C Communication

25.03.2021 Prof. Dr. Peter Nauth 68

 myRIO I2C Communication

25.03.2021 Prof. Dr. Peter Nauth 69

 Smart Optical
Proximity Sensor
and ADC

25.03.2021 Prof. Dr. Peter Nauth 70

 Optical Sensor with Analog Output

V0 ~ 1 / d

PSD = Position Sensitive

Detector

25.03.2021 Prof. Dr. Peter Nauth 71

 Optical Sensor with Analog Output

ADC
VIN

Atmega 128 L GP2D12 or GP2D120

25.03.2021 Prof. Dr. Peter Nauth 72

 Optical Sensor with Analog Output

• Far Range Sensor Sharp GP2D12

Range: d = 12 – 80 cm

• Near Range Sensor Sharp GP2D120

Range: d = 3 – 40 cm

25.03.2021 Prof. Dr. Peter Nauth 73

 Optical Sensor with Analog Output

Transfer Function Ua (d) des GP2D120 (near range)

25.03.2021 Prof. Dr. Peter Nauth 74

 Optical Sensor with Analog Output

Transfer Function Ua (d) des GP2D120 (near range)

25.03.2021 Prof. Dr. Peter Nauth 75

 Optical Sensor with Analog Output

Transfer Function Ua (d) des GP2D12 (far range)

25.03.2021 Prof. Dr. Peter Nauth 76

 Optical Sensor with Analog Output
Sensor function: Mathematical model of a 1/d function:
Ua = (a/(d-b)) + c

By regression analysis or 3 values of the transfer function a,

b, c are calculated for the far range sensor GP2D12:
d = 12 cm -> Ua = 2,0863 V
d = 20 cm -> Ua = 1,3958 V
d = 40 cm -> Ua = 0,7682 V

a = 33,14 Vcm b = -4 cm c = 0,015 V

25.03.2021 Prof. Dr. Peter Nauth 77

 Optical Sensor with Analog Output
VIN <- V0 = (a/(d-b)) + c
VIN <- V0 = (33,14 Vcm/(d+4 cm)) + 0,015 V)

ADC = VIN * ((2^N)-1) / VREF

If DigWidth N = 10 Bit, VREF = 5V

ADC = VIN ((2**10)/5V) = VIN * 204.8 / V
-> ADC = (6787cm/(d+4cm)) + 3)
d = (6787cm / (ADC -3)) – 4 cm

Example for Decision Making:

If (d < 500) action_obstacle_avoidance();
25.03.2021 Prof. Dr. Peter Nauth 78
 Operating Systems
and Development
Platforms
25.03.2021 Prof. Dr. Peter Nauth 79
 Operating Systems
Embedded Systems often process multiple tasks,
e.g. in an Autonomous Robot:
• Sensor data raeding from cameras, laser range
finder, Encoder, …
• Data Analysis and Decision Making
• Control of Actuators

25.03.2021 Prof. Dr. Peter Nauth 80

 Operating Systems
Scheduling of multiple tasks is done by
• Embedded Operating Systems (OS)
Such as
• Embedded Linux (Android, Debian, Ubuntu, ..)
• Windows Mobile
Embedded OS: compact and efficient
Realtime OS (RTOS) execute taks in realtime

25.03.2021 Prof. Dr. Peter Nauth 81

 Robot Operating System (ROS)
ROS in not an OS but a Robot development
application platform (!)
• Message Passing (Publisher – Subscriber)
• Distributed Computing of tasks (nodes)
• Libraries for Robotic functions such as SLAM
• Inclusion of libraries such as Open-CV for
Computer Vision
• Runs on Linux (PC or Embedded Systems)
25.03.2021 Prof. Dr. Peter Nauth 82
 Message Passing in ROS
ROS Master
• Has details of all tasks (nodes) running
• Tasks can be publisher, subscriber or others
If a node publishes
• Master is notified about publishing request (a)
• Master looks for potential Subscriber (b) and
• Connects Publisher and Subscriber (c)

25.03.2021 Prof. Dr. Peter Nauth 83

 Message Passing in ROS

25.03.2021 Prof. Dr. Peter Nauth 84

Smart Sensor: ROS, Arduino, SRF08

25.03.2021 Prof. Dr. Peter Nauth 85

Smart Sensor: ROS, Arduino, SRF08
Set up the ROS node and publisher
• #define CommandRegister 0x00
• #define ResultRegister 0x02
• int New_Address = 0xF8;
• char unit = ‘c‘ // c for centimeters
• float sensorReading;
• std_msgs::Float32 sonar_msg;
• ros::Publisher pub_sonar("sonar", &sonar_msg);
• ros::NodeHandle nh;
• Sonar_srf08 MySonar; //create MySonar object
25.03.2021 Prof. Dr. Peter Nauth 86
Smart Sensor: ROS, Arduino, SRF08
Request reading from sensor
• MySonar.setUnit(CommandRegister, New_Address, unit);
• delay(70);
Set register for reading
• MySonar.setRegister(New_Address, ResultRegister);
Read data from result register
• sensorReading = MySonar.readData(New_Address, 2);
• sonar_msg.data = sensorReading;
Publish
• pub_sonar.publish(&sonar_msg);
• publisher_timer = millis(); //Publish once a ms
25.03.2021 Prof. Dr. Peter Nauth 87
 Realtime Systems

25.03.2021 Prof. Dr. Peter Nauth 88

 Realtime Requirements
• Embedded Applications often must
- respond to events within a definite time
- be executed within a definite periode
• Period Pd: Time within the task can be executed
Initiation I: Start of task (within period)
Execution Time ET: Time the task needs
Deadline D: End of periode
• A Delay between the begin of the period and the
initiation can be caused by the Interrupt Latency
Time (TILZ) or others.

25.03.2021 Prof. Dr. Peter Nauth 89

 Realtime Requirements

action

Period

Delay Execution Time Buffer

25.03.2021 Prof. Dr. Peter Nauth 90

 Solution 1: Interrupts
• Realtime Tasks are started by Interrupts (IR),
requested by e.g. sensors or other tasks
• IR with higher priority interrupts those with lower
priority (Highest Priority: P=0)
• Execution Time (ET) must be limited that Realtime
Period Requirements are met:
• ET(P=0): Pd(P=0) >= ET(P=0) + Delay
ET(P=1): Pd(P=1) >= ET(P=1) + Delay +
n*(ET(P=0) + Delay)
with n= max possible IRs during Pd(P=1)

25.03.2021 Prof. Dr. Peter Nauth 91

 Solution 2: RTOS (Realtime OS)
• Priority Based Preemptive Scheduling (PPS)
Tasks with higher priority interrupt tasks with
lower priority.
+ High priority tasks are executed within their
period in any case

• Earliest Deadline First (EDF)

Tasks with earliest deadline are executed first.
+ As much as possible tasks can be executed
within their periods

25.03.2021 Prof. Dr. Peter Nauth 92

 Solution 3: FPGA

• Field Programmable Gate Arrays

• Program is executed in digital circuits

-> Execution time is almost = 0
-> Realtime demands met

25.03.2021 Prof. Dr. Peter Nauth 93

 Realtime Programming

25.03.2021 Prof. Dr. Peter Nauth 94

 Project Structure

PC-Program can
be placed here

RT-Program

FPGA-Program

25.03.2021 Prof. Dr. Peter Nauth 95

 Realtime Programming

Set Period

25.03.2021 Prof. Dr. Peter Nauth 96

 Configuration of Express-VI

25.03.2021 Prof. Dr. Peter Nauth 97

 I/O Configuration &FPGA-Program

Set Period

25.03.2021 Prof. Dr. Peter Nauth 98

 Smart Cameras
and
Computer Vision

25.03.2021 Prof. Dr. Peter Nauth 99

 Cameras
• 2-D cameras acquire
- a colour image C(x,y) = (R(x,y), G(x,y), B(x,y))
e.g. Logitec C310 (Resolution 1 MP, 1280x720 p)

• 3-D Kinect cameras acquire

- a colour image C(x,y) and
- a depth image z(x,y)
e.g. Kinect

25.03.2021 Prof. Dr. Peter Nauth 100

Smart Camera
 Raspberry Pi Camera Module
Camera Module 2 Specification

Resolution 8 Megapixels

Video modes 640x480p

Sensor 3280 x 2464 pixels

Pixel size 1.12 μm x1.12μm

Fixed focus 1m to infinity

Horizontal Field Of 62.2 degrees

View

Vertical Field Of 48.8 degrees

View

Focal length 3.04 mm +/- 0.01 mm

25.03.2021 Prof. Dr. Peter Nauth 102

Smart Camera
 RGB Colour Space

[1] [1]

25.03.2021 Prof. Dr. Peter Nauth 104

 HSI Colour Space

[1]

25.03.2021 Prof. Dr. Peter Nauth 105

 Image Formation

z is distance,
i.e. z = d

(1)
 Image Formation
• xc = f x/z yc = f y/z x,y: Object
xc,yc: Imaged Object

• xc = u h yc = v h h: pixel width and hight

f: focal length

• u = s x/z v = s y/z s = f/h; u,v: pixel

• (x,y,z) = (u z/s, v z/s, z)

Image Formation: View Angle

tan Θ = x / z Camera
z
Θ
= u z / (z s) x

=u/s Object

Θ = arctan (u/ s)
 Distance by View Angle
• The camera is mounted on the hight h.
• The object is recognized at pixel index v
related to the main camera axis
(parallel to the floor)
Camera
• tan Θ = hcamera / z and Θ
z

tan Θ = v / s hcamara

• z=hs/v Floor
Bottom of Object
 Distance by Stereo Vision
Stereo Vision uses images taken by two
cameras mounted in a distance a next to
each other. The same point P(x,y,z) is imaged
to different pixels in camera 1 (u1, v) and
camera 2 (u2, v).
u1 = s x/z and u2 = s (x+a)/z
->u2 – u1 = s a/z
z = s a/(u2 - u1)
Distance by Multiple Mesurements
• Measurement of the same object in
different distances z and z + Δz (Δz is
known, e.g. step length of a robot)

• u1 = s x/z and u2 = s x/(z + Δz )

->u2/u1 = 1 + Δz /z

• z = Δz /(u2/u1 – 1)
 Distance by Structured Illumination:
Kinect Camera

RGB Resolution: 640 x 480 Pixel

RGB -> C(x,y) = (R(x,y), G(x,y), B(x,y))
IR -> z(x,y)

25.03.2021 Prof. Dr. Peter Nauth 112

 Kinect Camera
• Depth image z(x,y)

• Colour image C(x,y)

25.03.2021 Prof. Dr. Peter Nauth 113

Kinect: Depth Image
 Computer Vision (CV)
• Result of Image Processing / Computer Vision:
- Object ID
- Object Position x,y or x, y, z
- Object Orientation Θ

• Methods:
- Feature based Image Processing
- Pattern Matching
- Deep Learning
25.03.2021 Prof. Dr. Peter Nauth 115
 Computer Vision (CV)

Libraries for Computer Vision:

• Programming in C, C++, Python:
Open CV
• Programming in LabVIEW:
IMAQ Vision

25.03.2021 Prof. Dr. Peter Nauth 116

 Image Acquisition from Camera
Open CV - Code

camera.start_preview()
sleep(5)
camera.capture(
‘/home/pi/Desktop/image.jpg’)
camera.stop_preview()

25.03.2021 Prof. Dr. Peter Nauth 117

 Transform RGB to Greyscale
Open CV - Code

Img =
cv2.imread(‘/home/pi/Desktop/image.jpg’,0)

Grey =
cv2.cvtColor(Img, cv2.CV_BGR2GRAY)

25.03.2021 Prof. Dr. Peter Nauth 118

 Image Acquisition from 2-D Camera
LabVIEW Code

25.03.2021 Prof. Dr. Peter Nauth 119

 Image Acquisition from File
LabVIEW Code

25.03.2021 Prof. Dr. Peter Nauth 120

 Pattern Matching
• Pattern Matching:
• Create a reference pattern, i.e.
take an image of the object to be detected
• Take an image of the scene to be analyzed
• Move the reference pattern over the image
and compare the corresponding pixels
• If most pixels are similar -> object detected

25.03.2021 Prof. Dr. Peter Nauth 121

 Pattern Matching

[1]

25.03.2021 Prof. Dr. Peter Nauth 122

 Pattern Matching
• Colour image C(x,y)

• Reference pattern for

Pattern Matching

25.03.2021 Prof. Dr. Peter Nauth 123

Pattern Matching:
 Featurebased CV

LabVIEW Colour Particle

IMAQ Grab Label Classifier
Threshold Analysis
Functions
Source Image Segmented Image Geometric Features Class

25.03.2021 Prof. Dr. Peter Nauth 125

 Featurebased CV

/ Machine Decision Making

25.03.2021 Prof. Dr. Peter Nauth 126

 Featurebased CV/ Preprocessing
u, v u v

[1]

25.03.2021 Prof. Dr. Peter Nauth 127

 Featurebased CV/ Preprocessing
Rectangular Lowpass Filter

f(u,v) f(u,v)
25.03.2021 Prof. Dr. Peter Nauth 128
 Featurebased CV/ Preprocessing
Rectangular Binominal Filter

f(m,n) f(m,n)

25.03.2021 Prof. Dr. Peter Nauth 129

 Featurebased CV/ Preprocessing
Binominal Coefficients

25.03.2021 Prof. Dr. Peter Nauth 130

 Featurebased CV/ Preprocessing
Noise Reduction by Low Pass and Median Filter

25.03.2021 Prof. Dr. Peter Nauth 131

 Featurebased CV/ Preprocessing
Median Filter (Rank Order Algorithm)

25.03.2021 Prof. Dr. Peter Nauth 132

 Featurebased CV/ Preprocessing
Laplace Highpass Filter

[1]

f(u,v)

25.03.2021 Prof. Dr. Peter Nauth 133

 Featurebased CV/ Preprocessing
Sobel Highpass Filter

I(u,v) filtered with fu

results in Ou(u,v)
I(u,v) filtered with fv
results in Ov(u,v)
fu fv [1]

Magnitude Image: O(u,v) = sqrt(Ou(u,v)^2 + Ov(u,v)^2)

Direction Image : α(u,v) = tan(Ov(u,v)/Ou(u,v))

25.03.2021 Prof. Dr. Peter Nauth 134

 Featurebased CV/ Preprocessing
• Erosion:
Goal: Eliminate small objects
Algorithmn: Rank Order Operation with smallest value
Disadvantage: Large objects get smaller.
• Dilatation:
Goal: Eliminate small holes in large objects
Algorithmn: Rank Order Operation with largest value
Disadvantage: Large objects get larger.
• Opening:
Goal: Eliminate small objects without affecting large objects
Algorithmn: 1.Erosion, 2. Dilatation
• Closing:
Goal: Eliminate small holes in large objects without affecting
large objects.
Algorithmn: 1. Dilatation, 2.Erosion

25.03.2021 Prof. Dr. Peter Nauth 135

 Featurebased CV/ Segmentation
1. Binarization
Dividing the image in foreground pixels
(=pixels representing objects) which are set
to 1 and background pixels which are set to 0
2. Labelling
Identifying groups (clusters) of foreground
pixels which belong together and setting
them to the object number (=label). The
pixels of the 1st cluster are set to 1, of the 2nd
cluster to 2, etc.
25.03.2021 Prof. Dr. Peter Nauth 136
 Preprocessing and Binarization
Open CV - Code

Img = cv2.imread(‘/home/pi/Desktop/image.jpg’,0)
kernel = np.ones((5,5),np.float32)/25
Img_filter = cv2.filter2D(Img,-1,kernel)
Thresh1 = cv2.threshold(Img_filter,90,225,
cv2.THRESH_BINARY_INV)

25.03.2021 Prof. Dr. Peter Nauth 137

 Featurebased CV/ Features
1. Region-based features derived from the pixel values of a region such as
Mean of Intensity (Brightness), Saturation or Hue (Colour)
Variance of Intensity (Brightness), Saturation or Hue (Colour)

2. Geometric features derived from the segment contours such as

Circularity factor
Ratio of main axis
Histogram of Freeman Code
Moments and Center of mass

3. Hough features derived from a contour image (Canny operator) such as

Number of lines or angle between lines
Number and / or radius of circles

4. Haar like features.

25.03.2021 Prof. Dr. Peter Nauth 138

 Featurebased CV / Center of Mass

sx = m10 / m00
sy = m01 / m00

Binary Image: f(x,y) = 1 if f(x,y) is object pixel,

else f(x,y) = 0
-> m00 is number of pixels of object

25.03.2021 Prof. Dr. Peter Nauth 139

 Features / Open CV - Code

contours,hierarchy =
cv2.findContours(thres,1,2)
cnt = contours[0]
M = cv2.moments(cnt)
cx = int(M[‘m10’]/M[‘m00’])
cy = int(M[‘m01’]/M[‘m00’])

25.03.2021 Prof. Dr. Peter Nauth 140

 Features / LabVIEW Code

IMAQ Particle Analysis

25.03.2021 Prof. Dr. Peter Nauth 141

 Machine Decision
Making

25.03.2021 Prof. Dr. Peter Nauth 142

 Methods of Learning
• No learning
e.g. Comparison of features with threshold
• Supervised Learning
e.g. Statistical Classifier Learning
• Unsupervised Learning
e.g. Cluster Analysis
• Reinforcement Learning (Forced Feedback)
e.g. Support Vector Machines (SVM)

25.03.2021 Prof. Dr. Peter Nauth 143

 Statistical Classifier

Feature vector of Assignment of feature

unknown object Classification vector x to a class
Phase i
i
x Ki
Known class to which
Learning feature vector x belongs to
Phase i
Feature vectors of
known objects
from training set

25.03.2021 Prof. Dr. Peter Nauth 144

Featurebased CV / Statistical Classifier

Discrimination function
of class i

P ( x |  i ) P ( i ) Bayes Theorem
Di (x) =
P( x)

25.03.2021 Prof. Dr. Peter Nauth 145

Featurebased CV / Statistical Classifier
A priori likelihood
P(i)
A posteriori likelihood
P(i|x))

P(i|x)) Diagnostic direction

Causal direction
P(x|i)

25.03.2021 Prof. Dr. Peter Nauth 146

Featurebased CV / Statistical Classifier
Distribution: F F F F
T F F F
T T F F
T T T F

Likelihoods for distribution:

P(Ω1) = 8/16 = 0.5; P(Ω2) = 8/16 = 0.5

P(x=T) = 6/16 = 0.375; P(x=F) = 10/16 = 0.625
P(x=T | Ω1) = 1/8 = 0.125; P(x=T | Ω2) = 5/8 = 0.625
P(x=F | Ω1) = 7/8 = 0.875; P(x=F | Ω2) = 3/8 = 0.375

25.03.2021 Prof. Dr. Peter Nauth 147

Featurebased CV / Statistical Classifier
Example:Baysian inference

A patient reports to a physician that he has a stiffed neck (symptom/ feature = s

and asks for a diagnosis, i.e. for the likelihood of having meningistis (class = m).
In order to solve this problem (1 feature, 1 class), data from medical research are available

P(s|m) = 0,7 Probab. of stiffed neck at patients with confirmed meningitis

P(m) = 1/50.000 Probability that a human has developed meningitis
P(s) = 0,01 Probability that a human shows the sypmtom stiffed neck.

Calculate the likelihood that the patient has developed meningistis

under the presence of the symptom/ feature s, i.e. calculate P(m|s).

The right answer is: P(m|s) = 0,0014.

25.03.2021 Prof. Dr. Peter Nauth 148

Featurebased CV / Statistical Classifier

P ( x |  i ) P ( i ) Bayes Theorem
Di (x) =
P( x)

Di(x) = P(x|i) Simplification

25.03.2021 Prof. Dr. Peter Nauth 149

Featurebased CV / Statistical Classifier

Multivariate
Gaussian
Distribution

i : Mean vector of features from learning phase

Ki : Co-variance matrix from learning phase

25.03.2021 Prof. Dr. Peter Nauth 150

Featurebased CV / Statistical Classifier

[1]

25.03.2021 Prof. Dr. Peter Nauth 151

Featurebased CV / Statistical Classifier
Multivariate
Gaussian
Distribution

Applying ln and neglecting the term independent of x yields:

Bayes Classifier =
Negative Squared
Mahalanobis Distance
Naive Bayes Classifier:
Feature are statistically independent, i.e. K is doagonal matrix

25.03.2021 Prof. Dr. Peter Nauth 152

Featurebased CV / Statistical Classifier

Decision Rule for Bayes Classifier

• Calculate Di (x) for each class

• Classify for the class with the highest Di (x)

classified = arg max Di (x)

25.03.2021 Prof. Dr. Peter Nauth 153

Featurebased CV / Statistical Classifier

Other statistical classifier

• Simplified Bayes classifier

• Nearest neighbor classifier

= Squared Euclidian Distance

25.03.2021 Prof. Dr. Peter Nauth 154

 Featurebased CV / Statistical Classifier

25.03.2021 Prof. Dr. Peter Nauth 155

 Featurebased CV / SVM

25.03.2021 Prof. Dr. Peter Nauth 156

 Featurebased CV / SVM
• Discrimination function:
D(x) = w T x

• Separation plane, i.e. D(x)=0:

x2 = -(w1/w2)x1 – w0/w2

• x = (x2, x1, 1) and w = (w2, w1, w0)

• Decision rule (D(x) = distance of x to separation plane):

If D(x) >= 0 -> Classification for class 1 (pos. outcome)
If D(x) < 0 -> Classification for class 2 (neg. outcome)

25.03.2021 Prof. Dr. Peter Nauth 157

 Featurebased CV / SVM Learning

Likelihood of false
classifications for
a given wi

25.03.2021 Prof. Dr. Peter Nauth 158

 Featurebased CV / SVM
• Learning aims to minimize the number of false
classifications, i.e. minimizing E{|wT xfalse|}

• 1st gradient optimization method applied:

wi+1 = wi – d/dw (E{|wT xfalse|})

• Classification rule: in case of a false classification

wi+1 = wi – x if false positive classification
wi+1 = wi + x if false negative classification
25.03.2021 Prof. Dr. Peter Nauth 159
 References
[1] Peter Cork:
Robotics, Vision and Control
ISBN 978-3-642-20143-1
[2] Sebastian Thrun and Wolfram Burgard
Probablisitc Robotics
ISBN 978-0-262-20162-9
[3] Stuart Russell and Peter Norvig
Artificial Intelligence
ISBN 978-0-136-04259-4

01.08.2014 Robotics / Prof. Nauth 160

 References
[4] E. Ivanjko, A.Kitanov, I. Petrovic:
Model based Kalman Filter Mobile Robot Self -
Localization
[5] Stuart Russell, Peter Norvig
Artificial Intelligence
[6] J.J.Craig
Introduction to Robotics
Pearson Prentice Hall, 2005

01.08.2014 Robotics / Prof. Nauth 161

 References
[7] Gonzalez R, Woods R (2008):
Digital image processing. Prentice Hall,
Englewood Cliffs
[8] Gary Bradski, Adrian Kaehler:
Learning Open CV
ISBN: 978-1-44-931465-1
[9] Johannes Steinmüller:
Bildanalyse, Springer Verlag
ISBN: 978-3-54-079742-5

01.08.2014 Robotics / Prof. Nauth 162

 References
[10] Barrett S., Thornton. M.
Embedded Systems Design with the Atmel
AVR Microcontroller ,
Morgan and Claypool Publishers, 2009
[11] Lentin Joseph
Mastering ROS for Robotics Programming
Packt Publishing Ltd., Birmingham, 2015
ISBN 978-1-78355-179-8

01.08.2014 Robotics / Prof. Nauth 163

 References
[12] ATmega640/V-1280/V-1281/V-2560/V-
2561/V [DATASHEET], Atmel
[13] NI myRIO-1900 User Guide and
Specifications , National Instruments
[14] https://www.raspberrypi.org/
[15] www.robot-
electronics.co.uk/htm/srf08tech.html

01.08.2014 Robotics / Prof. Nauth 164

 References
[16] http://www.ni.com/academic/students/
learn-labview/

25.03.2021 Prof. Dr. Peter Nauth 165

 Exercise: Position Estimation
Technical Data Logitech C310 camera
• Aspect Ratio: 16 : 9 (x:y)
• # Pixels: 1280 x 720 (HD)
• Focal Length: f = 4.4 mm
• Field of View: FOV = 60° (horizontal, in x-
direction)

25.03.2021 Prof. Dr. Peter Nauth 166

 Exercise: Position Estimation
Task 1
• Derive the formular in order to calculate the
real position (x, y) and view angles (Θx, Θy) from
the pixel co-ordinates (u,y) of an object imaged
with C310.
• The distance z is measured by a proximity
sensor.

25.03.2021 Prof. Dr. Peter Nauth 167

 Exercise: Position Estimation
Solution (Pixel width and hight are equal):
• Pixel width = CCD-Width/#Pixel
= 2 f tan(FOV/2)/#Pixel
= 4.4mm/1289 = 3.96 µm
• s = f/h = 4.4mm/ 3.4375µm = 1111
• x = u z /s = u * z / 1111
• y = v z /s = v * z / 1111
• Θx = arctan(u/s) = arctan(u/1111)
• Θy = arctan(v/s) = arctan(v/1111)
25.03.2021 Prof. Dr. Peter Nauth 168
 Exercise: Position Estimation
Task 2:
Calculate the position and view angles of an
object imaged with C310 at
• u = 320 pixel (horizontal), v = 360 (vertical)
• z = 2 m (measured by a proximity sensor)

25.03.2021 Prof. Dr. Peter Nauth 169

 Exercise: Position Estimation

Solution (Pixel width and hight are equal):

• x = u z /s = 320 * 2m / 1111 = 0.5 m
• y = v z /s = 360 * 2m / 1111 = 0.8 m
• Θx = arctan(u/s) = arctan(320/1111) = 16°
• Θy = arctan(u/s) = arctan(360/1111) = 18°

25.03.2021 Prof. Dr. Peter Nauth 170

 Exercise: Position Estimation
Remarks:
• Pixel co-ordinates are usually related to camera
co-ordinate systems with an origin in the left
upper corner of the image plane.
• If the real position and the angles are to be
calculated realative to the center of the image, a
shift of the origin is necessary.

25.03.2021 Prof. Dr. Peter Nauth 171

 Exercise: SRF08 setting
• Task: Set max gain to 524 (formulars: p. 51)
• Q1: Which value do you have to write to reg 1?
• Q2: At which distance dg is the gain limited to 524?
• Q3: Limit max. range to dg of Q2. Which value do
you have to write to reg 2 (range register)?

25.03.2021 Prof. Dr. Peter Nauth 172

 Exercise: SRF08 setting
Task:
Write a program that
- sets max gain to 524, i.e. write 0x1C to reg. 1 and
- Sets max range to 333 mm, i.e. Write 7 to reg. 2.
Adress of sensor is E4.
Use the function check()

25.03.2021 Prof. Dr. Peter Nauth 173

 Exercise: SRF08 setting
Solution (similar to p.57 – p.60):

// Set max gain to 524

TWCR = (1<<TWINT)|(1<<TWSTA)|(1<<TWEN);
Check(START);
TWDR = 0xE4;
TWCR = (1<<TWINT)|(1<<TWEN);
Check(MT_SLA_ACK);

25.03.2021 Prof. Dr. Peter Nauth 174

 Exercise: SRF08 setting
TWDR = 0x01;
TWCR = (1<<TWINT)|(1<<TWEN);
Check(MT_DATA_ACK);
TWDR = 0x1C;
TWCR = (1<<TWINT)|(1<<TWEN);
Check(MT_DATA_ACK);

25.03.2021 Prof. Dr. Peter Nauth 175

 Exercise: SRF08 setting
// Set max range to 333 mm
TWCR = (1<<TWINT)|(1<<TWSTA)|(1<<TWEN);
Check(REP_START);
TWDR = 0xE4;
TWCR = (1<<TWINT)|(1<<TWEN);
Check(MT_SLA_ACK);

25.03.2021 Prof. Dr. Peter Nauth 176

 Exercise: SRF08 setting
TWDR = 0x02;
TWCR = (1<<TWINT)|(1<<TWEN);
Check(MT_DATA_ACK);
TWDR = 0x07;
TWCR = (1<<TWINT)|(1<<TWEN);
Check(MT_DATA_ACK);
TWCR = (1<<TWINT)|(1<<TWST0)|(1<<TWEN);
Check(STOP);
25.03.2021 Prof. Dr. Peter Nauth 177
 Exercise: Analog Sensor
Transfer Function of Analog Sensor:
Vin = 4 Vcm /(d – 2 cm) – 0.5 V
AD Coverter: 10 bit, Vref = 5 V
-> ADC = (Vin * 2**10)/ 5V = Vin * 204.8/V
Q: What is the C-Instruction in order to calculate
distance d from digitized value stored in reg. ADC

25.03.2021 Prof. Dr. Peter Nauth 178

 Exercise: Analog Sensor, Solution
Transfer Function of Analog Sensor:
Vin = a/(d – b) + c
AD Coverter: 10 bit, Vref = 5 V
-> ADC = Vin * 204.8/V = Vin * k
ADC = Vin * k = (a/(d – b) + c) * k
ADC – ck = ak/(d – b)
ak /(ADC – ck ) = (d – b)
d = ak /(ADC – ck ) + b

25.03.2021 Prof. Dr. Peter Nauth 179

 Exercise: Analog Sensor, Solution
d = ak /(ADC – ck ) + b
d = (4 Vcm * 204.8/V) / (ADC + 0.5V*204.8/V) + 2 cm
d = 819.2 cm / (ADC + 102.4) + 2 cm

d = 819.2 / (ADC + 102.4) + 2; // Distance in cm

25.03.2021 Prof. Dr. Peter Nauth 180

 Exercise: RTOS
• Task 0: Priority = 0
Pd(0) = 10 ms, ET(0) = 3 ms, Delay = 1 ms
• Task 1: Priority = 1
Pd(1) = 20 ms, ET(1) = 5 ms, Delay = 1 ms, n = 2
• Q1: Can both tasks be executed in realtime if PPS
is used?
• Q2: Task 1 has been started at t = 0. After 4 ms
Task 0 is requested. Will an EDF with PPS system
continue running Task 1 or interrupt it?
25.03.2021 Prof. Dr. Peter Nauth 181
 Exercise: RTOS, Solution
• Task 0: Priority = 0
Pd(0) = 10 ms, ET(0) = 3 ms, Delay = 1 ms
• Task 1: Priority = 1
Pd(1) = 20 ms, ET(1) = 5 ms, Delay = 1 ms, n = 2
• Q1: Can both tasks be executed in realtime if PPS used
• A: Pd(0) > ET(0) + Delay
10 ms > 3 ms + 1 ms ok, RT-condition is met Task 0
Pd(1) > ET(1) + Delay
20 ms > 5 ms + 1 ms + 2 * (3 ms + 1 ms)
ok, RT-condition is met for Task 1
25.03.2021 Prof. Dr. Peter Nauth 182
 Exercise: RTOS
• Q2: Task 1 has been started at t = 0. After 4 ms Task 0
is requested. Will an EDF with PPS interrupt Task 1?
• A: t=4ms -> Remaining time trm1 for Task 1 is
trm1= (5ms + 1ms) – 4ms = 2 ms
Remaining time trm0 for Task 0 (if it‘d interrupt) is
trm0 = (3ms + 1ms) = 4 ms
• trm1 + (delay + ET(0)) = 6 ms, i.e. Task 0 needs 6 ms
after request if it does not interrupt Task 1
6 ms < Pd(0) , 6 ms < 10ms -> Task 0 will be finished
before deadline,
25.03.2021 Prof. Dr. Peter Nauth 183
 Exercise: RTOS
• Decision Step 1 (based on EDF):
Task1‘s deadline is shorter for 2 ms < 4 ms
-> Continue Task 1 running but only if Task 0 can
be executed in realtime
• Decision Step 2 (based on PPS):
6 ms < Pd(0) , 6 ms < 10ms -> Task 0 will be
finished before deadline, if Task 1 continues
-> Task 1 will not be interrupted and task 0 must
wait.
25.03.2021 Prof. Dr. Peter Nauth 184
 Exercise: RTOS
EDF combined with PPS
Earliest Run Time End (used by EDF in this exercise)
Run Time Task 1

Task 1

Period Task 1

Earliest Deadline as alternative:

Run Time Task 0
EDF in narrower sense
Task 0

Period Task 0

Request of Task 0

t/ms
4 8 12 16 20

25.03.2021 Prof. Dr. Peter Nauth 185

 Exercise: RTOS
PPS

Run Time Task 1

Task 1 Task 1

Period Task 1

Run Time Task 0

Task 0

Period Task 0

Request of Task 0

t/ms
4 8 12 16 20

25.03.2021 Prof. Dr. Peter Nauth 186

 Exercise: RTOS
• EDF combined with PPS
Run Time Task 1 = 6 ms < Pd(1)
Run Time Task 0 = 6 ms < Pd(0)
• PPS
Run Time Task 1 = 10 ms < Pd(1)
Run Time Task 0 = 4 ms < Pd(0)

25.03.2021 Prof. Dr. Peter Nauth 187

 Max. Gain

[15]

25.03.2021 Prof. Dr. Peter Nauth 188

 Sonic Power / Intensity Losses
• Attenuation: Losses due to conversion of sonic
power to heat.
• P(s) = P(0) e^(- 2 α f s)
with P(s): power
s: way sonic wave has traveled
f: frequency of sonic wave, e.g. 40 kHz
α: attenuation coefficient
• α = 0.19 Np/(Mhz cm) for air
• Np is relative value like dB, but related to base e
25.03.2021 Prof. Dr. Peter Nauth 189
 Sonic Power / Intensity Losses
• Exercise on attenuation:
f = 40 kHz, d = 2m,
• P(s) = P(0) e^(- 2 α f s)
= P(0) e^(- 2 * 0.19/(Mhz cm) * 0.04 MHz 200 cm)
= P(0) * e^(-3.04)
= 0.048 P(0)

25.03.2021 Prof. Dr. Peter Nauth 190

 Sonic Power / Intensity Losses
• Increase of beam width:
Beam width widens with distance due to beam
angle is larger than 0°.
• Hence, illuminated are increases and intensity
decreases with distance.

25.03.2021 Prof. Dr. Peter Nauth 191

 Sonic Power / Intensity Losses
Beam Pattern SRF08 α
(Radiation Pattern):
P(α)/P(α=0°) in dB

Beam Angle: -6db

αb = α-6db (i.e.1/4 Power)

= 2 * 27.5° = 55°

09.01.2019 Intelligent Sensors / Prof. Dr. Peter Nauth 192

 Sonic Power / Intensity Losses
P(z) of
wave front
Transducer

Δs z
Area of P(z)
s modelled as
surface of
s = z – Δs sphere with
s≈z if Δs << z center z = 0)
09.01.2019 Intelligent Sensors / Prof. Dr. Peter Nauth 193
 Sonic Power / Intensity Losses
Definitions
Pt: Power transmitted to object, Pt = P(s)
It: Intensity transmitted to object (Power/Area)
(Wave front propagates as spheric surface)
Pr: Power received at receiver plane
Ir: Intensity received at receiver plane
Pd: Power measured by receiver transducer
R0: Reflectivity of object
A0: Area of object
Ad: Area of transducer surface
09.01.2019 Intelligent Sensors / Prof. Dr. Peter Nauth 194
 Sonic Power / Intensity Losses
Power detected (received) by transducer
It = Pt / 4 π s² (Sphere Wave Front assumed)
Pr = It R A0 (Power in receiver plane = Power reflected)
Ir = Pr / 4 π s² (Sphere839 3057 4506
Wave Front assumed)
Pd = Ir Ad (Power detected by transducer)

Pd = Pt R A0 Ad / 16 π² s4

σ = 4 π x² Pr / Pt is reflection cross section

09.01.2019 Intelligent Sensors / Prof. Dr. Peter Nauth 195
 Exercise: SRF08 setting
Solution (similar to p.57 – p.60):

// Change Address from E0 to E4

TWCR = (1<<TWINT)|(1<<TWSTA)|(1<<TWEN);
Check(START);
TWDR = 0xE0;
TWCR = (1<<TWINT)|(1<<TWEN);
Check(MT_SLA_ACK);

25.03.2021 Prof. Dr. Peter Nauth 196

 Exercise: SRF08 setting
TWDR = 0x00;
TWCR = (1<<TWINT)|(1<<TWEN);
Check(MT_DATA_ACK);
TWDR = 0xA0;
TWCR = (1<<TWINT)|(1<<TWEN);
Check(MT_DATA_ACK);

25.03.2021 Prof. Dr. Peter Nauth 197

 Exercise: SRF08 setting
TWDR = 0xAA;
TWCR = (1<<TWINT)|(1<<TWEN);
Check(MT_DATA_ACK);
TWDR = 0xA5;
TWCR = (1<<TWINT)|(1<<TWEN);
Check(MT_DATA_ACK);

25.03.2021 Prof. Dr. Peter Nauth 198

 Exercise: SRF08 setting
TWDR = 0xE4;
TWCR = (1<<TWINT)|(1<<TWEN);
Check(MT_DATA_ACK);

TWCR = (1<<TWINT)|(1<<TWSTO)|(1<<TWEN);

25.03.2021 Prof. Dr. Peter Nauth 199