iPhone Programming

CS4347 Sound and Music Computing Zhou Yinsheng (yzhou86@comp.nus.edu.sg) Jan 22nd, 2011

Acknowledgement: Some materials are borrowed from the course of iPhone Application Development in Stanford University.
Total 67 Pages 1

Tools to create Apps in iPhone

Xcode

Interface Builder

Mac
Instruments Simulator/device
Total 67 Pages 2

Sound and Music Computing

We use iPhone to make music! Mobile Music Group

MOGFUN MOGCLASS MOGHEALTH

Total 67 Pages

MOGCLASS

MOGCLASS DEMO

Total 67 Pages

MOGCLASS

Do you want to create your own iPhone App or develop novel user interface for MOGCLASS?

Total 67 Pages

You need to know

Language: Objective-C and C/C++ Objective Oriented Programming Programming and testing in Xcode Digital Signal Processing basics Assembly Code (ARM) Computer Network Etc

Total 67 Pages

Today we will cover

Objective-C basics Building an Application Sensor programming (accelerometer, GPS, microphone) Audio Libraries

There are a lot of materials about iphone online. What we will cover today is more oriented for your final project.
Total 67 Pages 7

How to learn iPhone Programming

Recommended Book: None! Well use Apple documentation iOS Developer Center:

http://developer.apple.com/devcenter/ios/index.action

Download the sample code and learn.

Write the code by yourself Trial and error.

Total 67 Pages

iPhone SDK Technology Architecture

Cocoa Touch Media
Multi-Touch Events Multi-Touch Controls Accelerometer View Hierarchy Localization Core Audio OpenAL Audio Mixing Audio Recording Video Playback Collections Address Book Networking File Access SQLite OS X Kernel Mach 3.0 BSD Sockets Security Alerts Web View People Picker Image Picker Camera JPG, PNG,TIFF PDF Quartz (2D) Core Animation OpenGL ES Core Location Net Services Threading Preference URL utilities Power Mgmt Keychain Certificates File System Bonjour Total 67 Pages 9

Core Services
Core OS

Outlines
Objective-C basics Building an Application Sensor programming (accelerometer, GPS, microphone) Audio Libraries

Total 67 Pages

10

OOP Vocabulary
Class: defines the grouping of data and code, the type of an object. Instance: a specific allocation of a class. Method: a function that an object knows how to perform. Instance Variable (or ivar): a specific piece of data belonging to an object.

Total 67 Pages

11

OOP Vocabulary

Encapsulation
Keep implementing private and separate from interface

Polymorphism
Different objects, same interface

Inheritance
Hierarchical organization, share code, customize or extend behaviors

Total 67 Pages

12

Objective-C

Strict superset of C

A very simple language, but some new syntax Single inheritance, classes inherit from one and only one superclass Protocols define behavior that cross classes Dynamic runtime Loosely typed, if youd like
Total 67 Pages 13

Mix C with ObjC Or even C++ with ObjC (usually referred to as ObjC++)

Class and Instance Methods

Instance respond to instance methods

-(id)init; -(float)height; -(void)walk;

Classes respond to class methods

+(id)alloc; +(id)person; +(Person*)sharedPerson;

Total 67 Pages

14

Message Syntax
[receiver message]; [receiver message:argument]; [receiver message:arg1 andArg: arg2]

Total 67 Pages

15

Terminology

Message expression
[receiver method:argument]

Message
[receiver method:argument]

Selector
[receiver method:argument]

Method
The code selected by a message.

Total 67 Pages

16

Dot Syntax
Objective-C 2.0 introduced dot syntax Convenient shorthand for invoking accessor methods

float height = [person height]; float height = person.height; [person setHeight: newHeight]; person.height = newHeight;

Follows

the dots

[[person child] setHeight:newHeight]; //exact the same as person.child.height = newHeight;

Total 67 Pages 17

Dynamic and static typing

Dynamically-typed object
id anObject

just id Not id * (unless you really, really mean it)

Statically-typed object
Person *anObject

Objective-C provides compile-time, not runtime, type checking Objective-C always uses dynamic binding
Total 67 Pages 18

Selectors identify methods by name

A selector has type SEL

SEL action = [button action]; [button setAction: @selector(start:)];

Conceptually similar to function pointer Selectors include the name and all colons, for example:

-(void) setName:(NSString*)name age:(int)age;

would have a selector:

SEL sel = @selector(setName:age:);

Total 67 Pages

19

Working with selectors

You can determine if an object responds to a given selector

id obj; SEL sel = @selector(start:) if ([obj respondsToSelector:sel]){ [obj performSelector: sel withObject:self]; }

This sort of introspection and dynamic messaging underlies many Cocoa design patterns
-(void)setTarget:(id)target; -(void)setAction:(SEL)action;
Total 67 Pages 20

Working with Classes

You can ask an object about its class

Class myClass = [myObject class]; NSLog(@"My class is %@", [myObject className]);

Testing for general class membership (subclasses included):

if ([myObject isKindOfClass:[UIControl class]]) { // something }

Testing for specific class membership (subclasses excluded):

if ([myObject isMemberOfClass:[NSString class]]) { // something string specific }
Total 67 Pages 21

Working with Objects

Identity v.s. Equality Identitytesting equality of the pointer values

if (object1 == object2) { NSLog(@"Same exact object instance"); }

Equalitytesting object attributes

if ([object1 isEqual: object2]) { NSLog(@"Logically equivalent, but may be different object instances"); }

Total 67 Pages

22

-description

NSObject implements -description

- (NSString *)description;

Objects represented in format strings using %@ When an object appears in a format string, it is asked for its description
[NSString stringWithFormat: @The answer is: %@, myObject];

You can log an objects description with:

NSLog([anObject description]);

Your custom subclasses can override description to return more specific information
Total 67 Pages 23

Foundation Framework

Foundation Classes
NSObject
String
NSString / NSMutableString

Collection
NSArray / NSMutableArray NSDictionary / NSMutableDictionary NSSet / NSMutableSet NSNumber

Others
NSData / NSMutableData NSDate / NSCalendarDate
Total 67 Pages 24

More OOP Info

Tons of books and articles on OOP Objective-C 2.0 Programming Language

Total 67 Pages

25

Outlines
Objective-C basics Building an Application Sensor programming (accelerometer, GPS, microphone) Audio Libraries

Total 67 Pages

26

Anatomy of an Application

Compiled code
Your code Frameworks

Nib files
UI elements and other objects Details about object relations

Resources (images, sounds, strings, etc) Info.plist file (application configuration)

Total 67 Pages 27

App Lifecycle

Total 67 Pages

28

App Lifecycle
Main function UIApplicationMain which Info.plist to figure out what nib to load. MainWindow.xib contains the connections for our application. AppDelegate ViewController.xib View handle UI events

Total 67 Pages 29

Model-View-Controller

Total 67 Pages

30

Communication and MVC

Total 67 Pages

31

Model
Manages the app data and state Note concerned with UI or presentation Often persists somewhere Same model should be reusable, unchanged in different interfaces

Total 67 Pages

32

View
Present the Model to the user in an appropriate interface Allows user to manipulate data Does not store any data

(except to cache state)

Easily reusable & configurable to display different data

Total 67 Pages

33

Controller
Intermediary between Model & View Updates the view when the model changes Updates the model when the user manipulates the view Typically where the app logic lives

Total 67 Pages

34

Outlines
Objective-C basics Building an Application Sensor programming (accelerometer, GPS, microphone, ) Audio Libraries

Total 67 Pages

35

Accelerometer

What are the accelerometers?

Measure changes in force

What are the uses? Physical Orientation vs. Interface Orientation

Ex: Photos & Safari

Total 67 Pages

36

Orientation-Related Changes

Getting the physical orientation

UIDevice class
UIDeviceOrientationDidChangeNotification

Getting the interface orientation

UIApplication class
statusBarOrientation property

UIViewController class
interfaceOrientation property

Total 67 Pages

37

Shake Undo!

UIEvent type
@property(readonly) UIEventType type; @property(readonly) UIEventSubtype subtype; UIEventTypeMotion UIEventSubtypeMotionShake

Total 67 Pages

38

Getting raw Accelerometer Data

3-axis data Configurable update frequency (10-100Hz) Sample Code (AccelerometerGraph)

Class
UIAccelerometer UIAcceleration

Protocol
UIAccelerometerDelegate
Total 67 Pages 39

Getting raw Accelerometer Data

-(void)initAccelerometer{ [[UIAccelerometer sharedAccelerometer] setUpdateInterval: (1.0 / 100)]; [[UIAccelerometer sharedAccelerometer] setDelegate: self]; } -(void)accelerometer: (UIAccelerometer*)accelerometer didAccelerate: (UIAcceleration*) acceleration { double x, y, z; x = acceleration.x; y = acceleration.y; z = acceleration.z; //process the data } -(void)disconnectAccelerometer{ [[UIAccelerometer sharedAccelerometer] setDelegate: nil]; }
Total 67 Pages 40

Filtering Accelerometer Data

Low-pass filter
Isolates constant acceleration Used to find the device orientation

High-pass filter
Shows instantaneous movement only Used to identify user-initiated movement

Total 67 Pages

41

Filtering Accelerometer Data

Total 67 Pages

42

Filtering Accelerometer Data

Total 67 Pages

43

Applying Filters

Simple low-pass filter example

#define FILTERFACTOR 0.1 Value = (newAcceleration * FILTERFACTOR) + (previousValue * (1.0 FILTERFACTOR)); previousValue = value;

Simple high-pass filter example

lowpassValue = (newAcceleration * FILTERFACTOR) + (previousValue * (1.0 FILTERFACTOR)); previousLowPassValue = lowPassValue; highPassValue = newAcceleration lowPassValue;

Total 67 Pages

44

GPS

Classes
CLLocationManager CLLocation

Protocol
CLLocationManagerDelegate
Total 67 Pages 45

Getting a Location

Starting the location service

-(void)initLocationManager{ CLLocationManager* locManager = [[CLLocationManager alloc] init]; locManager.delegate = self; [locManager startUpdatingLocation]; }

Using the event data

-(void)locationManager: (CLLocationManager*)manager didUpdateToLocation: (CLLocation*)newLocation fromLocation: (CLLocation*)oldLocation{ NSTimerInterval howRecent = [newLocation.timestamp timeIntervalSinceNow]; if(howRecent < -10) return;
if(newLocation.horizontalAccuracy > 100) return; //Use the coordinate data. double lat = newLocation.coordinate.latitude; double lon = newLocation.coordinate.longitude; }
Total 67 Pages 46

Getting a Heading

Geographic North
CLLocationDirection trueHeading

Magnetic North
CLLocationDirection magneticHeading

-(void)locationManager: (CLLocationManager *)manager didUpdateHeading:(CLHeading*)newHeading{ //Use the coordinate data. CLLocationDirection heading = newHeading.trueHeading; CLLocationDirection magnetic = newHeading.magneticHeading; }

Total 67 Pages

47

Microphone

We will cover it in the audio library

Total 67 Pages

48

Outlines
Objective-C basics Building an Application Sensor programming (accelerometer, GPS, microphone) Audio Libraries

Total 67 Pages

49

Audio Libraries
System Sound API short sounds AVAudioPlayer ObjC, simple API Audio Session - Audio Toolbox Audio Queue - Audio Toolbox Audio Units OpenAL MediaPlayer Framework

Total 67 Pages

50

AVAudioPlayer

Play longer sounds (> 5 seconds) Locally stored files or in-memory (no network streaming) Can loop, seek, play, pause Provides metering Play multiple sounds simultaneously Cocoa-style API
Initialize with file URL or data Allows for delegate

Supports many more formats

Sample Code: avTouch

Everything the AudioFile API supports

Total 67 Pages

51

AVAudioPlayer

Create from file URL or data

AVAudioPlayer *player; NSString *path = [[NSBundle mainBundle] pathForResource: ofType:]; NSURL *url = [NSURL fileURLWithPath:path]; player = [[AVAudioPlayer allow] initWithContentsOfURL:url]; [player prepareToPlay];

Simple methods for starting/stopping

If(!player.playing){ [player play]; }else{ [player pause]; }

Total 67 Pages

52

Audio Sessions

Handles audio behavior at the application, inter-application, and device levels

Ring/Silent switch? iPod audio continue? Headset plug / unplug? Phone call coming?

Sample Code: avTouch & aurioTouch

Total 67 Pages

53

Audio Sessions

Six audio session categories

Ambient Solo Ambient (Default session) Media playback Recording Play and record Offline audio processing

Total 67 Pages

54

Audio Queue

Audio File Stream Services & Audio Queue Services Supports wider variety of formats Finer grained control over playback
Streaming audio over network Cf: AVAudioPlayer(local)

Allows queueing of consecutive buffers for seamless playback

Callback functions for reusing buffers

Sample code: SpeakHere

Total 67 Pages 55

Audio Units
For serious audio processing Graph-based audio

Rate or format conversion Real time input/output for recording & playback Mixing multiple streams VoIP (Voice over Internet Protocol).

Very, very powerful. Sample code: aurioTouch

Total 67 Pages 56

Audio Units

Lowest programming layer in iOS audio stack

Real-time playback of synthesized sounds Low-latency I/O Specific audio unit features

Otherwise, look first at the Media Player, AV Foundation, OpenAL, or Audio Toolbox!

Total 67 Pages

57

Audio Units

Ex: Karaoke app in iPhone real-time input from mic Real-time output to speaker Audio Unit provides excellent responsiveness Audio Unit controls audio flow to do pitch tracking, voice enhancement, iPod equalization, and etc.

Total 67 Pages

58

OpenAL

High level, cross-platform API for 3D audio mixing

Great for games Mimics OpenGL conventions

Models audio in 3D space

Buffers: Container for Audio Sources: 3D point emitting Audio Listener: Position where Sources are heard

Sample code: oalTouch

More information: http://www.openal.org/
Total 67 Pages 59

MediaPlayer Framework
Tell iPod app to play music Access to entire music library

For playback, not processing

Easy access through MPMediaPickerController Deeper access through Query APIs

Sample code: AddMusic

Total 67 Pages 60

Others

Accelerate Framework
C APIs for vector and matrix math, digital signal processing large number handling, and image processing vDSP programming guide

Bonjour and NSStream The Synthesis ToolKit in C++ (STK) and OSC
Total 67 Pages 61

More Info for Beginners

iPhone Application Programming (Standford University - iTunes University) Dan Pilone & Tracey Pilone. Head First iPhone Development.

Total 67 Pages

62

Thank you and good luck to your final projects!

Total 67 Pages

63