IBM Software Group

Lesson 11: Concurrency and locking

(DB2 on Campus Lecture Series)

Raul F. Chong
IBM Toronto Laboratory

2005 IBM Corporation

IBM Software Group | DB2 Information Management Software

Agenda
Concurrency & locking overview Concurrency Locking Lock escalation Lock Wait Deadlocks

IBM Software Group | DB2 Information Management Software

Agenda
Concurrency & locking overview Concurrency Locking Lock escalation Lock Wait Deadlocks

IBM Software Group | DB2 Information Management Software

What is a Transaction?
Savings Balance = $1000 Checking Balance = $200

Transfer $100 from Savings to Checking: - Debit $100 from Savings account - Credit $100 to Checking account

IBM Software Group | DB2 Information Management Software

Concurrency overview

App A
ID Name Peter John Mary Ann Age 33 23 22 55

App B App C App D

3 5 22 35

IBM Software Group | DB2 Information Management Software

Locking overview
App A
ID Name Peter John Mary Ann Age 33 23 22 55

App B App C App D

3 5 22 35

IBM Software Group | DB2 Information Management Software

Agenda
Concurrency & locking overview Concurrency Locking Lock escalation Lock Wait Deadlocks

IBM Software Group | DB2 Information Management Software

Concurrency
DB2 was designed as a multi-user database Access to data must be coordinated properly and transparently using a mechanism to ensure data integrity and consistency Without some form of concurrency control, the following problems may be encountered
Lost update Uncommitted read Non-repeatable read Phantom read

IBM Software Group | DB2 Information Management Software

Lost Update
Reservations Flight 512 512 ... Seat 7C 7B ... P_Name

... Update Reservations Set P-name = 'Manager' Where Flight = 512 and Seat = '7C' and P_Name is NULL 512 7C Manager ...
9

Update Reservations Set P-name = 'Instruct' Where Flight = 512 and Seat = '7C' and P_name is NULL 512 7C Instruct ...

IBM Software Group | DB2 Information Management Software

Uncommitted Read
Reservations Flight 512 512 ... Seat 7C 7B ... P_Name

1 Update Reservations Set P-name = 'Instruct' Where Flight = 512 and Seat = '7C' and P_Name is NULL 512 3 Roll back 7C

2 Select seat From Reservations Where P-name is NULL

Instruct 4 Incorrect results set

10

IBM Software Group | DB2 Information Management Software

Non-repeatable Read
FLIGHT 512 SEAT 7B NAME
DESTINATION

ORIGIN DALLAS

DENVER

.... ....
814 8A SAN JOSE DENVER

....
134 1C HONOLULU SAN JOSE

....
Book a flight from Dallas to Honolulu

....

11

IBM Software Group | DB2 Information Management Software

Phantom Read
Reservations Flight 512 512 ... Seat 7B 7A ... P_Name Susan Liu

1 Select seat From Reservations Where P-name is NULL

Update Reservations Set P-name = 'NULL' Where Flight = 512 and Seat = '7A' and P_Name = 'Susan Liu '

512 3 Repeat 1 now seat 7A is available

7B

12

IBM Software Group | DB2 Information Management Software

Agenda
Concurrency & locking overview Concurrency Locking Lock escalation Lock Wait Deadlocks

13

IBM Software Group | DB2 Information Management Software

Locking
DB2 uses locking to maintain data integrity Locks are acquired automatically as needed to support a transaction and are released when the transaction terminates (COMMIT/ROLLBACK) Locks can be acquired on tables or rows Two basic types of locks:
Share locks (S locks) acquired when an application wants to read and prevent others from updating the same row Exclusive locks (X locks) acquired when an application updates, inserts, or deletes a row

14

IBM Software Group | DB2 Information Management Software

Isolation Levels
DB2 provides different levels of protection to isolate data
Uncommitted Read (UR) Cursor Stability (CS) Read Stability (RS) Repeatable Read (RR)

Isolation level can be specified at many levels

Session (application). Defaults to CS Connection Statement

For embedded SQL, the level is set at bind time For dynamic SQL, the level is set at run time

15

IBM Software Group | DB2 Information Management Software

Isolation Levels Uncommitted Read

Uncommitted Read is also known as DIRTY READ
Lowest level of isolation Provides highest degree of concurrency No row locks are obtained on read operations unless other application attempts to drop or alter table Update operations act as if using Cursor Stability

Possible Situations
Uncommitted Read Non-repeatable Read Phantom Read

Situations Prevented
Loss of Update

16

IBM Software Group | DB2 Information Management Software

Isolation Levels Cursor Stability

Cursor Stability is the default isolation level
Minimal degree of locking Locks the "current" row of a cursor If the row is only read the lock is held until a new row is fetched or the unit of work is terminated If the row is updated the lock is held until the unit of work is terminated

Possible Situations
Non-repeatable Read Phantom Read

Prevented Situations
Loss of Update Uncommitted Read
17

IBM Software Group | DB2 Information Management Software

Isolation Levels Read Stability

Locks all the rows an application retrieves within a unit of work
For a given cursor, it locks all rows that qualify for the result set Moderate degree of locking

Possible Situations
Phantom Read

Prevented Situations
Loss of Update Uncommitted Read Non-repeatable Read

18

IBM Software Group | DB2 Information Management Software

Isolation Levels Repeatable Read

Highest isolation level, least concurrency
Same query issued by the application more than once in a unit of work will give the same result each time High degree of locking Locks held on all rows processed to build the result set i.e. rows not necessarily in the final result set may be locked No other application can update, delete, or insert a row that would affect the result set until the unit of work completes

Possible Situations
none

Prevented Situations
Loss of Update Uncommitted Read Non-repeatable Read Phantom Read
19

IBM Software Group | DB2 Information Management Software

Comparing isolation levels

20

IBM Software Group | DB2 Information Management Software

Comparison of Isolation Level Terminolgy

DB2 Uncommitted Read (UR)

.NET ReadUncommitted

JDBC TRANSACTION_READ_ UNCOMMITTED TRANSACTION_READ_ COMMITTED TRANSACTION_REPEA TABLE_READ TRANSACTION_SERIAL IZABLE

Cursor Stability (CS) Read Stability (RS) Repeatable Read (RR)

ReadCommitted RepeatableRead Serializable

21

IBM Software Group | DB2 Information Management Software

Statement Level Isolation

SELECT ... WITH {UR | CS | RS | RR} UR = Uncommitted Read CS = Cursor Stability RS = Read Stability RR = Repeatable Read Example Scenario:
Application needs to get a "rough" count of how many rows are in table. Performance is of utmost importance. Cursor Stability isolation level is required with the exception of one SQL statement: SELECT COUNT(*) FROM tab1 WITH UR

22

IBM Software Group | DB2 Information Management Software

Agenda
Concurrency & locking overview Concurrency Locking Lock escalation Lock Wait Deadlocks

23

IBM Software Group | DB2 Information Management Software

Lock Escalation
table

multiple row locks

table

lock escalation table table lock

24

IBM Software Group | DB2 Information Management Software

Lock Escalation
When optimizer thinks it is better to have one lock on the entire table, rather than multiple row locks Database configuration parameters that affect lock escalation:
LOCKLIST the amount of memory (4k pages) to manage locks for all connected applications Default is 50 * 4K pages on Windows MAXLOCKS Max percentage of the entire lock list that a single application can use up Default is 22 percent

25

IBM Software Group | DB2 Information Management Software

Lock Escalation (continued)

If the default values are used, lock escalation occurs when a single application requires more than 44K of lock memory If lock escalation, increase the value of LOCKLIST and MAXLOCKS Avoid lock escalations: Performance bottleneck Check the DB2 diagnostic log file in C:\Program Files\IBM\SQLLIB\DB2\db2diag.log

26

IBM Software Group | DB2 Information Management Software

Lock Escalation indication in the db2diag.log

2007-01-02-23.04.43.699000 Instance:DB2 Node:000 PID:984(db2syscs.exe) TID:1720 Appid:*LOCAL.DB2.011003030417 data_management sqldEscalateLocks Probe:1 Database:SAMPLE -- Start Table Lock Escalation. -- Lock Count, Target : 28, 14 7570 6461 7465 2065 6d70 6c6f 7965 6520 7365 7420 6669 7273 746e 6d65 3d27 6162 6327 c'

update employee set firstnme='ab

2007-01-02-23.04.43.699001 Instance:DB2 Node:000 PID:984(db2syscs.exe) TID:1720 Appid:*LOCAL.DB2.011003030417 data_management sqldEscalateLocks Probe:2 Database:SAMPLE -- Lock Count, Target : 28, 14 -- Table (ID) Name : (2;5) ADMINISTRATOR.EMPLOYEE -- Locks, Request Type : 25, X -- Result (0 = success): 0

27

IBM Software Group | DB2 Information Management Software

Lock Snapshot
View locks currently held by an application UPDATE MONITOR SWITCHES USING LOCK ON GET SNAPSHOT FOR LOCKS FOR APPLICATION AGENT ID <handle>

Application Lock Snapshot Snapshot timestamp Application handle Application ID Sequence number Application name Authorization ID Application status Status change time Application code page Locks held Total wait time (ms) = 11-05-2002 00:09:08.672586 =9 = *LOCAL.DB2.00B9C5050843 = 0001 = db2bp.exe = ADMINISTRATOR = UOW Waiting = Not Collected = 1252 =4 =0

List Of Locks Lock Name = 0x05000700048001000000000052 Lock Attributes = 0x00000000 Release Flags = 0x40000000 Lock Count = 255 Hold Count =0 Lock Object Name = 98308 Object Type = Row Tablespace Name = TEST4K Table Schema = ADMINISTRATOR Table Name = T2 Mode =X
28

IBM Software Group | DB2 Information Management Software

Agenda
Concurrency & locking overview Concurrency Locking Lock escalation Lock Wait Deadlocks

29

IBM Software Group | DB2 Information Management Software

Lock Wait
By default, an application waits indefinitely to obtain any needed locks LOCKTIMEOUT (db cfg):
Default value is -1 or infinite wait Change to specify the number of seconds to wait for a lock

A database connection also has a user-definable CURRENT LOCK TIMEOUT register

By default it inherits its value from the LOCKTIMEOUT parameter Use the SET LOCK TIMEOUT statement to change its value Once it is set for a connection, it persists across transactions e.g. SET LOCK TIMEOUT=WAIT n

30

IBM Software Group | DB2 Information Management Software

Agenda
Concurrency & locking overview Concurrency Locking Lock escalation Lock Wait Deadlocks

31

IBM Software Group | DB2 Information Management Software

Deadlock Causes and Detection

A deadlock occurs when two or more applications connected to the same database wait indefinitely for a resource The waiting is never resolved because each application is holding a resource that the other needs
RAISIN BRAN
USER A

MILK
USER B

within a UNIT OF WORK (UOW)

INSERT CEREAL AND MILK into BOWL

32

IBM Software Group | DB2 Information Management Software

Deadlock Settings
Deadlocks are an application design issue most of the time DLCHKTIME (db cfg) sets the time interval for checking for deadlocks
It defines the frequency that the database manager checks for deadlocks among all the applications connected to a database

If you are experiencing many deadlocks, you should re-examine your existing transactions and see if any re-structuring is possible

33

IBM Software Group | DB2 Information Management Software

Best Practices
Keep transactions as short as possible
Issue frequent COMMIT statements even for read-only transactions

Log transaction information only when required

Purge data quick using:
ALTER TABLE ACTIVATE NOT LOGGED INITIALLY WITH EMPTY TABLE Perform data modifications in batches/groups DELETE FROM ( SELECT * FROM tedwas.t1 WHERE c1 = FETCH FIRST 3000 ROWS ONLY ) Use concurrency features in DB2 data movement tools

Set the database level LOCKTIMEOUT parameter (usually between 30-120 seconds)
Can also use session-based lock timeout

Do not retrieve more data than is required

Use the FETCH FIRST n ROWS ONLY clause in SELECT statements

34

IBM Software Group | DB2 Information Management Software

Whats Next?
Application Developement e-Learning courses:
Lesson 11 Lesson 12 Lesson 13 Lesson 14 Lesson 15 Lesson 16 Concurrency & locking SQL PL stored procedures User-defined functions Triggers PureXML: SQL/XML & XQuery Java, Ruby on Rails, PHP COMPLETED!

Troubleshooting e-Learning course:

Lesson 17 Troubleshooting tips

35

IBM Software Group

Lesson 12: SQL PL Stored Procedures

(DB2 on Campus Lecture Series)

Raul F. Chong
IBM Toronto Laboratory

2005 IBM Corporation

IBM Software Group | DB2 Information Management Software

Agenda
Stored procedures overview IBM Data Studio SQL PL Stored Procedures basics Cursors Errors & condition handlers Flow control Dynamic SQL Calling Stored Procedures

IBM Software Group | DB2 Information Management Software

Agenda
Stored procedures overview IBM Data Studio SQL PL Stored Procedures basics Cursors Errors & condition handlers Flow control Dynamic SQL Calling Stored Procedures

IBM Software Group | DB2 Information Management Software

Stored Procedures
Client Application SQL #1 SQL #2 SQL #3 Network Server

myproc SQL #1 CALL myproc SQL #2 SQL #3

IBM Software Group | DB2 Information Management Software

Stored Procedures
Database objects that usually contain one or more SQL statements as well as procedural (business) logic Executed and managed by DB2 (server-side objects) Can be written using SQL PL, C/C++, Java, Cobol, CLR supported languages, and OLE Benefits for using stored procedures include:
Centralized business logic that promotes code re-use Improved security Improved performance

In this workshop, we focus on SQL PL procedures because of their popularity and simplicity
5

IBM Software Group | DB2 Information Management Software

Agenda
Stored procedures overview IBM Data Studio SQL PL Stored Procedures basics Cursors Errors & condition handlers Flow control Dynamic SQL Calling Stored Procedures

IBM Software Group | DB2 Information Management Software

IBM Data Studio

Replaces the DB2 Developer Workbench of DB2 9 Installed separately from DB2 9.5 Based on Eclipse Powerful IDE that allows you to:
Create, edit, debug, deploy, test Java/SQL Stores Procedures, UDFs Develop SQLJ applications Create, alter, and drop DB2 database objects (with impact analysis) Explore and edit data - relational and XML Visually build SQL and and XQuery statements Optimize queries using Visual Explain Develop queries and routines for pureXML applications Perform data movement tasks Collaborate and share projects with team members Quickly build SOAP and REST Web Services Discover database object relationships with physical data models (diagraming) Visualize data distribution across tables
7

IBM Software Group | DB2 Information Management Software

Agenda
Stored procedures overview IBM Data Studio SQL PL Stored Procedures basics Cursors Errors & condition handlers Flow control Dynamic SQL Calling Stored Procedures

IBM Software Group | DB2 Information Management Software

Creating your first SQL PL Stored Procedure

Using the Command Editor:
connect to sample create procedure p1 begin end

Using the IBM Data Studio

IBM Software Group | DB2 Information Management Software

Basic stored procedure structure

CREATE PROCEDURE proc_name [( {optional parameters} )] [optional procedure attributes] <statement>

<statement> is a single statement, or a set of statements grouped by BEGIN [ATOMIC] ... END

10

IBM Software Group | DB2 Information Management Software

Optional stored procedure attributes

LANGUAGE SQL RESULT SETS <n> (required if returning result sets) SPECIFIC my_unique_name can be same as procedure name highly recommended for manageability: GRANT EXECUTE ON SPECIFIC PROCEDURE ... DROP SPECIFIC PROCEDURE ...

11

IBM Software Group | DB2 Information Management Software

Parameters
CREATE PROCEDURE proc(IN p1 INT, OUT p2 INT, INOUT p3 INT) ... IN - Input parameter OUT - Output parameter INOUT - Input and Output parameter All params must be provided in CALL statement

12

IBM Software Group | DB2 Information Management Software

Comments in SQL PL Stored Procedures

-- This is an SQL-style comment

/* This is a C-style coment */ (valid within SQL procedures )

13

IBM Software Group | DB2 Information Management Software

Compound statements
BEGIN [ATOMIC] <declare variables> <declare conditions> <declare statements> <declare cursors> <declare handlers> <logic > END
Optionally atomic

Declarations

Compound Statement

Logic Can contain other compound stmts

14

IBM Software Group | DB2 Information Management Software

Variable declaration
DECLARE var_name <data type> [ DEFAULT value]; Note: Default value is NULL Examples: DECLARE temp1 SMALLINT DEFAULT 0; DECLARE temp2 INTEGER DEFAULT 10; DECLARE temp3 DECIMAL(10,2) DEFAULT 100.10; DECLARE temp4 REAL DEFAULT 10.1; DECLARE temp5 DOUBLE DEFAULT 10000.1001; DECLARE temp6 BIGINT DEFAULT 10000; DECLARE temp7 CHAR(10) DEFAULT 'yes'; DECLARE temp8 VARCHAR(10) DEFAULT 'hello'; DECLARE temp9 DATE DEFAULT '1998-12-25'; DECLARE temp10 TIME DEFAULT '1:50 PM'; DECLARE temp11 TIMESTAMP DEFAULT '2001-01-05-12.00.00'; DECLARE temp12 CLOB(2G); DECLARE temp13 BLOB(2G);

15

IBM Software Group | DB2 Information Management Software

Assignment statements
SET total = 100;
Same as VALUES(100) INTO total;

SET total = NULL;

any variable can be set to NULL

SET total = (select sum(c1) from T1);

Condition is raised if more than one row

SET first_val = (select c1 from T1 fetch first 1 row only)

fetch only the first row from a table

SET sch = CURRENT SCHEMA;

16

IBM Software Group | DB2 Information Management Software

Quicklab: Stored procedure with parameters

CREATE PROCEDURE P2 ( IN v_p1 INT, INOUT v_p2 INT, OUT v_p3 INT) LANGUAGE SQL SPECIFIC myP2 BEGIN -- my second SQL procedure SET v_p2 = v_p2 + v_p1; SET v_p3 = v_p1; END
To call the procedure from the Command Editor: call P2 (3, 4, ?)
17

IBM Software Group | DB2 Information Management Software

Cursors
cursor declaration and usage: DECLARE <cursor name> CURSOR [WITH RETURN <return target>] <SELECT statement>; OPEN <cursor name>; FETCH <cursor name> INTO <variables> CLOSE <cursor name>; result sets can also be directly returned to CLIENT or CALLER for processing
CLIENT: result set will return to client application CALLER: result set is returned to client or stored procedure that made the call

18

IBM Software Group | DB2 Information Management Software

Quicklab: Stored procedure returning result sets

CREATE PROCEDURE set () DYNAMIC RESULT SETS 1 LANGUAGE SQL BEGIN DECLARE cur CURSOR WITH RETURN TO CLIENT FOR SELECT name, dept, job FROM staff WHERE salary > 20000; OPEN cur; END

19

IBM Software Group | DB2 Information Management Software

SQLCODE and SQLSTATE

Access requires explicit declaration:
DECLARE SQLSTATE CHAR(5); DECLARE SQLCODE INT;

Can be declared ONLY at outermost scope and automatically set by DB2 after each operation SQLCODE
= 0, successful. > 0, successful with warning < 0, unsuccessful = 100, no data was found. i.e. FETCH statement returned no data

20

IBM Software Group | DB2 Information Management Software

SQLSTATE and conditions

SQLSTATE
Success: SQLSTATE '00000' Not found: SQLSTATE '02000' Warning: SQLSTATE '01XXX' Exception: Everything else A condition can be raised by any SQL statement General conditions: SQLWARNING, SQLEXCEPTION, NOT FOUND Specific conditions: SQLSTATE '01004' Can assign names to condition:
DECLARE trunc CONDITION FOR SQLSTATE '01004'
21

IBM Software Group | DB2 Information Management Software

Condition Handling
A condition handler must specify handled conditions where to resume execution (CONTINUE, EXIT or UNDO) action to perform to handle the condition Action can be any statement (including control structures) Upon SQLEXCEPTION condition,if no handler exists, the procedure terminates and returns to the client with error

raises exception

BEGIN [ATOMIC] DECLARE <type> HANDLER FOR <conditions> <handler-action> statement_1; CONTINUE point statement_2; statement_3; END UNDO or EXIT point

22

IBM Software Group | DB2 Information Management Software

Flow control statements

CASE (selects an execution path (simple / searched)) IF FOR (executes body for each row of table) WHILE ITERATE (forces next iteration. Similar to CONTINUE in C) LEAVE (leaves a block or loop. "Structured Goto") LOOP (infinite loop) REPEAT GOTO RETURN CALL (procedure call)

23

IBM Software Group | DB2 Information Management Software

Agenda
Stored procedures overview IBM Data Studio SQL PL Stored Procedures basics Cursors Errors & condition handlers Flow control Dynamic SQL Calling Stored Procedures

24

IBM Software Group | DB2 Information Management Software

Dynamic SQL
Useful when the final form of SQL is know only at RUN TIME example: if col1 and tabname are variables: 'SELECT ' || col1 || ' FROM ' || tabname; Also recommended for DDL to avoid dependency problems and package invalidation, and to implement recursion. Keywords:
EXECUTE IMMEDATE - ideal for single execution SQL PREPARE + EXECUTE - ideal for multiple execution SQL

25

IBM Software Group | DB2 Information Management Software

Quicklab: Dynamic SQL

prerequisite: create table T2 (c1 int, c2 int) CREATE PROCEDURE dyn1 (IN value1 INT, IN value2 INT) SPECIFIC dyn1 BEGIN DECLARE stmt varchar(255); DECLARE st STATEMENT; SET stmt = 'insert into T2 values (?, ?)'; PREPARE st FROM stmt; EXECUTE st USING value1, value1; EXECUTE st USING value2, value2; SET stmt = 'insert into T2 values (9,9)'; EXECUTE IMMEDIATE stmt; END

Compile stmt once Execute it many times

Compile and execute once

26

IBM Software Group | DB2 Information Management Software

Agenda
Stored procedures overview IBM Data Studio SQL PL Stored Procedures basics Cursors Errors & condition handlers Flow control Dynamic SQL Calling Stored Procedures

27

IBM Software Group | DB2 Information Management Software

Calling from CLI

SQLCHAR *stmt = (SQLCHAR *) SQLDOUBLE sal = 20000.0; SQLINTEGER salind = 0; "CALL MEDIAN_RESULT_SET( ? )" ; /* Bound to parameter marker in stmt */ /* Indicator variable for sal */

sqlrc = SQLPrepare(hstmt, stmt, SQL_NTS); sqlrc = SQLBindParameter(hstmt, 1, SQL_PARAM_OUTPUT, SQL_C_DOUBLE, SQL_DOUBLE, 0, 0, &sal, 0, &salind); SQLExecute(hstmt); if (salind == SQL_NULL_DATA) printf("Median Salary = NULL\n"); else printf("Median Salary = %.2f\n\n", sal ); sqlrc = StmtResultPrint(hstmt); sqlrc = SQLMoreResults(hstmt); /* Get first result set */ /* Check for another result set */

if (sqlrc == SQL_SUCCESS) { /* There is another result set */ sqlrc = StmtResultPrint(hstmt); }

See sqllib/samples/sqlproc/rsultset.c
28

IBM Software Group | DB2 Information Management Software

Calling Stored Procedures from a VB.NET Application

Try Create a DB2Command to run the stored procedure Dim procName As String = TRUNC_DEMO Dim cmd As DB2Command = conn.CreateCommand() Dim parm As DB2Parameter cmd.CommandType = CommandType.StoredProcedure cmd.CommandText = procName Register the parm parm.Direction parm parm.Direction output parameters for the DB2Command = cmd.Parameters.Add(v_lastname, DB2Type.VarChar) = ParameterDirection.Output = cmd.Parameters.Add(v_msg, DB2Type.VarChar) = ParameterDirection.Output

Call the stored procedure Dim reader As DB2DataReader = cmd.ExecuteReader Catch myException As DB2Exception DB2ExceptionHandler(myException) Catch UnhandledExceptionHandler() End Try
29

IBM Software Group | DB2 Information Management Software

Calling Stored Procedures from a Java Application

try { // Connect to sample database String url = jdbc:db2:sample; con = DriverManager.getConnection(url); CallableStatement cs = con.prepareCall(CALL trunc_demo(?, ?)); // register the output parameters callStmt.registerOutParameter(1, Types.VARCHAR); callStmt.registerOutParameter(2, Types.VARCHAR); cs.execute(); con.close(); } catch (Exception e) { /* exception handling logic goes here */ }
30

IBM Software Group | DB2 Information Management Software

Recommended Book

DB2 SQL PL: Essential Guide for DB2 UDB on Linux, UNIX, Windows, i5/OS, and z/OS, 2nd Edition More info:
http://www.software.ibm.com/data/developer/sqlplbook

Order from any online book seller using ISBN: 0-13-100772-6

31

IBM Software Group | DB2 Information Management Software

Sample Applications

C:\Program Files\IBM\SQLLIB\samples
32

IBM Software Group | DB2 Information Management Software

Whats Next?
Application Developement e-Learning courses:
Lesson 11 Lesson 12 Lesson 13 Lesson 14 Lesson 15 Lesson 16 Concurrency & locking SQL PL stored procedures COMPLETED! User-defined functions Triggers PureXML: SQL/XML & XQuery Java, Ruby on Rails, PHP

Troubleshooting e-Learning course:

Lesson 17 Troubleshooting tips

33

IBM Software Group

Lesson 13: User-Defined Functions

(DB2 on Campus Lecture Series)

Raul F. Chong
IBM Toronto Laboratory

2005 IBM Corporation

IBM Software Group | DB2 Information Management Software

Agenda
Inline SQL PL User-defined functions (UDFs) overview Scalar functions Table functions

IBM Software Group | DB2 Information Management Software

Agenda
Inline SQL PL User-defined functions (UDFs) overview Scalar functions Table functions

IBM Software Group | DB2 Information Management Software

Inline SQL PL
SQL PL = SQL Procedural Language Inline SQL PL = SQL PL subset supported within a dynamic SQL statement, triggers and UDFs

IBM Software Group | DB2 Information Management Software

Example: Randomly populating a table

Pre-req:
create sequence myseq create table T1 (id bigint, data char(100), insert_ts timestamp)

Inline SQL PL:

begin atomic declare cnt INT default 0; while (cnt < 20000) do insert into t1 values ( nextval for MYSEQ, (select case when (rand() < 0.5) then null else space(int(rand()*100)) end case from sysibm.sysdummy1), current timestamp); set cnt=cnt+1; end while; end

IBM Software Group | DB2 Information Management Software

Agenda
Inline SQL PL User-defined functions (UDFs) overview Scalar functions Table functions

IBM Software Group | DB2 Information Management Software

User-Defined Functions
Functions always return a value Some built-in functions already exist out-of-the-box
Eg: SUM(), AVG(), DIGITS(), etc.

Can create UDFs in:

SQL PL, C/C++, Java, CLR (Common Language Runtime), and OLE (Object Linking and Embedding) In this workshop, we focus on SQL PL functions because of their simplicity and popularity

IBM Software Group | DB2 Information Management Software

Types of functions
Scalar functions
Return a single value Cannot change database state (i.e. no INSERT, UPDATE, DELETE statements allowed) Example: COALESCE( ), SUBSTR( )

Table functions
Return values in a table format Called in the FROM clause of a query Can change database state (i.e. allow INSERT, UPDATE, DELETE statements) Example: SNAPSHOT_DYN_SQL( ), MQREADALL( )

Others type of functions (not covered in this course):

Row functions Column functions

IBM Software Group | DB2 Information Management Software

Agenda
Inline SQL PL User-defined functions (UDFs) overview Scalar functions Table functions

IBM Software Group | DB2 Information Management Software

Scalar function
Scalar functions take input values and return a single value Example:
CREATE FUNCTION deptname(p_empid VARCHAR(6)) RETURNS VARCHAR(30) SPECIFIC deptname BEGIN ATOMIC DECLARE v_department_name VARCHAR(30); DECLARE v_err VARCHAR(70); SET v_department_name = ( SELECT d.deptname FROM department d, employee e WHERE e.workdept=d.deptno AND e.empno= p_empid); SET v_err = 'Error: employee ' || p_empid || ' was not found'; IF v_department_name IS NULL THEN SIGNAL SQLSTATE '80000' SET MESSAGE_TEXT=v_err; END IF; RETURN v_department_name; END
10

IBM Software Group | DB2 Information Management Software

Invoking Scalar UDFs

Scalar UDFs can be invoked in SQL statements wherever a scalar value is expected, or in a VALUES clause SELECT DEPTNAME(000010) FROM SYSIBM.SYSDUMMY1 VALUES DEPTNAME(000010)

11

IBM Software Group | DB2 Information Management Software

Agenda
Inline SQL PL User-defined functions (UDFs) overview Scalar functions Table functions

12

IBM Software Group | DB2 Information Management Software

Table UDFs
Return a table of rows Used in the FROM clause of a query Similar to a view, except more powerful because data modification statements (INSERT/UPDATE/DELETE) can be performed Typically used to return a table and keep an audit record

13

IBM Software Group | DB2 Information Management Software

Table function example

A function which enumerates a set of employees of a department

CREATE FUNCTION getEnumEmployee(p_dept VARCHAR(3)) RETURNS TABLE (empno CHAR(6), lastname VARCHAR(15), firstnme VARCHAR(12)) SPECIFIC getEnumEmployee RETURN SELECT e.empno, e.lastname, e.firstnme FROM employee e WHERE e.workdept=p_dept

14

IBM Software Group | DB2 Information Management Software

Calling a Table function

Used in the FROM clause of an SQL statement The TABLE() function must be applied and must be aliased.

SELECT * FROM TABLE (getEnumEmployee('E01')) T

TABLE() function

alias

15

IBM Software Group | DB2 Information Management Software

QuickLab #13 Creating a Scalar UDF using IBM Data Studio

In this QuickLab, you will use the IBM Data Studio tool to create a new scalar user-defined function You will gain more familiarity with the IBM Data Studio as well as solidify your knowledge of SQL PL syntax for UDFs Refer to the QuickLab #13 instructions

16

IBM Software Group | DB2 Information Management Software

Recommended Book

DB2 SQL PL: Essential Guide for DB2 UDB on Linux, UNIX, Windows, i5/OS, and z/OS, 2nd Edition More info:
http://www.software.ibm.com/data/developer/sqlplbook

Order from any online book seller using ISBN: 0-13-100772-6

17

IBM Software Group | DB2 Information Management Software

Whats Next?
Application Developement e-Learning courses:
Lesson 11 Lesson 12 Lesson 13 Lesson 14 Lesson 15 Lesson 16 Concurrency & locking SQL PL stored procedures User-defined functions Triggers PureXML: SQL/XML & XQuery Java, Ruby on Rails, PHP COMPLETED!

Troubleshooting e-Learning course:

Lesson 17 Troubleshooting tips

18

IBM Software Group

Lesson 14: Triggers

(DB2 on Campus Lecture Series)

Raul F. Chong
IBM Toronto Laboratory

2005 IBM Corporation

IBM Software Group | DB2 Information Management Software

Agenda

Overview Before Trigger example After Trigger example

IBM Software Group | DB2 Information Management Software

Agenda

Overview Before Trigger example After Trigger example

IBM Software Group | DB2 Information Management Software

Triggers
A trigger is a database object defined on a table and fired when an INSERT, UPDATE, or DELETE operation is performed. Activate (fire) automatically Operations that cause triggers to fire are called triggering SQL statements

IBM Software Group | DB2 Information Management Software

Types of Triggers
BEFORE
activation before row is inserted, updated or deleted Operations performed by this trigger cannot activate other triggers (i.e. INSERT, UPDATE, and DELETE operations are not permitted)

AFTER
Activated after the triggering SQL statement has executed to successful completion May activate other triggers (cascading permitted up to 16 levels)

INSTEAD OF
Defined on views Logic defined in the trigger is executed instead of the triggering SQL statement

IBM Software Group | DB2 Information Management Software

Agenda

Overview Before Trigger example After Trigger example

IBM Software Group | DB2 Information Management Software

A Simple BEFORE Trigger

if no value provided on insert, column is NULL

define qualifier for new values

CREATE TRIGGER default_class_end NO CASCADE BEFORE INSERT ON cl_sched REFERENCING NEW AS n FOR EACH ROW MODE DB2SQL WHEN (n.ending IS NULL) SET n.ending = n.starting + 1 HOUR

optional WHEN

IBM Software Group | DB2 Information Management Software

Quicklab: BEFORE trigger using SQL PL

CREATE TRIGGER validate_sched NO CASCADE BEFORE INSERT ON cl_sched REFERENCING NEW AS n atomic, dynamic FOR EACH ROW compound SQL MODE DB2SQL BEGIN ATOMIC -- supply default value for ending time if null IF (n.ending IS NULL) THEN SET n.ending = n.starting + 1 HOUR; END IF; -- ensure that class does not end beyond 9pm IF (n.ending > '21:00') THEN SIGNAL SQLSTATE '80000' SET MESSAGE_TEXT='class ending time is beyond 9pm'; ELSEIF (n.DAY=1 or n.DAY=7) THEN SIGNAL SQLSTATE '80001' SET MESSAGE_TEXT='class cannot be scheduled on a weekend'; END IF; END

IBM Software Group | DB2 Information Management Software

Agenda

Overview Before Trigger example After Trigger example

IBM Software Group | DB2 Information Management Software

A Simple AFTER Trigger

Similar to BEFORE triggers, except that INSERT, UPDATE and DELETE are supported Prereq: CREATE TABLE audit (mytimestamp timestamp, comment varchar (1000))

CREATE TRIGGER audit_emp_sal AFTER UPDATE OF salary ON employee REFERENCING OLD AS o NEW AS n FOR EACH ROW MODE DB2SQL INSERT INTO audit VALUES ( CURRENT TIMESTAMP, ' Employee ' || o.empno || ' salary changed from ' || CHAR(o.salary) || ' to ' || CHAR(n.salary) || ' by ' || USER)
10

IBM Software Group | DB2 Information Management Software

QuickLab #11 Creating a Trigger Using Control Center

In this QuickLab, you will use Control Center to create a new trigger This will help you gain familiarity with triggers and solidify your knowledge of SQL PL syntax for triggers Refer to the QuickLab #11 instructions

11

IBM Software Group | DB2 Information Management Software

Final Notes
Triggers, support a single SQL statement. However, by using dynamic compound SQL, multiple statements can form a single statement. Another benefit of dynamic compound SQL is that inline SQL PL is supported Where possible, use the WHEN clause to limit trigger activation For more information, see DB2 Developer Domain:
Using SQL Procedural Language for Triggers in DB2 UDB http://www7b.boulder.ibm.com/dmdd/library/techarticle/yip/0111yip.html How to temporarily disable triggers in DB2 http://www7b.boulder.ibm.com/dmdd/library/techarticle/0211yip/0211yip.html Advanced SQL Procedural Scripting http://www7b.boulder.ibm.com/dmdd/library/techarticle/0203yip/0203yip.html

12

IBM Software Group | DB2 Information Management Software

Whats Next?
Application Developement e-Learning courses:
Lesson 11 Lesson 12 Lesson 13 Lesson 14 Lesson 15 Lesson 16 Concurrency & locking SQL PL stored procedures User-defined functions Triggers COMPLETED!

PureXML: SQL/XML & XQuery Java, Ruby on Rails, PHP

Troubleshooting e-Learning course:

Lesson 17 Troubleshooting tips

13

IBM Software Group

Lesson 15: pureXML SQL/XML & XQuery

(DB2 on Campus Lecture Series)

Raul F. Chong
IBM Toronto Laboratory

2005 IBM Corporation

IBM Software Group | DB2 Information Management Software

Agenda
Overview Inserting XML data XPath XQuery Querying XML data using SQL/XML Querying XML data using XQuery Update & Delete operations with XML XML Indexes Other XML support

IBM Software Group | DB2 Information Management Software

Agenda
Overview Inserting XML data XPath XQuery Querying XML data using SQL/XML Querying XML data using XQuery Update & Delete operations with XML XML Indexes Other XML support

IBM Software Group | DB2 Information Management Software

DB2 Commands & SQL/XML, Xquery Statements DB2 Tools Command Line Tools Command Editor Command Line Processor Command Window Development Tools DB2 Developer Workbench Project Deployment Tool General Administration Tools Control Center Journal License Center Replication Center Task Center Information Information Center Check for DB2 Updates Monitoring Tools Event Analyzer Health Center Indoubt Transaction Manager Memory Visualizer Activity Monitor Setup Tools Configuration Assistant Configure DB2 .Net Data Provider Default DB2 Selection Wizard First Steps Register Visual Studio Add-ins
4

DB2: The big picture

SQL/XML, Xquery Statements create bufferpool create tablespace create table alter bufferpool alter tablespace alter table select insert update delete ... DB2 System Commands db2set db2start db2stop db2ilist db2icrt db2idrop ... DB2 CLP Commands db2 update dbm cfg catalog db list node directory create database list applications list tablespaces ... <sql statement> xquery < >

DB2 Environment

Instance 'myinst'
Database Manager Configuration File (dbm cfg)

Database MYDB1
Database Configuration File (db cfg)

Port

IBM Software Group | DB2 Information Management Software

What is pureXML?
Two main characteristics:
XML is stored in parsed-hierarchical format in the database DB2 has a native XML interface to the engine

The storage format = the processing format For details, refer to:
Speech #2 (Part 2) of the DB2 on Campus Lecture Series: The role of DB2 in Web 2.0 and in the information on demand world

IBM Software Group | DB2 Information Management Software

Agenda
Overview Inserting XML data XPath XQuery Querying XML data using SQL/XML Querying XML data using XQuery Update & Delete operations with XML XML Indexes Other XML support

IBM Software Group | DB2 Information Management Software

Table definitions with XML columns

create table items ( id int primary key not null, brandname varchar(30), itemname varchar(30), sku int, srp decimal(7,2), comments xml ); create table clients( id int primary key not null, name varchar(50), status varchar(10), contact xml );

IBM Software Group | DB2 Information Management Software

Contents of C:\DB2workshop\Quicklabs\quicklab14a directory with XML documents

IBM Software Group | DB2 Information Management Software

Insert & Import of XML data

INSERT INTO clients VALUES (77, 'John Smith', 'Gold', '<addr>111 Main St., Dallas, TX, 00112</addr>') ;

IMPORT from "C:\DB2workshop\Quicklabs\quicklab14a\clients.del" of del xml from "C:\DB2workshop\Quicklabs\quicklab14a" INSERT INTO CLIENTS (ID, NAME, STATUS, CONTACT);

IMPORT from "C:\DB2workshop\Quicklabs\quicklab14a\items.del" of del xml from "C:\DB2workshop\Quicklabs\quicklab14a" INSERT INTO ITEMS (ID, BRANDNAME, ITEMNAME, SKU, SRP, COMMENTS);

IBM Software Group | DB2 Information Management Software

Agenda
Overview Inserting XML data XPath XQuery Querying XML data using SQL/XML Querying XML data using XQuery Update & Delete operations with XML XML Indexes Other XML support

10

IBM Software Group | DB2 Information Management Software

XPath

XML Query Language Subset of XQuery & SQL/XML

<dept bldg=101> <employee id=901> <name>John Doe</name> <phone>408 555 1212</phone> <office>344</office> </employee> <employee id=902> <name>Peter Pan</name> <phone>408 555 9918</phone> <office>216</office> </employee> employee </dept>

/ /dept /dept/employee /dept/employee/@id /dept/employee/name /dept/employee/phone /dept/employee/office (...)

dept

Each node has a path

employee

id=901

name

phone

office

id=902

name

phone

office

John Doe

408-555-1212

344

Peter Pan

408-555-9918

216

11

IBM Software Group | DB2 Information Management Software

XPath: Simple XPath Expressions

Use fully qualified paths to specify elements/attributes @ is used to specify an attribute use text() to specify the text node under an element

<dept bldg=101> <employee id=901> <name>John Doe</name> <phone>408 555 1212</phone> <office>344</office> </employee> <employee id=902> <name>Peter Pan</name> <phone>408 555 9918</phone> <office>216</office> </employee> </dept>

XPath /dept/@bldg /dept/employee/@id

Result 101 901 902

/dept/employee/name

<name>Peter Pan</name> <name>John Doe</name>

/dept/employee/name/text()

Peter Pan John Doe

12

IBM Software Group | DB2 Information Management Software

XPath: Wildcards
matches any tag name // is the descendent-or-self wildcard
*

<dept bldg=101> <employee id=901> <name>John Doe</name> <phone>408 555 1212</phone> <office>344</office> </employee> <employee id=902> <name>Peter Pan</name> <phone>408 555 9918</phone> <office>216</office> </employee> </dept>

XPath /dept/employee/*/text()

Result
John Doe 408 555 1212 344 Peter Pan 408 555 9918 216

/dept/*/@id //name/text() /dept//phone

901 902 Peter Pan John Doe <phone>408 555 1212</phone> <phone>408 555 9918</phone>

13

IBM Software Group | DB2 Information Management Software

XPath: Predicates
Predicates are enclosed in square brackets [] Can have multiple predicates in one XPath Positional predicates: [n] selects the n-th child

<dept bldg=101> <employee id=901> <name>John Doe</name> <phone>408 555 1212</phone> <office>344</office> </employee> <employee id=902> <name>Peter Pan</name> <phone>408 555 9918</phone> <office>216</office> </employee> </dept>

XPath /dept/employee[@id=902]/name
/dept[@bldg=101]/employee[office >300]/name //employee[office=344 OR office=216]/@id

Result <name>Peter Pan</name> <name>John Doe</name>

901 902

/dept/employee[2]/@id

902

14

IBM Software Group | DB2 Information Management Software

XPath: The Parent Axis

Current Parent

context: .

context: ..

<dept bldg=101> <employee id=901> <name>John Doe</name> <phone>408 555 1212</phone> <office>344</office> </employee> <employee id=902> <name>Peter Pan</name> <phone>408 555 9918</phone> <office>216</office> </employee> </dept>

XPath /dept/employee/name[../@id=902] /dept/employee/office[.>300] /dept/employee[office > 300]/office

/dept/employee[name=John Doe]/../@bldg /dept/employee/name[.=John Doe]/../../@bldg

Result <name>Peter Pan</name> <office>344</office> <office>344</office> 101 101

15

IBM Software Group | DB2 Information Management Software

Agenda
Overview Inserting XML data XPath XQuery Querying XML data using SQL/XML Querying XML data using XQuery Update & Delete operations with XML XML Indexes Other XML support

16

IBM Software Group | DB2 Information Management Software

What is XQuery?
A query language designed for XML data

XQuery Expressions XML Schema

www.w3.org/ XML/Schema www.w3.org/TR/XQuery & SQL/XML

XPath 2.0
www.w3.org/ TR/xpath20/

Functions & Operators

www.w3.org/TR/XQuery-operators/

XQuery 1.0 & XPath 2.0 Data Model

www.w3.org/TR/query-datamodel/

17

IBM Software Group | DB2 Information Management Software

XQuery
XQuery supports path expressions to navigate XML hierarchical structure XQuery supports both typed and untyped data XQuery lacks null values because XML documents omit missing or unknown data XQuery returns sequences of XML data

18

IBM Software Group | DB2 Information Management Software

FLWOR and PATH expressions

Like SELECT-FROM-WHERE expression in SQL an XQuery FLWOR expression may contain several clauses that begin with certain keywords.
for let where order Return

A Path expression in XQuery consists of a series of steps separated by slash characters

19

IBM Software Group | DB2 Information Management Software

XQuery: The FLWOR Expression

FOR: iterates through a sequence, bind variable to items LET: binds a variable to a sequence WHERE: eliminates items of the iteration ORDER: reorders items of the iteration RETURN: constructs query results

create table dept(deptID char(8),deptdoc xml); Input:

xquery for $d in db2-fn:xmlcolumn(DEPT.DEPTDOC')/dept let $emp := $d//employee/name where $d/@bldg > 95 order by $d/@bldg return <EmpList> {$d/@bldg, $emp} </EmpList>
<dept bldg=101> <employee id=901> <name>John Doe</name> <phone>408 555 1212</phone> <office>344</office> </employee> <employee id=902> <name>Peter Pan</name> <phone>408 555 9918</phone> <office>216</office> </employee> </dept>
20

IBM Software Group | DB2 Information Management Software

Agenda
Overview Inserting XML data XPath XQuery Querying XML data using SQL/XML Querying XML data using XQuery Update & Delete operations with XML XML Indexes Other XML support

21

IBM Software Group | DB2 Information Management Software

Two Worlds

XQuery Standard

SQL/XML Standards

22

IBM Software Group | DB2 Information Management Software

What you can and cant do with SQL

Plain SQL statements enable you to retrieve full XML documents but:
You cannot specify XML-based query predicates You cannot retrieve partial XML documents You cannot join, aggregate or order by fragments of XML documents using plain SQL

23

IBM Software Group | DB2 Information Management Software

SQL/XML queries
SQL/XML is designed to bridge between the SQL and XML worlds.
Part of the SQL standard includes specifications for embedding XQuery or XPath expressions within SQL statements

XPATH is a language for navigating XML documents to find elements or attributes XQuery includes support for XPATH

24

IBM Software Group | DB2 Information Management Software

New SQL/XML Functions in SQL 2006

XMLPARSE parses character/BLOB data, produces XML value XMLSERIALIZE converts an XML value into character/BLOB data XMLVALIDATE validates XML value against an XML schema and type-annotates the XML value XMLEXISTS determines if an XQuery returns a results (i.e. a sequence of one or more items) XMLQUERY executes an XQuery and returns the result sequence XMLTABLE executes an XQuery, returns the result sequence as a relational table (if possible) XMLCAST cast to or from an XML type

25

IBM Software Group | DB2 Information Management Software

xmlexists function
Restricts results based on an XML element value Syntax required prior to DB2 9.5:

select name from clients where xmlexists('$c/Client/Address[zip="95116"]' passing CLIENTS.CONTACT as "c")

New syntax allowed with DB2 9.5:

select name from clients where xmlexists('$CONTACT/Client/Address[zip="95116"]')

26

IBM Software Group | DB2 Information Management Software

Case sensitivity - Caution!

SQL is not case sensitive XPath and XQuery are case sensitive! DB2 objects (tables/columns) need to be put in upper case. Example:

select name from clients where xmlexists('$contact/Client/Address[zip="95116"]') select name from clients where xmlexists('$CONTACT/Client/Address[zip="95116"]')
27

IBM Software Group | DB2 Information Management Software

Curly quotes - Caution!

SQL, XPath and XQuery use straight quotes (for either single or double quotes depending the situation) Examples: The quotes in this example are curly, which is bad! 'The quotes in this example are straight, which is good!' The example below uses curly quotes, so it will fail: xquery db2-fn:xmlcolumn(CLIENTS.CONTACT)/Client/Address[zip=95116] It is typical to see this problem when copy/pasting from a Word or Powerpoint document

28

IBM Software Group | DB2 Information Management Software

xmlquery function
Retrieve one or more element values from our XML document

select xmlquery('$CONTACT/Client/email') from clients where status = 'Gold'

29

IBM Software Group | DB2 Information Management Software

Retrieving XML data using for and return clauses of XQuery

SELECT name, xmlquery ('for $e in $CONTACT/Client/email[1] return $e') FROM clients WHERE status = 'Gold'

30

IBM Software Group | DB2 Information Management Software

Retrieving and transforming XML into HTML

SELECT xmlquery('for $e in $CONTACT/Client/email[1]/text() return <p>{$e}</p>') FROM clients WHERE status = 'Gold'

31

IBM Software Group | DB2 Information Management Software

xmltable function: From XML to Relational

select t.comment#,i.itemname,t.customerID,Message from items i, xmltable('$COMMENTS/Comments/Comment' columns Comment# integer path 'CommentID', CustomerID integer path 'CustomerID', Message varchar(100) path 'Message') as t

32

IBM Software Group | DB2 Information Management Software

xmlelement function: From Relational to XML

select xmlelement (name "item",itemname), xmlelement (name "id", id), xmlelement (name "brand",brandname), xmlelement (name "sku",sku) from items where srp < 100

33

IBM Software Group | DB2 Information Management Software

Review: XMLTABLE vs XMLELEMENT

XML

XMLELEMENT function

XMLTABLE function

Relational

34

IBM Software Group | DB2 Information Management Software

Agenda
Overview Inserting XML data XPath XQuery Querying XML data using SQL/XML Querying XML data using XQuery Update & Delete operations with XML XML Indexes Other XML support

35

IBM Software Group | DB2 Information Management Software

Two Worlds

XQuery Standard

SQL/XML Standards

36

IBM Software Group | DB2 Information Management Software

Simple XQuery to return customer contact data xquery db2-fn:xmlcolumn('CLIENTS.CONTACT')

db2-fn:xmlcolumn is a function with a parameter that identifies the table name and column name of an XML column.

37

IBM Software Group | DB2 Information Management Software

FLWOR expression to retrieve client fax data

xquery for $y in db2-fn:xmlcolumn('CLIENTS.CONTACT')/Client/fax return $y

<fax>4081112222</fax> <fax>5559998888</fax>

38

IBM Software Group | DB2 Information Management Software

Path expression with additional filtering predicate

xquery db2-fn:xmlcolumn('CLIENTS.CONTACT')/Client/Address[zip="95116"]

39

IBM Software Group | DB2 Information Management Software

Querying DB2 XML data and returning results as HTML

xquery <ul> { for $y in db2-fn:xmlcolumn('CLIENTS.CONTACT')/Client/Address order by $y/zip return <li>{$y}</li> }</ul>

40

IBM Software Group | DB2 Information Management Software

Sample HTML

<ul> <li> <address> <street>9407 Los Gatos Blvd.</street> <city>Los Gatos</city> <state>ca</state> <zip>95302</zip> </address> </li> <Address> <street>4209 El Camino Real</street> <city>Mountain View</city> <state>CA</state> <zip>95302</zip> </address> </li> </ul>
41

IBM Software Group | DB2 Information Management Software

Embedded SQL within XQuery

db2-fn:sqlquery
A function which executes an SQL query and returns only the selected data The SQL Query passed to db2-fn:sqlquery must return XML data This XML data can then be further processed by XQuery xquery for $y in db2-fn:sqlquery('select comments from items where srp > 100')/Comments/Comment where $y/ResponseRequested='Yes' return ( <action> {$y//ProductID} {$y//CustomerID} {$y//Message} </action> )

42

IBM Software Group | DB2 Information Management Software

Joins with SQL/XML

create table dept (unitID char(8), deptdoc xml) create table unit (unitID char(8) primary key not null, name char(20), manager varchar(20), ... ) select u.unitID from dept d, unit u where XMLEXISTS ('$e//employee[name = $m] ' passing d.deptdoc as "e", u.manager as "m") select u.unitID from dept d, unit u where u.manager = XMLCAST(XMLQUERY('$e//employee/name' passing d.deptdoc as "e") as char(20))
43

IBM Software Group | DB2 Information Management Software

Joins with SQL/XML & Xquery

create table dept (unitID char(8), deptdoc xml) create table project(projectDoc xml) With SQL/XML: select XMLQUERY ('$d/dept/employee' passing d.deptdoc as "d") from dept d, project p where XMLEXISTS ('$e/dept[@deptID=$p/project/deptID]' passing d.deptdoc as "e", p.projectDoc as "p") Equivalent with Xquery: XQUERY for $dept in db2-fn:xmlcolumn("DEPT.DEPTDOC")/dept for $proj in db2-fn:xmlcolumn("PROJECT.PROJECTDOC")/project where $dept/@deptID = $proj/deptID return $dept/employee

44

IBM Software Group | DB2 Information Management Software

Agenda
Overview Inserting XML data XPath XQuery Querying XML data using SQL/XML Querying XML data using XQuery Update & Delete operations with XML XML Indexes Other XML support

45

IBM Software Group | DB2 Information Management Software

Update and Delete Operations

Update and Delete XML Data can be done in one of two ways:
SQL UPDATE and DELETE Statements Using the XQuery TRANSFORM expression

46

IBM Software Group | DB2 Information Management Software

Update example using SQL Update

UPDATE clients SET contact=( xmlparse(document <Client> <Address> <street>5401 Julio ave.</street> <city>San Jose</city> <state>CA</state> <zip>95116</zip> </Address> <phone> <work>4084633000</work> <home>4081111111</home> <cell>4082222222</cell> </phone> <fax>4087776666</fax> <email>newemail@someplace.com</email> </Client>')) where id = 3227
47

IBM Software Group | DB2 Information Management Software

Update example using XQuery TRANSFORM

UPDATE customers SET contactinfo = xmlquery( 'declare default element namespace "http://posample.org"; transform copy $newinfo := $c modify do insert <email2>my2email.gm.com</email2> as last into $newinfo/customerinfo return $newinfo' passing CONTACTINFO as "c") WHERE id = 100

48

IBM Software Group | DB2 Information Management Software

Agenda
Overview Inserting XML data XPath XQuery Querying XML data using SQL/XML Querying XML data using XQuery Update & Delete operations with XML XML Indexes Other XML support

49

IBM Software Group | DB2 Information Management Software

XML Indexing Examples

create unique index idx1 on customer(info) generate key using xmlpattern '/customerinfo/@Cid' as sql double; create index idx2 on customer(info) generate key using xmlpattern '/customerinfo/name' as sql varchar(40); create index idx3 on customer(info) generate key using xmlpattern '//name' as sql varchar(40);

create table customer( info XML);

<customerinfo Cid="1004"> <name>Matt Foreman</name> <addr country="Canada"> <street>1596 Baseline</street> <city>Toronto</city> <state>Ontario</state> <pcode>M3Z-5H9</pcode> </addr> <phone type="work">905-555-4789</phone> <phone type="home">416-555-3376</phone> <assistant> <name>Peter Smith</name> <phone type="home">416-555-3426</phone> </assistant> </customerinfo>
50 Matthias Nicola, IBM SVL

IBM Software Group | DB2 Information Management Software

XML Indexing Examples

create table customer( info XML);

Dont index everything! Too expensive for insert, update, delete !

<customerinfo Cid="1004"> <name>Matt Foreman</name> <addr country="Canada"> <street>1596 Baseline</street> <city>Toronto</city> <state>Ontario</state> <pcode>M3Z-5H9</pcode> </addr> <phone type="work">905-555-4789</phone> <phone type="home">416-555-3376</phone> <assistant> <name>Peter Smith</name> <phone type="home">416-555-3426</phone> </assistant> </customerinfo>
51 Matthias Nicola, IBM SVL

create index idx4 on customer(info) generate key using xmlpattern '//text()' as sql varchar(40);

IBM Software Group | DB2 Information Management Software

Agenda
Overview Inserting XML data XPath XQuery Querying XML data using SQL/XML Querying XML data using XQuery Update & Delete operations with XML XML Indexes Other XML support

52

IBM Software Group | DB2 Information Management Software

Other XML support

Based-table inlining and compression of small XML documents Can transform XML documents using XSLT functions XML Validation:
XML Schema support using XML Schema repositories XMLVALIDATE function during an INSERT BEFORE Trigger IS VALIDATED predicate on a CHECK constraint

Compatible XML Schema evolution using the UPDATE XMLSCHEMA command pureXML supported for UNICODE or non-UNICODE dbs with DB2 9.5 Annotated XML Schema Decomposition

53

IBM Software Group | DB2 Information Management Software

QuickLab #14a - SQL/XML and Xquery

Refer to the QuickLab #14a instructions

54

IBM Software Group | DB2 Information Management Software

Whats Next?
Application Developement e-Learning courses:
Lesson 11 Lesson 12 Lesson 13 Lesson 14 Lesson 15 Lesson 16 Concurrency & locking SQL PL stored procedures User-defined functions Triggers PureXML: SQL/XML & XQuery COMPLETED! Java, Ruby on Rails, PHP

Troubleshooting e-Learning course:

Lesson 17 Troubleshooting tips

55

IBM Software Group

Lesson 16: Java, Ruby on Rails, PHP

(DB2 on Campus Lecture Series)

Raul F. Chong
IBM Toronto Laboratory

2005 IBM Corporation

IBM Software Group | DB2 Information Management Software

Agenda

Java Ruby on Rails PHP

IBM Software Group | DB2 Information Management Software

Agenda

Java Ruby on Rails PHP

IBM Software Group | DB2 Information Management Software

IBM DB2 Driver for JDBC (aka JCC Driver)

Java driver optimized for all DB2 servers
DB2 for i5/OS (a.k.a iSeries or AS/400) DB2 for zOS (v8) DB2 for Linux, UNIX, Windows

Single driver can be used in type 2 and type 4 modes

db2jcc.jar (com.ibm.db2.jcc) Type 2 (Requires a DB2 client) Type 4 (Pure Java client, no need for a DB2 client)

db2jcc.jar is included in:

IBM Data Server client IBM Data Server Runtime Client IBM Data Server Driver for JDBC and SQLJ

IBM Software Group | DB2 Information Management Software

Basics
Can use IBM Data Studio
Based on Eclipse Use Java Perspective (Create a new project) For the Java project, add external JARs and class folders to the build path:
db2jcc.jar db2jcc_javax.jar db2jcc_license_cu.jar.

IBM Software Group | DB2 Information Management Software

Basics

IBM Software Group | DB2 Information Management Software

Database connection

IBM Software Group | DB2 Information Management Software

Inserting XML data from a file

IBM Software Group | DB2 Information Management Software

Inserting XML data from a character string

IBM Software Group | DB2 Information Management Software

Retrieving full XML documents with SQL

10

IBM Software Group | DB2 Information Management Software

Retrieving part of XML document with SQL/XML

11

IBM Software Group | DB2 Information Management Software

Retrieving part of XML document with SQL/XML

12

IBM Software Group | DB2 Information Management Software

Retrieving part of XML document with XQuery

13

IBM Software Group | DB2 Information Management Software

Update/delete part of XML document

14

IBM Software Group | DB2 Information Management Software

SQL PL Stored procedures

15

IBM Software Group | DB2 Information Management Software

Agenda

Java Ruby on Rails PHP

16

IBM Software Group | DB2 Information Management Software

What is Ruby on Rails?

Ruby: Object oriented language
Invented by Yukihiro Matsumoto (Matz) - 1995

Rails: Web Framework created using Ruby Creates database backed web-based applications Agile Software Development Hot new technology

17

IBM Software Group | DB2 Information Management Software

Ruby on Rails (RoR) highlights

Rails based on MVC (Model, View, Controller) architecture Convention over configuration Easy to perform testing In development, ships with a Ruby web server WEBrick Popular IDE: RadRail

18

IBM Software Group | DB2 Information Management Software

TPC Index http://www.tiobe.com/tpci.htm

19

IBM Software Group | DB2 Information Management Software

Job market (Source: indeed.com)

20

IBM Software Group | DB2 Information Management Software

Is Ruby on Rails the next Java?

Some of the most respected Java and PHP people think so Similar demographic as PHP but much faster adoption curve PHP community is trying to respond with their own MVC frameworks but most PHP programmers are not embracing MVC Enterprise qualities (MVC) but several orders of magnitude more productive than Java All-in-one framework: no assembly required, batteries included
21

IBM Software Group | DB2 Information Management Software

Productivity just an example

Credit: Java code by: Peter Szinek, MSc http://www.rubyrailways.com

22

IBM Software Group | DB2 Information Management Software

Ruby utilities

RI: Ruby Interactive IRB: Interactive Ruby Shell RubyGems

23

IBM Software Group | DB2 Information Management Software

Startup Toolkit for DB2 on Rails

Integrated installer that creates a complete DB2 Ruby on Rails development environment on a Windows PC. Source code available to do your own builds on other platforms Version 2.1 available on IBM alphaWorks http://www.alphaworks.ibm.com/tech/db2onrails Helps you install Ruby 1.8.4 and Rails 1.1.6, DB2 Express C 9 (includes pureXML support) IBM developed DB2 Ruby driver and DB2 Rails Adapter

24

IBM Software Group | DB2 Information Management Software

Agenda

Java Ruby on Rails PHP

25

IBM Software Group | DB2 Information Management Software

What is PHP?
PHP is the most popular Web language in the world today: 70% of the UNIX-based, Apache Web platform market 40% + of the overall market PHP is an open source, platform independent scripting language that is designed for web application development Originally created by Rasmus Lerdorf in 1995 Set of Perl scripts for tracking accesses to his online resume Original name was 'Personal Home Page Tools PHP 3.0 was a completed rewrite in 1997 Number of PHP developers has been growing ever since It is one of the most widely deployed web languages in the world today . Oh yes, PHP means (PHP Hypertext Preprocessor)
26

IBM Software Group | DB2 Information Management Software

Why PHP is popular ?

Rapid, iterative development cycles with a low learning curve

Robust, high-performance & scalable; stable & secure

Easily integrated into heterogeneous environments/systems

Proven through widespread deployment; vibrant community

27

IBM Software Group | DB2 Information Management Software

PHP - Key driver of LAMP Stack

LAMP Technical Linux, Apache HTTP Server, MySQL, PHP/Perl/Python Open Source web technology stack Often available on ISPs for reasonable monthly fees Pure open source software components LAMP Business (Appealing on many levels) 1. Rapid iterative development 2. Low cost of software acquisition 3. Scalable and robust scalable LAMP solutions requires significant investment PHP Concerns (CTOs) Lack of Professional level PHP developer skills Good software design skills are still important for large projects Considerable investment in .NET or J2EE already Many enterprises are not comfortable with MySQL

28

IBM Software Group | DB2 Information Management Software

DB2 Connection options for PHP

PHP Drivers for DB2 PHP Unified ODBC Extension (old, not good) ibm_db2 (only for DB2) PDO_ODBC Can be used with DB2 and other databases PDO_IBM Newer, more stable than PDO_ODBC

PDO = PHP Data Objects,

Standard objected-oriented db interface Introduced with PHP 5.1 Good for database agnostic applications

Use either ibm_db2 or PDO_IBM PHP drivers are offered under open-source licenses
http://www.php.net

29

IBM Software Group | DB2 Information Management Software

PHP extension for DB2 (ibm_db2)

Available from the PECL repository under the Apache 2.0 License
Developed and supported by IBM Full featured support for stored procedures and LOBs Fast, optimized for DB2 Works with both PHP 4 and PHP 5

30

IBM Software Group | DB2 Information Management Software

Your first ibm_db2 application

<?php $sql = "SELECT name, breed FROM ANIMALS WHERE weight < ?"; $conn = db2_connect($database, $user, $password); $stmt = db2_prepare($conn, $sql); $res = db2_execute($stmt, array(10)); while ($row = db2_fetch_assoc($stmt)) { print "{$row['NAME']} is a {$row['BREED']}.\n"; } ?>

31

IBM Software Group | DB2 Information Management Software

Configuring PHP for ibm_db2

Modify php.ini (Linux / UNIX):
extension=ibm_db2.so ibm_db2.instance_name=db2inst1

Modify php.ini (Windows):

extension=php_ibm_db2.dll

Or Check out Zend Core for IBM

32

IBM Software Group | DB2 Information Management Software

PHP Data Object (PDO) extension

Standard data access interface for PHP
Fast, light weight, and object oriented PDO_ODBC and PDO_IBM use DB2 libraries for native access Standard database API for multiple database servers Built into PHP 5.1 http://pecl.php.net/package/pdo http://pecl.php.net/package/PDO_ODBC

33

IBM Software Group | DB2 Information Management Software

What is Zend Core?

Zend Core is:

A seamless out-of-the-box supported PHP development and production environment
For:

Business-critical web applications

Delivering:

Reliability, Productivity and Flexibility needed for running PHP applications

Free download:
http://ibm.com/software/data/info/zendcore/
34

IBM Software Group | DB2 Information Management Software

Zend Core for IBM

Everything you need
Installs DB2 and IDS client Apache HTTP Server ( optional ) PHP 5.1.4 Popular PHP extensions including ibm_db2 PDO_IBM Many more DB2 Express-C server, IBM Cloudscape server ( optional ) Documentation complete PHP manual Sample applications for DB2

Easy to use and configure PHP environment

35

IBM Software Group | DB2 Information Management Software

Zend Core Management & Control

36

IBM Software Group | DB2 Information Management Software

Zend Core PHP Configuration

37

IBM Software Group | DB2 Information Management Software

Zend Core Remote Debug & Benchmark

38

IBM Software Group | DB2 Information Management Software

Whats Next?
Application Developement e-Learning courses:
Lesson 11 Lesson 12 Lesson 13 Lesson 14 Lesson 15 Lesson 16 Concurrency & locking SQL PL stored procedures User-defined functions Triggers PureXML: SQL/XML & XQuery Java, Ruby on Rails, PHP COMPLETED!

Troubleshooting e-Learning course:

Lesson 17 Troubleshooting tips

39

IBM Software Group

Lesson 17: Troubleshooting

(DB2 on Campus Lecture Series)

Raul F. Chong
IBM Toronto Laboratory

2005 IBM Corporation

IBM Software Group | DB2 Information Management Software

Agenda
Overview The help (?) command The Administration notification log The db2diag.log CLI Tracing DB2 Tracing The DB2 Information Center

IBM Software Group | DB2 Information Management Software

Agenda
Overview The help (?) command The Administration notification log The db2diag.log CLI Tracing DB2 Tracing The DB2 Information Center

IBM Software Group | DB2 Information Management Software

Problem Diagnosis

db2 ? <code>

Review Administration Notification Log

Review db2diag.log

Review the DB2 online manuals (Information Center)

Problem!

Review system with Operating System commands

Search for APARs, or known problems

Collect Traces, dumps, trap files, core files and contact IBM DB2 Tech Support
4

IBM Software Group | DB2 Information Management Software

Agenda
Overview The help (?) command The Administration notification log The db2diag.log CLI Tracing DB2 Tracing The DB2 Information Center

IBM Software Group | DB2 Information Management Software

db2 help: ?

db2 db2 db2 db2 db2

? ? ? ? ?

SQL0104N SQL104N SQL-0104 SQL-104 SQL-104N

IBM Software Group | DB2 Information Management Software

Finding More Information About Error Codes

To find out more information about an error code received, in the Command Editor input area, enter the code prefixed by a question mark and click the Execute button

Prefix error code by a question mark in the Input area

More information about the error code appears in the Output area

IBM Software Group | DB2 Information Management Software

SQLCODE and SQLSTATE Meaning

SQLCODE
SQLCODE = 0; the statement was successful SQLCODE > 0; the statement was successful, but returned a warning SQLCODE < 0; the statement was unsuccessful and returned an error

SQLSTATE
SQLSTATE is a five-character string that conforms to the ISO/ANSI SQL92 standard First two characters are known as the SQLSTATE class code Class code of 00 means the command was successful Class code of 01 implies a warning Class code of 02 implies a not found condition All other class codes are considered errors

IBM Software Group | DB2 Information Management Software

Agenda
Overview The help (?) command The Administration notification log The db2diag.log CLI Tracing DB2 Tracing The DB2 Information Center

IBM Software Group | DB2 Information Management Software

DB2 Administration Notification Log

Log with diagnosis information at the point failure On Linux/UNIX platforms, the administration notification log is a text file called instance.nfy On Windows, all administration notification messages are written to the Event Log The DBM configuration parameter notifylevel specifies the level of information to be recorded:
0 -- No administration notification messages captured (not recommended) 1 -- Fatal or unrecoverable errors 2 -- Immediate action required 3 -- Important information, no immediate action required (default) 4 -- Informational messages

10

IBM Software Group | DB2 Information Management Software

Agenda
Overview The help (?) command The Administration notification log The db2diag.log CLI Tracing DB2 Tracing The DB2 Information Center

11

IBM Software Group | DB2 Information Management Software

db2diag.log
The information in the db2diag.log includes:
The location reporting an error. Application identifiers allow you to match up entries pertaining to an application on the db2diag.logs of servers and clients. A diagnostic message (beginning with "DIA") explaining the reason for the error. Any available supporting data, such as SQLCA data structures and pointers to the location of any extra dump or trap files.

Entries in the db2diag.log:

Administrative events - i.e. backup/restore start and finished. Error information Other information, including pointers to dump files

12

IBM Software Group | DB2 Information Management Software

Locating the db2diag.log

WIN NT/2000 (default)
...\sqllib\<instance name>\db2diag.log

UNIX (default)
.../sqllib/db2dump/db2diag.log

optional: set alternate location in dbm cfg

db2 update dbm cfg using diagpath <path>

The verbosity of diagnostic text is determined by dbm cfg (diaglevel)

range is 0 to 4 (default is 3) most verbose is 4

13

IBM Software Group | DB2 Information Management Software

Agenda
Overview The help (?) command The Administration notification log The db2diag.log CLI Tracing DB2 Tracing The DB2 Information Center

14

IBM Software Group | DB2 Information Management Software

CLI Tracing
For CLI and Java Applications Configure db2cli.ini at the database CLIENT (application) Stored in /sqllib directory

[common] trace=0 tracerefreshinterval=300 tracepathname=/path/to/writeable/directory traceflush=1

15

IBM Software Group | DB2 Information Management Software

Agenda
Overview The help (?) command The Administration notification log The db2diag.log CLI Tracing DB2 Tracing The DB2 Information Center

16

IBM Software Group | DB2 Information Management Software

Low level Tracing (db2trc)

Only use when requested by DB2 Tech support You need to be able to reproduce the problem.
db2trc on do actions that lead to crash/error db2trc dump <outfile> dumps a binary datafile which can be analyzed db2trc fmt|flw <input dump file> <output file> turns binary outfile into readable text. db2trc off turn off tracing (dont forget this!)

Send files to DB2 Tech support for their analysis.

17

IBM Software Group | DB2 Information Management Software

Dealing with DB2 Technical Support

This section is not applicable if using the DB2 Express-C unwarranted (free) license. Before you do anything, ensure that clients and server are at the SAME Fixpak level. Easy to run into strange problems if they are not. If you are integrating several products, mention them all. Attempt to reduce the problem, provide scripts to setup a test case if possible COLLECT EVERYTHING NEEDED FOR SUPPORT:
db2support utility. example: db2support . -d MyDbName -c
18

IBM Software Group | DB2 Information Management Software

Agenda
Overview The help (?) command The Administration notification log The db2diag.log CLI Tracing DB2 Tracing The DB2 Information Center

19

IBM Software Group | DB2 Information Management Software

The DB2 Information Center

Online searchable manuals DB2 Information Center
V9: http://publib.boulder.ibm.com/infocenter/db2luw/v9/index.jsp V9.5: http://publib.boulder.ibm.com/infocenter/db2luw/v9r5/index.jsp

20

IBM Software Group | DB2 Information Management Software

Whats Next?
Application Developement e-Learning courses:
Lesson 11 Lesson 12 Lesson 13 Lesson 14 Lesson 15 Lesson 16 Concurrency & locking SQL PL stored procedures User-defined functions Triggers PureXML: SQL/XML & XQuery Java, Ruby on Rails, PHP

Troubleshooting e-Learning course:

Lesson 17 Troubleshooting tips COMPLETED!

21

IBM Software Group | DB2 Information Management Software

Whats Next?
Congratulations! You finished the e-learning DB2 on Campus Lecture series! Review the book, exercises and videos again Write the DB2 on Campus Exam (to be ready by July 2008) Keep checking this space (channeldb2.com, ibm.com/db2/express) for updates to this course, and other workshops/courses!

22