IBM System i

DB2 for i5/OS V5R4 Enhancements for Index Advisor and Autonomic Indexes Or Make Your SQL Code Fly in V5R4
March 2007

Satid Singkorapoom Advisory Technical Sales Specialist, System i Sales, IBM Thailand

i want stress-free IT. i want control. i want an i.

IBM System i

Background
This presentation is based on the following articles:
DB2 for i5/OS Redefines On Demand for Indexing by Scott Forstie and Jim

Flanagan, IBM Systems Magazine System i Edition, May 2006 at URL:

http://www.ibmsystemsmagazine.com/ME2/Audiences/dirmod.asp?sid=&nm=&type=Publishing&mod=Publi cations%3A%3AArticle&mid=8F3A7027421841978F18BE895F87F791&tier=4&id=00D363297CDB4F54BA F9FA42E9286218&AudID=1E8FEE745A284521B6CFB3FD70B49099

The Optimizer Takes Its Own Advice by Rob Downer and Shantan Kethireddy, IBM Systems Magazine System i Edition, June 2006 at URL:
http://www.ibmsystemsmagazine.com/ME2/Audiences/dirmod.asp?sid=&nm=&type=Publishing&mod=Publi cations%3A%3AArticle&mid=8F3A7027421841978F18BE895F87F791&tier=4&id=15C863E9852B45F696 0150532D43F2A5&AudID=1E8FEE745A284521B6CFB3FD70B49099

Simplify DB2 for i5/OS index advice by Scott Forstie, IBM developerWorks article, Jan 2007 at URL:
http://www-128.ibm.com/developerworks/db2/library/techarticle/dm-0701forstie/

i want an i.

2007 IBM Corporation

IBM System i

Background
SQL Performance Tuning facilities have been the strength of DB2 for i5/OS for many years. Such useful features have been appreciated by DB2 SQL programmers:
Detailed query optimizer feedback messages in the job log QAQQINI query control file Summary and Detailed SQL performance monitor System-wide SQL statement cache Visual Explain with Index and Statistics Advisors Automatic column statistics collection

This presentation describes three recent SQL performance enhancements to V5R4 of DB2 for i5/OS.

i want an i.

2007 IBM Corporation

IBM System i

Background
All V5R4 enhancements discussed in this presentation apply to the new SQE Engine. The enhanced index advisor and its condenser also support CQE.
Optimizer
Query Dispatcher CQE Optimizer

SQE Optimizer

DB2 Data Storage & Management

SLIC CQE Database Engine

SQE Optimizer SQE Primitives SQE Statistics Manager

i want an i.

2007 IBM Corporation

IBM System i

Agenda
New Index Advisor features in V5R4 Condensed Index Advisor for the creation of perfect indexes Benefits of Autonomic Indexes

i want an i.

2007 IBM Corporation

IBM System i

New Index Advisor features in V5R4 Condensed Index Advisor for the creation of perfect indexes Benefits of Autonomic Indexes

i want an i.

2007 IBM Corporation

IBM System i

Pre-V5R4 Index Advisor

Invoking Index and statistics advisors dialog Highlighting steps with advised Index and statistics

Pre-V5R4, Index Advisor is accessed by browsing job log for query optimizer messages, Visual Explain (above), and Detailed DB monitor.

i want an i.

2007 IBM Corporation

IBM System i

Note
Pre-V5R4, Index Advisor is accessed by browsing job log for query optimizer messages, Visual Explain (above), and Detailed DB monitor. Having a good indexing strategy is imperative to achieving high query performance on any database platform. A comprehensive and efficient indexing strategy can mean the difference between queries that complete in seconds rather than minutes or hours. Implementing such a strategy is an important task. DB2* Universal Database* (UDB) for i5/OS* wants to lend a hand in the form of Index Advised, which is a feature that enumerates definitions for nonexistent indexes with a high potential for improving a query's performance. Not only does Index Advised help guide the creation of permanent indexes, but it's also the basis for creating a new flavor of temporary index in V5R4 called a Maintained Temporary Index (MTI). An understanding of how Index Advised works, how it's used to create MTIs and which tools are available to analyze this advice is essential for anyone looking to create a highly effective indexing strategy.

i want an i.

2007 IBM Corporation

IBM System i

Enhanced Index Advisor in V5R4

Index Advisor is built directly into the SQE optimizer. As the queries execute, DB2 deposits index advice entries into a new V5R4 catalog table named SYSIXADV in QSYS2 schema. Always On - Index Advisor automatically analyzes all SQL queries based on the rules outlined in the "Indexing and Statistics Strategy" whitepaper.
(www.ibm.com/servers/enable/site/education/abstracts/indxng_abs.html)

SQE uses advised indexes and Plan Cache as the basis for creating Maintained Temporary Index (MTI) more on MTI later.

i want an i.

2007 IBM Corporation

IBM System i

Note
Index Advised itemizes the set of index definitions that the optimizer deems necessary to achieve better query performance. Because this feature is built directly into the optimizer, all SQL queries are automatically analyzed for new indexing opportunities based on the rules outlined in the "Indexing and Statistics Strategy" whitepaper (at http://www.ibm.com/servers/enable/site/education/abstracts/indxng_abs.html)

i want an i.

2007 IBM Corporation

IBM System i

Index Advisor
Advised indexes can be found:
1. Pre-V5R4: within database monitor output under Table Scan (type 3000) records. Only one advised index allowed per one record of this type. 2. V5R4 onwards: within database monitor output under the new type 3020 records. All advised indexes for the same table are now recorded. 3. Within Visual Explain under the Index and Statistics Advisor 4. From iSeries Navigator's Index Advisor facility 5. Via the SQL Plan Cache by filtering on "Queries with index advised

In iSeries Navigator, advised indexes can be displayed:

At database instance level Within a specific schema (library) of a specific database instance For a specific table

i want an i.

2007 IBM Corporation

IBM System i

Note
With V5R4, index advice can be found in four main areas: 1. Within database monitor output under 3020 records 2. Within Visual Explain under the Index and Statistics Advisor 3. From iSeries* Navigator's Index Advisor facility 4. Via the SQL Plan Cache by filtering on "Queries with index advised" The first two areas existed prior to V5R4. The latter two, introduced with V5R4, are the focus of this article. First, we'll explain the new Index Advisor that's shipped with V5R4 and detail, through an example, how it can be used to implement an improved index strategy. Next, we'll show how to locate the advice and elaborate on what it means. Then we'll examine how this index advice information can be used in conjunction with the Index Evaluator to provide a more complete picture of your query environment. Index Advisor Information in DBMON Output Prior to V5R4, one way to extract information on advised indexes is from the DB Monitor output by looking for the Table Access records (type 3000). But each type-3000 record only contains a single advised index. If a query receives multiple advised indexes from the optimizer when scanning a table, there will be only one index advisor entry in the output. Moreover, if the query is implemented without accessing a table but an index is still advised, there will not be any entry recorded because there will not be a type 3000 record for that table. Starting in V5R4, all advised indexes are recorded in DB monitor output with the new record type 3020.

i want an i.

2007 IBM Corporation

IBM System i

Invoking Index Advisor from iSeries Navigator

i want an i.

2007 IBM Corporation

IBM System i

Index Advice Output Example

3 Levels of Index Advice

Default sorting

i want an i.

By default, the list is sorted by 3 columns: Times Advised for Query Use + Most Expensive Query + Table Size, all in descending 2007 IBM Corporation order.

IBM System i

Complete Index Advice Attributes

Number of currently displayed entries against total number of entries

i want an i.

2007 IBM Corporation

IBM System i

Note: Index Advice Attributes Some Highlights

Column Name
Table for Which Index was advised

Description
The optimizer is advising creation of a permanent index over this table. This is the long name format for the table. The advice was generated because the table was queried and no existing permanent index could be used to improve the performance of the query. Schema or library name for the table. Column names for the advised index. The order of the column names is important. The names should be listed in the same order on the CREATE INDEX SQL statement, unless the leading, order independent key information indicates that the ordering can be changed. The keys at the beginning of the KEY_COLUMNS_ADVISED field that could be reordered and still satisfy the index being advised. Binary Radix or Encoded Vector Index The timestamp representing the last time this index was advised for a query. The cumulative number of times this index has been advised. This count should stop increasing when a matching permanent index is created. The row of advice will remain in this table until the user removes it.

Schema Keys Advised

Leading Keys Order Independent Index Type Advised Last Advised for Query Use Times Advised for Query Use

i want an i.

2007 IBM Corporation

IBM System i

Note: Index Advice Attributes (continued)

Column Name
Estimated Index Creation Time Reason Advised Most Expensive Query Estimate Average of Query Estimates (seconds) Rows in Table when Advised MTI USED MTI CREATED MTI LAST USED

Description
Estimated time required to create this index. Coded reason why index was advised. Possible values: Row selection, Ordering/Grouping, Row selection and Ordering/Grouping. Execution time in seconds of the longest-running query that generated this index advice. Average execution time in seconds of all queries that generated this index advice. Number of rows in the table for which the index is being advised, for the last time this index was advised. The number of times that this specific MTI has been used by the optimizer. The optimizer will stop using a matching MTI when a permanent index is created. The number of times that this specific MTI has been created by the optimizer. MTIs do not persist across system IPLs. The timestamp representing the last time this specific MTI was used by the optimizer to improve the performance of a query. The MTI Last Used field can be blank, which indicates that an MTI that exactly matches this advice has never been used by the queries that generated this index advice.
2007 IBM Corporation

i want an i.

IBM System i

Keys Advised and Leading Keys Order Independent

When generating advice, SQE first looks for local selection equalpredicates (e.g., table.column=constant).
Columns against equal-predicates can be moved around in their position with other equal-predicate columns (Order Independent). Non-equal selection columns are generally order dependent.

For table join, SQE adds join columns to the advised column list following the columns from local selection predicates. For grouping or ordering, SQE may include the grouping and ordering columns after the local selection equal predicates.
Only when all of the columns in the "group by or "order by" list are in the same table.

i want an i.

2007 IBM Corporation

IBM System i

Note
In general, Index Advised first looks for local selection equal predicates (e.g., table.column=constant). (Note: Index Advised can advise radix indexes or encoded vector indexes, or EVIs, to handle selection.) For joining, the optimizer adds join columns to the advised column list following the columns from local selection equal predicates. If present, columns from in-list predicates or a column from a single inequality local selection predicate (e.g., >, <, != etc.) are appended to the list. These columns appear last because the number of records they select can vary widely. Also, whereas columns against equal predicates can be moved around in their position with other equal predicate columns, non-equal selection columns don't provide this flexibility. For grouping or ordering, Index Advised may include the grouping and ordering columns after the local selection equal predicates, but this is only true when all of the columns in the "group by or "order by" list come from the same table. Additionally, the optimizer can recommend using a multiple-key column EVI to handle star-schema-type joins or simple grouping.

i want an i.

2007 IBM Corporation

IBM System i

Keys Advised and Leading Keys Order Independent

Although all entries in index advice are unique, some entries can present repetitive group of key columns.
Example: when the same group of columns are advised in more than one entry but in a different order and the order of these keys is unimportant. This opportunity for reordering of the advised keys is indicated via the columns displayed in "Leading Keys Order Independent" column.

These columns can be rearranged to create only one index that can still satisfy all instances of advice. This is when the new Condensed Advised Indexes function is useful more on this in the next section.

i want an i.

2007 IBM Corporation

IBM System i

Note
In its raw form, index advice can be unnecessarily verbose. Although all entries in the table are unique, some entries can be redundant. One good example of this occurs when the same columns are advised but in a different order and the order of these keys is unimportant. In this case, the columns can be arranged to create one index that covers both instances of advice. This opportunity for reordering of the advised keys is indicated via the "Leading Keys Order Independent" column. The keys listed in this column are the leading keys which could be reordered while still satisfying the index being advised.

i want an i.

2007 IBM Corporation

IBM System i

iSeries Navigator - User Actions on Index Advice

Clear All Advised Indexes: as indexes are created or tables are dropped, the old advice entry isn't removed automatically by the system. It can be considered a historical table and as such will keep accumulating rows, so you may want to clear out the table occasionally. Condense Advised Indexes: provides an intelligent view for the creation of advised perfect index. Prune Advised Indexes: removes all index advice rows where the table that the advice was created for no longer exists. Create Index: provides a view of the recommended index to be created from the New Index dialog, including index type, key columns, column order, etc. Remove from List: delete the selected entries from SYSIXADV table. Show statements: display all the SQL statements that have advised index for the table referenced in the Table Which Index Was Advised column. Table functions of the one on which the entry is based.

i want an i.

2007 IBM Corporation

IBM System i

iSeries Navigator - User Actions on Index Advice (cont.)

i want an i.

2007 IBM Corporation

IBM System i

Note
Many actions can be taken on the entries in the list. First we'll explain the actions that work against the entire list. For example, the underlying datastore for the advice is an SQL table (QSYS2/SYSIXADV), which is populated by the database but not maintained. As indexes are created or tables are dropped, the old advice isn't removed automatically by the system. It can be considered an historical table and as such will keep accumulating rows, so you may want to clear out the table occasionally. You can use the Clear Advised Indexes menu option. For the databasewide and schema-wide views you have an additional option to "Prune Advised Indexes." This option removes all index advice rows where the table that the advice was created for no longer exists. There's also a menu option to remove the selected row of index advice from the list. Let's describe the menu options for each of the indexes advised in the list: The Create Index option, which you can use to display the advised index in the New Index dialog, provides a view of the recommended index as it would be if you created it from the New Index dialog, including index type, key columns, column order, etc. Additionally, on the New Index dialog you can use the Show SQL button to display the CREATE INDEX DDL statement in Run SQL Scripts. From there, you can copy and paste this SQL statement into your change management system. In our INDEXADVIC example, two rows of advice can be satisfied by creating an index with key columns SALES_PERSON ASC and SALES_DATE ASC. A third row is satisfied by creating an index with key columns SALES_DATE ASC, REGION ASC and SALES_PERSON ASC.

i want an i.

2007 IBM Corporation

IBM System i

Note
Show Statements shows all statements in the Query Plan Cache (QPC) that referenced this table and generated index advice. With this information, determine which other query statements have generated index advice over the same table, and what additional positive impact may be gained by creating the index. (Note: This option only displays SQL statements that were executed using the SQE query engine.) A cascading Table menu is available for each index advice that allows you to perform many of the table actions that you can perform for a particular table in the list - view the contents, definition, or description and several other options. The most intriguing of these options may be the Show Indexes menu option. If you select this option, you can launch the Show Index list to see all of the indexes currently defined over the table. By using the Index Advice list in conjunction with the Show Indexes list, you can work on your table's indexing strategy. For example, if one index-advice row shows that you should have an index with certain columns but you already have some of those columns in an existing index, try adding a column recommended by the Index Advisor to your existing index..

i want an i.

2007 IBM Corporation

IBM System i

New Index Advisor features in V5R4 Condensed Index Advisor for the creation of perfect indexes Benefits of Autonomic Indexes

i want an i.

2007 IBM Corporation

IBM System i

In Search of the Perfect Indexes

A perfect index is an index created to satisfy as many queries as possible that run against the table on which this index is built.
Less indexes implies less complex DB environment to manage

In the past, DB2 for i5/OS provided Index Advisor to help SQL programmers in identifying the set of efficient candidate key columns
Each query must be examined by an SQL programmer for the advised indexes in the job log or Visual Explain Practically, too many queries impose daunting amount of time and effort for programmers to analyze and collapse many advised indexes to as few as possible not an easy task!

Here comes a new tool in V5R4 for this task

i want an i.

2007 IBM Corporation

IBM System i

Condense Advised Indexes for perfect indexes

When you see columns displayed in Leading Keys Order Independent of the Index Advisor, there is an opportunity for you to collapse multiple entries into one to facilitate creation of a perfect index. Use the new Condensed Advised Indexes function in iSeries Navigator to do this hard work for you!
Requires PTFs: SI25391, SI25469, and SI25470 for 5722-SS1 at V5R4M0 Also the latest Service Pack for iSeries Access for Windows V5R4M0 Like Index Advisor, it is available at database instance, schema, and table levels

Condensed advice entries are materialized from a new V5R4 view named CondensedIndexAdvice provided in QSYS2 schema.
CondensedIndexAdvice view uses a new V5R4 UDTF named Condense_Advice() to process QSYS2/SYSIXADV for the materialized result Example query for result: SELECT * FROM QSYS2.CONDENSEDINDEXADVICE WHERE table_schema = .. ;

i want an i.

2007 IBM Corporation

IBM System i

Note
The condense index advice technology requires the following V5R4M0 5722-SS1 PTFs to be installed on the server: SI25391, SI25469, and SI25470. When index advice is condensed, the raw index advice is placed into a summary form. Some fields within the raw advice are specific to the instance of advice and have no value in a summarized form. Other fields are quite important to preserve and weight the importance of the condensed advice. The condensed index advice permits the same iSeries Navigator actions as those found under the Index Advisor, with one exception. Since the condensed index advice does not exist in a permanent form, there is no "Remove from List" action.

i want an i.

2007 IBM Corporation

IBM System i

CondensedIndexAdvice view definition

QSYS2.CONDENSEDINDEXADVICE ( TABLE_NAME FOR COLUMN TABLE_SCHEMA FOR COLUMN SYSTEM_TABLE_NAME FOR COLUMN PARTITION_NAME FOR COLUMN KEY_COLUMNS_ADVISED FOR COLUMN INDEX_TYPE LAST_ADVISED FOR COLUMN TIMES_ADVISED FOR COLUMN ESTIMATED_CREATION_TIME FOR COLUMN LOGICAL_PAGE_SIZE FOR COLUMN MOST_EXPENSIVE_QUERY FOR COLUMN AVERAGE_QUERY_ESTIMATE FOR COLUMN TABLE_SIZE NLSS_TABLE_NAME FOR COLUMN NLSS_TABLE_SCHEMA FOR COLUMN TABNAME TABSCHEMA SYS_TNAME TABPART KEYSADV LASTADV TIMESADV ESTTIME "PAGESIZE" QUERYCOST QUERYEST NLSSNAME NLSSSCHEMA VARCHAR(258) CCSID 37 NOT NULL , CHAR(10) CCSID 37 NOT NULL , CHAR(10) CCSID 37 NOT NULL , VARCHAR(128) CCSID 37 DEFAULT NULL , VARCHAR(16000) CCSID 37 DEFAULT NULL , CHAR(14) CCSID 37 DEFAULT NULL , TIMESTAMP DEFAULT NULL , BIGINT DEFAULT NULL , INTEGER DEFAULT NULL , INTEGER DEFAULT NULL , INTEGER DEFAULT NULL , INTEGER DEFAULT NULL , BIGINT DEFAULT NULL , CHAR(10) CCSID 37 DEFAULT NULL , CHAR(10) CCSID 37 DEFAULT NULL )

-- Example: Condensed index advice for a range of schemas, where the average -- query estimate driving the index advice is greater than 10 seconds SELECT * FROM QSYS2.CONDENSEDINDEXADVICE WHERE table_schema LIKE 'Q%' AND average_query_estimate > 10 ;

i want an i.

2007 IBM Corporation

IBM System i

Note
Besides providing the user with the condensed column key order, the condensed advice includes contextual information to aide in determining the importance of the advice. Columns, such as "Times Advised for Query Use" and "Average of Query Estimates", provide a sense of how beneficial the permanent index could be to this environment. While the "Estimated Index Creation Time" column, helps to determine whether the creation of the index needs to be a scheduled activity.

i want an i.

2007 IBM Corporation

IBM System i

Condense_Advice() UDTF definition

CREATE FUNCTION QSYS2.Condense_Advice(TABLE_SCHEMA VARCHAR(128), TABLE_NAME VARCHAR(128) ) RETURNS TABLE(TABLE_PARTITION VARCHAR(128), KEY_COLUMNS_ADVISED VARCHAR(16000), INDEX_TYPE CHAR(14), LAST_ADVISED TIMESTAMP, TIMES_ADVISED BIGINT, ESTIMATED_CREATION_TIME INTEGER, LOGICAL_PAGE_SIZE INTEGER, MOST_EXPENSIVE_QUERY INTEGER, AVERAGE_QUERY_ESTIMATE INTEGER, TABLE_SIZE BIGINT, NLSS_TABLE_NAME CHAR(10),NLSS_TABLE_SCHEMA CHAR(10)) LANGUAGE C NOT DETERMINISTIC READS SQL DATA CALLED ON NULL INPUT SCRATCHPAD 325064 DISALLOW PARALLEL FINAL CALL CARDINALITY 1 EXTERNAL NAME 'QSYS/QDBSSUDF2(CONDENSE_ADVICE)' PARAMETER STYLE DB2SQL; -- Example: Query the condenser UDTF directly, providing selection criteria -- and ordering the results of the 15 most important condensed entries SELECT * FROM TABLE(QSYS2.CONDENSE_ADVICE('CRPDTA','B123456')) AS a WHERE a.table_size >(1024*1024) AND MONTH(last_advised)= MONTH(CURRENT TIMESTAMP) ORDER BY average_query_estimate DESC FETCH FIRST 15 ROWS ONLY ;

i want an i.

2007 IBM Corporation

IBM System i

Note
The Condense_Advice UDTF used by the new CondensedIndexAdvice view can also be accessed directly by user queries.

i want an i.

2007 IBM Corporation

IBM System i

How Condense Advised Indexes works example 1

Advised Indexes Entries
Keys Advised C1, C2, C3 C1, C3, C2 Leading Keys Order independent C1, C2, C3 C1

In this example, the columns in the second entry cannot be reordered. To satisfy the index advice, a permanent index would need to have keys in the order { C1, C3, C2 }. However, the first entry of advice has greater flexibility. The Leading Keys Order Independent detail shows that they can be in any order, so you can condense the two entries of advice into a single permanent index by reordering { C1, C2, C3 } into { C1, C3, C2 }. CREATE INDEX ON TABLE_A(C1, C3, C2) will cover both entries

i want an i.

2007 IBM Corporation

IBM System i

Note
In this example, the second row of advice is not pliable. To satisfy the index advice, a permanent index would need to have keys in the order { C1, C3, C2 }. However, the first row of advice has great flexibility. The leading key order independent detail shows that the keys could be in any order, so you can condense the two rows of advice into a single permanent index by reordering { C1, C2, C3 } into { C1, C3, C2 }.

i want an i.

2007 IBM Corporation

IBM System i

How Condense Advised Indexes works example 2

Advised Indexes Entries
Keys Advised C1, C2, C3 C1, C3, C2 C1, C3, C2, C4 Leading Keys Order independent C1, C2, C3 C1 C1, C2, C3, C4

If you extend the previous example to include a third entry of advice as indicated above, you are still able to conclude that the addition of a single permanent index { C1, C3, C2, C4 } should provide SQE all it needs to permanently improve the performance of these three queries. CREATE INDEX ON TABLE_A(C1, C3, C2, C4) will cover all three entries

i want an i.

2007 IBM Corporation

IBM System i

Note
If you extend the preceding example to include a third row of advice, you are still able to conclude that the addition of a single permanent index { C1, C3, C2, C4 } should provide the optimizer all it needs to permanently improve the performance of these queries.

i want an i.

2007 IBM Corporation

IBM System i

So, from these sample Advised Indexes

You Select This Function

i want an i.
2007 IBM Corporation

IBM System i

You get the following Condensed Advice

The Index Advisor Condenser produces this result for you at a mouse click!
The condensed advice includes contextual information to aide in determining the importance of the advice. Columns, such as "Times Advised for Query Use" and "Average of Query Estimates", provide a sense of how beneficial the permanent index could be to this environment. While the "Estimated Index Creation Time" column helps to determine whether the creation of the index needs to be a scheduled activity. The number of times advised is summed together for display, and the longest estimated creation time is used. So, not only are the key columns condensed, but other columns are processed as well.

i want an i.

2007 IBM Corporation

IBM System i

Managing Existing Indexes with Index Evaluator

With Index Advisor and its Condenser, you are capable of creating a few perfect indexes that can satisfy as many queries as possible for a table What about other existing indexes of that table that have been around?
Are there any of them that are rarely used or not used at all since they were created? Is there a tool to help identify these ? Yes, use Index Evaluator.

Index Evaluator is invoked by right-clicking the table of your interest in iSeries Navigator and select option Show Indexes
Available since V5R2 Show all of the indexes currently defined over the table of your interest.

i want an i.

2007 IBM Corporation

IBM System i

Invoking Index Evaluator

i want an i.

2007 IBM Corporation

IBM System i

Index Evaluator Provides Usage Information

When both columns Query Used Count and Query Statistics Used Count of any index entry display a value of 0 (zero), this indicates a candidate index to be deleted due to lack of use.
Except for Temporary Indexes. Leave them to the system. Make sure you know how long these counts have been active. If in doubt, you can restart the index usage counting by issuing the following command against the index object to reset the counting to zero: CHGOBJD OBJ(schema/index) OBJTYPE(*FILE) USECOUNT(*RESET)). You then return to display the usage counting some times later.

Also look at the Last Used Date column. This column is updated when native read/write operation is done to a keyed logical file or physical file. So, a date displayed in this column indicates how long ago the index was used by the native operation.
This column information was newly added in V5R4.

i want an i.

2007 IBM Corporation

IBM System i

Note
For existing indexes, it might be difficult to gauge the effect of a change in index strategy because the usage statistics have accumulated over time. Use the Change Object Description (CHGOBJD) command to reset the statistics for existing indexes: CHGOBJD OBJ(schema/index) OBJTYPE(*FILE) USECOUNT(*RESET))

i want an i.

2007 IBM Corporation

IBM System i

New Index Advisor features in V5R4 Condensed Index Advisor for the creation of perfect indexes Benefits of Autonomic Indexes

i want an i.

2007 IBM Corporation

IBM System i

Autonomic Indexes
Also called Maintained Temporary Index (MTI) Classic Query Engine (CQE) has had the ability to create a temporary index but its usage is restricted to a single job and single query only. MTIs created by SQE can be shared across queries and jobs like permanent indexes.

i want an i.

2007 IBM Corporation

IBM System i

Maintained Temporary Index (MTI)

SQE automatically controls the creation, maintenance and deletion of MTIs.
Creation occurs when SQE can justify that a query's performance can be enhanced sufficiently by using an MTI.

An MTI is updated as the base table is updated.

The system cost of maintaining an MTI is similar to the cost of maintaining a permanent index.

An MTI is deleted when:

The last access plan, in the system plan cache, that refers to that MTI is removed (e.g. IPL). Needs QAQQINI parameter CACHE_RESULTS = *SYSTEM When a permanent index is created that covers the same columns as the MTI

i want an i.

2007 IBM Corporation

IBM System i

Note
While Index Advised is a highly effective end-user tool for improving query performance, it's also the basis for the creation of another new tool in the Structured Query Engine (SQE) optimizer's toolbox the MTI. An MTI is a temporary radix index that the SQE optimizer can create to improve a query plan's performance. It has an advantage over other temporary result sets created by the optimizer: it's maintained. Like a permanent radix index, an MTI is updated as the underlying table is updated. Therefore, the cost of maintaining an MTI is similar to the cost of maintaining a permanent radix index. The optimizer creates an MTI for a query plan for two main reasons. One, an index is functionally necessary to implement a query when a temporary result set isn't allowed. A query running under a sensitive cursor is one example where a non-maintained temporary result set can't be used to implement a query. The other reason the optimizer can choose to create an MTI is that the query will perform better with an index. While the Classic Query Engine (CQE) has the ability to create a temporary index, its usage is restricted to a single job and single query only. SQE's MTIs, on the other hand, can be shared across queries and jobs just like a permanent index. SQE automatically controls the creation, sharing and deletion of MTIs. Creation of an MTI occurs when the optimizer determines that a query's performance can be enhanced sufficiently by using the MTI, thus justifying the MTIs create cost. An MTI is deleted when the last plan in the system plan cache referencing it is removed (e.g., IPL) or when a permanent index is created that covers the same columns as the MTI. To avoid incurring MTI create costs after an IPL, consider creating permanent indexes for regularly used MTIs. MTI usage information can be viewed using iSeries* Navigator or the Database Monitor.

i want an i.

2007 IBM Corporation

IBM System i

MTI Information Example

i want an i.

2007 IBM Corporation

IBM System i

MTI Control Mechanism

When SQE sees that a query runs frequently, it is more likely to create the advised MTI for that query.
Justification for creation : the one-time cost of creating the MTI must be more than offset by the runtime savings in subsequent queries SQE interrogates the system plan cache to garner this "historical use information

ALLOW_TEMPORARY_INDEXES parameter of QAQQINI control file:

*YES = SQE is free to create MTIs as it sees fit *ONLY_REQUIRED = SQE will create MTIs only when they are functionally necessary for sensitive (live data) queries

To minimize overhead of creating MTIs at query run-time, SQE create them for relatively small and moderately sized files only.
There is still no substitute for a good indexing strategy.

i want an i.

2007 IBM Corporation

IBM System i

Note
Now that the SQE optimizer has the ability to create maintained, shareable indexes, it relies on feedback from Index Advised to create those MTIs. When Index Advised recommends an index that may help a query perform better, the optimizer considers the effect on the query performance of creating that index. The cost of creating the index is considered, but it's discounted the more times a query runs. So, the first time a given query runs, creating an MTI based on information from Index Advised may not be justified. Other techniques such as hash join and table scan are typically more appropriate for a query that has only run once. If the optimizer sees that a query runs frequently, it's more likely to create the advised MTI. The justification is based on the fact that the one-time cost of creating the MTI will be more than offset by the runtime savings in subsequent query runs of this query and others. To garner this "historical use information," the optimizer interrogates the system plan cache. The QAQQINI file gives users the ability to override some of the initialization settings used by the optimizer during optimization. Two QAQQINI parameters affect MTI creation. One is the ALLOW_TEMPORARY_INDEXES parameter, which has two values: *YES and *ONLY_REQUIRED. The default value for this parameter is *YES. When *YES is specified, the optimizer is free to create MTIs as it sees fit. When the *ONLY_REQUIRED value is specified, the optimizer will create MTIs only when they're functionally necessary for sensitive (live data) queries. By specifying *ONLY_REQUIRED, indexes recommended by Index Advised aren't created. The other QAQQINI parameter that affects MTIs is the CACHE_RESULTS option, which helps determine how long MTIs stay in the system. When the default value of *SYSTEM is in effect, MTIs stay in the system as long as there's a referencing plan in the system plan cache. When the *JOB or *NONE values are in effect, an MTI is deleted when all job's consuming it end. Note that the CACHE_RESULTS QAQQINI parameter only affects reusable MTIs. MTIs created for sensitive cursor queries are typically deleted when the job ends.

i want an i.

2007 IBM Corporation

IBM System i

DB2 for i5/OS V5R4 Enhancements - Summary

Take Back Control System is Simplicity
Every component of the System i was designed specifically to support business needs, and it integrates those components for superior functionality. Hardware, OS, database, security, middleware and storage - an all-in-one sum that's truly greater than its parts.

SQE in V5R4 delivers more versatility in SQL performance tuning area These enhancements are easy to use or automated

i want an i.

2007 IBM Corporation

IBM System i

Additional Information on Index Advisor Condenser

APAR SE27116 - OSP-DB ADD INDEX ADVICE CONDENSER This is an add function PTF to deploy a Database Index Advice Condenser. The Index Advisor, a feature introduced to DB2 for i5/OS V5R4, records statistics on a system's indexing needs. Each time a query is run in which a permanent index is recommended, the Index Advisor records the index which could be used to improve that query's performance. Each time a unique index is advised, a new entry is created in the table. If multiple advice occurs on the same unique index, the information is added to the existing entry. For example, the "Number of Times Advised entry is incremented, and the "Last Advised for Query Use" is updated with the current time. In its raw form, index advice can be unnecessarily verbose. Although all entries in the table are unique, some entries can be redundant. One good example of this occurs when the same columns are advised but in a different order and the order of these keys is unimportant. In this case, the columns can be arranged to create one index that covers both instances of advice. The following examples assume all advice is on the same table, same partition, with the same index type and NLSS schema and table. Keys Advised Leading Keys, Order Independent C1, C2, C3 C1, C2, C3 C1, C3, C2 C1, C3, C2 In this case, since all columns are in both sets and all are order independent, the two instances of advice would be condensed into either { C1, C3, C2 } or { C1, C2, C3 }, depending upon the order of the two instances of advice in the Index Advisor table.

i want an i.

2007 IBM Corporation

IBM System i

APAR SE27116 (cont.)

Objects added by this PTF: QSYS2/Condense_Advice() UDF and QSYS2/CondensedIndexAdvice View. The CondensedIndexAdvice view can be queried directly to condense advice. iSeries Navigator users will be given a graphical solution in the form of Database Schema action. Example usage: -- This query returns uncondensed index advice for a specific table: select * from qsys2.sysixadv where TABLE_SCHEMA = 'MYlibrary' AND TABLE_NAME = 'Mytable'; -- This query returns condensed index advice for a specific table: select * from qsys2.CondensedIndexAdvice where TABLE_SCHEMA = 'MYlibrary' AND TABLE_NAME = 'Mytable';

i want an i.

2007 IBM Corporation

IBM System i

Additional Information: more reference

Indexing and Statistics Strategies whitepaper: http://www.ibm.com/servers/enable/site/education/abstracts/indxng_abs.html iSeries* Navigator: http://public.boulder.ibm.com/infocenter/iseries/v5r4/index.jsp?topic=/rzajq/queryopt.html DB2* UDB for i5/OS SQL and Query Performance Tuning and Monitoring Workshop: http://www.ibmsystemsmag.com/ME2/Audiences/Segments/Publications/www.ibm.com/eserver/iseries/ service/igs/db2performance.html QAQQINI Information: http://publib.boulder.ibm.com/pubs/html/as400/v4r4/ic2924/info/db2/rbafymst02.htm#ToC_689 Viewing the Query Plan Cache Through iSeries Navigator: http://publib.boulder.ibm.com/infocenter/iseries/v5r4/index.jsp?topic=/rzajq/rzajqdisplayplancache.html

i want an i.

2007 IBM Corporation

IBM System i

Additional Information: more reference

"Preparing for and Tuning the SQL Query Engine on DB2 for i5/OS" (IBM Redbooks): Minimize any SQL or database performance issues when upgrading to OS/400 V5R2 or V5R3 or to IBM i5/OS V5R4: http://www.redbooks.ibm.com/abstracts/sg246598.html

"OnDemand Index Advice for DB2 for i5/OS": Read additional information on the DB2 for i5/OS Index Advisor: http://www.ibm.com/servers/eserver/iseries/db2/index_advisor.html

i want an i.

2007 IBM Corporation