HERMES SoftLab EMC

Documentum SMART Plug-In

for HP Operations Manager
(SPI for EMC Documentum)

Version 02.10

________
User’s Guide
Contents
Notices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii

Trademarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii

Document Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
Edition History . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

Conventions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

Product Documentation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

Customer Support. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
Licensing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
Contacting Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
Before Contacting Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
General Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
Product Web Sites . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

Chapters Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

Getting Started . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
About SPI for EMC Documentum. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

High-level Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

Quick Introduction to SPI for EMC Documentum . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

Templates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
Service Map . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
Reports and Graphs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

Setting Up the Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

Setting Up the Environment. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
Where to Deploy the Templates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

Autodiscovery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

i
 How to Perform Autodiscovery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

Monitoring Documentum Content Server . . . . . . . . . . . . . . . . . . . . . . 25

About Monitoring Documentum Content Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
How to Monitor the Status of the Documentum Server Process . . . . . . . . . . . . . . . . . . . . . . . . . 27
How to Monitor CPU Utilization of the Documentum Server Process. . . . . . . . . . . . . . . . . . . . 27
How to Monitor Memory Usage of the Documentum Server Process . . . . . . . . . . . . . . . . . . . . 27
How to Collect Resource Consumption Information of the Documentum Server Process . . . 28
How to Check the Status of the Documentum Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
How to Check Last Check-in of the Documentum Server and if the Server will be Dropped from
the List of Known Servers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
How to use a template if more servers reside on the system . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

Monitoring Documentum Connection Broker . . . . . . . . . . . . . . . . . . . 31

About Monitoring Documentum Connection Broker . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
How to Monitor the Status of the Documentum Connection Broker Process . . . . . . . . . . . . . . 32
How to Monitor CPU Utilization of the Documentum Connection Broker Process. . . . . . . . . 33
How to Monitor Memory Usage of the Documentum Connection Broker Process . . . . . . . . . 33
How to Collect Resource Consumption Information of the Documentum Connection Broker
Process. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
How to use a template if more connection brokers reside on the system . . . . . . . . . . . . . . . . . . . . . . . . 34

Monitoring Database Availability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

About Monitoring Database Availability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
How to Monitor Database Availability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
How to Monitor Database Login Time . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
How to Collect Database Login Time Performance Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
How to use a template if more servers reside on the system . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

Monitoring Jobs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
About Monitoring Jobs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
How to Check the Job Status . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
How to Check the Status of the 'dm_agent_exec' Process. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
How to Monitor CPU Utilization and Memory Usage of the 'dm_agent_exec' Process . . . . . 42

ii User Guide
Contents
How to Collect CPU Utilization and Memory Usage Information of the 'dm_agent_exec' Process
43
How to Monitor Job Execution Time . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
How to Monitor Jobs that were Scheduled but Failed to Start. . . . . . . . . . . . . . . . . . . . . . . . . . . 44
How to Collect Job Execution Time Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
How to Collect Information About Failed Jobs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
How to Collect Information About Running Jobs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
How to Collect Information about Jobs that were Scheduled but Failed to Start . . . . . . . . . . . 46
How to Display the Job’s Last Execution Time. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
How to Display the Currently Running Jobs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
How to Display Jobs that were Scheduled but Failed to Start . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
How to use a template if more repositories reside on the system . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

Monitoring Documentum Sessions and User Activity . . . . . . . . . . . . 49

About Monitoring Documentum Sessions and User Activity. . . . . . . . . . . . . . . . . . . . . 50
How to Monitor User Sessions for Extensive CPU Utilization. . . . . . . . . . . . . . . . . . . . . . . . . . . 50
How to Monitor User Sessions for Extensive Memory Usage . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
How to Collect User Session CPU and Memory Utilization. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
How to Collect Information About User Sessions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
How to use a template if more servers reside on the system . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52

Monitoring Client Response . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55

About Monitoring Client Response . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
How to Monitor Client Response Time . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
How to Collect Client Response Time Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
How to Display Client Response Statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
How to configure end-user or application server systems to monitor client response . . . . . . . . . . . . . 58
How to monitor response time of a specific server/repository . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59

Monitoring Log Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61

About Monitoring Log Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
How to Monitor the Server Log File. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
How to Monitor the agentexec Log File. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63

i
 How to Monitor the Lifecycle Log File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
How to Monitor the User Session Log File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
How to use a template if more repositories reside on the system . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65

Monitoring File Stores . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67

About Monitoring File Stores . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
How to Monitor Documentum File Stores . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
How to Collect File Store Statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
How to Display File Stores Statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
How to use a template if more servers reside on the system . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70

Monitoring Database Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71

About Monitoring Database Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
How to Monitor the Table Size . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
How to Monitor the Number of Specific Events in the Table. . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
How to Collect Table Size Growth Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
How to Collect Event Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
How to Collect Unfinished Workflows Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
How to use a template if more repositories reside on the system . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76

Monitoring Index Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77

About Monitoring Index Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78
How to Monitor Index Server Availability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80
How to Monitor Index Server Log File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82
How to Monitor Indexing Latency . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82
How to Monitor the FT Query Response Time. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82
How to Monitor the Index Queue Size . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
How to Monitor the Index Queue for Failed Items . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
How to Monitor the FT Index Disk Usage. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
How to Collect Index Server statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84
How to Collect Indexing Performance Statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
How to Collect the Index Queue Size Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86
How to Collect Information on Failed Items in the Index Queue . . . . . . . . . . . . . . . . . . . . . . . . 86
How to Collect the FT Index Disk Usage Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87

iv User Guide
Contents
How to Monitor the Index Agent Availability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87
How to Monitor the Index Agent Log File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87
How to Collect the Index Agent Process Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88
How to Display Index Server Status. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88
How to Display Index Server Disk Usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89
How to Display Index Agent Status . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89
How to Display Indexing Latency . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89
How to Display FT Query Response Time . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90
How to use a template if more repositories reside on the system . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90

SPI for EMC Documentum Integration with HP Reporter . . . . . . . . . . 91

SPI for EMC Documentum Integration with HP Reporter . . . . . . . . . . . . . . . . . . . . . . . 92
Integration Requirements. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92
Deploying Templates and Collecting Performance Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92

How HP Reporter Creates Reports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93

Perform System Discovery. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93
Gather Performance Data. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94
Generate Reports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95
Generate Reports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95
Reports Implemented in SPI for EMC Documentum . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96

Analyzing Historical Data Using HP Performance Manager . . . . . . . 109

Analyzing Historical Data Using HP Performance Manager. . . . . . . . . . . . . . . . . . . . . 110
HP Performance Manager User Defined Graph Templates . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110
Sample Graphs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112

Troubleshooting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117
Troubleshooting Assistance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118
Documentum SPI Service Stops Responding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118
Configuration File (dmspi.cfg) Is Overwritten if Redeployment on the Node Is Performed 118
Autodiscovery on DCE Agents Fails . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118
Autodiscovery Fails if There Are no Application Groups in a Service Tree . . . . . . . . . . . . . . 119

v
Metric Lists . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121
Metric Lists. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122

vi User Guide
Notices
The information contained in this document is subject to change without notice.
HERMES SOFTLAB D.D. PROVIDES THIS MATERIAL "AS IS" AND MAKES
NO WARRANTY OF ANY KIND, EXPRESSED OR IMPLIED, INCLUDING, BUT
NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY
AND FITNESS FOR A PARTICULAR PURPOSE. HERMES SOFTLAB D.D.
SHALL NOT BE LIABLE FOR ERRORS CONTAINED HEREIN OR FOR
INCIDENTAL OR CONSEQUENTIAL DAMAGES IN CONNECTION WITH
THE FURNISHING, PERFORMANCE OR USE OF THIS MATERIAL WHETHER
BASED ON WARRANTY, CONTRACT, OR OTHER LEGAL THEORY.

Trademarks
This document contains proprietary information, which is protected by
copyright. All rights are reserved. No part of this document may be photocopied,
reproduced or translated to another language without the prior written consent of
HERMES SoftLab d.d.
Documentum and the Corporate Logo are trademarks of Documentum, a division
of EMC Corporation.
Microsoft®, Windows®, Windows® 2000, and Windows NT® are registered
trademarks of Microsoft Corporation in the United States and/or other countries.
UNIX is a registered trademark of The Open Group in the United States and other
countries.
Solaris Operating Environment, Java, and all Java-based trademarks are
trademarks of Sun Microsystems, Inc. in the United States, other countries, or
both.
IBM® and AIX® are registered trademarks of the IBM Corporation.
All other product names are the property of their respective trademark or service
mark holders and are hereby acknowledged.

vii
 HERMES SoftLab d.d.
Litijska 51
SI-1000 Ljubljana
Slovenia, Europe
www.hermes-softlab.com
© Copyright HERMES SoftLab d.d. 2008

viii User Guide

Chapter 1

Document Overview
Edition History
New editions are complete revisions of the manual. The printing dates for each
edition are listed below.

Edition Date
First Edition April 2006
Second Edition October 2006
Third Edition December 2007
Fourth Edition December 2008

2 User Guide
Conventions
The following typographical conventions are used in this manual:

Font Definition Example

Italic Product names, book or manual titles, Refer to the SPI for EMC Documentum
man page names, and section, table, Installation and Configuration Guide for
and figure titles. additional information.
Emphasis. You must follow these steps.
Window and dialog box names. In the HPOM Application Bank
window, select an application.
Bold Commands on menus and buttons, Expand Services and select SPI for
dialog box titles and options, menu EMC Documentum.
and icon names.
Computer File names, syntax, directory names, or The following file is located on the root
text that should be entered on screen directory of the SPI for EMC
or that is displayed on the monitor. Documentum installation CD:
readme.txt.

Document Overview 3
Product Documentation
With SPI for EMC Documentum, the following documentation is provided:
• HERMES SoftLab EMC Documentum SMART Plug-In for HP Operations Manager
Installation and Configuration Guide
Installation and Configuration Guide is available in printed and PDF format
(DMSPI_Install_GuideUNIX.pdf).
• HERMES SoftLab EMC Documentum SMART Plug-In for HP Operations Manager
User’s Guide
This guide is available in printed and PDF format (DMSPI_User_GuideUNIX.pdf).
• Release notes
Release notes are available in TXT format (dmspi-release-notes-unix.txt).
• Readme
Readme file is available in TXT format (dmspi-readme-unix.txt).
• License file
License file is available in TXT format (hsl_license_terms.txt)

4 User Guide
Customer Support
Use the following e-mail and Web page addresses if you need help with the licensing
process or while using the product, and if you would like additional information
about this or other HERMES SoftLab products.

Licensing
To obtain the license activation file you can visit HERMES SoftLab licensing portal:
http://spi.hermes-softlab.com/licensing/
or send an e-mail to the following address:
spi-licensing@hermes-softlab.com
For more information on licensing and licensing procedure refer to SPI for EMC
Documentum Installation and Configuration Guide.
If you encounter any problems with the licensing process, contact the HERMES
SoftLab licensing department at:
spi-licensing@hermes-softlab.com

Contacting Support

IMPORTANT NOTE

Should you require additional assistance or information while using the product,
contact the vendor that shipped the software.

If you have purchased the software directly from HERMES SoftLab, send e-mail to:
support-dmspi@hermes-softlab.com

Before Contacting Support

Before you contact the support department, have the following information available
so that a technical support analyst can work on your problem more efficiently:
• the support file dmspi_supp.zip
To create the support file dmspi_supp.zip, run the Collect Support Information
application on one or more nodes. To run the application, perform the following

Document Overview 5
 steps:
Go to Application Bank/SPI for EMC Documentum/DMSPI-Maintenance/
DMSPI-Support application group and run the Collect Support Information
application on the managed nodes for which you would like to collect the
information. The files with the support information are created in the following
directory on the node(s):
%OvDataDir%\dmspi\support (Windows nodes)
$OvAgentDir/dmspi/support (Unix nodes)
• sequence of events leading to the problem
• commands and options that you used
• messages you have received (a description with the time and date)

General Information
For marketing or business-related issues in reference to this or other HERMES
SoftLab SPIs, send e-mail to:
spi-info@hermes-softlab.com

Product Web Sites

Visit HERMES SoftLab SMART Plug-In Web site at:
http://www.hermes-softlab.com/products/SPI/about_SPI.html
and the company Web site at:
http://www.hermes-softlab.com/

6 User Guide
Chapters Summary
This guide describes how to use HERMES SoftLab EMC Documentum SMART Plug-
In for HP Operations Manager to monitor and manage Documentum application
resources.
The guide contains the following chapters:
• “Getting Started” on page 9
This chapter gives you a short overview of SPI for EMC Documentum
functionality and describes four main components of the product.
• “Setting Up the Environment” on page 22
This chapter provides information on how to perform autodiscovery and where to
deploy the SPI for EMC Documentum templates.
• “Monitoring Documentum Content Server” on page 25
This chapter provides information on how to monitor Documentum content
server.
• “Monitoring Documentum Connection Broker” on page 31
This chapter provides information on how to monitor Documentum connection
broker.
• “Monitoring Database Availability” on page 35
This chapter provides information on how to monitor database availability.
• “Monitoring Jobs” on page 39
This chapter provides information on how to monitor Documentum
administrative jobs.
• “Monitoring Documentum Sessions and User Activity” on page 49
This chapter provides information on how to monitor Documentum sessions and
user activity.
• “Monitoring Client Response” on page 55
This chapter provides information on how to monitor client response.
• “Monitoring Log Files” on page 61
This chapter provides information on how to monitor log files.
• “Monitoring File Stores” on page 67
This chapter provides information on how to monitor file stores.
• “Monitoring Database Tables” on page 71
This chapter provides information on how to monitor database tables.

Document Overview 7
 • “Monitoring Index Server” on page 77
This chapter provides information on how to monitor index server.
• “SPI for EMC Documentum Integration with HP Reporter” on page 91
This chapter provides instructions on how to use the product for monitoring, how
to integrate SPI for EMC Documentum with HP Reporter, and how to create
reports.
• “Analyzing Historical Data Using HP Performance Manager” on page 109
This chapter provides instructions on how to use HP Performance Manager to
analyze historical data.
• “Troubleshooting” on page 117
This chapter provides instructions on what to do if you encounter any problems
while using the SPI for EMC Documentum.
• Appendix A, “Metric Lists” on page 122
This chapter provides metrics lists that are created for SPI for EMC Documentum
reports.

8 User Guide
Chapter 2

Getting Started
About SPI for EMC Documentum
HERMES SoftLab EMC Documentum SMART Plug-In for HP Operations Manager
(SPI for EMC Documentum) is designed specifically for use with EMC Documentum
products from HP Software environments. Developed by HERMES SoftLab Group,
it proactively and securely manages a distributed, Documentum environment from
one, centralized point. SPI for EMC Documentum works with other HP Software
solutions to help you gain control over your EMC Documentum environment.
Additionally, it helps you to manage your IT infrastructure in order to increase your
system availability and performance. Benefits of using SPI for EMC Documentum
include the following:
• Alerts on failures of critical Documentum processes. Server, connection broker,
and job processes are monitored for availability and performance.
• Tracks the time that has elapsed since the server’s last check-in and the total time
that connection broker will wait before dropping the server from the list of
known servers.
• Alerts on Documentum job failures.
• Monitors job execution time and detects possible hung jobs.
• Monitors jobs that were scheduled to run but never started.
• Collects job execution statistics.
• Measures response times for business transactions from end-user perspective.
Sources of potential response time problems can be detected.
• Identifies user sessions that are causing excessive CPU and memory
consumption.
• Monitors file stores and helps with capacity planning.
• Alerts if database is not available or connection time takes too long.
• Immediately identifies internal Documentum issues by forwarding relevant log
messages.
• Monitors table size growth and the number of unprocessed renditions, the
number of failed indexing items, and other events specified by user.
• Monitors if index server is available and all necessary components are running.
• Monitors if it is possible to search for documents, add new documents to full-text
index, and how fast is the search.
• Monitors index server performance (save-to-search latency).
• Collects indexing performance.

10 User Guide
• Monitors index agent health, availability, and resource consumption.
• Enhances an administrator's toolbox by providing various informational
applications including information about client response time, file stores capacity,
user sessions, and so on.
• Regularly records resource consumption of critical Documentum processes.
• Collects information about session activity (number of active/inactive sessions,
session resource consumption).
• Collects and reports client response time statistics.
• Collects and reports information about file store and database table growth.
• Reports resource consumption for Documentum processes.
• Provides insight into session activity by generating numerous reports.
• Automatically discovers Documentum environment, creates a corresponding
service map model, and displays dependencies between servers, repositories, and
connection brokers.

Getting Started 11
High-level Architecture
Below is a high-level architecture diagram of SPI for EMC Documentum.

12 User Guide
Quick Introduction to SPI for EMC Documentum
SPI for EMC Documentum integrates the following HP Operations Manager
components:
• HP Operations Manager/UNIX Templates
• HP Operations Manager/UNIX Applications
• HP Operations Manager/UNIX Reports
• HP Operations Manager/UNIX Service Map

Templates
Templates instruct HP Operations Manager how to solve system management
issues, for example, "Monitor CPU Utilization".
SPI for EMC Documentum templates enable you to define complex rules and
instructions for monitoring different systems and services. Templates are
instructions for HP Operations Manager (HPOM) agents that monitor your
Documentum environment. When an issue is detected, a message is sent to the
HPOM console.
In addition, you can define automated actions to be performed upon detecting a
specific issue, or include instructions for operators as part of the message. Every
templates can start automatic or operator initiated action when issue is detected. You
can check which actions were launched when issue was detected, and check the
action outcome in the message annotation.

Getting Started 13
 SPI for EMC Documentum adds a SPI for EMC Documentum template group and
several templates to HP Operations Manager for UNIX.

These templates enable you to:

• Monitor Documentum Content Server and connection broker availability and
resource consumption.
• Detect job failures and track the number of failed jobs per Repository.
• Monitor job execution time and detect possible hung jobs.
• Monitor user activity and identify user sessions that are causing excessive CPU
and memory consumption.
• Monitor client response time for standard operations (for example, connect,
check-in, check-out, delete) against Repositories from the client systems.
• Monitor file stores.
• Monitor log files (server, session, job, life-cycle) .
• Monitor the size and growth of Documentum DB tables (for example,
dmi_queue_item, dm_audittrail) .

14 User Guide
 • Monitor the number of unprocessed renditions , number of failed indexing items
and other events specified by user.
• Monitor if index server is available and all necessary components are running.
• Monitor if it is possible to search for documents, add new documents to full-text
index, and how fast is the search.
• Monitor index server performance (save-to-search latency).
• Monitor index sgent health, availability and resource consumption.
• Collect performance statistics.

Applications
SPI for EMC Documentum supplies powerful applications for monitoring and
managing your EMC Documentum environment. SPI for EMC Documentum
applications enable you to inspect, analyze and manage your EMC Documentum
environment, check the current status, and react to issues in your environment.

Getting Started 15
 SPI for EMC Documentum adds a SPI for EMC Documentum applications group
and several applications to HP Operations Manager for UNIX.

16 User Guide
 These applications enable you can perform the following tasks:
• Show users that are causing excessive CPU and memory consumption.
• Display file store info.
• Display documents with largest content and most versions.
• Display workflows that are not finished yet and are older than the specified time
period.
• Display current running jobs and their execution times.
• Display full-text index query response.
• Display Index Agent status.
• Display save-to-search time of a document.
• Display jobs that were scheduled to run but never started.
• Display Index Server disk usage.
• Display Index Server status.
• Perform discovery of your EMC Documentum environment and update the
Documentum servicemodel.
• Troubleshoot the SPI for EMC Documentum installation and configuration.
• License SPI for EMC Documentum.
• Manage Documentum nodes.

Service Map
Service map is a component of the HP Operations Manager for Windows. This
component enables you to manage your IT (information technology) environment
while focusing on the IT services that you provide.
SPI for EMC Documentum integrates into the HP Operations Manager for Windows
service map component. It automatically generates a Documentum service map,
which offers complete graphical representation of your Documentum environment
and its hierarchical organization. All Documentum object types (connection brokers,
Content Servers, repositories, file stores, and so on) in your environment are
displayed in a tree view, which shows the components and their hierarchical
dependencies.
In addition, graphical representation of your EMC Documentum environment
clearly indicates relations between EMC Documentum environment components,
how they impact one another, and which business services are affected. This enables
you to effectively manage services within your information technology framework.

Getting Started 17
 Example of the service map view:

Reports and Graphs

SPI for EMC Documentum offers reporting functionality, where collected
performance data can be represented in web-based reports or real-time graphs.
SPI for EMC Documentum integrates with the HP Operations Manager for
Windows reporting component (Reporter Lite) and with the HP Reporter products.
Additionally, SPI for EMC Documentum integrates with the HP Operations
Manager for Windows graphing component (Performance Manager Lite) and with
the HP Performance Manager products.

18 User Guide
You can generate reports and graphs on the following information:
• Documentum process availability and resource consumption
• File store statistics
• Database availability
• User activity
• Client response time
• Table size growth and the number of unprocessed events (renditions, indexing)
• Job timeline report and top jobs by longest execution time
• The number of running/failed jobs in the specified interval
• Index agent/server process availability and resource consumption
• Save-to-search latency - the time it takes to add a new document into the index
and make it searchable
• Full-text index search response time

Getting Started 19
20 User Guide
Chapter 3

Setting Up the Environment

Setting Up the Environment
Setting up your environment involves installing, licensing and configuring SPI for
EMC Documentum on the HPOM management server and on the managed nodes.
For more information on how to install, license, and configure managed nodes refer
to SPI for EMC Documentum Installation and Configuration Guide.
After you successfully configured your environment, you can start using the SPI for
EMC Documentum product. For procedural information on how to:
• perform autodiscovery of Documentum components and update your
configuration, refer to “How to Perform Autodiscovery” on page 24.
To see the SPI for EMC Documentum templates, open the Message Source Template
window and expand the SPI for EMC Documentum group.

Where to Deploy the Templates

• DMSPI-Client
This template group contains templates for monitoring response time from server
or end-user systems. Install the templates from this group on the servers or on the
nodes where the dmcl API is installed.
• DMSPI-Connection Broker
This template group contains templates for monitoring Documentum connection
broker process. Install the templates from this group only on the nodes where one
or more connection brokers reside.
• DMSPI-Content Server
This template group contains templates for monitoring Documentum server and
server components. Install the templates from this group only on the nodes where
one or more Documentum servers reside.
• DMSPI-Index Server
This template group contains templates for monitoring Documentum Indexing
Server. Install the templates from this group only on the nodes where
Documentum index servers reside.
• DMSPI-Licensing
This template group contains templates for generating, merging, installing license
Install the templates from this group on all nodes where SPI for EMC
Documentum is installed.
•

22 User Guide
• DMSPI-SPI Self Monitoring
This template group contains templates for intercepting error messages from SPI
for EMC Documentum. Install the templates from this group on all nodes where
SPI for EMC Documentum is installed.

Setting Up the Environment 23

Autodiscovery
To perform automatic discovery of the Documentum topology and generate a
service view that displays your Documentum environment, you can use the
DMSPI-SvcDiscMapUplExt template. The template is located in the Discovery
group under Connection Broker, Content Server, or Index Server group.

How to Perform Autodiscovery

To perform autodiscovery, do the following:
1. Deploy the DMSPI-SvcDiscMapUplExt template on the node where SPI for EMC
Documentum is installed.
2. On the system where DMSPI-SvcDiscMapUplExt template was deployed, go to
Application Bank/SPI for EMC Documentum/DMSPI-Documentum/DMSPI-
Discovery and run the Autodiscovery application. The application will send a
message with the Documentum environment topology to the HPOM server.
Automatic action on the HPOM server will upload the Documentum service tree.
By default, services are assigned to the opc_op user. If you want these services to also
be assigned to another operator or user, you must execute the following command:
/opt/OV/bin/OpC/opcservice -ass operator dmspi_root

24 User Guide
Chapter 4

Monitoring Documentum Content

Server
About Monitoring Documentum Content Server
Servers are processes that provide client access to the repository. Every repository
must have at least one active server. If a repository does not have an active server,
then users cannot access that repository. Servers are accessible for users via
connection broker, which keeps a list of active Servers. To stay on the list, each
Server has to check-in with the connection broker at regular intervals otherwise it is
dropped from the list after a specified period of time. Availability of the Server
process is vital for the health of your Documentum environment.
To monitor the Documentum server with SPI for EMC Documentum, you can use
the following templates located in the SPI for EMC Documentum/DMSPI-EMC
Documentum 5.3 /DMSPI-D5 ContentServer and SPI for EMC Documentum/
DMSPI-EMC Documentum 6 /DMSPI-D6 ContentServer template groups:
DMSPI-DxCS Quick Start template group
• DMSPI-SrvrProcess
This is a Monitor template that monitors Documentum Server service and daemon
availability.
• DMPSI-SrvrAvailability
It is a Measurement Threshold template that monitors the current state of the
Documentum server.
• DMPSI-SrvrCheckIn
It is a Measurement Treshold template that monitors server’s last chec-in time
before the server will be dropped from the list of known servers.
DMSPI-DxCS Additional template group.
• DMSPI-SrvrProcessCpu
It is a Measurement Threshold template that monitors CPU utilization of the
Documentum server process.
• DMSPI-SrvrProcessMem
It is a Measurement Threshold template that monitors memory usage of the
Documentum server process.
DMSPI-DxCS Data Collection template group
• DMSPI-SrvrProcessPerf
It is a Scheduled Task template that collects server process MEM and CPU usage
and stores this information to HP Performance Agent.

26 User Guide
How to Monitor the Status of the Documentum Server Process
To check if the Documentum server process is running, deploy the DMSPI-
SrvrProcess template on the system. The template runs dmspi_proc(.exe) with the
following arguments:
dmspi_proc(.exe) --srvr_proc
A message is sent each time the process state changes:
• Not running - critical message
• Running - normal message
If no servers are found on the system, a DMSPI internal error message is sent.
See also “How to use a template if more servers reside on the system” on page 29.

How to Monitor CPU Utilization of the Documentum Server

Process
To monitor CPU utilization of the server process, use the DMSPI-SrvrProcessCpu
template. The template runs dmspi_proc(.exe) with the following arguments:
dmspi_proc(.exe) --srvr_proc --cpu
A message is sent if:
• Server process CPU utilization > 50% - warning message
• Server process CPU utilization > 70% - minor message
• Server process CPU utilization > 90% - critical message
If no servers are found on the system, a DMSPI internal error message is sent.
See also “How to use a template if more servers reside on the system” on page 29.

How to Monitor Memory Usage of the Documentum Server

Process
To monitor memory usage of the server process, use the DMSPI-SrvrProcessMem
template. The template runs dmspi_proc(.exe) with the following arguments:
dmspi_proc(.exe) --srvr_proc --mem
A message is sent if:
• Server process memory usage > 30000 kb - warning message
• Server process memory usage > 50000 kb - minor message

Monitoring Documentum Content Server 27

 • Server process memory usage > 100000 kb - major message
If no servers are found on the system, a DMSPI internal error message is sent.
See also “How to use a template if more servers reside on the system” on page 29..

How to Collect Resource Consumption Information of the

Documentum Server Process
To collect performance data about server CPU and memory utilization, deploy the
DMSPI-SrvrProcessPerf template to the node. The template runs
dmspi_proc(.exe) with the following arguments:
dmspi_ proc(.exe) --srvr_proc --perf
Information about server process CPU and memory usage will be logged to HP
Performance Agent where DMSPI_SRVR_PROCESS class will be added to the DMSPI
datasource with the following metrics:
• Repository name
• Server name
• Server process status
• Server process memory usage (KB)
• Server process virtual memory usage (KB)
• Server process CPU utilization (%)
If no servers are found on the system, a DMSPI internal error message is sent.
See also “How to use a template if more servers reside on the system” on page 29.

How to Check the Status of the Documentum Server

To check the current state of the server, deploy the DMSPI-SrvrAvailability
template on the node. The template runs dmspi_srvr(.exe) with the following
arguments:
dmspi_srvr(.exe) --availability
A message will be generated each time the server status changes. The following
states are possible:
• Starting, Open
• Stopped
• Presumed down
If no servers are found on the system, a DMSPI internal error message is sent.

28 User Guide
 See also “How to use a template if more servers reside on the system” on page 29.

How to Check Last Check-in of the Documentum Server and if the

Server will be Dropped from the List of Known Servers
To check if the server did not check in and will be dropped from the connection
broker's list of known servers, deploy the DMSPI-SrvrCheckIn template on the
node. This tracks the time that has elapsed since the server's last check-in
(LastCheckInTime) and the total time that connection broker will wait before
dropping the server from the list of known servers (WaitTime). The template runs
dmspi_srvr(.exe) with the following arguments:
dmspi_srvr(.exe) -checkin
The percentage of how much WaitTime has been used by LastCheckInTime is
calculated as:
ElapsedTimePerc = ((CurrentTime - LastCheckInTime) / WaitTime) * 100
A message is sent if ElapsedTimePerc exceeds:
• 1% of total keep entry interval - warning message
The threshold is so low because we want to notify the administrator immediately
if the server does not check-in.
• 70% of total keep entry interval - minor message
• 90% of total keep entry interval - major message
The message also contains information about how much time (in minutes) remains
until the server will be dropped.

How to use a template if more servers reside on the system

It is possible to have more than one server on the same system. By default, all
templates for monitoring Documentum server check only the first server specified in
the DMSPI configuration file (%OvDataDir%\dmspi\conf\dmspi.cfg on Windows
or $OvAgentDir/dmspi/conf/dmspi.cfg on Unix systems) or the first one found
on the system.
To monitor a specific server on the system, make a copy of the template and add --
server and --template arguments to the program name.
Example:
dmspi_srvr(.exe) --availability --server=server_name --
template=my_new_template_name
If the wrong server is specified, a DMSPI internal error message is sent.

Monitoring Documentum Content Server 29

30 User Guide
Chapter 5

Monitoring Documentum
Connection Broker
About Monitoring Documentum Connection Broker
The Documentum connection broker is a process that provides client sessions with
connection information. When a client session is opened, the client contacts the
connection broker and requests the information it needs to connect with a server for
the requested repository.
To monitor Documentum connection broker with SPI for EMC Documentum, use
the following templates located in the SPI for EMC Documentum/DMSPI-EMC
Documentum 5.3/DMSPI-D5 Connection Broker and SPI for EMC Documentum/
DMSPI-EMC Documentum 6/DMSPI-D6 Connection Broker template groups:
DMSPI-DxCB Quick Start template group
• DMSPI-BrokerProcess
It is a Measurement Threshold template that monitors availability of the
Documentum connection broker process.
DMSPI-DxCB Additional template group
• DMSPI-BrokerProcessCpu
It is a Measurement Threshold template that monitors CPU utilization of the
Documentum connection broker process.
• DMSPI-BrokerProcessMem
It is a Measurement Threshold template that monitors memory usage of the
Documentum connection broker process.
DMSPI-DxCB Data Collection template group
• DMSPI-BrokerProcessPerf
It is a Scheduled Task template that collects connection broker process MEM and
CPU usage and stores this information to HP Performance Agent.

How to Monitor the Status of the Documentum Connection Broker

Process
To check if the Documentum connection broker process is running, deploy the
DMSPI-BrokerProcess template on the system. This checks if the
dmdocbroker(.exe) process is running. The template runs dmspi_proc(.exe) with
the following arguments:
dmspi_proc(.exe) --docbroker
A message is sent each time the process state changes:

32 User Guide
 • Not running - critical message
• Running - normal message
If no connection brokers are found on the system, a DMSPI internal error message is
sent.
See also “How to use a template if more connection brokers reside on the system” on
page 34.

How to Monitor CPU Utilization of the Documentum Connection

Broker Process
To monitor CPU utilization of the connection broker process, use the DMSPI-
BrokerProcessCpu template. The template runs dmspi_proc(.exe) with the
following arguments:
dmspi_proc(.exe) --docbroker --cpu
A message is sent to the console if:
• process CPU utilization > 50% - warning message
• process CPU utilization > 70% - minor message
• process CPU utilization > 90% - critical message
If no connection brokers are found on the system, a DMSPI internal error message is
sent.
See also “How to use a template if more connection brokers reside on the system” on
page 34.

How to Monitor Memory Usage of the Documentum Connection

Broker Process
To monitor memory usage of the connection broker process, use the DMSPI-
BrokerProcessMem template. The template runs dmspi_proc(.exe) with the
following arguments:
dmspi_proc(.exe) --docbroker --mem
A message is sent to the console if:
• process memory usage > 30000 kb - a warning message
• process memory usage > 50000 kb - a minor message
• process memory usage > 100000 kb - a major message

Monitoring Documentum Connection Broker 33

 If no connection brokerrs are found on the system, a DMSPI internal error message is
sent.
See also “How to use a template if more connection brokers reside on the system” on
page 34.

How to Collect Resource Consumption Information of the

Documentum Connection Broker Process
To collect performance data about connection broker CPU and memory utilization,
deploy the DMSPI-BrokerPerf template to the node. The template runs
dmspi_proc(.exe) with the following arguments:
dmspi_proc(.exe) --docbroker --perf
Information about connection broker process CPU and memory usage is logged to
HP Performance Agent where DMSPI_DOCBROKER_PROC class is added to the DMSPI
datasource with the following metrics:
• Connection broker process name and port number
• Connection broker process status
• Connection broker process memory usage (KB)
• Connection broker process virtual memory usage (KB)
• Connection broker process CPU utilization (%)
If no connection brokers are found on the system, a DMSPI internal error message is
sent.
See also “How to use a template if more connection brokers reside on the system” on
page 34.

How to use a template if more connection brokers reside on the system

It is possible to have more than one connection broker on the same system. By
default, all templates for monitoring Documentum connection broker check only the
first connection broker specified in the DMSPI configuration file
(%OvDataDir%\dmspi\conf\dmspi.cfg on Windows or $OvAgentDir/dmspi/
conf/dmspi.cfg on Unix systems) or the first one found on the system.
To monitor a specific connection broker on the system, make a copy of the template
and add --port and --template arguments to the program name.
Example:
dmspi_proc(.exe) --docbroker --port=port_number --
template=my_new_template_name
If the wrong connection broker is specified, a DMSPI internal error message is sent.

34 User Guide
Chapter 6

Monitoring Database Availability

About Monitoring Database Availability
Repositories comprise object type tables, type indexes, content files, and full-text
indexes. The type tables and type indexes are tables in an underlying relational
database. Because the Documentum database usually resides on a standalone
machine, it is crucial that the Documentum servers can connect to the database.
To monitor the Documentum database availability, use the following templates
located in the SPI for EMC Documentum/DMSPI-EMC Documentum 5.3/DMSPI-
D5 Content Server and SPI for EMC Documentum/DMSPI-EMC Documentum 6/
DMSPI-D6 Content Server templates groups:
DMSPI-DxCS Quick Start template group
• DMSPI-DBAvailability
It is a Measurement Threshold template that checks if Documentum database is
available.
• DMSPI-DBLogin
It is a Measurement Threshold template that checks the database login time from
the Documentum server.
DMSPI-DxCS Data Collection template group
• DMSPI-DBLoginPerf
This is a Schedule template that collects the information about the database login
time and stores it to HP Performance Agent.

How to Monitor Database Availability

To check the database availability, deploy the DMSPI-DBAvailability template on
the node. A message is sent each time the process state changes:
• Available - normal message
• Not available - critical message
The template runs dmspi_db(.exe) with the following arguments:
dmspi_db(.exe) --availability
See also “How to use a template if more servers reside on the system” on page 37.

36 User Guide
How to Monitor Database Login Time
To check the login time of the database, deploy the DMSPI-DBLogin template on
the node. A message will be generated if the database login time from the server
exceeds the threshold value:
• login time > 5 seconds - minor message
• login time > 3 seconds - warning message
The template runs dmspi_db(.exe) with the following arguments:
dmspi_db(.exe) --login
See also “How to use a template if more servers reside on the system” on page 37.

How to Collect Database Login Time Performance Data

To collect performance data about database login response, deploy the DMSPI-
DBLoginPerf template to the node. The template runs dmspi_db(.exe) with the
following arguments:
dmspi_db(.exe) --login --perf
Information about database login time is logged to HP Performance Agent where
DMSPI_DB_LOGIN class is added to the DMSPI datasource with the following metrics:
• Server name
• Repository name
• Database login time (ms)
See also “How to use a template if more servers reside on the system” on page 37.

How to use a template if more servers reside on the system

It is possible to have more than one server on the same system. By default, all
templates for monitoring database availability check only database availability of the
first server specified in the DMSPI configuration file
(%OvDataDir%\dmspi\conf\dmspi.cfg on Windows or $OvAgentDir/dmspi/conf/
dmspi.cfg on Unix systems) or the first one found on the system.
To monitor database availability of a specific server on the system, make a copy of
the template and add --server and --template arguments to the program name.
Example:
dmspi_db(.exe) --availability --server=server_name --
template=my_new_template_name
If the wrong server is specified, a DMSPI internal error message is sent.

Monitoring Database Availability 37

38 User Guide
Chapter 7

Monitoring Jobs
About Monitoring Jobs
Jobs are repository objects that automate method object execution. Methods
associated with jobs are executed automatically on a user-defined schedule. Jobs are
invoked by the dm_agent_exec process. At regular intervals, the dm_agent_exec
process examines the job objects in the repository and runs those jobs that are ready
for execution.
To monitor Documentum job status, job execution time, the number of running jobs,
and processes responsible for the job execution with SPI for EMC Documentum, you
can use the following templates located in the SPI for EMC Documentum/DMSPI-
EMC Documentum 5.3/DMSPI-D5 Content Server and SPI for EMC Documentum/
DMSPI-EMC Documentum 6/DMSPI-D6 Content Server template groups:
DMSPI-DxCS Quick Start template group:
• DMSPI-JobStatus
It is a Scheduled Task template that monitors the status of Documentum jobs.
• DMSPI-JobStatus
It is a Open Message Interface template that monitors the status of Documentum
jobs.
• DMSPI-JobProcess
It is a Measurement Threshold template that checks availability of the
dm_agent_exec process. The dm_agent_exec process is responsible for executing
jobs.
• DMSPI-JobTime-dm_DBWarning
It is a Measurement Threshold template that checks the Documentum
dm_DBWarning job execution time.

• DMSPI-JobTime-dm_DataDictionaryPublisher
It is a Measurement Threshold template that checks the Documentum
dm_DataDictionaryPublisher job execution time.

• DMSPI-JobTime-dm_ContentWarning
It is a Measurement Threshold template that checks the Documentum
dm_ContentWarning job execution time.

• DMSPI-JobTime-dm_StateOfDocbase
It is a Measurement Threshold template that checks the Documentum
dm_StateOfDocbase job execution time.

40 User Guide
 DMSPI-DxCS Additional template group:
• DMSPI-JobProcessMem
It is a Measurement Threshold template that monitors memory usage of the
dm_agent_exec process.
• DMSPI-JobProcessCpu
It is a Measurement Threshold template that monitors CPU utilization of the
dm_agent_exec process.

• DMSPI-JobsFailedToStart
It is a Scheduled Task template that monitors jobs that were scheduled but for
some reason failed to start.
• DMSPI-JobsFailedToStart
It is a Open Message Interface template that monitors jobs that were scheduled but
for some reason failed to start.

DMSPI-DxCS Data Collection template group:

• DMSPI-JobProcessPerf
This template collects MEM and CPU usage of the dm_agent_exec process and
stores this information to HP Performance Agent.
• DMSPI-JobExecTimePerf
It is a Scheduled Task template that collects job execution time statistics and stores
this information to HP Performance Agent.
• DMSPI-JobStatusPerf
It is a Scheduled Task template that collects statistics about the number of failed
jobs and stores this information to HP Performance Agent.
• DMSPI-RunningJobsPerf
It is a Scheduled Task template that collects statistics about the number of jobs that
are currently running and stores this information to HP Performance Agent.

How to Check the Job Status

To check the status of the job, deploy the DMSPI-JobStatus (Schedule template) and
DMSPI-JobStatus (Message template) templates on the node. A message is sent if
the job finishes with an error.
The template DMSPI-JobStatus (Schedule) runs dmspi_jobs(.exe) with the
following arguments:
dmspi_job(.exe) --status

Monitoring Jobs 41
 See also “How to use a template if more repositories reside on the system” on page
47.

How to Check the Status of the 'dm_agent_exec' Process

To check if the process responsible for executing jobs is running, deploy the DMSPI-
JobProcess template on the system. This checks if the dm_agent_exec(.exe)
process is running. The template runs dmspi_proc(.exe) with the following
arguments:
dmspi_proc(.exe) --job
A message is sent each time the process state changes:
• Not running - critical message
• Running - normal message
See also “How to use a template if more repositories reside on the system” on page
47.

How to Monitor CPU Utilization and Memory Usage of the

'dm_agent_exec' Process
To check the impact of the dm_agent_exec(.exe) process on the system, use the
DMSPI-JobProcessCpu and DMSPI-JobProcessMem templates.
With the DMSPI- JobProcessCpu template, you can monitor CPU utilization of the
dm_agent_exec process. The template runs dmspi_proc(.exe) with the following
arguments:
dmspi_proc(.exe) --job --cpu
A message is sent if:
• 'dm_agent_exec' process CPU utilization > 50% - warning message
• 'dm_agent_exec' process CPU utilization > 70% - minor message
• 'dm_agent_exec' process CPU utilization > 90% - critical message
With the DMSPI-JobProcessMem template, you can monitor virtual memory usage
of the dm_agent_exec process. The template runs the dmspi_proc(.exe) executable
with the following arguments:
dmspi_proc(.exe) --job --mem
A message is sent if:
• 'dm_agent_exec' process memory usage > 5000 kb - warning message

42 User Guide
 • 'dm_agent_exec' process memory usage > 10000 kb - minor message
• 'dm_agent_exec' process memory usage > 15000 kb - major message
See also “How to use a template if more repositories reside on the system” on page
47.

How to Collect CPU Utilization and Memory Usage Information of

the 'dm_agent_exec' Process
To collect performance data about dm_agent_exec CPU and MEM utilization,
deploy the DMSPI-JobProcessPerf template to the node. The template runs
dmspi_proc(.exe) with the following arguments:
dmspi_proc(.exe) --job --perf
Information about the dm_agent_exec process CPU and memory usage is logged to
HP Performance Agent.
DMSPI_JOB_PROCESS class is added to the DMSPI datasource with the following
metrics:
• Repository name
• Server name
• dm_agent_exec process status
• dm_agent_exec process memory usage (KB)
• dm_agent_exec process virtual memory usage (KB)
• dm_agent_exec process CPU utilization (%)
See also “How to use a template if more repositories reside on the system” on page
47.

How to Monitor Job Execution Time

To monitor the job execution time and detect possible hung jobs, deploy the
following Monitor templates on the node:
• DMSPI-JobTime-dm_ContentWarning, if you would like to monitor execution
time of the dm_ContentWarning job.
• DMSPI-JobTime-dm_DataDictionaryPublisher, if you would like to monitor
execution time of the dm_DataDictionaryPublisher job.
• DMSPI-JobTime-dm_DBWarning, if you would like to monitor execution time of
the dm_DBWarning job.

Monitoring Jobs 43
 • DMSPI-JobTime-dm_StateOfDocbase, if you would like to monitor execution
time of the dm_StateOfDocbase job.
To monitor the execution time of any Documentum job, copy one of the listed
templates, change the template name (for example, DMSPI-JobTime-JobName) and
the --job and --template parameters:
dmspi_job (.exe) --exectime --job=job_name --template=DMSPI-JobTime-
JobName
The message will be sent if execution time of monitored job exceeds the predefined
thresholds:
• Job execution time > 3600s - warning message
• Job execution time > 7200s - minor message
• Job execution time > 14400s - major message
See also “How to use a template if more repositories reside on the system” on page
47.

How to Monitor Jobs that were Scheduled but Failed to Start

To monitor jobs that were scheduled but for some reason failed to start, deploy the
DMSPI-JobsFailedToStart (Scheduled Task) and DMSPI- JobsFailedToStart (Open
Message Interface) templates on the node.
The template DMSPI-JobsFailedToStart (Scheduled Task) runs dmspi_jobs(.exe)
with the following arguments:
dmspi_jobs(.exe) --failed
If more repositories reside on the system, the template monitors the jobs only for the
first repository specified in the DMSPI configuration file
(%OvDataDir%\dmspi\conf\dmspi.cfg on Windows or $OvAgentDir/dmspi/
conf/dmspi.cfg on Unix systems).
To monitor the jobs for the specific repository, make a copy of the template and add
--docbase argument.
Example:
dmspi_ job(.exe) --failed --docbase_{repository name}

How to Collect Job Execution Time Information

To collect the job execution time information, deploy the DMSPI-JobExecTimePerf
template on the node. The template runs dmspi_job with the following parameters:

44 User Guide
 dmspi_job --exectime --perf
Information about job execution time is logged to HP Performance Agent, where
DMSPI_JOBS_EXECTIME class is added to the DMSPI datasource with the following
metrics:
• Repository name
• Target server
• Job name
• Job start time in number of seconds since 01/01/1970
• Job stop time in number of seconds since 01/01/1970
• Job exit status
• Job status message
• Job owner

NOTE

To gather accurate statistics, this template should run more frequently than the
most frequent Documentum job.

See also “How to use a template if more repositories reside on the system” on page
47.

How to Collect Information About Failed Jobs

To collect statistics about failed jobs, deploy the DMSPI-JobStatusPerf template on
the node. The template runs dmspi_job with the following arguments:
dmspi_job --status --perf
The template collects and stores job statistics to HP Performance Agent where the
DMSPI_JOBS class is added to the DMSPI datasource with the following metrics:
• Docbase name
• Number of failed jobs
See also “How to use a template if more repositories reside on the system” on page
47.

Monitoring Jobs 45
How to Collect Information About Running Jobs
To collect statistics about how many jobs are runnning at the moment, deploy the
DMSPI-JobRunningPerf template on the node. The template runs dmspi_job with
the following arguments:
dmspi_job --current --perf
The template collects and stores job statistics to HP Performance Agent where
DMSPI_JOBS_RUNNING class is added to the DMSPI datasource with the following
metrics:
• Docbase name
• Target server
• The number of jobs that are running at the moment
See also “How to use a template if more repositories reside on the system” on page
47.

How to Collect Information about Jobs that were Scheduled but

Failed to Start
To collect information about jobs that were scheduled, but for some reason failed to
start, deploy the DMSPI-JobsFailedToStartPerf template on the node. The template
runs dmspi_job with the following arguments:
dmspi_job --failed --perf
The template collects and stores job information to HP Performance Agent where
DMSPI_JOBS_NO_START class is added to the DMSPI datasource with the following
metrics:
• Job name
• Target server
• Expected execution

How to Display the Job’s Last Execution Time

To display information about the job's last execution time, go to Application Bank/
SPI for EMC Documentum/DMSPI-Documentum/DMSPI-Content Server and run
the Job Execution Times application. The following information is displayed:
• Job name

46 User Guide
 • Target server
• Job owner
• Job start time
• Job finish time
• Job duration
• Job exit status
• Job status message

How to Display the Currently Running Jobs

To display jobs that are currenly running, go to Application Bank/SPI for EMC
Documentum/DMSPI-Documentum/DMSPI-Content Server and run the Running
Jobs application. The following information is displayed:
• Job name
• Target server
• Job owner
• Job start time
• Job duration till now

How to Display Jobs that were Scheduled but Failed to Start

To display jobs that were scheduled but for some reason failed to start, go to
Application Bank/SPI for EMC Documentum/DMSPI Documentum/DMSPI-
Content Server and run the JobsFailedToStart application. The following
information is displayed:
• Job name
• Object ID
• Target server
• Expected execution

How to use a template if more repositories reside on the system

It is possible to have more than one repository on the system. By default, all
templates for monitoring jobs check only jobs for the first repository specified in the
DMSPI configuration file (%OvDataDir%\dmspi\conf\dmspi.cfg on Windows or
$OvAgentDir/dmspi/conf/dmspi.cfg on Unix systems).

Monitoring Jobs 47
 To monitor the jobs of a specific repository, make a copy of the template and add 
--docbase and --template arguments to the program name.
Example:
dmspi_job(.exe) --status --docbase=repository_name --
template=my_new_template_name
If the wrong repository is specified, a DMSPI internal error message is sent

48 User Guide
Chapter 8

Monitoring Documentum Sessions

and User Activity
About Monitoring Documentum Sessions and User Activity
A repository session is opened when an end user or application establishes a
connection to a server. Inactive repository sessions are sessions in which the server
connection has timed out but the client has not specifically disconnected from the
server. If the client sends a request to the server, the inactive session automatically
reestablishes its server connection and becomes active.
To monitor Documentum sessions and user activity with SPI for EMC Documentum,
you can use the following templates located in the SPI for EMC Documentum/
DMSPI-EMC Documentum 5.3/DMSPI-D5 Content Server and SPI for EMC
Documentum/DMSPI-EMC Documentum 6/DMSPI-D6 Content Server:
DMSPI-DxCS Quick Start template group:
• DMSPI-SessionsCpu
It is a Measurement Threshold template that monitors CPU utilization of user
sessions.
DMSPI-DxCS Additional template group:
• DMSPI-SessionsMem
It is a Measurement Threshold template that monitors memory usage of user
sessions (should be deployed on UNIX nodes only).
DMSPI-DxCS Data Collection template group:
• DMSPI-SessionsPerf
It is a Scheduled Task template that collects user session CPU and memory
utilization and stores information to HP Performance Agent.
• DMSPI-NumberOfSessionsPerf
It is a Scheduled Task template that collects the number of active and inactive
sessions and stores information to HP Performance Agent.

How to Monitor User Sessions for Extensive CPU Utilization

To check the user sessions for extensive CPU utilization and discover the users that
make the most load on the Documentum server, deploy the DMSPI-SessionsCpu
template on the node. The template runs dmspi_sess with the following arguments:
dmspi_sess --cpu
A message is sent if CPU utilization of all sessions is over:
• 50% - warning message

50 User Guide
 • 70% - minor message
• 90% - major message
You can add an additional filter switch --max_cpu={session cpu threshold} to
turn on monitoring of the most CPU intensive sessions. If the switch is specified, the
monitor template will send the message only if sum of all sessions CPU utilization
exceeds the threshold and the session with max CPU utilization exceeds the specified
--max_cpu option value.
Additionally, an automatic action is prepared that displays top 10 user sessions by
CPU utilization.
See also “How to use a template if more servers reside on the system” on page 52.

How to Monitor User Sessions for Extensive Memory Usage

To check the user sessions for extensive memory usage and discover the users that
make the highest load on the Documentum server, deploy the 
DMSPI-SessionsMem template on the node. The template runs dmspi_sess with the
following arguments:
dmspi_sess --mem
A message is sent if memory usage of the session with highest memory usage
exceeds:
• 70000KB - warning message
• 100000KB - minor message
• 150000KB - major message
Additionally, an automatic action is prepared that displays top 10 user sessions by
memory usage.

NOTE

This feature is available only for UNIX nodes.

See also “How to use a template if more servers reside on the system” on page 52.

How to Collect User Session CPU and Memory Utilization

To collect performance data about user session CPU utilization, deploy the DMSPI-
SessionsPerf template on the node. The template runs dmspi_sess with the
following arguments:

Monitoring Documentum Sessions and User Activity 51

 dmspi_sess --top_cpu=10 --perf
The template collects and stores information about top 10 user sessions with the
extensive CPU utilization. The information is stored to HP Performance Agent
where DMSPI_SESS_UTIL class is added to the DMSPI datasource with the following
metrics:
• Session ID
• User name
• System name
• Server name
• Repository name
• Session CPU utilization (%)
• Session memory usage in KB (for Unix systems only)

How to Collect Information About User Sessions

To collect performance data about current server sessions, deploy the DMSPI-
NumOfSessionsPerf template on the node. This Schedule template runs
dmspi_sess with the following arguments:
dmspi_sess --num_sess --perf
The template stores information about server sessions to HP Performance Agent.
The DMSPI_SESSIONS class will be added to the DMSPI datasource with the
following metrics:
• Docbase name
• Server name
• Number of active sessions
• Number of inactive sessions
• Maximum number of concurrent sessions

How to use a template if more servers reside on the system

It is possible to have more than one server on the system. By default, all templates
for monitoring Documentum sessions monitor only Documentum sessions of the
first server specified in the DMSPI configuration file
(%OvDataDir%\dmspi\conf\dmspi.cfg on Windows or $OvAgentDir/dmspi/
conf/dmspi.cfg on Unix systems).
To monitor the Documentum sessions of a specific server, make a copy of the
template and add --server and --template arguments to the program name.
Example:

52 User Guide
dmspi_sess(.exe) --mem --server=server_name --
template=my_new_template_name
If the wrong server is specified, a DMSPI internal error message is sent.

Monitoring Documentum Sessions and User Activity 53

54 User Guide
Chapter 9

Monitoring Client Response

About Monitoring Client Response
A client is an end user, application, or process that uses Content Server to access the
repository.
To monitor client response time for basic transactions (connect, check-in, check-out,
and so on) with SPI for EMC Documentum, you can use the following templates
located in the SPI for EMC Documentum/DMSPI-EMC Documentum 5.3/DMSPI-
D5 Client and SPI for EMC Documentum/DMSPI-EMC Documentum 6/DMSPI-D6
Client template groups:
DMSPI-DxC Quick Start template group:
• DMSPI-ClientResponseCheckin
It is a Measurement Threshold template that monitors the time needed to checkin
the dummy document to the repository.
• DMSPI-ClientResponseCheckout
It is a Measurement Threshold template that monitors the time needed to
checkout the dummy document from the repository.
• DMSPI-ClientResponseConnect
It is a Measurement Threshold template that monitors the repository connect time.
DMSPI-DxC Additional template group:
• DMSPI-ClientResponseDelete
It is a Measurement Threshold template that monitors the time needed to delete
the dummy document from the repository.
• DMSPI-ClientResponseDisconnect
It is a Measurement Threshold template that monitors the repository disconnect
time.
• DMSPI-ClientResponseImport
It is a Measurement Threshold template that monitors the time needed to import
the dummy document to the repository.
DMSPI-DxC Data Collection template group:
• DMSPI-ClientResponsePerf
It is a Scheduled Task template that collects information about the response time
of standard Documentum operations and stores it to HP Performance Agent.

56 User Guide
How to Monitor Client Response Time
To monitor client response time, deploy all or some of the following templates on the
system (depends on which operations would you like to monitor):
• DMSPI-ClientResponseCheckin
• DMSPI-ClientResponseCheckout
• DMSPI-ClientResponseConnect
• DMSPI-ClientResponseDelete
• DMSPI-ClientResponseDisconnect
• DMSPI-ClientResponseImport
All templates use the dmspi_conn executable with the different arguments:
dmspi_conn --checkin
dmspi_conn --checkout, and so on
A message will be sent if the response is longer than:
• 3000 ms - warning message
• 5000 ms - minor message
All templates, except DMSPI-ClientResponseConnect and DMSPI-
ClientResponseDisconnect, use the dummy document (dmspi_test.txt stored in
the HPOM Agent instrumentation directory) to measure the response times from the
repository. It is recommended to set the dm_DMClean job as active to remove
orphaned content objects.
See also “How to monitor response time of a specific server/repository” on page 59.

How to Collect Client Response Time Data

To collect performance data about the client response time, deploy the DMSPI-
ClientResponsePerf template on the node. The template runs dmspi_conn with the
following arguments:
dmspi_conn --perf
The template collects and stores the following information to HP Performance Agent
where DMSPI_CLIENT_RESP class is added to the DMSPI datasource with the
following metrics:
• Repository name
• Server name
• User name

Monitoring Client Response 57

 • Connect time (ms)
• Import time (ms)
• CheckOut time (ms)
• CheckIn time (ms)
• Delete time (ms)
• Disconnect time (ms)
See also “How to monitor response time of a specific server/repository” on page 59.

How to Display Client Response Statistics

To display client response time statistics, go to Application Bank/SPI for EMC
Documentum/DMSPI-Documentum/DMSPI-Content Server and run the Client
Response Time application. The following information is displayed:
• Repository name
• Server name
• User name
• Connect time (ms)
• Disconnect time (ms)
• Checkin time (ms)
• Checkout time (ms)
• Delete time (ms)
• Import time (ms)
• Total time (ms)

How to configure end-user or application server systems to monitor client

response
If you deploy the templates to end-user systems, you must perform the following
steps to configure the systems:
1. Deploy instrumentation (SPI for EMC Documentum, Data Collector) on the
node.
2. Create the %OvDataDir%\dmspi directory on Windows or $OvAgentDir/dmspi
on Unix (for example, /var/opt/OV/dmspi).
3. Go to the %OvDataDir%\dmspi on Windows or $OvAgentDir/dmspi on Unix and
extract the dmspi.zip file with the following command:
unzip %OvInstrumentationDir%\dmspi.zip (or $OvInstrumentationDir/
dmspi.zip on Unix).

58 User Guide
 4. Create the dmspi.cfg file and save it to the %OvDataDir%\dmspi\conf directory
on Windows or $OvAgentDir/dmspi/conf directory on Unix.

NOTE

The dmcl.ini file should contain all connection brokers that you specify in the
dmspi.cfg configuration file.

Example:
If you want to connect to the D1_eagle2 repository via eagle2 server, you will
need the following information:
• connection broker system and port number to which eagle2 sends its
connection information
• Documentum admin username and password
The dmspi.cfg file should look like this:
[SERVER_eagle2]
docbrokers=nova@1489
docbase=D1_eagle2
[SERVERS]
servers=eagle2
[DOCBASE_D1_eagle2]
servers=eagle2
docbrokers=nova@1489
user=dmuser
password=2D23E19CF1A07811
[DOCBASES]
docbases=D1_eagle2
5. Start the service manually as the Documentum admin user:
/$OvAgentDir/dmspi/bin/dmspi_svc --start

How to monitor response time of a specific server/repository

By default, all templates for monitoring client response times monitor only the
response from the first server (repository) specified in the DMSPI configuration file
(%OvDataDir%\dmspi\conf\dmspi.cfg on Windows or $OvAgentDir/dmspi/
conf/dmspi.cfg on Unix systems).

Monitoring Client Response 59

 If more servers or repositories are specified in the DMSPI configuration file and you
want to monitor response time of a specific server (repository), make a copy of the
template and add --server (--docbase) and --template arguments to the
program name.
Example:
dmspi_conn(.exe) --checkin --server=server_name --
template=my_new_template_name
dmspi_conn(.exe) --checkin --docbase=repository_name --
template=my_new_template_name
If the wrong server (repository) is specified, a DMSPI internal error message is sent.

60 User Guide
Chapter 10

Monitoring Log Files

About Monitoring Log Files
To periodically monitor Server, connection broker, lifecycle, and user session log
files for error messages with SPI for EMC Documentum, use the templates listed
below. There are several patterns prepared that are searched in the log files.
Additional custom patterns for searching can be added later.
Log file templates are located in the SPI for EMC Documentum/DMSPI-EMC
Documentum 5.3/DMSPI-D5 Content Server and SPI for EMC Documentum/
DMSPI-EMC Documentum 6/DMSPI-D6 Content Server template groups.
DMSPI-DxCS Quick Start template group:
• DMSPI-ServerLog
It is a Logfile Entry template that periodically scans the server log file for new
messages (logs) with the content that needs attention.
• DMSPI-AgentexecLog
It is a Logfile Entry template that periodically scans the agentexec log file for new
messages (logs) with the content that needs attention.
DMSPI-DxCS Additional template group:
• DMSPI-LifecycleLog
It is a Logfile Entry template that periodically looks for the lifecycle log files and
if they exist scans them for new messages (logs) with the content that needs
attention.
• DMSPI-UserSessionLog
It is a Logfile Entry template that periodically scans user session log files for new
messages (logs) with the content that needs attention.

How to Monitor the Server Log File

To monitor the server log file ($DOCUMENTUM/dba/log/server_log_name.log - log
file name is usually a server name, but can be a name that you specify in the server
ini file), deploy the DMSPI-ServerLog template on the node. The template is
started periodically and scans the server log content. The template runs
dmspi_log(.exe) with the following arguments:
dmspi_log(.exe) --srvr
A message is sent if the following strings are found:
• FATAL ERROR - major message
• ERROR - minor message

62 User Guide
 • WARNING - warning message
It is possible to have more than one server on the same system. By default, DMSPI-
ServerLog template will scan only the log file of the first server specified in the
DMSPI configuration file (%OvDataDir%\dmspi\conf\dmspi.cfg on Windows or
$OvAgentDir/dmspi/conf/dmspi.cfg on Unix systems) or the first one found on
the system.
To scan the log file of the specific server, make a copy of the template and add --
server and --template arguments to the program name.
Example:
dmspi_log(.exe) --srvr --server=server_name --
template=my_new_template_name
If the wrong server is specified, a DMSPI internal error message is sent.

How to Monitor the agentexec Log File

To monitor the agentexec log file agentexec.log (located in $DOCUMENTUM/dba/
log/docbase_id_hex/agentexec/), deploy the DMSPI-AgentexecLog template on
the node. The template is started periodically and scans the agentexec log content.
The template runs dmspi_log(.exe) with the following arguments:
dmspi_log(.exe) --agentexec
A message is sent if the following strings will be found:
• FATAL ERROR - major message
• ERROR - minor message
• WARNING - warning message
See also “How to use a template if more repositories reside on the system” on page
65.

How to Monitor the Lifecycle Log File

To monitor the lifecycle log files bp_transition and bp_schedule (located in
$DOCUMENTUM/dba/log/docbase_id_hex/), deploy the DMSPI-LifecycleLog
template on the node. The template is started periodically and scans the lifecycle log
content. The template runs dmspi_log(.exe) with the following arguments:
dmspi_log(.exe) -lifecycle
A message will be sent if the following strings will be found:
• FATAL ERROR - major message

Monitoring Log Files 63

 • ERROR - minor message
• WARNING - warning message
By default, the template checks the log files that were created in the last hour only to
avoid monitoring of too many files. You can specify how old log files you would like
to monitor with an optional argument --hours=number_of_hours. With this
parameter, only the log files that were created after the specified hour will be
scanned.
The template runs dmspi_log(.exe) with the following arguments:
dmspi_log(.exe) --lifecycle --hours=NrOfHrs
See also “How to use a template if more repositories reside on the system” on page
65.

How to Monitor the User Session Log File

To monitor the user session log file (located in $DOCUMENTUM/dba/log/
docbase_id_hex/user_name/), deploy the DMSPI-UserSessionLog template on
the node. The template is started periodically and scans user sessions log content.
The template runs dmspi_log(.exe) with the following arguments:
dmspi_log(.exe) --user=dmuser (replace dmuser with an existing Documentum
username)
A message will be sent if the following strings will be found:
• FATAL ERROR - major message
• ERROR - minor message
• WARNING - warning message
By default, the template checks the log files that were created in the last hour only to
avoid monitoring of too many files. You can specify how old log files you would like
to monitor with an optional argument --hours=number_of_hours. With this
parameter, only the log files that were created after the specified hour will be
scanned.
The template runs dmspi_log(.exe) with the following arguments:
dmspi_log(.exe) --user=dmuser --hours=NrOfHrs
You can monitor log files from any user by replacing dmuser with the user’s name.
See also “How to use a template if more repositories reside on the system” on page
65.

64 User Guide
How to use a template if more repositories reside on the system
It is possible to have more than one repository on the system. By default, all
templates for monitoring log files scan only log files of the first repository specified
in the DMSPI configuration file (%OvDataDir%\dmspi\conf\dmspi.cfg on
Windows or $OvAgentDir/dmspi/conf/dmspi.cfg on Unix systems).
To monitor the log files of a specific repository, make a copy of the template and add
--docbase and --template arguments to the program name.
Example:
dmspi_log(.exe) --dmuser --docbase=repository_name
--template=my_new_template_name
If the wrong repository is specified, a DMSPI internal error message is sent.

Monitoring Log Files 65

66 User Guide
Chapter 11

Monitoring File Stores

About Monitoring File Stores
File store storage areas are the basic building blocks of your storage strategy. In most
installations, the majority of the content files are stored in file store storage areas.
To monitor file store growth with SPI for EMC Documentum, you can use the
following templates located in the SPI for EMC Documentum/DMSPI-EMC
Documentum 5.3/DMSPI-D5 Content Server and SPI for EMC Documentum/
DMSPI-EMC Documentum 6/DMSPI-D6 Content Server template groups:
DMSPI-DxCS Quick Start template group:
• DMSPI-FileStores-filestore_01
It is a Measurement Threshold template that monitors available space on
Documentum file store filestore_01.
DMSPI-DxCS Additional template group:
• DMSPI-FileStores-thumbnail_store_01
It is a Measurement Threshold template that monitors available space on
Documentum file store thumbnail_store_01.
DMSPI-DxCS Data Collection template group:
• DMSPI-FileStoresPerf
It is a Scheduled Task template that collects file stores statistics and stores them to
HP Performance Agent.

How to Monitor Documentum File Stores

SPI for EMC Documentum provides two templates to monitor default Documentum
file stores (filestore_01 and thumbnail_store_01). To monitor other default or
custom-named file stores, you should make a copy of the template and change the
parameters of the new template. The template runs dmspi_filestores with the
following parameters:
dmspi_filestores --freespace --filestore="filestore_name" 
--template="new_template_name"
The file store template periodically scans the disk on which the specified file store
resides and returns used space in percentage. The message is sent to the console if:
• Used space > 95% - major message
• Used space > 90% - minor message
The DMSPI-FileStores-filestore_01 template for monitoring the filestore_01 file
store runs dmspi_filestores with the following parameters:

68 User Guide
 dmspi_filestores --freespace --filestore="filestore_01" 
--template="DMSPI-FileStores-filestore_01"
Example:
If you want to monitor the streaming_storage_01 filestore, perform the
following steps:
1. Make a copy of the DMSPI-Filestores-flestore_01 template and change its name
to DMSPI-Filestores-streaming_storage_01.
2. Change the parameters in the new template to:
dmspi_filestores --freespace --filestore="streaming_storage_01" --
template="DMSPI-FileStores-streaming_storage_01"
See also “How to use a template if more servers reside on the system” on page 70.

How to Collect File Store Statistics

To collect performance statistics about file stores, deploy the DMSPI-FilestorePerf
template on the node. The template runs dmspi_filestores with the following
arguments:
dmspi_filestores --perf
The template collects and stores all file stores statistics to HP Performance Agent.
The DMSPI_FILESTORES class is added to the DMSPI datasource with the following
metrics:
• Repository name
• Server name
• File store name
• Name of the distributed store
• Available space for the file store [KB]
• Total size of the disk on which file store resides [KB]
• File store used space [KB]
• Percentage of used space occupied by full text index
If you want to collect statistics for the specific file store, add the parameter
--filestore="filestore_name" to the dmspi_filestores executable:
dmspi_filestores -perf -filestore="filestore_name"
See also “How to use a template if more servers reside on the system” on page 70.

Monitoring File Stores 69

How to Display File Stores Statistics
To display file stores statistics, go to Application Bank/SPI for EMC Documentum/
DMSPI-Documentum/DMSPI-Content Server and run the Filestore Info
application. The following information will be displayed for each file store:
• Repository name
• Server name
• File store name
• Name of the distributed store
• File store Available space [KB]
• Filesystem capacity [KB]
• File store used space [KB]
• Percentage of occupied space:
• Content storage [%]
• Full text index [ %]

How to use a template if more servers reside on the system

It is possible to have more than one server on the system. By default, all templates
for monitoring file stores monitor only file stores of the first server specified in the
DMSPI configuration file (%OvDataDir%\dmspi\conf\dmspi.cfg on Windows or
$OvAgentDir/dmspi/conf/dmspi.cfg on Unix systems).
To monitor the file stores of a specific server, make a copy of the template and add
--server and --template arguments to the program name.
Example:
dmspi_filestores(.exe) --freespace --filestore=filestore_name --
server=server_name --template=my_new_template_name
If the wrong repository is specified, a DMSPI internal error message is sent.

70 User Guide
Chapter 12

Monitoring Database Tables

About Monitoring Database Tables
Documentum database tables can grow rapidly if something goes wrong in your
Documentum environment. For example, requests that are sent from client
applications to the Documentum Content Server are stored in a special queue called
dm_queue (database table). Documentum applications (for example, Rendition
services) poll dm_queue at predefined intervals, and retrieve any pending requests. If
Rendition server is down or does not work properly, the requests are accumulating
and queue item table can grow rapidly. With SPI for EMC Documentum, you can
monitor the size and the growth of any Documentum database table. Additionally, it
is possible to get information about events that occurred in your Documentum
environment - for example, number of unprocessed renditions, number of login
failures, and so on.
Templates for monitoring database tables are located in the SPI for EMC
Documentum/DMSPI-EMC Documentum 5.3/DMSPI-D5 Content Server and SPI
for EMC Documentum/DMSPI-EMC Documentum 6/DMSPI-D6 Content Server
template groups:
DMSPI-DxCS Quick Start template group:
• DMSPI-DmAudittrailSize
It is a Measurement Threshold template that monitors the size of the
dm_audittrail table.

• DMSPI-DmiQueueItemSize
It is a Measurement Threshold template that monitors the size of the
dmi_queue_item table.

• DMSPI-LogonFailure
It is a Measurement Threshold template that monitors the number of logon
failures.
• DMSPI-RenditionQueue
It is a Measurement Threshold template that monitors the number of unprocessed
renditions.
DMSPI-DxCS Data Collection template group:
• DMSPI-DmAudittrailSizePerf
This is a Schedule template that collects information about dm_audittrail table
size growth.
• DMSPI-DmiQueueItemSizePerf
This is a Schedule template that collects information about dmi_queue_item table
size growth.

72 User Guide
 • DMSPI-LogonFailurePerf
This is a Schedule template that collects information about number of logon
failures.
• DMSPI-RenditionQueuePerf
This is a Schedule template that collects information about the number of
unprocessed rendition.
• DMSPI-UnfinishedWorkflowsPerf
It is a Schedule template that collects data on workflows that are not finished yet
and are older than the specified time period.

How to Monitor the Table Size

SPI for EMC Documentum provides two templates to monitor the size of the
Documentum tables:
• DMSPI-DmAudittrailSize
• DMSPI-DmiQueueItemSize
Both templates use dmspi_table with the following arguments:
dmspi_table --table="dm_audittrail"
dmspi_table --table="dmi_queue_item"
A message will be sent if the table size is bigger than:
• 5000 - warning message
• 8000 - minor message
To monitor the size of any other Documentum table, you should make a copy of the
DMSPI-DmAudittrailSize or DMSPI-DmiQueueItemSize template and change the
--table parameter of the new template:
dmspi_table --table="new_table_name"
See also “How to use a template if more repositories reside on the system” on page
76.

How to Monitor the Number of Specific Events in the Table

The events that occurred in the Documentum environment (for example, rendition
requests, logon failures, and so on) are stored to database tables. To monitor the
number of such events, SPI for EMC Documentum provides two templates:
• DMSPI-LogonFailure, to monitor the number of logon failures

Monitoring Database Tables 73

 • DMSPI-RenditionQueue, to monitor the number of unprocessed renditions
Both templates use dmspi_table with the following arguments:
dmspi_table --dql_query="dmspi_logfail.dql"
dmspi_table --dql_query="dmspi_rendqueue.dql"
A message will be sent to the console if:
• number of logon failures > 50 - warning message
• number of logon failures > 80 - minor message
• number of unprocessed renditions > 30 - warning message
• number of unprocessed renditions > 50 - minor message
To monitor the number of events in the Documentum database table, you should
write your own DQL query and store it to the instrumentation directory. In this
directory, you can also find the DQL example queries that get the number of
unprocessed rendition events (dmspi_rendqueue.dql) and logon failures
(dmspi_logfail.dql). Then make a copy of the DMSPI-LogonFailure or DMSPI-
RenditionQueue template and change the following:
• Specify the name of the file that contains your DQL query (--
dql_query="new_dql_query")
• Specify the template name (--template="new_template_name")
• In the modified template, change the threshold levels and messages if necessary
dmspi_table --dql_query="new_dql_query" --
template="new_template_name"
See also “How to use a template if more repositories reside on the system” on page
76.

How to Collect Table Size Growth Information

To collect information about the size and growth of the audittrail and
dmi_queue_item database tables, deploy the DMSPI-DmAudittrailSizePerf and
DMSPI-DmiQueueItemSizePerf templates on the node. The templates will run the
dmspi_table executable with the following arguments:
dmspi_table --table="dm_audittrail" -perf
dmspi_table --table="dmi_queue_item" --perf
To collect the size and growth of any Documentum table, make a copy of the
DMSPI-DmAudittrailSizePerf or DMSPI-DmiQueueItemSizePerf template and
change the --table parameter of the new template:
dmspi_table --table="table_name" --perf

74 User Guide
 The DMSPI_EVENT_SIZE class will be added to the DMSPI datasource with the
following metrics:
• Repository name
• Table name
• Table size
See also “How to use a template if more repositories reside on the system” on page
76.

How to Collect Event Information

To collect information about the number of unfinished renditions and logon failures,
deploy the DMSPI-RenditionQueuePerf and DMSPI-LogonFailurePerf templates
on the node. The template will run dmspi_table with the following arguments:
dmspi_table --dql_query="dmspi_logfail.dql" --perf
dmspi_table --dql_query="dmspi_rendqueue.dql" --perf
To collect the number of any events in the Documentum database table, you should
write your own DQL query and store it to the $OvInstrumentationDir directory. In
this directory, you can also find the DQL example queries that get the number of
unprocessed events (dmspi_rendqueue.dql) and logon failures
(dmspi_logfail.dql). Then make a copy of the DMSPI-LogonFailurePerf or
DMSPI-RenditionQueuePerf template and change or add the --dql_query and --
event parameters of the new template:
dmspi_table --dql_query="new_dql_query" --event="event_name" --perf
The DMSPI_EVENT_SIZE class will be added to the DMSPI datasource with the
following metrics:
• Repository name
• Table name
• Event name
• Number of events
See also “How to use a template if more repositories reside on the system” on page
76.

How to Collect Unfinished Workflows Information

To collect data on workflows that are not finished yet and are older than a specified
time period, deploy the DMSPI-UnfinishedWorkflowsPerf template on the node.
The template runs dmspi_table with the following arguments:

Monitoring Database Tables 75

 dmspi_table --perf --workflow=50
To change the predefined time period (50 days), change the --workflow parameter
value.
The DMSPI_U_WORKFLOWS class will be added to the DMSPI datasource with the
following metrics:
• Repository name
• Workflow ID
• Workflow name
• Workflow state
• Workflow start date
• Supervisor name
• Workflow age
See also “How to use a template if more repositories reside on the system” on page
76.

How to use a template if more repositories reside on the system

It is possible to have more than one repository on the system. By default, all
templates for monitoring database tables monitor only tables of the first repository
specified in the DMSPI configuration file (%OvDataDir%\dmspi\conf\dmspi.cfg on
Windows or $OvAgentDir/dmspi/conf/dmspi.cfg on Unix systems).
To monitor the database tables of a specific repository, make a copy of the template
and add --docbase and --template arguments to the program name.
Example:
dmspi_table (.exe) --table="table_name" --docbase=repository_name --
template=my_new_template_name
If the wrong repository is specified, a DMSPI internal error message is sent.

76 User Guide
Chapter 13

Monitoring Index Server

About Monitoring Index Server
Full-text indexes enable users to search a repository for specific text found in stored
documents or the attributes of documents. The full-text indexing software consists of
three components: Content Server, the index agent, and the index server.
Content Server manages the objects in a repository, generates the events that trigger
full-text indexing operations, queries the full-text indexes, and returns query results
to client applications.
The index agent exports documents from a repository and prepares them for
indexing.
The index server creates full-text indexes and responds to full-text queries from
Content Server.
With SPI for EMC Documentum you can monitor:
• If is it possible to add new documents to full-text index
• If is it possible to search for documents and how fast is the search
• What's the index server performance (save-to-search latency)
• Index server availability and resource consumption
• If there are any internal issues
• Full-text index disk usage
• Index agent availability and resource consumption
• Index queue size and number of failed indexing items
To monitor the index server with SPI for EMC Documentum, you can use the
following templates that have to be deployed on the Content Server node and are
located in the SPI for EMC Documentum/DMSPI-EMC Documentum 5.3/DMSPI-
D5 Content Server and SPI for EMC Documentum/DMSPI-EMC Documentum 6/
DMSPI-D6 Content Server template groups:
DMSPI-DxCS Additional template group:
• DMSPI-FTQueryResponse
It is a Measurement Threshold template that monitors full-text query response
time.
• DMSPI-FTSearchAvailability
It is a Measurement Threshold template that checks Index Server search
availability.
• DMSPI-IdxAgtStatus
It is a Measurement Threshold template that checks index agent status.

78 User Guide
• DMSPI-IdxQueueFailed
It is a Measurement Threshold template that checks failed items in index queue.
• DMSPI-IdxQueueSize
It is a Measurement Threshold template that monitors the size of index queue.
• DMSPI-IndexingLatency
It is a Measurement Threshold template that monitors how long does it take for a
document to become searchable.
DMSPI-DxCS Additional Data Collection template group:
• DMSPI-IdxQueueFailedPerf
It is a Scheduled Task template that collects failed items statistics.
• DMSPI-IdxQueueSizePerf
It is a Scheduled Task template that collects queue size statistics.
• DMSPI-IndexingPerf
It is a Scheduled Task template that collects full-text indexing performance
statistics.
Additionally, the following templates located in the SPI for EMC Documentum/
DMSPI-EMC Documentum 5.3/DMSPI-D5 Index Server and SPI for EMC
Documentum/DMSPI-EMC Documentum 6/DMSPI-D6 Index Server template
groups have to be deployed on the index server node:
DMSPI-DxIS Quick Start template group:
• DMSPI-NctrlStatus
It is a Measurement Threshold template that checks index server's node controller
module status.
• DMSPI-ProcServerStatus
It is a Measurement Threshold template that checks index server's document
processor module status.
• DMSPI-IdxAgtLog
It is a Logfile Entry template that periodically scans the index agent log file for new
messages (logs) with the content that needs attention.
• DMSPI-IdxSrvrDiskUsage
It is a Measurement Threshold template that checks index server's full-text index
disk usage (in %).
• DMSPI-IdxSrvrLog
It is a Logfile Entry template that periodically scans the index server log file for
new messages (logs) with the content that needs attention.

Monitoring Index Server 79

 DMSPI-DxIS Additional template group:
• DMSPI-ContentDistributorStatus
It is a Measurement Threshold template that checks index server's content
distributor module status.
• DMSPI-QRServerStatus
It is a Measurement Threshold template that checks index server's QRServer
module status.
• DMSPI-RTSTopDispatcherStatus
It is a Measurement Threshold template that checks index server's RTS Top
Dispatcher module status.

DMSPI-DxIS Data Collection template group:

• DMSPI-IdxAgtProcPerf
It is a Scheduled Task template that collects performance data for index agent
process.
• DMSPI-IdxSrvrDiskUsagePerf
It is a Scheduled Task template that collects index server's full-text index disk
usage.
• DMSPI-IdxSrvrProcPerf
It is a Scheduled Task template that collects performance data for index server
processes.

How to Monitor Index Server Availability

To monitor the index server availability and detect possible cases when the index
server is not functioning properly, deploy the DMSPI-{Index Server
component}Status templates to the index server node and the DMSPI-
FTSearchAvailability template on the Content Server, index server, application
server, or Documentum client node. A message is sent if index server does not work
properly or there is no connectivity with the host where index server resides.
The template DMSPI-FTSearchAvailability runs dmspi_idxsrvr(.exe) with the
following arguments:
dmspi_idxsrvr(.exe) --availability

80 User Guide
See also “How to use a template if more repositories reside on the system” on page
90.
To monitor the status of main components in the FAST data search system, deploy
the following templates to index server node:
• DMSPI-NctrlStatus for monitoring Node Controller component
• DMSPI-QRServerStatus for monitoring QRServer component
• DMSPI-RTSTopDispatcherStatus for monitoring Top Level Fdispatch
component
• DMSPI-ProcServerStatus for monitoring Document Processors component
• DMSPI-ContentDistributorStatus for monitoring Content Distributors
component
To monitor the status of any index server component, copy the DMSPI-NctrlStatus
template and change the --module={module process name} parameter of the new
template. A general index server component status monitoring template would look
like this:
dmspi_idxsrvr(.exe) --module={module process name} --status --
template={template_name}
The template DMSPI-NctrlStatus runs dmspi_idxsrvr(.exe) with the following
arguments:
dmspi_idxsrvr(.exe) --module=nctrl --status --template=DMSPI-
NctrlStatus
The template DMSPI-QRServerStatus runs dmspi_idxsrvr(.exe) with the
following arguments:
dmspi_idxsrvr(.exe) --module=qrserver --status --template=DMSPI-
QRServerStatus
The template DMSPI-RTSTopDispatcherStatus runs dmspi_idxsrvr(.exe) with
the following arguments:
dmspi_idxsrvr(.exe) --module=topfdispatch --status --template=DMSPI-
RTSTopDispatcherStatus
The template DMSPI-ProcServerStatus runs dmspi_idxsrvr(.exe) with the
following arguments:
dmspi_idxsrvr(.exe) --module=procserver_n --status --template=DMSPI-
ProcServerStatus
The template DMSPI-ContentDistributorStatus runs dmspi_idxsrvr(.exe) with
the following arguments:
dmspi_idxsrvr(.exe) --module=contentdistributor --status --
template=DMSPI-ContentDistributorStatus

Monitoring Index Server 81

How to Monitor Index Server Log File
To monitor the index server log file, deploy the DMSPI-IdxSrvrLog template on the
index server node. The template is started periodically and scans the index server
log content. The template runs dmspi_log(.exe) with the following arguments:
dmspi_log(.exe) --index_server
A message is sent if the following string is found:
• FATAL - major message
• ERROR - minor message

How to Monitor Indexing Latency

To monitor indexing latency (the time needed to merge a new document into the FT
index and to make it searchable), deploy the DMSPI-IndexingLatency template on
the Content Server node. The template runs dmspi_idxsrvr(.exe) with the
following arguments:
dmspi_idxsrvr(.exe) --latency
A message is sent if:
• Create/delete latency > 180 seconds - warning message
• Create/delete latency > 360 seconds - minor message
• Create/delete latency > 600 seconds - major message
See also “How to use a template if more repositories reside on the system” on page
90.

How to Monitor the FT Query Response Time

To monitor if your full text queries come back in a timely fashion, deploy the
DMSPI-FTQueryResponse template to the Content Server node. The template runs
dmspi_idxsrvr(.exe) with the following arguments:
dmspi_idxsrvr(.exe) --query
A message is sent if:
• Query response time > 2 seconds - warning message
• Query response time > 5 seconds - minor message
• Query response time > 10 seconds - major message

82 User Guide
 See also “How to use a template if more repositories reside on the system” on page
90.

How to Monitor the Index Queue Size

To monitor the size of Index queue, deploy the DMSPI-IdxQueueSize template on
the Content Server node. The template runs dmspi_table(.exe) with the following
arguments:
dmspi_table(.exe) --dql_query=dmspi_indsize.dql
A message is sent if:
• Index queue size > 500 - warning message
• Index queue size > 5000 - minor message
• Index queue size > 10000 - major message
See also “How to use a template if more repositories reside on the system” on page
90.

How to Monitor the Index Queue for Failed Items

To monitor the Index queue and detect possible failed queue items, deploy the
DMSPI-IdxQueueFailed template on the Content Server node. The template runs
dmspi_table(.exe) with the following arguments:
dmspi_table(.exe) --dql_query=dmspi_indfail.dql
Message will be sent if there are Index queue items with failed state:
• Number of failed items > 500 - warning message
• Number of failed items > 5000 - minor message
• Number of failed items > 10000 - major message
See also “How to use a template if more repositories reside on the system” on page
90.

How to Monitor the FT Index Disk Usage

To monitor the full-text index disk usage, deploy the DMSPI-IdxSrvrDiskUsage
template on the index server node. The DMSPI-IdxSrvrDiskUsage template runs
dmspi_idxsrvr(.exe) with the following arguments:
dmspi_idxsrvr(.exe) --disk_usage

Monitoring Index Server 83

 A message is sent if:
• FT index disk usage > 90% - minor message
• FT index disk usage > 95% - major message

How to Collect Index Server statistics

To collect index server statistics, deploy the DMSPI-IdxSrvrProcPerf template on
the index server node. The template runs dmspi_proc(.exe) with the following
arguments:
dmspi_proc(.exe) -perf --index_server
Process performance metrics (CPU & MEM utilization) for the following index
server processes will be collected:

ProcessDescription Module name

FASTDataSearch Data Search 4.0 Node Controller

Service Fast Search & Transfer
ASA
ominNames Name Service
logserver Log Server
configserver Config Server
contentdistributor Content Distributor
frtsobj RTS Indexer
fsearchctrl RTS Top Dispatcher
fsearchctrl RTS Search
qrserver QRServer
statusserver Status Server
procserver Document Processor
Index server information will be stored to HP Performance Agent. The
DMSPI_IDXSRVR_PROC class is added to the DMSPI datasource with the following
metrics:
• Process label
• Process name
• Process description
• Process status

84 User Guide
 • Process memory usage (KB)
• Process virtual memory usage (KB)
• Process CPU utilization (%)

How to Collect Indexing Performance Statistics

Indexing process consists of three major phases:
• Waiting to be indexed - after Save, Checkin, Destroy, Readonlysave, or
MoveContent operation is performed on a SysObject in the repository, a new
queue item is created in the dmi_queue_item table
• Document preprocessing - Index agent and index server create their own
representation of the document (DFTXML, FIXML) before the document is
indexed
• Indexing of the content file and its properties
To collect indexing performance statistics, deploy the DMSPI-IndexingPerf
template to the node. The template runs dmspi_idxsrvr(.exe) binary with the
following arguments:
dmspi_idxsrvr(.exe) --latency --perf
Statistics about the different stages of indexing is logged to HP Performance Agent.
The DMSPI_IDX_PERFORMANCE class is added to the DMSPI datasource with the
following metrics:
• Index agent name
• Repository name
• Awaiting time when adding new document to the FT index (ms)
• Preprocessing time when adding new document to the FT index (ms)
• Indexing time when adding new document to the FT index (ms)
• Awaiting time when removing a document from the FT index (ms)
• Preprocessing time when removing a document from the FT index (ms)
• Time to remove a document from the FT index (ms)
• FT query response time (ms)
See also “How to use a template if more repositories reside on the system” on page
90.

Monitoring Index Server 85

How to Collect the Index Queue Size Information
To collect the size of Index queue, deploy the DMSPI-IdxQueueSizePerf template
on the Content Server node. The template runs dmspi_table(.exe) with the
following arguments:
dmspi_table(.exe) --dql_query=dmspi_indsize.dql --perf
Index queue size information will be stored to HP Performance Agent. The
DMSPI_EVENT_SIZE class is added to the DMSPI datasource with the following
metrics:
• Repository name
• Table name
• Event name
• Number of events
See also “How to use a template if more repositories reside on the system” on page
90.

How to Collect Information on Failed Items in the Index Queue

To collect information about the number of failed items in the index queue, deploy
the DMSPI-IdxQueueFailedPerf template on the Content Server node. The
template runs dmspi_table(.exe) with the following arguments:
dmspi_table(.exe) --dql_query=dmspi_indfail.dql --perf
Information about number of failed items in Index queue will be stored to HP
Performance Agent. The DMSPI_EVENT_SIZE class is added to the DMSPI datasource
with the following metrics:
• Repository name
• Table name
• Event name
• Number of events
See also “How to use a template if more repositories reside on the system” on page
90.

86 User Guide
How to Collect the FT Index Disk Usage Information
To collect statistics for full-text index disk usage, deploy the DMSPI-
IdxSrvrDiskUsagePerf template on the index server node. The DMSPI-
IdxSrvrDiskUsagePerf template executes dmspi_idxsrvr(.exe) executable with
the following arguments:
dmspi_idxsrvr(.exe) --disk_usage --perf
Information about disk usage will be stored to HP Performance Agent. The
DMSPI_IDXSRVR_DU class is added to the DMSPI datasource with the following
metrics:
• Index server node name
• Full-text index location
• Disk usage (%)
• Disk usage (GB)
• Disk size (GB)
• Disk free size (GB)

How to Monitor the Index Agent Availability

To monitor the index agent status (running, not responding), deploy the DMSPI-
IdxAgtStatus template on the Content Server node. The DMSPI-IdxAgtStatus
template executes dmspi_idxsrvr(.exe) executable with the following arguments:
dmspi_idxsrvr(.exe) --index_agent="" --status
This template will also detect if the index agent Web application, that is running as
Apache Tomcat servlet, is responsive. The default template will monitor only the
first found index agent for the first repository in the local configuration file. To
monitor another index agent, you will have to create a copy of the DMSPI-
IdxAgtStatus template and set the following parameters in the new template:
--docbase={repository name} and --index_agent={Index Agent name}
A message is sent, if the index agent does not work properly or there is no
connectivity with the host where index agent resides.

How to Monitor the Index Agent Log File

To monitor the index agent log file, deploy the DMSPI-IdxAgtLog template on the
index server node. The template is started periodically and scans the index agent log

Monitoring Index Server 87

 content for error or warning messages. The template runs dmspi_log(.exe) with
the following arguments:
dmspi_log(.exe) --index_agent="Index Agent name"

How to Collect the Index Agent Process Information

To collect index agent process statistics, deploy the DMSPI-IdxAgtProcPerf
template on the index server node. The template runs dmspi_proc(.exe) with the
following arguments:
dmspi_proc(.exe) -perf --index_agent={index agent name}
Process performance information (CPU & MEM utilization) will be stored to HP
Performance Agent. The DMSPI_IDXAGT_PROC class is added to the DMSPI
datasource with the following metrics:
• Process label
• Process name
• Repository name
• Process status
• Process memory usage (KB)
• Process virtual memory usage (KB)
• Process CPU utilization (%)

How to Display Index Server Status

To display information about the status of index server components, go to
Applications/SPI for EMC Documentum/DMSPI-Documentum/DMSPI-Index
Server and run the Index Server Status application. The following information is
displayed for each index server component:
• Module name
• Process name
• PID
• Statusapplication

88 User Guide
How to Display Index Server Disk Usage
To display information about the index server disk usage, go to Applications/SPI for
EMC Documentum/DMSPI-Documentum/DMSPI-Index Server and run the Index
Server Disk Usage application. The following information is displayed:
• Fulltext location
• Current diska usage
• Used disk space
• Total disk space
• Free disk space

How to Display Index Agent Status

To display index agent status go to Applications/SPI for EMC Documentum/
DMSPI-Documentum/DMSPI-Content Server and run the Index Agent Status
application. The following information is displayed:
• Index agent name
• Index agent URL
• Repository
• Index name
• Status

How to Display Indexing Latency

To display information about indexing latency (the time needed to merge a new
document into the FT index and to make it searcheable) go to Applications/SPI for
EMC Documentum/DMSPI-Documentum/DMSPI-Content Server and run the
Indexing latency application. The following information is displayed:
• Repository
• Index agent
• Awaiting time
• Processing time
• Indexing time

Monitoring Index Server 89

How to Display FT Query Response Time
To display information about current full-text query response time, go to
Applications/SPI for EMC Documentum/DMSPI-Documentum/DMSPI-Content
Server and run the FT Query Response application. The following information is
displayed:
• Repository
• Index agent
• Query response time

How to use a template if more repositories reside on the system

It is possible to have more than one repository on the system. By default, all
templates for monitoring FT index availability and performance monitor only FT
index of the first repository specified in the DMSPI configuration file
(%OvDataDir%\dmspi\conf\dmspi.cfg on Windows or $OvAgentDir/dmspi/
conf/dmspi.cfg on Unix systems).
To monitor the FT index of a specific repository, make a copy of the template and
add --docbase and --template arguments to the program name.
Example:
dmspi_idxsrvr(.exe) --query --docbase=repository_name --
template=my_new_template_name
If the wrong repository is specified, a DMSPI internal error message is sent.

90 User Guide
Chapter 14

SPI for EMC Documentum

Integration with HP Reporter
SPI for EMC Documentum Integration with HP Reporter

Integration Requirements
Before reports can be installed on a system and SPI for EMC Documentum can be
integrated with HP Reporter, check if:
• The HP Reporter is installed on the same system as the HPOM Management
server or it is installed as a standalone system.
• The Reporter Service is running.
• An HPOM Agent is installed on the managed nodes from which you want to
generate reports.

Deploying Templates and Collecting Performance Data

To produce reports, templates must be deployed. All templates related to collecting
performance data have the Perf extension (for example, DMSPI-SrvrProcessPerf).
After a template is successfully deployed, the HP Performance Agent begins to
collect performance data on regular intervals. HP Reporter can be used to generate
reports for all Documentum systems, where HP Performance Agents are collecting
data.

92 User Guide
How HP Reporter Creates Reports
Reporter performs the following steps when producing reports:
“Perform System Discovery” on page 93
“Gather Performance Data” on page 94
“Generate Reports” on page 95

Perform System Discovery

HP Reporter creates Web-based reports from data derived from the targeted systems
that it "discovers". During a system discovery, Reporter looks for systems that are
specified in the Discovery Area, and which have a HP Performance Agent installed.
It then adds those systems to the Discovered Systems group:

Discovered systems are placed in groups for easier access and reporting. Systems can
be assigned to groups as they are discovered as specified in the Automatic Grouping
facility. By default, systems are automatically assigned to groups based on the name
of their operating system. However, you can also manually assign systems to groups.
A system can belong to any number of different groups. You can also create your
own groups and assign systems to them manually using the "drag and drop" action.
The following group is created automatically for SPI for EMC Documentum
purposes:
• SPI for EMC Documentum
There is also auto-grouping defined for this group. Discovered system which
contains a DMSPI Data Source is automatically added to the SPI for EMC
Documentum discovered systems group in Reporter.

SPI for EMC Documentum Integration with HP Reporter 93

IMPORTANT

The auto-grouping feature only works with newly discovered systems. If your
systems have already been discovered by Reporter prior to the Reporter part of
SPI for EMC Documentum installation, they may not have been added to the SPI
for EMC Documentum Reporter group; it may even occur that the group itself has
not been created.
If a SPI for EMC Documentum group was not created automatically, you can create
it manually:
1. In the left pane, right-click Discovered Systems.
2. Select Add Group.
3. In the Add Group dialog box, enter the new group name SPI for EMC
Documentum (note that the group name is case-sensitive) and click Add.
If a specific Documentum system has not been put under the SPI for EMC
Documentum group, you can add it manually.

Gather Performance Data

Once Reporter has run through its discovery, it gathers performance data from each
discovered system and places it in a local database. Additionally, Reporter gathers
data only for those metrics that it knows about. These metrics are specified under
Metrics Lists:

Metric lists control what information is gathered from a system into the Reporter's
database. A metric list groups metrics from a single metric class supplied by the
performance agent for UNIX or Windows. The metric list can also select the degree
of summarization (points every 5 minutes, hour, day, and so on) and how much data
to gather and retain in the database. The shorter the interval, the more records
collected. The default summarization level is one hour. Metric lists are tightly
connected to Data Source and Objects within that Data Source on each system.

94 User Guide
 For a list of metrics lists, refer to “Metric Lists” on page 121.

Generate Reports
Reporter generates HTML reports based on the data available from the local
Reporter database. Additionally, Reporter creates HTML reports from the set of
reports that it knows about; reports which are located in the Reports group:

Available reports are listed in families under the Reports icon. To see the reports,
expand the Reports hierarchy in the Reporter window, and click on a report family
to display its reports in the right pane.
From the right pane, you can delete a report, display its definition details, and edit
the report definition details including the Date Range and Shift Name.
The following report family is created for SPI for EMC Documentum purposes:
• SPI for EMC Documentum

SPI for EMC Documentum Integration with HP Reporter 95

Reports Implemented in SPI for EMC Documentum
Several report categories and related individual reports exist under the SPI for EMC
Documentum report family as specified in the following tables.

Category Report

Documentum process Server process statistics for yesterday

availability Server process statistics for last 28 days
The reports show availability of the Documentum server
processes during the previous day or the last 28 days. The first
parts of the reports are focused on determining top/bottom 5
servers by server process memory usage, CPU utilization, and
availability. The second parts of the reports show in detail
how the resource usage and availability of the server processes
varied in time.
IMPORTANT: The reports take into account only those
Documentum servers that have the following template
deployed (for at least 2 days):
DMSPI-SrvrProcessPerf
Documentum process Connection broker process statistics for yesterday
availability Connection broker process statistics for last 28 days
The reports show availability of the connection broker
processes during the previous day or the last 28 days. The first
parts of the reports are focused on determining top/bottom 5
connection broker processes by memory usage, CPU
utilization, and availability. The second parts of the reports
show in detail how the resource usage and availability of the
connection broker processes varied in time.
IMPORTANT: The reports take into account only those
Documentum connection broker systems that have the
following template deployed (for at least 2 days):
DMSPI-BrokerProcessPerf

96 User Guide
Documentum process Job process statistics for yesterday
availability Job process statistics for last 28 days
The reports show availability of the 'dm_agent_exec' process
during the previous day or the last 28 days. The first parts of
the reports are focused on determining top/bottom 5 servers
by 'dm_agent_exec' process memory usage, CPU utilization,
and availability. The second parts of the reports show in detail
how the resource usage and availability of the
'dm_agent_exec' processes varied in time.
IMPORTANT: The reports take into account only those
Documentum servers that have the following template
deployed (for at least 2 days):
DMSPI-JobProcessPerf
Documentum process Index server process statistics for yesterday
availability Index server process statistics for last 28 days
The reports show availability and resource consumption of the
index server processes during the previous day or last 28 days.
The first parts of the reports are focused on determining top/
bottom index servers by index server process memory usage,
CPU utilization, and availability.
The second parts of the reports show in detail how the
resource usage and availability of the index server processes
varied in time.
IMPORTANT: The reports take into account only those index
servers that have the following template deployed (for at least
2 days):
DMSPI-IdxSrvrProcPerf
Documentum process Index agent process statistics for yesterday
availability Index agent process statistics for last 28 days
The reports show availability and resource consumption of the
index agent processes during the previous day or last 28 days.
The first parts of the reports are focused on determining top/
bottom index agents by index agent process memory usage,
CPU utilization, and availability.
The second parts of the reports show in detail how the
resource usage and availability of the index agent processes
varied in time.
IMPORTANT: The reports take into account only those index
servers that have the following template deployed (for at least
2 days):
DMSPI-IdxAgtProcPerf

SPI for EMC Documentum Integration with HP Reporter 97

 File store statistics File store statistics
This report shows file store distribution and statistics. The first
part of the report focuses on determining top 10 file stores
with largest content and least available space, and the second
part show how much of the total file store space is occupied by
individual file store. Additionally, detailed statistics for each
file store are provided. They show free and used space for the
disk, on which the file store resides. For each file store you can
also see how much used space is occupied by content and how
much by full text index. Use this report to precisely monitor
the file store usage and available disk space.
IMPORTANT: This report takes into account only those
Documentum servers that have the following template
deployed (for at least 2 days):
DMSPI-FilestoresPerf
Database availability Database availability for yesterday
Database availability for last 28 days
The reports show average, minimal, and maximal login time
for Documentum database during the last day or the last 28
days. Login time is measured with the Documentum
dmdbtest application and shows the time needed for the
Documentum server to login to the Documentum database.
IMPORTANT: The reports take into account only those
Documentum servers that have the following template
deployed (for at least 2 days):
DMSPI-DBLoginPerf
User sessions User session statistics for yesterday
User session statistics for last 28 days
The reports present the average session activity on
Documentum servers during the previous day or the last 28
days. Based on their state, sessions are categorized as active or
inactive. Use these reports to identify trends in server session
activity and to compare inter-server session activities.
IMPORTANT: The reports take into account only those
Documentum servers that have the following template
deployed (for at least 2 days):
DMSPI-NumberOfSessionsPerf

98 User Guide
User sessions User session CPU and memory consumption for yesterday
User session CPU and memory consumption for last 28 days
The reports show the load of user sessions on the system
where the Documentum server is installed. The graph shows
average CPU utilization of user sessions during the previous
day or the last 28 days. Additionally, the table shows top 10
users with extensive CPU utilization for each server.
For UNIX servers the reports also show memory usage for
user sessions and a table with top 10 users with extensive
memory usage.
IMPORTANT: The reports take into account only those
Documentum servers that have the following templates
deployed (for at least 2 days):
DMSPI-SessionsPerf
DMSPI-SessionsMemPerf
Client response Client response time for yesterday
Client response time for last 28 days
The reports present average, minimum, and maximum
response time values for standard Documentum user
operations from client systems during the previous day or the
last 28 days for each Documentum repository. The first parts
of the reports are focused on determining top 5 client systems
by each Documentum operation, and the second parts of the
reports show in detail how the response time varied in time.
Use these reports to get insight into how fast the documents
are accessed or stored to the repository.
IMPORTANT: The reports take into account only those
Documentum systems that have the following template
deployed (for at least 2 days):
DMSPI-ClientResponsePerf

SPI for EMC Documentum Integration with HP Reporter 99

 Client response Server availability for yesterday
Server availability for last 28 days
The reports present the availability of Documentum
infrastructure (connection brokers and repositories) from the
client perspective. Availability is tested for clients that connect
trough DMCL. The top graphs show servers that were
unavailable for most of the time. The following graphs show
availability of servers during the previous day or the last 28
days.
IMPORTANT: The reports take into account only those
Documentum systems that have the following template
deployed (for at least 2 days):
DMSPI-ClientResponsePerf
Table size growth Database table size growth for yesterday
Database table size growth for last 28 days
The reports show the table size growth during the previous
day or the last 28 days. Tables are grouped by repositories and
sorted by table names. Reports show the size of a specific table
at the beginning of the reporting period and at the end of the
period and calculate the table growth. They also show
minimum, maximum, and average table size.
IMPORTANT: The reports take into account only those
Documentum systems that have the following templates
deployed (for at least 2 days):
DMSPI-DmiQueueItemSizePerf
DMSPI-DmAudittrailSizePerf
Table size growth Number of events for yesterday
Number of events for the last 28 days
The reports show the number of events for every event that
you are monitoring. Events are grouped by repositories and
sorted by the event name. Report show the number of events
that have occurred in a repository at the beginning of the
period and at the end of the period, and calculates the
difference between them. Additionally, the minimum,
maximum, and average number of events are shown.
IMPORTANT: The reports take into account only those
Documentum systems that have the following templates
deployed (for at least 2 days):
DMSPI-RenditionQueuePerf
DMSPI-LogonFailurePerf

100 User Guide

Jobs Number of failed jobs for yesterday
Number of failed jobs for last 28 days
The reports show the average number of failed jobs per
repository during the previous day or the last 28 days.
IMPORTANT: The reports take into account only those
Documentum servers that have the following template
deployed (for at least 2 days):
DMSPI-JobStatusPerf
Jobs Job execution time for yesterday
Job execution time for last 28 days
The reports show the job execution time for all executed jobs
during the previous day or the last 28 days. The first parts of
the reports show top 5 jobs by job duration. The second parts
of the reports show in details when and how long the jobs
were running and if the jobs were finished successfully.
IMPORTANT: The reports take into account only those
Documentum servers that have the following template
deployed (for at least 2 days):
DMSPI-JobExecTimePerf
Jobs Number of running jobs for yesterday
Number of running jobs for last 28 days
The reports show the number of running jobs per repository
during the previous day or the last 28 days. The first parts of
the reports show repositories with the most running jobs. The
second parts of the reports show in details the number of
running jobs per repository during the time interval.
IMPORTANT: The reports take into account only those
Documentum servers that have the following template
deployed (for at least 2 days):
DMSPI-RunningJobsPerf

SPI for EMC Documentum Integration with HP Reporter 101

 Full-Text Indexing Indexing latency for yesterday
Indexing latency for last 28 days
The reports present indexing latency statistics. Save-to-search
latency is the time needed to merge a new document into the
FT index and to make it searchable. Destroy latency is the time
needed to remove a document from the FT index.
The first parts of the reports are focused on determining top
repositories by indexing latency, and the second parts of the
reports show in detail average, minimum and maximum times
for all three major phases of indexing process: waiting to be
indexed, document preprocessing, and indexing of the content
file and it's properties. Use these report to get insight into how
fast the documents are stored to or removed from FT index.
IMPORTANT: The reports take into account only those
Documentum servers that have the following template
deployed (for at least 2 days):
DMSPI-IndexingPerf
Full-Text Indexing FT query response for yesterday
FT query response for last 28 days
The reports show average, minimal, and maximal FT query
response values for Documentum repositories during the
previous day or the last 28 days. Full text query response
shows if your FT queries come back in a timely fashion.
IMPORTANT: The reports take into account only those
Documentum servers that have the following template
deployed (for at least 2 days):
DMSPI-IndexingPerf
Full-Text Indexing Jobs that were scheduled but failed to start for yesterday
Jobs that were scheduled but failed to start for last 28 days
The reports show list of administrative jobs that were
scheduled but for some reason failed to start for Documentum
repositories during the previous day or the last 28 days.
IMPORTANT: The reports take into account only those
Documentum servers that have the following template
deployed (for at least 2 days):
DMSPI-JobsFailedToStartPerf

102 User Guide

Full-Text Indexing Index events for yesterday
Index events for last 28 days
The reports show the number of events for every indexing
event that you are monitoring. Index events are grouped by
repositories and sorted by the event name. Reports show the
number of Index events that have occurred in a repository at
the beginning of the period and at the end of the period, and
calculate the difference between them. Additionally, the
minimum, maximum, and average number of events is
shown.
IMPORTANT: The reports take into account only those
Documentum servers that have the following templates
deployed (for at least 2 days):
DMSPI-IdxQueueSizePerf
DMSPI-IdxQueueFailedPerf
Full-Text Indexing FT index disk usage
The reports show index server disk usage statistics. The first
parts of the reports are focuses on determining top index
servers with largest disk usage and least available space, and
the second parts show detailed statistics for each index server.
They show free and used space for the disk, on which the FT
index resides. Use these reports to precisely monitor the FT
index usage and available disk space.
IMPORTANT: The reports take into account only those index
servers that have the following template deployed (for at least
2 days):
DMSPI-IdxSrvrDiskUsagePerf
Workflow statistics Unfinished workflows
This report shows a list of unfinished workflows that are older
than the specified time period in the template.
IMPORTANT: This report takes into account only those
Documentum servers that have the following template
deployed (for at least 2 days):
DMSPI-UnfinishedWorkflows

SPI for EMC Documentum Integration with HP Reporter 103

 Threshold assistant Average CPU and MEM consumption of server process for
yesterday
Average CPU and MEM consumption of server process for
last 28 days
The reports provide statistical information that helps adjust
thresholds in server process monitoring templates for an
individual server. For each corresponding monitoring
template, the reports dump out the average and minimum/
maximum values for PRIME and ALL shifts for the individual
server (PRIME shift needs to be defined in HP Reporter).
With these reports you can determine server-specific
threshold levels for server process monitoring templates, fine-
tune them for the individual server or group of servers, and
greatly reduce the number of false alarms that need
administrative attention.
IMPORTANT: The reports take into account only those
Documentum systems that have the following template
deployed (for at least 2 days):
DMSPI-SrvrProcessPerf
Threshold assistant Average CPU and MEM consumption of connection broker
process for yesterday
Average CPU and MEM consumption of connection broker
process for last 28 days
The reports provide statistical information that helps adjust
thresholds in connection broker process monitoring templates
for an individual Documentum connection broker. For each
corresponding monitoring template, the reports dump out the
average and minimum/maximum values for PRIME and ALL
shifts for the individual connection broker (PRIME shift needs
to be defined in HP Reporter).
With these reports you can determine connection broker-
specific threshold levels for connection broker process
monitoring templates, fine-tune them for the individual
connection broker or group of connection brokers, and greatly
reduce the number of false alarms that need administrative
attention.
IMPORTANT: These reports take into account only those
Documentum connection broker systems, that have the
following template deployed (for at least 2 days):
DMSPI-BrokerProcessPerf

104 User Guide

Threshold assistant Average CPU and MEM consumption of job process for
yesterday
Average CPU and MEM consumption of job process for last
28 days
The reports provide statistical information that helps adjust
thresholds in job process monitoring templates for an
individual server. For each corresponding monitoring
template, the reports dump out the average and minimum/
maximum values for PRIME and ALL shifts for the individual
job process (PRIME shift needs to be defined in HP Reporter).
With these reports it is possible to determine server-specific
threshold levels for job process monitoring templates, fine-
tune them for the individual server or group of servers, and
greatly reduce the number of false alarms that need
administrative attention.
IMPORTANT: The reports take into account only those
Documentum systems that have the following template
deployed (for at least 2 days):
DMSPI-JobProcessPerf
Threshold assistant Average database login time for yesterday
Average database login time for last 28 days
The reports provide statistical information that helps adjust
thresholds in database login time monitoring templates for an
individual server. For each corresponding monitoring
template, the reports dump out the average and minimum/
maximum values for PRIME and ALL shifts for the individual
Documentum server (PRIME shift needs to be defined in HP
Reporter).
With these reports it is possible to determine server-specific
threshold levels for database login time monitoring templates,
fine-tune them for the individual server or group of servers,
and greatly reduce the number of false alarms that need
administrative attention.
IMPORTANT: These reports take into account only those
Documentum servers that have the following template
deployed (for at least 2 days):
DMSPI-DBLoginPerf

SPI for EMC Documentum Integration with HP Reporter 105

 Threshold assistant Average CPU and MEM consumption of user sessions for
yesterday
Average CPU and MEM consumption of user sessions for
last 28 days
The reports provide statistical information that helps adjust
thresholds in session monitoring templates for an individual
server. For each corresponding monitoring template, the
reports dump out the average and minimum/maximum
values for PRIME and ALL shifts for the individual server
(PRIME shift needs to be defined in HP Reporter).
With these reports it is possible to determine server-specific
threshold levels for session monitoring templates, fine-tune
them for the individual server or group of servers, and greatly
reduce the number of false alarms that need administrative
attention.
IMPORTANT: The reports take into account only those
Documentum systems that have the following templates
deployed (for at least 2 days):
DMSPI-SessionsPerf
DMSPI-SessionsMemPerf
Threshold assistant Average client response time for yesterday
Average client response time for last 28 days
The reports provide statistical information that helps adjust
thresholds in client response monitoring templates for an
individual server. For each corresponding monitoring
template, the reports dump out the average and minimum/
maximum values for PRIME and ALL shifts for the individual
Documentum servers (PRIME shift needs to be defined in HP
Reporter).
With these reports it is possible to determine repository-
specific threshold levels for client response monitoring
templates, fine-tune them for the individual Documentum
server or group of servers, and greatly reduce the number of
false alarms that need administrative attention.
IMPORTANT: The reports take into account only those
Documentum systems that have the following template
deployed (for at least 2 days):
DMSPI-ClientResponsePerf

106 User Guide

Threshold assistant Average table size growth for yesterday
Average table size growth for last 28 days
The reports provide statistical information that helps adjust
thresholds in table size monitoring templates for an individual
repository. For each corresponding monitoring template, the
reports dump out the average and minimum/maximum
values for PRIME and ALL shifts for the individual repository
(PRIME shift needs to be defined in HP Reporter).
With these reports it is possible to determine repository-
specific threshold levels for table size monitoring templates,
fine-tune them for the individual repository, and greatly
reduce the number of false alarms that need administrative
attention.
IMPORTANT: The reports take into account only those
Documentum systems that have the following templates
deployed (for at least 2 days):
DMSPI-DmiQueueItemSizePerf
DMSPI-DmAudittrailSizePerf
Threshold assistant Average number of events for yesterday
Average number of events for last 28 days
The reports provide statistical information that helps adjust
thresholds in event monitoring templates for an individual
repository. For each corresponding monitoring template, the
reports dump out the average and minimum/maximum
values for PRIME and ALL shifts for the individual repository
(PRIME shift needs to be defined in HP Reporter).
With these reports it is possible to determine repository-
specific threshold levels for events monitoring templates, fine-
tune it for the individual repository, and greatly reduce the
number of false alarms that need administrative attention.
IMPORTANT: These reports take into account only those
Documentum systems that have the following templates
deployed (for at least 2 days):
DMSPI-RenditionQueuePerf
DMSPI-LogonFailurePerf

SPI for EMC Documentum Integration with HP Reporter 107

 Threshold assistant Average index queue size growth for yesterday
Average index queue size growth for last 28 days
These reports provide statistical information that helps adjust
thresholds in index queue monitoring templates for an
individual repository. For each corresponding monitoring
template, the reports dump out the average and minimum/
maximum values for PRIME and ALL shifts for the individual
repository (PRIME shift needs to be defined in HP OV
Reporter).
With these reports it is possible to determine repository-
specific threshold levels for index queue monitoring
templates, fine-tune them for the individual repository, and
greatly reduce the number of false alarms that need
administrative attention.
IMPORTANT: The reports take into account only those
Documentum systems that have the following templates
deployed (for at least 2 days):
DMSPI-IdxQueueSizePerf
DMSPI-IdxQueueFailedPerf
Threshold assistant Average indexing latency and FT query response for
yesterday
Average indexing latency and FT query response for last 28
days
These reports provide statistical information that helps adjust
thresholds in indexing performance monitoring templates for
an individual repository/index agent. For each corresponding
monitoring template, the reports dump out the average and
minimum/maximum values for PRIME and ALL shifts for the
individual repository (PRIME shift needs to be defined in HP
OV Reporter).
With these report it is possible to determine server-specific
threshold levels for indexing performance monitoring
templates, fine-tune them for the individual repository, and
greatly reduce the number of false alarms that need
administrative attention.
IMPORTANT: The reports take into account only those
Documentum systems that have the following templates
deployed (for at least 2 days):
DMSPI-IndexingLatency
DMSPI-FTQueryResponse

108 User Guide

Chapter 15

Analyzing Historical Data Using HP

Performance Manager
Analyzing Historical Data Using HP Performance Manager
HP Performance Manager (HPPM) provides a central point from where you can
monitor and manage performance of all networked systems in your environment.
Using HP Performance Manager you can analyze historical data from HP
Performance Agent systems, receive alarms generated by HP Performance Agent,
and predict future resource usage. HP Performance Manager also allows you to
perform the following functions:
• Select a data source and list the graphs associated with it
• Choose a graph to view, select how the graph will display, and change the
metrics graphed
• Draw graphs
• Drill down to view detail over a period of time
• Export and import systems and graph policies
• Design graphs and save them as policies
• Receive and view alarms
• Create forecasts
For additional information on HP Performance Manager, refer to the HP
Performance Manager documentation.

HP Performance Manager User Defined Graph Templates

After installation, performance graphs are added on the management server to the
following text files:
/etc/opt/OV/share/dmspi/graph/VPI_GraphsSPI for EMC Documentum.txt
/etc/opt/OV/share/dmspi/graph/VPI_GraphsSPI for EMC Documentum-OVPA-
RPC.txt
Copy the file into your Performance Manager's data directory to fully integrate
them. Note that the available graphs are listed under the family SPI for EMC
Documentum.

110 User Guide

 The following table describes SPI for EMC Documentum graphs.

Category Name Description

Client response Client response time Shows the average client response
time for standard user operations
Documentum availability Shows the availability of the
Documentum from the client
perspective
Connection broker Connection broker process Shows the availability of the
process availability connection broker process
Connection broker process Shows the average memory usage
memory usage of the connection broker process
Connection broker process CPU Shows the average CPU utilization
utilization of the connection broker process
Content Server Content Server process availability Shows the availability of the
process Content Server process
Content Server process memory Shows the average memory usage
usage of the Content Server process
Content Server process CPU Shows the average CPU utilization
utilization of the Content Server process
Database Average DB Login time Shows the average login time for
availability Documentum database
File store statistics Distribution of used space Shows the file store distribution for
the repository
Filestore usage Shows the file store usage and
available disk space
Full-Text Indexing Destroy latency Shows the destroy latency statistics
- the time needed to remove a
document from the FT index.
FT query response Shows the FT query response
values for Documentum
repositories
Index agent process Index agent process availability Shows the availability of the index
agent process
Index agent process memory usage Shows the average memory usage
of the index agent process

Analyzing Historical Data Using HP Performance Manager 111

 Index agent process CPU Shows the average CPU utilization
utilization of the index agent process
Index server process Index server process availability Shows the availability of the index
server process
Index server process memory Shows the average memory usage
usage of the index server process
Index server process CPU Shows the average CPU utilization
utilization of the index server process
Job process Job process availability Shows the availability of the
dm_agent_exec process

Job process average memory usage Shows the average memory usage
of the dm_agent_exec process
Job process average CPU Shows the average CPU utilization
utilization of the dm_agent_exec process
Jobs Average number of failed jobs Shows the average number of
failed jobs per repository
Table size growth Average database table size Shows the average size of a specific
database table
Average number of events Shows the number of events that
have occurred in a repository
User sessions activity Shows the average session activity
on Documentum servers
User session CPU utilization Shows the average CPU utilization
of user sessions
Index Server Disk Usage Show the index server disk usage
statistics
Save-to-search latency Shows the save-to-search latency
statistics - the time needed to
merge a new document into the FT
index and to make it searchable.

Sample Graphs
Below are some examples of the graphs that can be produced.

NOTE

All graphs will have the format similar to that shown in the sample graphs here.

112 User Guide

The following graph displays average client response times.

Analyzing Historical Data Using HP Performance Manager 113

 The following graph displays average database login time.

114 User Guide

The following graph displays file store statistics.

Analyzing Historical Data Using HP Performance Manager 115

116 User Guide
Chapter 16

Troubleshooting
Troubleshooting Assistance
This chapter describes possible errors that can occur while using SPI for EMC
Documentum and gives instructions on how to solve them.

Documentum SPI Service Stops Responding

Documentum SPI service can stop responding after some time. The reason for this
problem is described on Documentum support web site:
http://softwaresupport.emc.com/support/bugs/dmfrmdefectview.asp?id=95130
To solve the problem, install SP4 if you are using Documentum 5.2.5 or SP1 if you
are using Documentum 5.3.

Configuration File (dmspi.cfg) Is Overwritten if Redeployment on

the Node Is Performed
To prevent automatic updates of the SPI for EMC Documentum configuration, set
the manual_configuration field under the DMSPI tab in the configuration file to true.
Note that the default value is false.
Example:
[DMSPI]
manual_configuration=true

Autodiscovery on DCE Agents Fails

Autodiscovery automatic action fails to execute on the management server.
The problem is that the opcdeploy binary does not transfer the generated service tree
definition file from node to management server correctly.
To solve this do the following:
• manually transfer the dmspi_svctree.xml file located in
$OvAgentDir/dmspi/bin to the management server folder /opt/OV/
dmspi/bin and
• upload the service tree definition file by executing
opcservice -replace /opt/OV/dmspi/bin/dmspi_svctree.xml

118 User Guide

 Once the service definition is uploaded, you can assign the service view by executing
the following command:
opcservice -assign {operator} dmspi_root

Autodiscovery Fails if There Are no Application Groups in a

Service Tree
You may get the following error:
# /opt/OV/dmspi/bin/dmspi_updsvc.sh eximus.hermes.si "/opt/OV/dmspi/
bin/dmspi_svctree.xml"
Error: Service definition file is not ok.
Target element SVCDISC:Applications referenced in association does not exist
(SVC10-122)
Warning: Check operation found 1 problem(s) in source file:
/opt/OV/dmspi/bin/dmspi_svctree.xml'. (SVC10-167) (SVC10- 123)
Add an application group to the services root or perform the following to append
Documentum service tree directly to the services root:
1. Delete this code from XML on the node:
<Association>
<Composition/>
<TargetRef>SVCDISC:Applications</TargetRef>
<SourceRef>dmspi_root</SourceRef>
</Association>
2. Run the automatic command on the Management server manually:
/opt/OV/dmspi/bin/dmspi_updsvc.sh {node_name} "/opt/OV/dmspi/bin/
dmspi_svctree.xml"
3. Assign the service to the suitable operator:
opcservice -assign opc_adm dmspi_root

Troubleshooting 119
120 User Guide
Appendix A

Metric Lists
Metric Lists
The following metric lists are created for SPI for EMC Documentum reports on
Documentum processes:
• DMSPI_SRVR_PROCESS
• DMSPI_DOCBROKER_PROC
• DMSPI_JOB_PROCESS
Points in metric lists are summarized with an 1-hour interval and retained for 30
days in the Reporter database.

DMSPI_SRVR_PROCESS metric list

Metric name Description

SP_DOCBASE_NAME Repository name

SP_SERVER_NAME Server name

SP_PROCESS_STATE Server process status

SP_MEMORY_USAGE Server process memory usage (KB)

SP_VMEMORY_USAGE Server process virtual memory usage (KB)

SP_CPU_UTIL Server process CPU utilization (%)

DMSPI_DOCBROKER_PROC metric list

Metric name Description

DP_DOCBROKER Connection broker process name and port
number

DP_PROCESS_STATE Connection broker process status

DP_MEMORY_USAGE Connection broker process memory usage (KB)

DP_VMEMORY_USAGE Connection broker process virtual memory usage

(KB)

DP_CPU_UTIL Connection broker process CPU utilization (%)

122 User Guide

DMSPI_JOB_PROCESS metric list

Metric name Description

JP_DOCBASE_NAME Repository name

JP_SERVER_NAME Server name

JP_PROCESS_STATE dm_agent_exec process status

JP_MEMORY_USAGE dm_agent_exec process memory usage (KB)

JP_VMEMORY_USAGE dm_agent_exec process virtual memory usage

(KB)

JP_CPU_UTIL dm_agent_exec process CPU utilization (%)

The following metric lists are created for SPI for EMC Documentum reports on
Documentum server sessions:
• DMSPI_SESSIONS
• DMSPI_SESS_UTIL
Points in metric lists are summarized with an 1-hour interval and retained for 30
days in the Reporter database.

DMSPI_SESSIONS metric list

Metric name Description

SE_DOCBASE_NAME Repository name

SE_SERVER_NAME Server name

SE_ACTIVE_SESSIONS Number of active sessions

SE_INACTIVE_SESSIONS Number of inactive sessions

SE_MAX_CONCURRENT Maximum number of concurrent sessions

DMSPI_SESS_UTIL metric list

Metric name Description

SU_SESSION_ID Session ID

SU_USER_NAME User name

SU_HOST_NAME System name

SU_SERVER_NAME Server name

Metric Lists 123

 SU_DOCBASE_NAME Repository name

SU_CPU_UTIL Session CPU utilization (%)

SU_MEM_UTIL Session memory usage in KB (for Unix systems
only)

The following metric lists are created for SPI for EMC Documentum reports on
Documentum database availability:
• DMSPI_DB_LOGIN
Points in metric lists are summarized with an 1-hour interval and retained for 30
days in the Reporter database.

DMSPI_DB_LOGIN metric list

Metric name Description

DL_SERVER_NAME Server name

DL_DOCBASE_NAME Repository name

DL_LOGIN_TIME Database login time (ms)

The following metric lists are created for SPI for EMC Documentum reports on
Documentum file stores:
• DMSPI_FILESTORES
Points in metric lists are summarized with an 1-hour interval and retained for 30
days in the Reporter database.

DMSPI_FILESTORES metric list

Metric name Description

FS_DOCBASE_NAME Repository name

FS_SERVER_NAME Server name

FS_FILESTORE_NAME File store name

FS_DISTRIBUTED_STORE Name of the distributed store

FS_AVAILABLE_SPACE Available space for the file store (KB)

FS_FS_CAPACITY Total size of the disk on which file store resides

(KB)

124 User Guide

 FS_USED_SPACE File store used space (KB)

FS_FULL_TEXT_PERC Percentage of used space occupied by full text

index

The following metric lists are created for SPI for EMC Documentum reports on
Documentum client response:
• DMSPI_CLIENT_RESP
• DMSPI_CLIENT_AVAIL
Points in metric lists are summarized with an 1-hour interval and retained for 30
days in the Reporter database.

DMSPI_CLIENT_RESP metric list

Metric name Description

CR_DOCBASE_NAME Repository name

CR_SERVER_NAME Server name

CR_USER_NAME User name

CR_CONNECT_TIME Connect time (ms)

CR_IMPORT_TIME Import time (ms)

CR_CHECKOUT_TIME CheckOut time (ms)

CR_CHECKIN_TIME CheckIn time (ms)

CR_DELETE_TIME Delete time (ms)

CR_DISCONNECT_TIME Disconnect time (ms)

DMSPI_CLIENT_AVAIL metric list

Metric name Description

CA_DOCBASE_NAME Repository name

CA_SERVER_NAME Server name

CA_USER_NAME User name

CA_AVAILABILITY Repository status (available/not available)

The following metric lists are created for SPI for EMC Documentum reports on
Documentum jobs:

Metric Lists 125

 • DMSPI_JOBS
• DMSPI_JOBS_EXECTIME
• DMSPI_JOBS_RUNNING
Points in metric lists are summarized with an 1-hour interval and retained for 30
days in the Reporter database.

DMSPI_JOBS metric list

Metric name Description

JB_DOCBASE_NAME Repository name

JB_FAILED Number of failed jobs

DMSPI_JOBS_EXECTIME metric list

Metric name Description

JE_DOCBASE_NAME Repository name

JE_TARGET_SERVER Target server

JE_JOB_NAME Job name

JE_JOB_START Job start time in number of seconds since 01/01/

1970

JE_JOB_STOP Job stop time in number of seconds since 01/01/

1970

JE_JOB_EXIT_STATUS Job exit status

JE_JOB_STATUS_MSG Job status message

JE_JOB_OWNER Job owner

DMSPI_JOBS_RUNNING metric list

Metric name Description

JR_DOCBASE_NAME Repository name

JR_TARGET_SERVER Target server

JR_JOBS_RUNNING The number of jobs that are running at the

moment

126 User Guide

DMSPI_JOBS_NO_START metric list

Metric name Description

IN_OBJECT_NAME Job name

IN_TARGET_SERVER Target server

IN_NEXT_INVOCATION Expected execution of the job

The following metric lists are created for SPI for EMC Documentum reports on
Documentum database table size growth:
• DMSPI_TABLE_SIZE
• DMSPI_EVENT_SIZE
Points in metric lists are summarized with an 1-hour interval and retained for 30
days in the Reporter database.

DMSPI_TABLE_SIZE metric list

Metric name Description

TS_DOCBASE_NAME Repository name

TS_TABLE_NAME Table name

TS_TABLE_SIZE Table size

DMSPI_EVENT_SIZE metric list

Metric name Description

ES_DOCBASE_NAME Repository name

ES_TABLE_NAME Table name

ES_EVENT_NAME Event name

ES_NUMBER_OF_EVENTS Number of events

The following metric list is created for SPI for EMC Documentum reports on
Documentum documents and workflows:
• DMSPI_U_WORKFLOWS
Points in metric lists are not summarized and retained for 30 days in the Reporter
database.

Metric Lists 127

 DMSPI_U_WORKFLOWS metric list

Metric name Description

UW_DOCBASE_NAME Repository name

UW_WORKFLOW_ID Workflow ID

UW_WORKFLOW_NAME Workflow name

UW_RUNNING_STATE Workflow state

UW_START_DATE Workflow start date

UW_SUPERVISOR_NAME Supervisor name

UW_WORKFLOW_AGE Workflow age

The following metric lists are created for SPI for EMC Documentum reports on index
server:
• DMSPI_IDXSRVR_PROC
• DMSPI_IDXAGT_PROC
• DMSPI_IDX_PERF
• DMSPI_IDXSRVR_DU
Points in metric lists are summarized with an 1-hour interval and retained for 30
days in the Reporter database.

DMSPI_IDXSRVR_PROC metric list

Metric name Description

IS_PROCESS_LABEL Process label

IS_PROCESS_NAME Process name

IS_PROCESS_DESC Process description

IS_PROCESS_STATE Process status

IS_MEMORY_USAGE Process memory usage (KB)

IS_VMEMORY_USAGE Process virtual memory usage (KB)

IS_CPU_UTILProcess CPU utilization (%)

128 User Guide

DMSPI_IDXAGT_PROC metric list

Metric name Description

IA_PROCESS_LABEL Process label

IA_PROCESS_NAME Process name

IA_DOCBASE_NAME Repository name

IA_PROCESS_STATE Process status

IA_MEMORY_USAGE Process memory usage (KB)

IA_VMEMORY_USAGE Process virtual memory usage (KB)

IA_CPU_UTIL Process CPU utilization (%)

DMSPI_IDX_PERF metric list

Metric name Description

IP_AGENT_NAME Index Agent name

IP_DOCBASE_NAME Repository name

IP_CREATE_AWAITING Awaiting time when adding new document to the
FT index (ms)

IP_CREATE_PREPROC Preprocessing time when adding new document

to the FT index (ms)

IP_CREATE_INDEXING Indexing time when adding new document to the

FT index (ms)

IP_DESTROY_AWAITING Awaiting time when removing a document from

the FT index (ms)

IP_DESTROY_PREPROC Preprocessing time when removing a document

from the FT index (ms)

IP_DESTROY_INDEXING Time to remove a document from the FT index

(ms)

IP_QUERY_TIME FT query response time (ms)

DMSPI_IDXSRVR_DU metric list

Metric name Description

IS_DU_NODE Index server node name

IS_DU_LOCATION Full-text index location

Metric Lists 129

 IS_DU_USAGE_PERCENT Disk usage (%)

IS_DU_USAGE Disk usage (GB)

IS_DU_SIZE Disk size (GB)

IS_DU_FREE Disk free size (GB)

130 User Guide