Advanced Configuration Training 2007

Advanced Configuration
Training
Additional Material

Enginuity & E-Packs, Dynamic Drive

Database, Introduction to Scripts for CS,
Primus, Powerlink, E-Lab Navigator and
Channel Express.

EMC CONFIDENTIAL—INTERNAL USE ONLY 1

1
Advanced Configuration Training 2007

Enginuity & E-Packs

Enginuity Information, Code Fixes,

SymmWin Patches and the E-Pack
Database.

EMC CONFIDENTIAL—INTERNAL USE ONLY 2

2
 Advanced Configuration Training 2007

Enginuity & E-Packs

Testing Your Knowledge

What is the difference between an Enginuity “Maintenance Release”
and an Enginuity “Service Release”?

What Enginuity code level should I upgrade to?

What is the “Enginuity Matrix” used for?

What is the difference between an Enginuity “fix” and a SymmWin
“patch”?

What is an “Enginuity Pack”?

How do I request an “E-Pack”?

What is a “GS Serviceability Patch”?

How can I tell if a “fix” is available in the current GA code?

How can I tell if a SymmWin “patch” is currently installed?

EMC CONFIDENTIAL—INTERNAL USE ONLY 3

Instructors: The above are rhetorical questions. Basically these are the questions
that should be answered by the following presentation.

3
 Advanced Configuration Training 2007

Types of Enginuity Releases

Introduction

Microcode or “Enginuity” Releases are either in the form of
either Service Releases (SR), Maintenance Releases (MR)
or Service Packs (E-Pack).

In the case of a Service Release the code is “frozen” 6
weeks prior to GA date for testing. It will only be released
once the testing has been successfully completed.

A Maintenance Release may be distributed between
Service Releases if a number of critical fixes have been
identified and need to be urgently released. The
Maintenance Release is not fully regression tested.

Service Packs or E-Packs are site specific fixes that contain
fewer fixes than a Maintenance Releases, again these do
not go through full regression testing.
EMC CONFIDENTIAL—INTERNAL USE ONLY 4

As above.

4
 Advanced Configuration Training 2007

Types of Enginuity Releases

Summary

Service Releases:
– Scheduled for release 4 times per year.
– Include major & minor “bug” fixes, plus new functionality.
– Are fully performance, regression and “Serviceability” tested.
– The code is “locked down” and tested ~6 weeks before release.
– Exhaustive testing must be completed in accordance with
documented serviceability test plans before the code can go GA.

Maintenance Release:
– Contains a number of critical fixes (no new features). Generally only
affects limited numbers of customers. Similar to E-Pack but
typically contains multiple fixes.
– Some regression testing (not as much as a Service Release).
– Release Notes are available (same as for Service Release).
EMC CONFIDENTIAL—INTERNAL USE ONLY 5

Again Enginuity Releases described as “Service Releases” typically contain new

features & functionality (as well as “bug” fixes). These are fully regression tested
versions of code released with extensive Release Notes on a scheduled quarterly
basis. “Maintenance Release” are unscheduled interim releases of code to address
critical “issues” - these “bug fixes” may affect or potentially affect significant number
of customers. In a situation like this a “Maintenance Release” is considered a better
option than an E-Pack (see next slide). A Maintenance Release DOES receive
limited regression testing (this is much less than a Service Release but MORE than
just the “focus” testing received by E-Packs (“focus testing” just confirms that the
included “fix” addresses the stated bug). A Maintenance Release is provided with
Release Notes and is an official GA release of code (with it own unique
XXXX.YY.ZZ release number for CCA & SYR tracking purposes).
To determine if a release is a Service Release or a Maintenance Release go to the
Enginuity Release Notes. Look for the statement “There are no new features and
changes implemented by this Enginuity release”. Maintenance Releases rarely
contain enhancements and typically only a few fixes. The Enginuity Release Notes
of a full Service Release will certainly contain numerous enhancements as well as
bug fixes.
A quick indicator of an Maintenance Release is a small XXXX.YY.ZZ increment from
earlier Enginuity releases…

5
 Advanced Configuration Training 2007

Types of Enginuity Releases

Summary (cont)

Enginuity Service Packs (or Enginuity Packs / E-Packs):
– Can be provided more quickly than a Maintenance Release.
– Contain “site specific” Enginuity “fixes”.
– These fixes are added to the specified GA release.
– “E-Packs” receive only “focus testing” on the specific fixes (i.e.
much less testing than a Maintenance or Service Release).
– Availability carefully controlled via E-Pack Database.
– Available from Engineering upon approved request.
– Some E-Pack requests are so “common” that the updated code is
made “Generally Available” from the SSR Web Site (NO additional
Engineering approval required - only “standard” CCA5 approvals).
– No Release Notes (refer to Primus for the “fix” details).

EMC CONFIDENTIAL—INTERNAL USE ONLY 6

Enginuity Packs are “special builds” of GA Enginuity code that have been compiled
with specifically requested “bug” fixes (necessary to address specific issues at
specific sites). E-Packs must be requested on a “per Symmetrix / per Site” basis via
the “Enginuity Pack Database” page and they are then approved by SSR Microcode
Support & EMC Engineering. Note that NOT all requested E-Packs will be approved
/ supplied.
If numerous customers are affected and numerous requests are received (or
anticipated) then the E-Pack may be placed on the SSR Enginuity Web for “general
download” (i.e. E-Pack requests for the specific fixes will no longer be required).
Note that E-Packs ONLY receive “focus testing” – only to confirm that the specific
bug or bugs for which the E-Pack was requested are fixed. Also Release Notes are
NOT provided, refer the Primus Solutions associated with the “bug” fix numbers.
NOTE that is an E-Pack corrected “issue” or “issues” is significant enough the “E-
Pack” may be re-released as a Maintenance Release (as noted on the last slide this
makes tracking the implementations a lot easier).

6
 Advanced Configuration Training 2007

Enginuity Releases
What is “Target” Code?

EMC Microcode Engineering has certain criterion that it
employs when deciding if / when an Enginuity level should
become “target”.
– For example, an Enginuity code becomes a candidate for “target”
when it has been installed on at least 500 Symmetrix systems and
has been GA for at least 30 days.
– It must have no more than five* “Severity 1” OPTs opened against it
AND it must successfully complete at least 10 of the 10 most
common configuration change types (and it must not be blocked by
the Regulator). It can be either a Service or a Maintenance Release.

Note that the final decision for making a code “target”
belongs to EMC Engineering (the Enginuity Product
Management Team) and NOT SSR.

EMC CONFIDENTIAL—INTERNAL USE ONLY 7

*This OPT count was three in earlier documents and it have been raised to five.
However, if the other criterion are met the Enginuity PMT will consider the impact of
the open OPTs before deciding if the code can still be made target.
Refer to the
http://www.cs.isus.emc.com/config/Enginuity/SymmetrixEnginuityImplementationPro
cess.pdf document (there is a link from the “Microcode \ Enginuity Home” page).

7
 Advanced Configuration Training 2007

Enginuity Releases
What Enginuity Level Should I Upgrade to?

SSR has ALWAYS recommend that you should upgrade to
“target” code (or higher). Refer to the SSR Web.
– This has been a standing CS Management decision reinforced by
CCA Approvals “Mgt approval is required for all changes not at
target code. A journal entry is required as proof of approval”.
– Note all the listed criterion BEFORE a code can become “target”.
There is no longer any need to wait and “let the code settle in”.
– If there is a Service Release above the target code consider
upgrading to this higher Service Release (since it has been fully
regression tested). This way you will benefit from any additional
“fixes”. Otherwise you can wait until this SR becomes “target”.
– If there is a Maintenance Release above the target release then
read the Release Notes / Errata Sheets to see if the code contains
fixes relevant to your site. If it does then upgrade to this MR.
EMC CONFIDENTIAL—INTERNAL USE ONLY 8

Refer to the
http://www.cs.isus.emc.com/config/Enginuity/TargetEnginuity/Target.htm page for
details (again there is a link from the “Microcode \ Enginuity Home” page).

8
 Advanced Configuration Training 2007

Enginuity Information
Microcode \ Enginuity Home Page

The http://www.cs.isus.emc.com/config/Enginuity/enginuity.htm page
has links to all things code related e.g. FTP Download Sites, SCSI Code
Information Page, Errata Sheets, Enginuity Release Notes, the Enginuity
Matrix, E-Pack Database, Regulator scripts, Fire Drill Information, etc.

EMC CONFIDENTIAL—INTERNAL USE ONLY 9

As above.

9
 Advanced Configuration Training 2007

Enginuity Information
Enginuity Release Notes

The Enginuity Release Notes are available from Powerlink.
– For example, go to “Home > Resources/Tools > CS Support >
Documentation/White Paper Library > Hardware/Platforms > Symmetrix
DMX Family > Symmetrix DMX-3 > Release Notes”.

Or go to the SSR Web and follow the “Enginuity Links”.

EMC CONFIDENTIAL—INTERNAL USE ONLY 10

As above.

10
 Advanced Configuration Training 2007

Enginuity Information
Enginuity Errata Sheets

The Enginuity Errata Sheets for each code release are available from
the SSR Web. Again follow the “Enginuity Links”.

EMC CONFIDENTIAL—INTERNAL USE ONLY 11

As above.

11
 Advanced Configuration Training 2007

Enginuity Information
Enginuity Errata Sheets (cont)

The Errata Sheets are available in Excel format. Note that they list
individual microcode features and state whether they are “Supported” or
“Not Supported” AND the “Impact to Customers”. This information (in an
abridged format) is also in the Limitations section of the Release Notes.

EMC CONFIDENTIAL—INTERNAL USE ONLY 12

Note that the “Limitations” section of the Enginuity Release Notes will have a
“Number”. The same reference number may also be present in the corresponding
Errata Sheet under the “RN” column. The Excel document typically has MORE
information, OPT#’s, Workarounds, Primus Solution numbers (if available), etc.

12
 Advanced Configuration Training 2007

Enginuity Information
Change Control Reference Sheet

The displayed information can be sorted by the various columns Online
(whether online or offline), OS (CKD or FBA or both), Notes (the type of
change), Primus (solution number), and date Listed.

EMC CONFIDENTIAL—INTERNAL USE ONLY 13

We don’t discuss the CCREF Page here in Advanced Configuration Training. BUT
please remind that the displayed information can be sorted by the various columns
Online (whether online or offline), OS (CKD or FBA or both), Notes (the type of
change), Primus (solution number), and date Listed.

13
 Advanced Configuration Training 2007

Enginuity Information
Enginuity Matrix

The Enginuity Matrix lists the “vital statistics”

for your specified Enginuity Release.

However, its main function is to detail

whether hardware replacements are
supported at this code level AND what
conditions (Primus Solutions) will apply.
EMC CONFIDENTIAL—INTERNAL USE ONLY 14

As above.

14
 Advanced Configuration Training 2007

Enginuity Packs
Testing Your Knowledge

Question: What is Enginuity “code”?

Answer: The Enginuity “code” loaded onto the Symmetrix Service
Processor is composed of TWO parts - the actual Symmetrix director
emulation microcode (i.e. the *.bin files in the C:\SymmWin UCODE
directory and shown on the E7,D,A display) AND the SymmWin
program or “application” (which is our interface into the System & bin
files and ALL the maintenance & configuration change scripts).

Compare this
to an E7,D,A
display from
the same box.
What do you
notice about
the files in the
These emulation files are in EEPROM &
the Service Processor C:\ RUNNING
SymmWin UCODE directory. CODE?

EMC CONFIDENTIAL—INTERNAL USE ONLY 15

The Enginuity “code” - for the purpose of discussing “Enginuity Packs” – is made up
of two distinct parts.
The ACTUAL emulation code loaded to the SP then into the directors by a “hot code
load” AND the SymmWin program OR application that runs on the SP. We discuss
SymmWin in later slides…
As above. We are simply demonstrating here that the director emulation binary files
or emulation code (found under the SymmWin UCODE directory) on the Service
Processor ARE the same files loaded into the actual EEPROM & Running Code on
the Symmetrix – this is the purpose / result of the “Hot Code Load” script. In the
above example we have no “T” for temporary fixes (no “+” sighs on the E7,D,A
(althoughthis can be

15
 Advanced Configuration Training 2007

Enginuity Packs
What is a Enginuity “Temporary” Fix?

Question: So what is an Enginuity code “Temporary” fix?

Answer: The “bug” fixes we typically discuss are “Temporary fixes”
applied to the director emulation code.

“Temporary” means that the compiled emulation code has been
modified by the addition of a NEW fix or by a revision change to an
existing fix. The updated emulation code is loaded into the Symmetrix
directors via an online (hot) code load. Eventually the designation of this
fix or revised fix will become “Permanent” i.e. included in a subsequent
GA code release (either a Maintenance or Service Release).

They are commonly denoted as “Fix # XXXXX” in the Enginuity Release
Notes, in Primus Solutions, in OPT, etc.

ALL Permanent and Temporary fix numbers are also displayed on the
E7,SYRN display AND in the C:\ SymmWin UCODE directory
REVISION.LOG file on the Service Processor (see next slide).

EMC CONFIDENTIAL—INTERNAL USE ONLY 16

Again note the 100% consistency between what is loaded from the SP to the
Symmetrix during the “Online (Hot) Code Load” process.

16
 Advanced Configuration Training 2007

Enginuity Packs
What is a Enginuity “Temporary” Fix? (cont)
Here is a sample E7,SYRN output at 5671.60.65. Here is a sample REVISION.LOG
file from the SP at 5671.60.65.

Note that E7,SYRN,,,’T’ is used to display

“Temporary” Enginuity code fixes only.
In this example there are NO
“Temporary fixes”. Note the “P” for
“Permanent” – indicating that ALL of
the listed “fixes” are part of the GA
code AND note the associated fix
revision numbers “-v00x”.

EMC CONFIDENTIAL—INTERNAL USE ONLY 17

As above.

17
 Advanced Configuration Training 2007

Enginuity Packs
What is a SymmWin “Temporary” Patch?

Question: So what is a “temporary” SymmWin “fix” or “patch”?

Answer: A temporary SymmWin “patch” or “fix” is a modification to the
SymmWin “application”. This may take the form of an updated
SymmWin library or *.smp file and or an updated symmwin.exe file.

These “patches” change SymmWin functionality - either when building
or editing the bin file (e.g. adding the SPC-2 flag to 5670) OR more
commonly the patch changes the operation of SymmWin maintenance
or configuration change scripts (e.g. correcting Permanent Sparing at
5671). These patches are commonly denoted as “Fix # 2000xxxx” in the
Enginuity Release Notes, in Primus Solutions, in OPT, etc.

NOTE that SymmWin fixes or patches are NEVER loaded to the
Symmetrix directors. They are ONLY visible by displaying the SymmWin
Help “Show me the makeup of this build” (at 5x7x). See next slide.

Again a “Temporary” fix will become “Permanent” in a future release.

EMC CONFIDENTIAL—INTERNAL USE ONLY 18

Note that the term “fixes” and “patches” are used interchangeably by Engineering
and in documentation. To try and be more consistent we (SSR) try to refer to
Enginuity code changes as “fixes” and SymmWin application changes as “patches”
but there is NO distinction between the terms (i.e. “fix” and “patch” are just words
with NO special EMC meaning).
The distinction we need you to be clear about is the difference between a microcode
“fix” or Enginuity “fix” and a SymmWin application “fix”.
You NEED to be aware of the difference if you are asked to check for “FIX #
2000XXXX” and “FIX # XXXXX”.

18
 Advanced Configuration Training 2007

Enginuity Packs
What is a SymmWin “Temporary” Patch? (cont)

Go to SymmWin
Help & “Show me
the makeup of this
build”. This opens
the SymmWin
This is an example at 5771.94.102. “Table Viewer”.

“102” is the SymmWin Release number, hence

the “fixes” are listed for 101 and below. Use
Expand and Autosize to see the details, GFS is
useful if looking for a particular patch number.
This is GA code so ALL of the listed fixes are
“PERMANENT” (note the associated Revision
numbers).

EMC CONFIDENTIAL—INTERNAL USE ONLY 19

Note that “Show me the makeup of this build” is a DMX (or 5x7x) only enhancement
(not in 556x).

19
 Advanced Configuration Training 2007

Enginuity Packs
Introduction

Question: So what is an E-Pack?

Answer: An E-Pack is a special build of Enginuity code
designed to address one or more specific customer issues.
This “special” release contains recompiled director
emulation code and or a modified version of SymmWin.
– These “temporary” fixes will eventually be included in a future GA
Enginuity release (at which time the E-Pack is withdrawn).
– Again, we have differentiated between an Enginuity “code fix” and
SymmWin “patch” since an “Enginuity Special build” or “Enginuity
Pack” or “Service Pack” or “E-Pack” may contain either or both
types of these “temporary” fixes. If you order or install an “E-Pack”
you need to know what you are installing, what can be “combined”
and how to confirm whether it has been successfully installed.

EMC CONFIDENTIAL—INTERNAL USE ONLY 20

Note that E-Packs are known by a number of different names - they ALL mean the
same thing – a “special build” of code (either the microcode OR the SymmWin
application OR both).

20
 Advanced Configuration Training 2007

Enginuity Packs
Obtaining an E-Pack

If an Enginuity Pack is required and available you can
download it from the SSR Web Site (see next slide).

Go to the Home page and the Enginuity Links to the
“Enginuity Packs available from the SSR Web”.

These are “Generally Available” E-Packs and DO NOT
require an “E-Pack Request” (through the “Enginuity Pack
Database” - this is discussed shortly).

Note the Primus links and detailed installation instructions.

Again the E-Packs available from the SSR Web can be
downloaded and installed on your sites - as long as all
standard / local CCA approvals have been met.

EMC CONFIDENTIAL—INTERNAL USE ONLY 21

As above.

21
 Advanced Configuration Training 2007

Enginuity Packs
E-Packs Available from the SSR Web

EMC CONFIDENTIAL—INTERNAL USE ONLY 22

As above.

22
 Advanced Configuration Training 2007

Enginuity Packs
Enginuity Pack Database

If you require an E-Pack that is NOT available from the SSR
Web then you MUST submit an “E-Pack Request”.
– Go to the SSR Web Home Page and follow the Enginuity Links to
the “EPack Request Database”. Click on “Enter Request” and follow
the instructions (or open / review an existing request).

EMC CONFIDENTIAL—INTERNAL USE ONLY 23

As above.

23
 Advanced Configuration Training 2007

Enginuity Pack Database (cont)

NOTE that ALL “E-Pack Requests” MUST have TS2 Platforms or TS2
Solutions involvement. You CANNOT just request a “fix” based on a
Primus Solution (or a PSE Lab “recommendation”). Technical Support
MUST confirm that the fix is applicable to your customers environment.

NOTE that ALL “E-Pack Requests” MUST have the name of the actual
“EMC Engineer” that was contacted by TS2 – again this is NOT the name
of the PSE or TS2 or SSR representative that may have been contacted.

EMC CONFIDENTIAL—INTERNAL USE ONLY 24

You must follow ALL of the instructions and complete all of the required fields. The
above text is just to highlight TWO common mistakes / misunderstandings which
cause unnecessary rejections, resubmissions & general delays.

24
 Advanced Configuration Training 2007

Enginuity Pack Database (cont)

Remember that “separate” E-Packs CANNOT be combined

(see next slide) and they MUST be built at a specified Enginuity
level (the default is the latest MR or SR for that code family).

EMC CONFIDENTIAL—INTERNAL USE ONLY 25

As above.

25
 Advanced Configuration Training 2007

Enginuity Packs
Rules & Regulations

You CANNOT “combine” Enginuity code fix E-Packs.
– The required code “fixes” MUST be compiled WITH the Enginuity
source code (of the specified level) to produce an “E-Pack”. Another
Enginuity code load or another E-Pack load will ALWAYS
“overwrite” the previously installed code fixes. If your customer
wants to upgrade code then you need a NEW E-Pack. If your
customer needs another “fix” then you need a NEW E-Pack.

You CANNOT combine SymmWin patch E-Packs.
– These are (typically) compiled version of the SymmWin application
and or modified *.SMP files which will overwrite each other.
– You CAN combine an Enginuity code E-Pack (that updates the
emulation binaries) AND a SymmWin patch E-Pack (since these
only affect SymmWin). Again some E-Packs require that you MUST
install BOTH types of fixes to correct a “problem”.
EMC CONFIDENTIAL—INTERNAL USE ONLY 26

The new “temporary” Enginuity fixes or revised (new revision) Enginuity fixes MUST
be “compiled” with the Enginuity source code (i.e. build into a specified Enginuity
level). You cannot combine E-Packs that contain compiled director emulation code –
even if they were built at the SAME Enginuity level.
A subsequent hot code load or new E-Pack simply “wipes out” the previous fixes by
overwriting the previously loaded emulation binaries.
Similarly SymmWin “patch” files will also overwrite each other. Remember that
these patches are also Enginuity level specific – you cannot apply SymmWin
patches from different Enginuity levels, you cannot combine SymmWin “E-Packs”
from the SAME Enginuity level.
However, having said all this (as will be discussed shortly) the GS Serviceability
“patches” ALSO modify SymmWin behavior and they CAN be combined with
SymmWin “E-Packs”. A GS Serviceability “patch” (typically) does NOT update the
SymmWin Release level – they are (usually) not SymmWin “fixes” as shown on the
“Show me the makeup of this build” screen. They address different areas of the
SymmWin application and add functionality (e.g. additional drive support, an SRDF
data collection script, a special power down script, etc). These scripts are discussed
later in this presentation.

26
 Advanced Configuration Training 2007

Enginuity Packs
Installing an E-Pack

Download ALL of your E-Pack files from the SSR Web Site
or FTP Site (or \\C4\public share) as required.

Simple follow the included instructions (note that a DMX-
800 at 5670 require a special procedure due to “Launcher”).

For an Enginuity code E-Pack the “E-Pack Installer”
program will update the emulations binaries (*.bin files) and
REVISION.LOG file in the SymmWin UCODE directory.

Once you have installed an Enginuity code “fix” E-Pack onto
the Service Processor (i.e. updated the emulation binaries
in UCODE) you MUST install these files onto the Symmetrix
directors with a hot code load.

EMC CONFIDENTIAL—INTERNAL USE ONLY 27

As above.

27
 Advanced Configuration Training 2007

Enginuity Packs
Installing an E-Pack (cont)

For a SymmWin patch E-Pack first stop SymmWin and any
specified Windows Services.

The “SymmWin Patch Installer” program will update the
SymmWin *.SMP files in the UCODE directory.

Reboot the Service Processor.

You DO NOT need to run ANY SymmWin scripts to update
the Symmetrix.

EMC CONFIDENTIAL—INTERNAL USE ONLY 28

As above.

28
 Advanced Configuration Training 2007

Enginuity Packs
Verifying an E-Pack

Follow any included instructions.

To confirm that your Enginuity code E-Pack has been
successfully installed on the Service Processor look in the
SymmWin UCODE directory. This will now contain *.old and
a REVISION.LOG.old file. These are the original emulation
binary files and the original REVISION.LOG. Examine the
current REVISION.LOG file.

AFTER the hot code load this will match the E7,SYRN
display. E7,SYRN,,,’T’ will show you that “Temporary” fixes
are now present. E7,D,A should show a “+” (to be sure you
should run the SYRN command from the same type of
director that has the modified emulation binaries).

EMC CONFIDENTIAL—INTERNAL USE ONLY 29

As above.

29
 Advanced Configuration Training 2007

Enginuity Packs
Verifying an E-Pack (cont)

Again follow any included instructions.

To confirm that your SymmWin patch E-Pack has been
successfully installed on the Service Processor click on
SymmWin, Help, “Show me the makeup of this build”.

As shown earlier, go to the top of the display, expand the
“Minor Release” ZZ and look for the required Fix #
2000xxxx.

This will be displayed as Status “TEMPORARY”.

EMC CONFIDENTIAL—INTERNAL USE ONLY 30

As above.

30
 Advanced Configuration Training 2007

Enginuity Packs
An Example
I downloaded the E-Packs for emc153615 and emc148716 at 5671.60.65.

This is for
SymmWin
patch #
20005072
and
Enginuity
code fixes
31801,
32117,
32342.

Can I install BOTH E-Packs on the SAME DMX?

Yes
EMC CONFIDENTIAL—INTERNAL USE ONLY 31

As above.

31
 Advanced Configuration Training 2007

Enginuity Packs
An Example (cont) – No E-Packs Installed

E7,D,A & E7,SYRN,,,’T’

show no “Temporary” fixes
have been installed and
“Show me the makeup of
this build” confirms no
“TEMPORARY” SymmWin
patches have been applied.

EMC CONFIDENTIAL—INTERNAL USE ONLY 32

As above.

32
 Advanced Configuration Training 2007

Enginuity Packs
An Example (cont) – SymmWin “fix” Installed

Here I have simply installed
the SymmWin patch #
20005072:

Follow the detailed
instructions:
– Stop SymmWin &
Optimizer.
– Stop the specified
Windows Services via
the Control Panel &
Administrative Services. SymmWin Help “Show me the makeup of
this build” now confirms that
– Run the executable. “TEMPORARY” SymmWin patch #
– Reboot the Service 20005072 has been successfully applied.
Processor. A hot code load is NOT required.

EMC CONFIDENTIAL—INTERNAL USE ONLY 33

As above.

33
 Advanced Configuration Training 2007

Enginuity Packs
An Example (cont) – Original UCODE Directory

This shows the content of the UCODE directory and

REVISION.LOG file BEFORE installing the Enginuity code fixes.

EMC CONFIDENTIAL—INTERNAL USE ONLY 34

As above.

34
 Advanced Configuration Training 2007

Enginuity Packs
An Example (cont) - Updated UCODE Directory

This shows the content of the UCODE directory and “new” REVISION.LOG file
AFTER installing the Enginuity code fixes on the SP. NOTE the *.old backup files.
EMC CONFIDENTIAL—INTERNAL USE ONLY 35

As above.

35
 Advanced Configuration Training 2007

Enginuity Packs
“Not same EMULation files on PC and SYMM!”

Once you have installed an Enginuity “fix” E-Pack onto the
Service Processor you must install these files onto the
Symmetrix directors with a hot code load.

If you do not update the running emulation code all
configuration changes (CE or SIL) and replacement scripts
will fail the step “Verify_Samemul_PcSymm” (or equivalent)
i.e. the size & timestamp & checksums of the emulation
binaries on the SP (UCODE dir) and the Running emulation
code on the Symmetrix directors will be different.

All changes (or part replacements) will “red box” with a “Not
same EMULation files on PC and SYMM! Files Tested:
xxxx.BIN”. Refer to emc65635 and emc28626.

EMC CONFIDENTIAL—INTERNAL USE ONLY 36

As above.

36
 Advanced Configuration Training 2007

Service Processor Replacement

“Not same EMULation files on PC and SYMM!”

Note that the error “Not same EMULation files on PC and
SYMM!” will also occur (again preventing any CE or SIL
scripts) if the Service Processor has been replaced AND the
Enginuity code “E-Pack” running on the Symmetrix directors
has NOT been re-installed on the SP (i.e. UCODE dir).

Contact the Account CE to locate the required E-Pack.

Otherwise use the E7,SYRN,,,’T’ command and take note of
the listed “Temporary” fixes. Then look at the E-Packs
available from the SSR Web. If you do not find a precise
match you need to contact the PSE Lab for assistance*.

Any E-Pack related SymmWin patches should also be re-
installed BUT they WON’T cause the above “red box”.
EMC CONFIDENTIAL—INTERNAL USE ONLY 37

*As per emc28626 the PSE Lab can read the emulation code from a director back
into Global Memory (GM) and then run a “PSE Services” script to read this
emulation code from GM and resave it (as file name XXXXX.bin) on the SP. While
this process may eliminate the above “mismatch” it does NOT mean that the
SymmWin installation on the SP is “correct”.
You still need to locate the FULL E-Pack release and re-install this on the SP. If the
installed E-Pack looks to be unique you must contact the SSR Group with the box
serial number details and we can look up the “Enginuity Pack Database” to
determine if an E-Pack WAS requested.
In the absolute “worst case scenario” you can perform a hot code load to eliminate
ANY emulation code discrepancies (i.e. whatever is in the SymmWin UCODE
directory of the SP will be loaded into the directors) – BUT – this will remove ANY
previously installed “temporary” fixes and expose the customer to the issues
prevented by the previously installed E-Pack.

37
 Advanced Configuration Training 2007

Hot Code Load Performed

Example (cont) – E-Pack Now Installed

The E7,D,A &

E7,SYRN,,,’T’ now show
that 3 “Temporary” fixes
have been successfully
installed:
31801, 32117, 32342
This matches the
REVISION.LOG file on
the Service Processor.
EMC CONFIDENTIAL—INTERNAL USE ONLY 38

As above.

38
 Advanced Configuration Training 2007

Enginuity & SymmWin “Fixes”

Testing Your Knowledge

Question: How do you check if Enginuity “fix” # XXXXX is
currently installed at your site? How can you check whether
it is included in target (or higher) code?

Answer: From Inlines use “E7,SYRN,XXXXX” or search the
UCODE REVISION.LOG file on the SP or your CE laptop (if
you have the SAME code installed). To confirm if the fix is in
target or higher code you need to install the code on your
CE laptop and again search the REVISION.LOG file.

Question: How do you check whether SymmWin “patch” #
2000XXXX is currently installed or is in target / higher code?

Answer: Same as above. This time you go to SymmWin
Help tab & select “Show me the makeup of this build”.
EMC CONFIDENTIAL—INTERNAL USE ONLY 39

As above.

39
 Advanced Configuration Training 2007

GTS STAT Serviceability Scripts

Introduction

The SSR Web & Enginuity Pack page (and Primus) have
links to the available “GTS Serviceability Scripts”.

Supplied by the GTS S.T.A.T. Group these scripts may be
SymmWin “enhancements” to correct an existing upgrade
or maintenance script OR (more commonly) these will be
“special” standalone scripts created for a pre-activity test /
data collection OR to correct a specific problem OR add a
new function (e.g. to provide support for a newly released
drive model or to test RMS cabling, etc).

Often these scripts are distributed & installed automatically
via SYR. They are also available from the E-Pack page for
sites with NO dial-up access or for CE’s to re-install when
SymmWin is upgraded (or the SP replaced).
EMC CONFIDENTIAL—INTERNAL USE ONLY 40

As above.

40
 Advanced Configuration Training 2007

GTS STAT Serviceability Scripts

Where are they located?

These are a “sample” from the Enginuity Pack page.

Note the different issues resolved by the GTS STAT scripts. First follow the
Primus link to confirm if the patch or script is applicable to your customer. Special
“approval” is NOT required. Always take note of the installation instructions.
EMC CONFIDENTIAL—INTERNAL USE ONLY 41

As above.

41
 Advanced Configuration Training 2007

GTS STAT Serviceability Scripts

Installing & Verifying a Script

The GTS STAT scripts are supplied for specific Enginuity
levels or Enginuity families. The Serviceability scripts are
also all “different”. Meaning that the specific instructions
supplied with every GTS STAT script must be followed.

For example:
– The “Format Pending” script that must be run before any power
down at 567x (emc121835) simply requires the appropriate
FP7x.SMP file to be copied into the Service Processor and
SymmWin UCODE directory. The script is run “manually”.
– The “SRDF Upgrade Data Collection Script” (emc112530) is
distributed as an executable file and it must be installed since it
modified the SymmWin program and adds a menu item to the
SymmWin, Procedures, PSE Utilities.

EMC CONFIDENTIAL—INTERNAL USE ONLY 42

As above.

42
 Advanced Configuration Training 2007

GTS STAT Serviceability Scripts

Installing & Verifying a Script (cont)
– For details on the various “GS Drive Patches” refer to emc121668,
emc107228, emc104802, emc157271 (for 5771) & emc161027 (for
5772). These provide support for the latest drives in typically “older”
Enginuity levels. These are distributed as an executable file. This
will update any / all SymmWin files that require updating.

Note that the

installation program
To confirm that the won’t run unless
installation was you are at the
successful go to correct minimum
the C:\Install_Logs Enginuity level for
directory and the new drive type.
examine the
GS_Tools.log file.

EMC CONFIDENTIAL—INTERNAL USE ONLY 43

*Note that below Enginuity 5772 any “new” drive models typically required that you
either upgrade your Enginuity level AND / OR installed a GS Serviceability Drive
patch. If a GS Serviceability patch is required this will typically update the files
shown above.
For example, fibrcode.dat & fibrcode.zip (new fibre drive SCSI code),
parts_symm6.txt (new drive part numbers), fibrblk.dat (new drive capacities), etc.
Remember that the basic / raw support for the new drive type must already be in the
DA emulation included with the currently running Enginuity code. The above
additional files are required by SymmWin running on the SP. NO mock code load or
additional scripts are required below 5772.

43
 Advanced Configuration Training 2007

GTS STAT Serviceability Scripts

GS Serviceability “Drive Patches”
Note that the installation program
determines which files (if any) are
required. Compare this file list to
the previous slide. The same
GS70NDrv.exe patch installer was
run at a later 5670 release.

This GS_Tools.log example is for

5771. Again note the updating of
the files for SymmWin. Note that
the “firmware7.zip” file is
updated. This file contains the
SCSI code for the new drives.
Remember that at 567x & 5771
no further actions are required
(i.e. no additional scripts or
“mock” code loads are required
to support the new drive types).
EMC CONFIDENTIAL—INTERNAL USE ONLY 44

At Enginuity 5772 there will no longer be any need to upgrade to the latest Enginuity
code to support a “new” drive type. The basic / raw drive support will be via a
separate “Dynamic Drive Data Base” file in the UCODE directory. This is loaded into
the DMX-3 by a special “Load Drive DB” script (discussed shortly).
HOWEVER a GS Serviceability patch “download” and installation on the SP
will still be required – this will distribute the updated “Dynamic Drive Data
Base” file (contained in the firmware7.zip) PLUS any other ancillary files
necessary (required by SymmWin) for this new drive type.
In the above example for 5771 note that the “firmware7.zip” file IS included. On
DMX-3 this contains all of the binary emulation files INCLUDING the fibre SCSI
code (on DMX / DMX2 these are the fibrcode.dat & fibrcode.zip files).
AT 5772 the “firmware7.zip” file is also needed because it will contain the
NEW driveDB.bin (the “Dynamic Drive Data Base” file).

44
 Advanced Configuration Training 2007

GTS STAT Serviceability Scripts

AND Enginuity & SymmWin “Fixes”

GS Serviceability “scripts” are independent of the actual
running Enginuity code or any additional Enginuity code fix
“E-Packs”. They only affect the SymmWin application.

Therefore you CAN safely install any “microcode related” E-
Packs and any GS Serviceability “patch”.

SymmWin “patches” or temporary “fixes” are also typically
independent of the GS Serviceability scripts – since the
functional areas are different.
– However, since both the SymmWin E-Packs and Serviceability
Scripts update the SymmWin application we (SSR) cannot
guarantee that there will never be an unwanted interaction.
– Contact your local RTS for any necessary confirmations. Your RTS
can contact the GTS STAT Team through the PSE Lab or SSR.
EMC CONFIDENTIAL—INTERNAL USE ONLY 45

As above.

45
Advanced Configuration Training 2007

EMC CONFIDENTIAL—INTERNAL USE ONLY 46

46
 Advanced Configuration Training 2007
EMEA Tech Ops Business Team - Symmetrix

Dynamic Drive Database

Dynamic Drive Database

5772 New Script Feature

Introduced at Enginuity 5772

EMC CONFIDENTIAL—INTERNAL USEEMC

ONLYConfidential – Restricted Distribution 1

EMC CONFIDENTIAL—INTERNAL USE ONLY 47

Based on the 5772 Dynamic Drive Database presentation (supplied by Wolfgang Timov in the
UPDATES.ZIP).

47
 Advanced Configuration Training 2007
Go to View/Master/Slide Master to type in header

Agenda
Dynamic Drive Database
Contents

Drive support

What is the at Enginuity
Dynamic Drive5771 and below.
Database

Why

Drive was it created?
support changes at Enginuity 5772.

How is it implemented?

Dynamic Drive Database.

Displaying the Drive DB File Version.

How to determine if a new Drive DB file is required.

How to update the Dynamic Drive Database.
– Download & Install the GTS New Drive Support Package.
– Load Drive DB script.
– EMCOther ways of USE
CONFIDENTIAL—INTERNAL updating
EMC the– Restricted
ONLYConfidential DriveDistribution
DB file. 2

EMC CONFIDENTIAL—INTERNAL USE ONLY 48

As above.

48
 Advanced Configuration Training 2007
Project Motivation
Go to View/Master/Slide Master to type in header

At Enginuity 5771 and below

In 5771 and below, drive support is statically compiled into
DA binary image at build time

In 5771 and below, drive support is “statically” compiled into
A Syren
DA
binary fix is required
image at buildtotime.
support new drives post-GA
– Limits field usage to drive upgrades with minimum revision Enginuity

A full recompilation
releases or higherof the microcode (SYREN fix* applied)
is required to support any new drives post-GA.
– Therefore a simple drive upgrade MAY require BOTH an Enginuity
upgrade (to a higher minimum Enginuity release) AND the
Entry 1: Device id
device type, queue depth

application of a “GS Serviceability Patch”.

Drive feature mode page

Part of Need to update

Entry 2: Device id Ucode in order to
device type, queue depth d70f.bin support new drive
Drive feature, mode page
Entry 1: Device id
device type, queue depth
Drive
Drive Database
feature mode page
Need to update
Entry 2: Device id
Part of Enginuity to support
device type, queue depth d70f.bin new drive model**.
Drive feature, mode page

EMC CONFIDENTIAL—INTERNAL USEEMC

ONLYConfidential – Restricted Distribution 3
Drive Database

EMC CONFIDENTIAL—INTERNAL USE ONLY 49

Currently, only specific drive models can be supported by particular Enginuity Releases. If a
newly qualified drive needs to added to an existing system - which doesn’t support this drive
model – BOTH the Enginuity microcode and files used by the SymmWin program must be
upgraded to an appropriate level. Below 5772 drive related information is “embedded” in the
DA emulation code (note the above drive attributes).
*SYREN is the acronym for "Symmetrix Release Engineering". The “SYREN fix” process is
used by Engineering and Microcode QA to generate new emulation BIN files.
**The statement “Need to update Enginuity to support new drive model” is typically correct for
Enginuity 5771 and below . However, there are some exceptions. Again, FULL support for a
particular drive model (ALL of the requisite files) are normally included in the specified
minimum Enginuity Release. However, while this is true it is possible for a minimum Enginuity
Release to NOT have all of the requisite files to support a new drive model and these
additional files will still need to be distributed via a GS Serviceability patch (refer back to the
earlier presentation).

49
 Advanced Configuration Training 2007
Project Overview
Go to View/Master/Slide Master to type in header

Changes at Enginuity 5772

At
5772 we now
Dynamic drivehave a Dynamic
database Drive Database.
is now a separate file that is
saved along with other emulation files and allows a

This Symmetrix
file is nowtoseparate
add new drivefromsupport
the DA emulation
without code
requiring an and
this allows us to add support for new drive models without
Enginuity upgrade
a fullNote:
Enginuity upgrade.
Original static drive DB still present in DA image if needed
Entry 1: Device id
device type, queue depth
Drive feature mode page
No need to update
Entry 1: Device
Ucode in order to
Entry 2: id
Device id Download to Symm through
device type, queue
device type, depth
queue depth support new drive
Drive feature, mode page Symmwin script
Drive feature mode page
No need to update
Driveid Database Enginuity to support
Entry 2: Device Download to DMX-3 through
device type, queue depth new model drive.
Drive feature, mode page SymmWin script

External file
Drive Database

External file
EMC CONFIDENTIAL—INTERNAL USEEMC
ONLYConfidential – Restricted Distribution 4

Note: The original “static” drive database is still present in the DA emulation code if required.

EMC CONFIDENTIAL—INTERNAL USE ONLY 50

Starting with Enginuity 5772, a new central repository of drive information will now be kept in
director FLASH. This repository can be updated “dynamically” through a SymmWin script. So
if a new drive model needs to be installed in the Symmetrix, the new drive model information
can first be added to this “drive database”. The Symmetrix will then be able to retrieve the new
drive model information, configure the drive, and communicate with it (without the need for a
full Enginuity upgrade).

50
 Advanced Configuration Training 2007
Configuration Notes
Go to View/Master/Slide Master to type in header

Dynamic Drive Database

Drive database file is called “drivedb.bin”
– Drive
Located in firmware7.zip, alongDB
with File
required SCSI codes

The
Updates to drive DB

drive database orwritten
DriveinDB
Syren
file fixes
is called drivedb.bin and
is stored

Drive (along withcan
DB updates director emulation
be accessed & fibre
on demand as SCSI
neededcode
files,byetc)
CE’sin the firmware7.zip repository.
– Packaged as part of GS releases

The Drive DBfirmware7.zip
– Includes can still ONLY be updated
and all other required filesthrough SYREN
(part #’s, etc)
applied fixes. However, updated Drive DB files can be
downloaded and applied on demand.

EMC CONFIDENTIAL—INTERNAL USEEMC

ONLYConfidential – Restricted Distribution 5

Go to O:\EMC\<sn>\SymmWin\UCODE\firmware7.zip.
EMC CONFIDENTIAL—INTERNAL USE ONLY 51

As above.

51
 Advanced Configuration Training 2007

Dynamic Drive Database

Displaying the Drive DB Version

An E7,D,A will show the currently running Drive DB version:

EMC CONFIDENTIAL—INTERNAL USE ONLY 52

As above.

52
 Advanced Configuration Training 2007
Go to View/Master/Slide Master to type in header

Dynamic Drive Database

Dynamic DB file verification
Is a new Drive DB file required?

Primus and the

Currently CCAprocess
the CCA Reference Page
through willReadiness
Service highlight will
if a later
provide a primus solution the specifies the latest
Drive DB version is required. The PSE Lab will have a revision
required of the DB file. If necessary the CE would pull the
Compatibility
file from theMatrix charttoas
GS website thewell.
SP.

The CE can

PSE then download
lab maintains the latest
a compatibility matrix“GS
chartNew Drive
with the
Support
latestPackage” (as required) from the “special builds”
revision information.
page of the

Long termSSR Web
a push .
via SYR will be automated to verify the
revision of the service processor and push the updated
version if necessary.

EMC CONFIDENTIAL—INTERNAL USEEMC

ONLYConfidential – Restricted Distribution 10

EMC CONFIDENTIAL—INTERNAL USE ONLY 53

At 5772 GA the CE must verify if an updated Drive DB file is required BEFORE a new install
or drive upgrade or drive replacement is performed. If so then the latest “GS New Drive
Support Package” must be downloaded first and the Drive DB file must be updated manually.

53
 Advanced Configuration Training 2007

Dynamic Drive Database

Download & Install the GS Drive Support Package
1. Download the latest “GS New Drive Support Package”
from the SSR Web “special builds” page. The installation
file name is GS772NDRV.x.x.x.x.EXE.
2. Go to Windows, File, Properties to make sure the “File
version” is the one you want. The “Description” should
state “New Drive Support Package 5772”
3. Copy the file to the C:\ drive of the Service Processor.
4. Run the executable
GS772NDRV.x.x.x.x.EXE:

EMC CONFIDENTIAL—INTERNAL USE ONLY 54

NOTE that the “GTS New Drive Support Package for 5772” Version number will NOT
correlate with the required Drive DB version number. The package numbers above refer to the
version of the GS installation program - this must match the Primus Solution to ensure that
you are using the correct executable / installation package.

54
 Advanced Configuration Training 2007

Dynamic Drive Database

Download & Install the GS Drive Package (cont)
5. Go to the C:\Install_Logs directory and examine the
GS_Tools.log file to confirm that “GS New Drive Support
Package Version x.x.x.x is now Installed”.
– Note that the installation of the GS Drive Package does not simply
overwrite the firmware7.zip file. It checks if the Drive DB is
supported by the current Enginuity level then it will load some or all
new files to the SP (depending on this Enginuity level).

Note: The installation of the

GS Patch to the Service
Processor does NOT load the
updated Drive DB file to
System!.
An E7,D,A will confirm that
this file NOT has changed.

EMC CONFIDENTIAL—INTERNAL USE ONLY 55

As above.

55
 Advanced Configuration Training 2007
Go to View/Master/Slide Master to type in header

How to update drive DB?

Dynamic Drive Database
The “Load Drive DB” Script

Next

Gorun
to the “Load Drive
GS website DB” script
and download to load this
firmware7.zip new Drive
to ucode
(O:\EMC\190300142\SYMMWIN\ucode) directory of Service
DB file from the Service Processor into the DA director.
Processor.

Run stand alone “Load Drive DB” script in Procedures
Procedure Wizard Disk Maintenance Utilities Load
Drive DB

Use E7,D,A to make sure drive DB revision is the one
desired.

Note: The Load Drive DB script updates a portion

of the DF emulation code and therefore it
performs a “Fast IML” of all DF processors. From
a host impact perspective please treat this like
EMC CONFIDENTIAL—INTERNAL USEEMC any online
ONLYConfidential code
– Restricted upgrade.
Distribution 8

EMC CONFIDENTIAL—INTERNAL USE ONLY 56

There is a standalone “Disk Maintenance Utilities” script called “Load Drive DB” that actually
loads the Drive DB file into the DA director FLASH.

56
 Advanced Configuration Training 2007
Go to View/Master/Slide Master to type in header

Dynamic Drive Database

How to update drive DB?
Verify Drive DB Version (again)

An
E7,D,A will
Go to GS show
now and
website the new
download Drive DB to
firmware7.zip version:
ucode
(O:\EMC\190300142\SYMMWIN\ucode) directory of Service
Processor.

Run stand alone “Load Drive DB” script in Procedures
Procedure Wizard Disk Maintenance Utilities Load
Drive DB

Use E7,D,A to make sure drive DB revision is the one
desired.

EMC CONFIDENTIAL—INTERNAL USEEMC

ONLYConfidential – Restricted Distribution 8

EMC CONFIDENTIAL—INTERNAL USE ONLY 57

The E7,D,A shows that the new drive information has been loaded. The new drives can now
be added to the system. An Enginuity upgrade is NOT required.

57
 Advanced Configuration Training 2007
Configuration Notes
Go to View/Master/Slide Master to type in header

Dynamic Drive Database

The following Symmwin scripts loads the drive DB:
Updating the Drive DB File

Online Code Load

In
addition to the
Offline Code Load“Load Drive DB” script the following
SymmWin scripts all contain new steps to check if the Drive

SOS Code Load
DB version on the SP is newer than the current version on
Config
the
DA & Install New Symm
directors.

Online Config Change

They will automatically update the Drive DB as part of the

NDU Upgrade
requested change:
Stand-alone
–
Configure drive New
& Install DB download
Symmetrixscript
A periodic
–
Online task LoadDriveDB
or Offline Code Load runs every 24 hours to check
for a new drive DB and loads it if necessary
– NDU Upgrade to 5772
– Online Configuration Change
EMC CONFIDENTIAL—INTERNAL USEEMC
ONLYConfidential – Restricted Distribution

Note that SYMCLI changes do not update the database.
6

EMC CONFIDENTIAL—INTERNAL USE ONLY 58

The above scripts compares the Drive DB version on System with the Service Processor. If
there is a new version on the SP, then the script will load the new version. If there is an
OLDER Drive DB version on the SP, then the script will ask for a “user decision” on how to
proceed.

58
Advanced Configuration Training 2007

EMC CONFIDENTIAL—INTERNAL USE ONLY 59

59
Advanced Configuration Training 2007

CS Introduction to Scripts

“What Not To Do and Why Not To Do It.”

EMC CONFIDENTIAL—INTERNAL USE ONLY 60

60
 Advanced Configuration Training 2007

Online Configuration Change Scripts

Testing Your Knowledge

Question: My drive upgrade script failed. But I can finish it
out next month (my customer doesn’t need the storage until
then). True / False?

Question: My configuration change script failed, I can just
restart it from the beginning (after all, my customer does this
through ECC/SYMCLI software all the time). True / False?

Question: I have seen the PSE Lab resume and redo the
failed step before doing anything else - since contacting the
Lab takes time I can first try this myself. True / False?

Question: The Service Processor C: drive needs “cleaning” I
can delete the contents of the C:\SymmWin\USER directory
(since there is nothing valuable in there). True / False?
EMC CONFIDENTIAL—INTERNAL USE ONLY 61

Instructors: Try the above quiz on your students and discuss the
results before going onto the next slide.
All of the above scenario’s seem plausible and reasonable. The answer
in every case is “FALSE” (and they reflect popular misconceptions).

61
 Advanced Configuration Training 2007

Online Configuration Change Scripts

Introduction

All script recovery decisions must be made by the PSE Lab.

All script recoveries should be performed in a timely
manner.

The PSE Lab may escalate to a Senior PSE or a Script
SME or to a member of the SSR Group.

If the issue is already fully detailed in Primus the PSE Lab
may refer the issue to the local RTS.

Note that the CE is never to take any corrective actions
without the express permission of the PSE Lab (or local
RTS).

EMC CONFIDENTIAL—INTERNAL USE ONLY 62

All CE’s should be aware they must call the PSE Lab whenever a
script “red boxes”
However, since configuration change scripts are so easy to apply (i.e.
only ONE bin file is needed for multiple changes) and can be confirmed
to be online safe (i.e. the online simulation “green boxes”) and the
actual activities ARE typically successful (i.e. script failures ARE
relatively uncommon) that field staff have become MUCH less aware of
the potential pitfalls and have started to exhibit “bad habits”.

62
 Advanced Configuration Training 2007

Online Configuration Change Scripts

The Golden “Rules”

Always contact the PSE Lab first (before attempting any
corrective actions).

You must never “Resume from failed step” and re-try the
step (without explicit PSE Lab or RTS permission).

You must never bypass a script “red box” i.e. skip a step
(without explicit PSE Lab or RTS permission).

You must never try to manually issue a Fast_IML or other
command via Inlines (to “manually” complete a script).

You must never “Abort” a failed script and restart from the
beginning (without explicit PSE Lab or RTS permission).

EMC CONFIDENTIAL—INTERNAL USE ONLY 63

The safest course of action is to do nothing and ring the PSE

Lab.
The listed restrictions apply to all CE’s (and typically most RxS’s)
however the reason for these “rules” are not necessarily clear to the
field. While some actions are obviously dangerous (or should be
obviously dangerous such as retrying a failed step OR issuing
manual Inlines commands OR skipping a step, etc) some seemingly
minor actions can also prevent a failed script from being
completed….
The purpose of this presentation is NOT to encourage the field /
CS staff to “try and fix” script issues themselves – it is to
explain the above rules and try and show them the sort of
potential disaster that awaits themselves and their customer if
they don’t stop what they are doing and call the PSE Lab first…..

63
 Advanced Configuration Training 2007

Online Configuration Change Scripts

How can a change fail when the simulation passes?

All configuration changes contain numerous scripted checks
that are run (and are only valid to be run) during the actual
implementation of the configuration change.

These “checks” will look for Enginuity related issues AND
change specific issues such as active IO, sufficient cache,
invoked spare or BCV, active SRDF, SNAP sessions, etc.

All current configuration changes take the System bin file
and submitted IMPL.BIN file and resave these to check their
validity. This check is performed at the SymmWin level AND
it is performed at the actual microcode or Enginuity level
(during the actual activity). All changes MUST pass BOTH
levels of validation before the new configuration is applied.

EMC CONFIDENTIAL—INTERNAL USE ONLY 64

First of all, as you are all aware, a requested configuration change can pass all of the
“standalone” SymmWin policing checks and online simulations on your CE laptop AND
the Service Processor BUT it can still fail when actually applied to the Symmetrix. This
is because the activity may fail ANY of the scripted checks that look at the
current state of the running Symmetrix at the time of the activity. For example,
you cannot perform ANY configuration changes if the Enginuity versions on the
Service Processor and actual Symmetrix are different (say due to an earlier SP
replacement or an uninstalled E-Pack). These initial checks are to first ensure that it is
feasible & safe to attempt to change the current configuration. Checks like the Script
Regulator look at the issues specific to the current Enginuity level. Secondly the
scripted checks look for specific conditions that may prevent the desired change from
being successfully applied. Depending on the requested configuration change specific
“box related” issues will also prevent the activity from proceeding. These scripted
“Avoid_XYZ_Corruption” conditions CANNOT be checked by an online simulation. For
example, you cannot add an RDF mirror to a STD volume if a hot spare or BCV is
currently established, you cannot change the RDF attributes of a dynamic RDF volume
that has a entry in the GDAT, you cannot delete a volume that has active SNAP
sessions established against it, etc (AND therefore “skipping a failed step” may result
in cache corruption, GDAT corruption, directors dropping “DD”, etc, these and many
more disasters can result from the incorrect corrective actions taken during a script).
Finally while the actual configuration change may pass these SymmWin applied
checks it can still fail at the Fast FRD compare or Ucode_Compare_MboxImpl stage -
this is the microcoded check on the validity of the “new” bin file.....

64
 Advanced Configuration Training 2007

Online Configuration Change Scripts

Configuration Change Basics

Any / all current configuration scripts start the actual
“change” by posting a new IMPL.BIN file from the Service
Processor into the Symmetrix Global Memory “mailbox”.

Once the new bin file is in the mailbox and verified the new
bin is either loaded into “Running” (director Control Store)
first OR loaded into “Flash” (director EEPROM) first.

The above sequence is code family dependent (although
the sequence may vary with specific scripts).

At 5670 and we load the new bin into “Running” first (the F4
IML is used) then update “Flash” later. At 5x71 we load
“Flash” first and “Running” later (the F2 IML is used).

EMC CONFIDENTIAL—INTERNAL USE ONLY 65

• The Fast IML will take the new configuration (bin file) and load it into “Running”. This can be done a number of ways
depending on what script is being run / what is specified in the procstep.log file.
• For online configuration change scripts will use some form of the F4 or F2 utility accompanied with the DVUP option (or
similar parameter). Once the Fast IML is complete the system WILL be running the new configuration. This can be verified by
the E7,D,A display.
• The director EEPROM (“Flash”) may not be updated at this stage (this depends on what script is being run and the Enginuity
family). A second fast IML in the script may also be required – this may be a FD (again this depends on the script).
• For all online configuration change scripts at 5x71 you should see where we previously loaded EEPROM). This was at the
step LOAD_ALL_FLASHES.
• At 5x71 the Fast IML step will either load the new configuration from Global Memory or from EEPROM (depending on what
utility is being used). Should all the directors IML correctly then the E7,D,x displays (see later slides) should match between
Global Memory, Running and EEPROM.
RTS ONLY (expanded excerpt from PSE Training):
1. The currently running emulation binary files and the current IMPL.BIN are loaded into Global Memory mailboxes from each
director type into their respective mailbox number (as shown on the E7,D,,1 display e.g. mbox 2 for fibre DA or FA directors).
2. The Global Memory mailboxes then have their contents compared to the emulation files on the PC (in the SymmWin ucode
directory on the Service Processor) and the current IMPLORG.BIN file (this is the System IMPL.BIN file that was read in
earlier during the script and saved in the PC). Red box: Not same EMULation files on PC and SYMM! commonly due to an
SP replacement or incomplete E-Pack installation.
3. If Step 2 passes, the files in the Global Memory mailboxes are erased and the emulation files from the PC (i.e. for a code
upgrade) and the IMPL.BIN (i.e. for a code upgrade or a new IMPL.BIN for a configuration change) are loaded into the
respective GM mailboxes.
4. The new IMPL.BIN file (at this point ONLY in GM mailbox) is compared to the current System IMPL.BIN file in each director
processor slice. The comparison is done by a synchonized broadcast command to all A slices first (where only A slices
respond), then all B slices, and so on. If any “discrepancies” are detected we will post a F4EE error on the slice (say due to
an “illegal” configuration change not detected by SymmWin). For example, the step Ucode_Compare_MboxImpl fails with
“Fast IML will fail on the following directors!” (use E2,,EEEE to display the detected discrepancy).
5. If the comparison was successful at Step 4, we then burn the IMPL.BIN from the GM mailbox into each director's EEPROM
(Flash). This is done by a synchronized broadcast command to the A slices (there is only one EEPROM on each physical
director board).
6. We do another check like Step 4, but this time, we compare the IMPL.BIN in the Global Memory to the newly changed
IMPL.BIN in each slice's EEPROM (e.g. step Ucode_Compare_EEImpl). We will check each slice (first the A slices, then B
slices, then C, etc) to ensure that each director slice is communicating after the load all Flash process is complete. If any
discrepancies are found we again post F4EE commands and stop the script.
7. If Step 6 was successful, then we load the bin file into a special location in NVD for each slice called the Control Store
Mailbox or CS_Mailbox. This is again performed via a synchronized broadcast command, and done slice by slice (so a total
of 4 commands). At the same time as this step, we compare the bin file in EEPROM to what is in CSRAM (current “Running”
code), and if it is a valid configuration change, then we put the slice into “Upgrade in Progress” mode. For example, the step
Ucode_Verify_EEImpl will allow the new configuration (new IMPL.BIN file) to be loaded into CSRAM and executed (i.e.
loaded into “Running”) on the director at the next scripted step.
8. If Step 7 is successful, then a full box synchronized command is issued to all slices simultaneously to load the new IMPL.BIN
from the CS_Mailbox into CSRAM and allow the new file and configuration to be used by the processors on all director
slices. The Do_Fast_IML step (typically) takes from 2 to 5 seconds on all processors. The I/O managed by the processor is
suspended for at least 1 second during the synchronized IML process (so that the CPU can spend time building and deleting
or overwriting any tables that it needs for the new configuration to work). We do a final check to ensure all the files are
properly in place (e.g. another Ucode_Compare_EEImpl).
9. In earlier code (below latest 5x71 and 5772) we also did another final synchronized Fast IML on all director slices (literally a
FD,CE) to “clean up”. This step is not required any later codes.
• Symm 4.x / Symm5 and DMX at 5670 did much the same configuration change process except the order of the bin load is
different. They load CSRAM (Running) first from the Global Memory mailbox, then load Flash (EEPROM) from Global
Memory, and then we execute the masked & synchronized Do Fast IML step on all director slices (typically with a follow up
$FD,CE).
• At Enginuity 5773, front-end channel assignment changes and meta configuration changes will NOT require director IML’s.
Engineering have written new tables into Global Memory and the NVD of the directors to allow syscalls to make the required
changes to the assignment and meta information - rather then loading a new file to all directors.

65
 Advanced Configuration Training 2007

Online Configuration Change Scripts

SymmWin User Directory File

The “step” bins built by SymmWin step engine (and listed in
the OnlUpg.log) are stored in the C:\ SymmWin user or O:\
Public user directories.

These “impl_step_xx.bins” will be saved as IMPL.BIN then
loaded into System as the change progresses.

When a configuration change is started the script structure
is populated with the date and time (and checksums) of all
the required files. Temporary files may also be created and
these may be tracked.

Therefore you must never alter / delete the contents of the
USER directory - unless you are absolutely certain that
there are NO active / outstanding or incomplete scripts.
EMC CONFIDENTIAL—INTERNAL USE ONLY 66

As above.

66
 Advanced Configuration Training 2007

Online Configuration Change Scripts

SymmWin User Directory Files (cont)

The progress of all scripts are stored in the SymmWin user
directory at 567x and in the script directory at 577x.

The status and progress of OnlUpg, OnlUpg_SIMUL,
OflUpg, OflUpg_SIMUL, DiskRpl, DirRpl, MemRpl,
MemRmv, HotCode, SwapvSil, SprRepl, GlobalSparing, etc
are stored in *.DAT files at 5x71 (*.DAT & *.PRC files at 70).

The *.DAT (*.PRC) files are incremented as the script
progresses (as *.DA1, *.DA2, etc). These files may
“rollover” and be lost if a script is restarted or resumed from
a failed step (making it impossible to resume an activity).

Therefore never attempt to restart or resume from a failed
step (without explicit PSE Lab or RTS permission).
EMC CONFIDENTIAL—INTERNAL USE ONLY 67

As above.

67
 Advanced Configuration Training 2007

Online Configuration Change Scripts

Resuming a previously failed script

A common misconception is that a failed script (any online
upgrade) can be safely completed days or weeks after the
initial failure. This is NOT true.

A failed script cannot be ignored, it MUST be completed in a
timely manner OR the change must be “backed out” and the
Symmetrix left in a “clean state” - for the next configuration
change OR to avoid any possible customer impact.

Depending on the failed activity and at what point the script
failed it is possible that the Symmetrix directors are still in
“upgrade mode” and / or that the “online cache tests” are
still disabled, etc. This may cause serious host impact (i.e.
performance degradation, host timeouts, etc).

EMC CONFIDENTIAL—INTERNAL USE ONLY 68

A common misconception is that a failed script (online upgrade, hot code load,
replacement script) can be safely completed days or weeks after the initial failure.
This is NOT true. There are TWO reasons for completing a script in a timely manner.
The first reason is to avoid any customer inconvenience or impact (as noted here).
The second reason is to ensure that the script CAN be completed successfully…(see
next slide).
A failed script cannot be ignored, it MUST be completed in a timely manner OR the
failed change must be “backed out” and the Symmetrix left in a “clean state”. Either
for the next customer change OR to avoid any possible customer impact.
What was highlighted on the previous slide is that it may NOT be possible to resume
a script if any necessary files are lost or changed BUT a much more serious issue is
potential customer impact. Depending on the type of failed activity and which step in
the script failed it is also possible that the Symmetrix directors are still in “upgrade
mode” or that the “online cache tests” are still disabled. In this state the customer may
well be seriously impacted (e.g. AB3E errors) as BCV’s and RDF mirrors will NOT
copy and channel I/O may be severely restricted (i.e. < 25 I/O per second per
director) - refer to emc11562, emc100780 and emc133080 for details. As already
noted any mismatch between director FLASH (EEPROM) and Running (CS) will
generate F4EE errors and any director that takes a hardware exception (i.e. drops
“DD”) cannot be recovered until this discrepancy is resolved.

68
 Advanced Configuration Training 2007

Online Configuration Change Scripts

Resuming a previously failed script (cont)

Failed scripts must be completed in a timely manner.

While the PSE Lab can run the “Recover Failed Step
Procedure” this script MUST be run in a timely manner (<48
hours) and NONE of the requisite files must be have been
altered.

The “Recover Failed Step Procedure” will either find NO
step controlled scripts to resume OR it will abort without any
messages.

While the SSR Group or Script SME can attempt to resume
a script failure beyond 48 hours the risk of a critical file
having been lost or altered during this time is very high and
it may be impossible to resume the failed script.
EMC CONFIDENTIAL—INTERNAL USE ONLY 69

The longer a script is left incomplete the more likely that a critical file will be lost or
altered by EMC / CE activities, SYR, subsequent customer SIL script, etc.
The PSE “Recover Failed Step Procedure” was deliberately designed NOT to resume
a script failure more that 48 hours old – this is to force SSR or Script SME
intervention (hopefully to guarantee that nothing critical has changed and that the
previously failed script CAN be safely completed).

69
 Advanced Configuration Training 2007

Online Configuration Change Scripts

Testing Your Knowledge

Question: You are onsite performing a multi-step online
configuration change at 5671 and it “red boxes”. Before you
call the PSE Lab you decide to re-run the online simulation
(just to recheck the upgrade bin file). Is this O.K.?

Answer: NO. The online simulation will pass and the PSE
Lab can resume from the failed step. BUT the script will now
resume from “Step 00 Enforce_Regulator_Process”. Why?
– The online configuration change and online simulation DO use
different files to track their progress and status (i.e. OnlUpg.DAT &
OnlUpg_SIMUL.DAT). But the online simulation will create new
impl_step_xx.bin files AND it will have updated the IMPL.BIN file in
the user directory. At 5671 this discrepancy prevents the script from
being resumed. Script SME assistance will now be required.

EMC CONFIDENTIAL—INTERNAL USE ONLY 70

Question: So what is the best course of action in the situation listed above?
Answer: Do nothing and call the PSE Lab. If you do something please tell the PSE
Lab what was done….. Any missing information or misinformation can cause the
WRONG corrective action to be taken.

70
 Advanced Configuration Training 2007

Online Configuration Change Scripts

Example of a 5671 script failure
We have failed at the Do_Fast_IML step (i.e. Fast IML failed) and
have the typical symptom of “red X’s” and a ”CODE MISMATCH !!!”.
F4EE errors are likely (confirming
the mismatch and giving the
reason for the IML failure). D80B
errors will be logged if a director
had dropped “DD” during the
change (e.g. at the IML step).
The E7,D,A command shown
here confirms that the IMPL.BIN
does not match between Flash
(EEPROM) and Running (CS).
We must be extremely careful
since we are (as expected) past
the “Point of No Return”.

EMC CONFIDENTIAL—INTERNAL USE ONLY 71

The PSE Lab must be involved to complete this online configuration change.
Question: With the directors in the above state (CODE MISMATCH !!!) what would
happen if any director encountered a “fatal error” (i.e. took a hardware exception) and
dropped “DD”. Could the Lab recover this dead director with a fast IML (FD,CE) OR
full offline IML (FE,CE)? Could it be successfully replaced using the director
replacement script? Answer: No director cannot be IML’ed OR replaced while a
CODE MISMATCH exists. The bin in Flash (EEPROM) does not match the bin file
currently in Running, if the director were IML’ed (Fast or Full) it would attempt to
implement the bin file from EEPROM - this does NOT match the running configuration
on all of the other active directors. The director being IML’ed would immediately &
correctly drop “DD” (with further D80B & F4EE errors) - we understandably CANNOT
operate a Symmetrix with inconsistent configurations in memory…..
Aside: Two common field questions. 1. The RUNNING CODE above also lists the
EMULATION code downloaded from the Service Processor during the hot code load
process – these are the d61f.bin, s61f.bin & r61F.bin shown above (d5xx for Symm5,
d7xx for Symm7 etc). These files currently loaded to the director must match the
same files on the SP in the SymmWin UCODE directory or NO configuration changes
are possible. Note that the code family & level is also shown above (NO “+” here
typically indicates that NO E-Packs are installed but run an E7,’SYRN’,,,’T’ to be
100% sure). 2. The EPROM CODE files above are loaded by the Procedures,
Director Maintenance, Download EPROM script (from the contents of the firmware.zip
file in UCODE). These files are NOT updated by the configuration change or hot code
load process.

71
 Advanced Configuration Training 2007

Online Configuration Change Scripts

Example of a 5671 script failure (cont)

The E7,D,,1 confirms that the

IMPL.BIN loaded into the
Global Memory mailbox is the
same bin now loaded into
Flash. And again this does
NOT match the bin currently
active in Running (CS). Note
the checksum & timestamp.

Note we must NEVER redo a step without understanding the reason

for the original failure and the consequences of redoing a failed step.

You would NEVER do this BUT if you

did attempt to redo the Fast IML step
you would get another “red box”.
EMC CONFIDENTIAL—INTERNAL USE ONLY 72

Since we are running Enginuity 5671 the configuration change scripts perform the
“Load_ALL_Flashes” step first. THEN it performs the Do_Fast_IML step to place the
new configuration into Running (i.e. make the change active). We are past the “Point
of No Return” (a lock.all file has been created) since we now have a mismatch been
the active running bin file (old) and the bin loaded into EEPROM (new). You
CANNOT abort the change or restart from the beginning.
The reason that the E7,D,A and E7,D,,1 commands are highlighted is because these
commands tell us the current state of the scripted bin file load, i.e. whether the NEW
IMPL.BIN (note the timestamp above – it should match the IMPL.BIN in the
SymmWin User directory) has been loaded in GM yet, whether it has been placed
into Running by the Fast IML step, whether it has been burnt into EEPROM by the
Load Flash step. Remember that the configuration change sequence depends on the
code family AND the type of configuration change being performed. The information
above dictates how we (i.e. Engineering, PSE Lab, SSR, RTS, etc) proceed with the
script recovery.
Also NOTE that I am NOT suggesting that you run these commands. The
intention is just to show you some of information that the Lab will need (and give you
some idea of the complexities involved in updating the actual Symmetrix
configuration). Again the PSE Lab must be involved to complete any failed online
configuration change.

72
 Advanced Configuration Training 2007

Online Configuration Change Scripts

Testing Your Knowledge

Question: In the earlier example we failed at the
Do_Fast_IML step. The EEPROM had already been loaded
with the new bin. So why can’t I just issue a $FD,CE?

Answer: NEVER consider doing this. You need to consider
the complexity of “changing” the Symmetrix configuration.
– We are attempting to “change” an active running configuration
online without impact to the hosts or the loss of a single I/O. First
this means that all directors need to be in the correct “mode” to be
IML’ed and instantaneously implement the “new” configuration.
– The required changes need to be “masked through” so that the
director IML process will be successful – the correct “IML
Parameters” MUST be loaded into memory by the script. Finally the
scripted Do_Fast_IML is a fully “Synchronized FAST IML” since ALL
directors must implement the new configuration simultaneously.
EMC CONFIDENTIAL—INTERNAL USE ONLY 73

This is again a frequently asked question.

73
 Advanced Configuration Training 2007

Online Configuration Change Scripts

Best Practice / Summary

Always call the PSE Lab when a script fails. Do nothing
unless instructed by the PSE Lab.

Never try to “Resume from failed step” and re-try the step or
skip the step (do not try to bypass a “red box”).

Never try to manually issue a Fast_IML or other command
via Inlines (do not try to “manually” complete a script).

Never “Abort” a failed script and try to restart from the
beginning (always obtain explicit PSE Lab permission).

Never delete SymmWin user directory files from the Service
Processor (unless you are certain that no CE or customer
SIL or automated scripts are running or incomplete).

EMC CONFIDENTIAL—INTERNAL USE ONLY 74

As above.

74
 Advanced Configuration Training 2007

Online Configuration Change Scripts

Other Issues – The “Not Ready” Bit

You are familiar with the setting the “Not Ready” bit in the
bin file at 5265 to 5671 for the online VTOC stage of
OFFLINE configuration changes.

This feature is NOT explained here - refer to Primus
1.0.48478608.2504991.

However, drives & volumes can also be set “Flag NR” at
567x and 5771 – due to the action of SymmWin scripts.

This may be due to the action of Permanent Member
Sparing at 5671 and 5771.

This may be due to a “failed” ONLINE upgrade script
(Volume Add) at 5670, 5671 & 5771.

EMC CONFIDENTIAL—INTERNAL USE ONLY 75

The NR bit for OFFLINE configuration changes is not discussed here. The following
slides are ONLY to highlight that spare drives may be set NR on the DA & in the bin
at 5x71 AND that SymmWin Engineering use this code / script feature to perform the
ONLINE VTOC during drive / volume adds at 567x and 577x.

75
 Advanced Configuration Training 2007

Online Configuration Change Scripts

The “Not Ready” Bit (cont)

Situation 1: Permanent Member Sparing at 5671 and 5771.
– After a successful Permanent Member “swap” the original “bad”
data drive will now be set “NR” on the DF (since it is a bad drive)
AND it is also configured in the bin file as a “Not Ready” drive.

Use the Disk Map

“Show: Not Ready”
feature to display any
NR Spares (OR drives
with “Not Ready” data
volumes).

EMC CONFIDENTIAL—INTERNAL USE ONLY 76

As above.

76
 Advanced Configuration Training 2007

Online Configuration Change Scripts

The “Not Ready” Bit (cont)

NOTE that this “Not Ready” state in the bin file of Dynamic
Spare(s) does NOT cause any issues during any online (or
offline) configuration changes.

The bin file will save cleanly and all changes are correctly
“masked” through (at latest 5x71 - refer to emc133505).

You cannot “clear” the “Not Ready” state of a dynamic
spare via a configuration change. The bin file can be
modified but it cannot be loaded online – the simulation &
actual script will fail.

You must use the SymmWin Disk Maintenance Utilities,
Replace Spare drive script (refer to Primus emc150287).

EMC CONFIDENTIAL—INTERNAL USE ONLY 77

As above. Refer to Primus emc133505 for an issue at earlier 5671 & 5771 where a
successful Permanent Member swap (or the presence of a “Not Ready” spare) would
cause an IML failure during a subsequent online volume add.
Aside: The question has been asked “How do I reverse a Permanent Member
Sparing swap?” It is relatively straight forward. You need to use the SymmWin “QA
Tools” “Global Sparing: Replace Disk Drive” script. This is effectively a Permanent
Member Sparing “manual” swap. Assuming that the earlier Permanent Sparing swap
has been completed successfully the original defective drive has been physically
replaced and it is now a “good” spare in the System bin file. Unfortunately the Global
Sparing: Replace Disk Drive script will look at all “Ready” hot spares as candidates
for a Permanent Member swap so you may need to reduce its choices to get the
exact “reverse swap” you require. First use the QA Tools “Set/Reset Not Ready Bit
on Spare Drives” script to set all other hot spares “Not Ready” on the DF and “Not
Ready” in the bin. This build an upgrade bin but doesn’t do the bin load. There is NO
need to load the bin built by this particular script (note the issues detailed in
emc150287). Once all other spares are “Not Ready” on the DF’s you can do the
manual “Global Sparing: Replace Disk Drive” swap back to what you want. Once the
swap script completes successfully please ensure that all drives are “Ready” on the
Disk Map of the System bin). Then re-run the “Set/Reset Not Ready Bit on Spare
Drives” script to make sure all spares are made available on the DF’s again.

77
 Advanced Configuration Training 2007

Online Configuration Change Scripts

The “Not Ready” Bit (cont)

Situation 2: Online Volume Add at 567x and 5771.
– During ALL online volume adds for DMX the Add_Volumes sub-
procedure will set the “Not Ready” bit for all new devices (at step
Drvup_Set_Not_Ready_Bit).
– This flag is set in the IMPL.BIN file by SymmWin and this bin is
loaded prior to the online VTOC steps.
– Once the online VTOC steps have been successfully completed the
step LookFor_Nr_Bit checks for the presence of the NR bit in the
System bin. If present the Add_Volumes script load another bin file
to clear the “Not Ready” bit.
– This script “mechanism” is NOT visible in the OnlUpg.log and is not
a concern UNLESS the Add_Volumes script fails and this additional
scripted bin load has NOT been performed.

EMC CONFIDENTIAL—INTERNAL USE ONLY 78

As above.

78
 Advanced Configuration Training 2007

Online Configuration Change Scripts

The “Not Ready” Bit (cont)

In some cases the Add_Volumes script may be interrupted
(typically the online VTOC steps fail and the VTOC has
been completed “manually”).
– If the script is then “aborted” (manual clean up performed) the “Not
Ready” bit will remain set in the System bin.

No CE or SIL configuration changes will now be allowed
(step Verify_No_NrBit_Set will always fail).

The Not Ready bit indicates an earlier script failure. Contact
the PSE Lab or your local RTS before taking any action.
– The presence of the “Not Ready” bit on a customer data volume is
NOT masked through and will prevent the bin file being saved.
– This will cause a DU issue at the next power cycle or full offline IML.
EMC CONFIDENTIAL—INTERNAL USE ONLY 79

As above.

79
 Advanced Configuration Training 2007

Online Configuration Change Scripts

The “Not Ready” Bit (cont)
To confirm this issue
resave the System bin file:

To determine which volumes are affected examine the Volumes Map

(look for “black” volumes – these should be the newly added devices):

Again, this can also be seen on the

Disk Map with Show: Not Ready:

EMC CONFIDENTIAL—INTERNAL USE ONLY 80

As above.

80
 Advanced Configuration Training 2007

Online Configuration Change Scripts

The “Not Ready” Bit (cont)

The System bin file WILL need to be corrected.
With PSE Lab / RTS approval toggle the
NR data volumes to “Ready” on the
SymmWin Volumes Map and load this bin
file ONLINE (Change_Global_Flags script).

The NR data volumes

(entire disk) can ALSO
be toggled on the
SymmWin Disk Map.

EMC CONFIDENTIAL—INTERNAL USE ONLY 81

As above.

81
 Advanced Configuration Training 2007

Online Configuration Change Scripts

ProcStep.log
Remember: Always call the PSE Lab when a configuration
change script fails. Do nothing unless instructed by the PSE Lab.
The following advanced information is provided for the benefit of
Senior CS staff / RxS’s and is to be used with all due caution.
To display all steps in the current configuration change script. Login
to SymmWin as “PSE” or higher. Use the “Recover Failed Step
Procedure”, “Resume from Failed Step” and “Show Proc Step”. This
has been generated on my CE laptop using an online simulation.

EMC CONFIDENTIAL—INTERNAL USE ONLY 82

Instructors: The following slides are OPTIONAL and intended ONLY for more
advanced students (Senior CE’s / RTS’s).
The above information relates to resuming from a failed step.
Again we DO NOT expect CE’s to ever attempt to do this on a “live” box
(UNLESS specifically directed to do so by the PSE Lab or local RTS).
BUT the information is useful if they are asked by the PSE Lab or their RTS to
resume the script AND generate / view the “ProcsStep.log” file. As the name
suggests – this file lists all the steps in the current configuration change script. In
some Primus Solutions you are asked to look for certain step names or numbers in
the ProcStep file.
The above information can also be applied to online simulations. In some instances –
to determine exposure to certain “bugs” (again detailed in Primus) - it may be
necessary to determine if a certain step will be executed by the planned configuration
change script. Finally generating the SAME steps in the change script on your CE
laptop from the ORG & UPG bins can also be very useful for RCA’s.

82
 Advanced Configuration Training 2007

Online Configuration Change Scripts

Example of ProcStep.log

The resulting log file is displayed

immediately during an active change (and
it is stored in the SymmWin logs directory).

EMC CONFIDENTIAL—INTERNAL USE ONLY 83

The “procstep.log” has now been generated for this failed activity and it can be
viewed by the Log Browser (it is stored in the SymmWin logs directory).

83
 Advanced Configuration Training 2007

Online Configuration Change Scripts

C:\...\SymmWin\help\symplstep.chm
You can view the available script “Step Documentation”. This gives a
brief functional overview of each scripted step (you can also go to the
SymmWin HELP directory and just open the symplstep.chm file).

EMC CONFIDENTIAL—INTERNAL USE ONLY 84

Instructors:
Again this is only for advanced users / attendee’s only:
The “symplstep.chm” help file is available with every current SymmWin release. This
file is locate in the C:\SymmWin HELP directory on the Service Processor AND CE
laptop.
We again DO NOT expect you to make ANY script recovery decisions based on
this advice.
We are only providing this information to assist you in UNDERSTANDING why a script
might fail.
For example, note all of the Avoid Corruption checks that are applied during an actual
configuration change activity. If performing an RCA for a “red box” at one of these
check steps you can use this documentation (in addition to Primus) to determine
precisely what condition the script was testing for.

84
 Advanced Configuration Training 2007

Online Configuration Change Scripts

CCPI & RCA
At later Enginuity 5x7x we have a procslog.log file. This details
ALL CE or SIL instigated scripts and summarizes when they were
instigated AND whether they were successful (i.e. “Script History”).

All “automated” scripts such

as SYR, Optimizer or Global
Sparing AND any customer
SIL script will be listed as an
“Unattended” procedure.
Everything else will be a CE
run activity. “Real runs” &
online simulations are listed.
Note that ALL script failures
(and attempts to “Resume
from Failed Step”) will also
be shown here.
EMC CONFIDENTIAL—INTERNAL USE ONLY 85

Finally we cannot leave a discussion on configuration change scripts and script

failures without mentioning Root Cause Analysis (RCA) or Configuration Change
Process Improvement (CCPI). While the LOGALL.log SILCONFIG.log & ONLUPG.log
are well known as places to obtain information you may also find the above NEW log
file very useful as well…..

85
 Advanced Configuration Training 2007

Online Configuration Change Scripts

CCPI & RCA (cont)
Finally please note that at later Enginuity ALL script failures that
generate a “red box” will automatically create a zipped log file.

These script failure (i.e. red box) generated zip

files (including log & bin files) are stored as
"zipped_files_date_time_script-name.zip" in the
C:\EMC\<serial number>\Log_Zip directory at
567x and O:\EMC\<serial number>\Public\User
directory at 5771. Refer to emc126449 for details.

All archived logall.log files can be found in

the C:\EMC\Archive\SymmWin directory (at
567x) and the O:\EMC\Archive\SymmWin
directory (at 5771).

EMC CONFIDENTIAL—INTERNAL USE ONLY 86

Again extremely useful for Root Cause Analysis (RCA) or Configuration Change
Process Improvement (CCPI).

86
Advanced Configuration Training 2007

EMC CONFIDENTIAL—INTERNAL USE ONLY 87

87
 Advanced Configuration Training 2007

Configuration Changes via

ECC / SYMCLI

Troubleshooting on the Symmetrix

Service Processor (SP)

EMC CONFIDENTIAL—INTERNAL USE ONLY 88

Instructors: Note that this presentation is OPTIONAL. It is aimed at RTS, RSS

and Senior CE’s only. Only present this topic if there is appropriate time &
interest (the average CE will NOT find this useful).

Note that as you go through this presentation ECC GUI configuration changes or
SYMCLI (symconfigure) change sessions, etc, ALL refer generically to configuration
changes performed from the host via the Symmetrix Interface Layer (SIL).

88
 Advanced Configuration Training 2007

Configuration Changes via ECC / SYMCLI

Introduction

The Solutions Enabler CLI Configuration Change
Component and Symmetrix Management Console (SMC)
are all designed to allow you to perform online configuration
changes via software.

The use of these software tools is taught elsewhere but
troubleshooting is typically NOT covered.

Any issues will continue to be forwarded to the Solution
Support Centre (SSC) and PSE Lab however the following
introduction is to assist you in understanding what
(currently) happens on the Service Processor (SP).

The ECC GUI performs changes via the SP in the same
way as the SYMCLI so the troubleshooting is the same.
EMC CONFIDENTIAL—INTERNAL USE ONLY 89

From a Service Processor perspective the ECC GUI performs configuration

changes in the same way as the SYMCLI.
However, ECC does not use an explicit symconfigure –file command file. The GUI
will correctly parse the requested changes and forwards this information to the SP.
GUI implemented changes avoid syntax / typographical errors. That is, each change
option has a “drop down” window with only specific legal choices.
These optional choices are strung together to form a “pseudo” command file that
generates the change request submitted to the SP.
The changes can be viewed before being committed, however, no physical
command file is generated or archived - to do this the ECC Data Collection policy
must be correctly set to log ALL host configuration changes.
Similarly the ECC command history file needs to be set up to track the changes
implemented by the GUI otherwise host troubleshooting may be much more difficult.
Note also that the GUI does not have separate “preview”, “prepare” and “commit”
stages like the CLI. Once the pseudo command file is constructed the only option is
to “commit” the change.

89
 Advanced Configuration Training 2007

Configuration Changes via ECC / SYMCLI

1. The ECC/SYMCLI instructions are entered via the Host Operating System.
2. These commands are linked to SYMAPI Library Function calls.
3. These library functions generate the low level SCSI commands (syscalls) sent
to the Symmetrix via the SIL over the front-end channel directors.
4. These syscalls are then forwarded to the Symmetrix Service Processor (SP).
5. The syscalls build the “new” bin file and invoke the scripted steps (SYMPL or
Delphi) that perform the actual configuration change on the Symmetrix.
EMC CONFIDENTIAL—INTERNAL USE ONLY 90

As above. The syscalls generated by the SYMAPI at the host must reach the
Service Processor to create the new bin file.
Once this new bin file has been successfully built by SymmWin it then it become the
responsibility of the SymmWin script step engine to apply the change.
This original flow has NOT changed (originally true at Enginuity 5x66 and above, still
true at 5x71).
Again we do not discussed ECC or Solutions Enabler errors or software
issues in this presentation. We do not investigate actual “red box” script
failures, These issues / areas remain the responsibility of the Solution
Support Centre (SSE’s) and PSE Lab (PSE’s) respectively.
We are focused on what happens on the Service Processor.

90
 Advanced Configuration Training 2007

Configuration Changes via ECC / SYMCLI

The 3 Stages of a SIL Configuration Change
1. Preview:

This does a symconfigure verify. Checks for communication
and compatible software versions (i.e. Solutions Enabler,
SYMAPI, Enginuity & SymmWin). Looks for the Config Lock
(15). Preview checks the syntax of the command line /
command file before sending the change request to the SP.
– This request is the DSSIL.DAT file generated on the Service
Processor and located in the C:\SymmWin\UCODE or
O:\SymmWin\SCRIPTS directory.

SymmWin verifies the correctness of the requested change
by building and successfully saving the “new” bin file.
– The “new” bin (IMPLSIL.BIN) is built and saved on the SP but NO
actual scripts are run on the Symmetrix.

EMC CONFIDENTIAL—INTERNAL USE ONLY 91

The “preview” stage checks syntax and basic command file legality (at the host
level) before submitting the change request to the SP. If SymmWin successfully
builds and saves the new bin file based on the submitted change request then this
stage has successfully “passed”.

91
 Advanced Configuration Training 2007

Configuration Changes via ECC / SYMCLI

The 3 Stages of a SIL Configuration Change (cont)
2. Prepare:

This does a preview and then verifies the appropriateness
of loading a new configuration at this time.
– This stage runs the “CheckEnv” script (~17 steps). This script looks
for generic “box” issues that might prevent a configuration change
being performed (e.g. runs the EMC Script Regulator, verifies no
microcode mismatches, looks for invoked hot spare, etc).
3. Commit:

Does both the preview and prepare stages before
“activating” (i.e. applying) the requested configuration
change.
– The IMPLSIL.BIN is passed to the SymmWin step engine as
IMPL.BIN. An OnlUpg plan is built and the step bin(s) are loaded.
EMC CONFIDENTIAL—INTERNAL USE ONLY 92

The “prepare” does another preview and then runs the CheckEnv script on the
attached Symmetrix.
The invoked CheckEnv script is specifically written for SIL changes to “generically”
confirm whether an online configuration change can be performed at this time.
Basically it resaves the bin files, runs the EMC Script Regulator, checks for
matching emulation on the SP & directors, verifies no “DD” directors, checks for
invoked hot spares (however this restriction has changed at 5671.45 and above) &
checks for directors in “upgrade mode”, etc.
The CheckEnv is like a “Healthcheck” prior to ANY configuration change.
The actual configuration change script invoked at the “commit” stage will still have
its own additional scripted checks.
The “commit” does yet another prepare and then renames the “new” bin file
(IMPLSIL.BIN) as IMPL.BIN.
The new IMPL.BIN is then handed over to the SymmWin step engine for
implementation.
SymmWin (depending on the Enginuity level) will build one or more step bins (as it
would with any submitted bin file) and load these in sequence.

92
 Advanced Configuration Training 2007

Troubleshooting SIL Configuration Changes

Where Do I Start?

The best place to start is in the SP SymmWin LOGS
directory and examine the SILCONFIG.LOG.
– This shows the SIL request received from the host, the stage that
the configuration change session has reached and the steps
executed in the current script. All related errors are also shown.
– Note that script and Symmetrix error messages will also be in the
SymmWin LOGALL.LOG files. The PROCSLOG.LOG (if available)
is also a good activity summary.

From the host examine the SYMAPI.LOG.
– This details the request sent to the Symmetrix and Symmetrix
response to this request plus all script & API error messages.
– For UNIX hosts look in the “/var/symapi/logs/symapi-yyyymmdd.log”
files and for Windows hosts refer to the “c:\program
files\emc\symapi\log\symapi-yyyymmdd.log” files.
EMC CONFIDENTIAL—INTERNAL USE ONLY 93

To determine where the problem lies start with the SILCONFIG.LO* files.
These files are on the SP and therefore typically available for inspection by
the CE or RTS. These files will list the SIL requests received from the host
and the response from the Symmetrix, it will show the scripted steps that
have been executed. This should give you a good idea if the problem is on
the host side (in ECC or Solutions Enabler) OR whether you have an issue
with SymmWin or the API services on the Service Processor.
Note that the SYMAPI.LOG on the host contains everything issued from this
host and all the responses from the Symmetrix.
Note that only SIL invoked configuration changes are listed in the
SILCONFIG.LOG (other SYMCLI commands are NOT logged here). CE
applied configuration changes (via the SymmWin Procedures menu) DO
NOT get logged in the SILCONFIG.LOG.
ALL scripts, CE or SIL, or summarized in the PROCSLOG.LOG (available at
later Enginuity). This is a good summary (all SIL scripts are listed as
“UNATTENDED” – refer to the “Introduction to Scripts” presentation).
The LOGALL.LOG is also useful to see what scripts have been run (CE or
SIL) on the Symmetrix. NOTE that the “preview” phase does NOT invoke
any scripts so there will be no evidence of this stage in the LOGALL.LOG.

93
 Advanced Configuration Training 2007

Troubleshooting SIL Configuration Changes

What can be examined on the SP?

Preview takes the bin from ‘System’ and applies the
changes in the DSSIL.DAT file to builds the IMPLSIL.BIN
file in the SymmWin USER directory.
– Backup copies of this bin file (e.g. IMPLSIL.BI*) are created by each
“stage” of a symconfigure session (saved in the USER directory).

Commit will rename the current IMPL.BIN in the USER).
directory as IMPLORG.BIN and rename IMPLSIL.BIN as
IMPL.BIN and performs the change.
– The invoked SIL script is identical to the CE implemented script,
however, all CE user interaction / acknowledgement has been
removed (i.e. SIL_REQUEST set to YES in the script structure
“common variables” section).

EMC CONFIDENTIAL—INTERNAL USE ONLY 94

As above.

94
 Advanced Configuration Training 2007

Troubleshooting SIL Configuration Changes

What can be examined on the SP? (cont)

Remember the IMPLSIL.BIN is like any other bin file and it
can be copied, viewed, compared or simulated.

The latest IMPLSIL.BIN will have been generated from the
latest DSSIL.DAT (i.e. applied to the running “System” bin).
There will always be multiple IMPLSIL.BIN files in the USER
directory IMPLSIL.BI1, IMPLSIL.BI2, etc.

You can use the Windows Explorer & time / date stamp for
the DSSIL.DA*, IMPLSIL.BI*, IMPLORG.BI*, IMPL.BI* files
to determine if a bin was actually loaded to the Symmetrix.

The E7,D,A command will show the time / date stamp of the
last IMPL.BIN to be loaded into the directors (this time
stamp is the file creation date of the IMPL.BIN from the SP).
EMC CONFIDENTIAL—INTERNAL USE ONLY 95

As above. As example follows which make the above clearer.

95
 Advanced Configuration Training 2007

Troubleshooting SIL Configuration Changes

What can be examined on the SP? (cont)
• Again at the preview stage the DSSIL.DAT file is generated
and it can also be examined. (Remember at later Enginuity
levels backup copies of this file are also created (e.g.
DSSIL.DA*). The steps are outlines below:
Login as PCGroup & Load the DSSIL.DAT file then Edit this request
“Create SIL Request”

EMC CONFIDENTIAL—INTERNAL USE ONLY 96

An example follows shortly which will make this clearer. Note that at Enginuity 5772
you need to login to SymmWin as “ENG/Engineering Services”. Go to “Tools” & “SIL
Tools”. This should open on the “SIL Test” tab. However, you cannot run any of the
optional functions on your CE laptop i.e. “No service now running” error on a CE
laptop. You can use this menu and “Edit Submit Request” on the Service Processor
to examine DSSIL.DAT.

96
 Advanced Configuration Training 2007

Troubleshooting SIL Configuration Changes

Example

In this example the customer has requested the creation of
new meta volumes. According to the SYMAPI log (see next
slide) the change was successfully submitted to the
Symmetrix but the meta’s were NOT created.

A SR was logged with the Solution Support Centre. The
SSE confirmed that there are no software errors and
escalated this to the PSE Lab (assuming a script failure).

The PSE confirmed that there were NO incomplete scripts
(i.e. nothing to “Resume from Failed Step”). No errors were
logged on the Symmetrix. All “Health Checks” passed.

However, the requested meta volumes had not been
created (confirmed via the host and from the Symmetrix).
EMC CONFIDENTIAL—INTERNAL USE ONLY 97

As above.

97
 Advanced Configuration Training 2007

Troubleshooting SIL Configuration Changes

Example SYMAPI.LOG excerpt
The SYMAPI.LOG excerpt shows the previous successful SIL change:
03/05/2007 15:05:09.009 3180 5096 Establishing session with Local cfg srvr (000187721531).
03/05/2007 15:07:47.117 3180 5140 STARTING a 'WRITE_DISABLE' control operation.
.
03/05/2007 15:08:17.180 3180 4972 {
03/05/2007 15:08:17.695 3180 4972 unmap dev 027 from dir 10B:0, lun=01C; (generated)
03/05/2007 15:08:17.695 3180 4972 unmap dev 027 from dir 10B:0;
.
03/05/2007 15:08:17.727 3180 4972 unmap dev 019 from dir 8A:1, lun=019; (generated)
03/05/2007 15:08:17.727 3180 4972 unmap dev 019 from dir 8A:1;
03/05/2007 15:08:17.727 3180 4972 }
03/05/2007 15:08:21.414 3180 4972 Submitting configuration changes................. Submitted.
03/05/2007 15:08:35.789 3180 5968 Validating configuration changes................. Validated
03/05/2007 15:08:37.680 3180 5240 VALIDATE......................................... Done.
03/05/2007 15:08:48.023 3180 4264 Initiating PREPARE of configuration changes...... Queued.
03/05/2007 15:09:22.164 3180 4264 PREPARE requesting required resources............ Obtained.
03/05/2007 15:09:23.804 3180 5704 Step 006 of 017 steps............................ Executing.
03/05/2007 15:09:45.163 3180 5832 Step 014 of 017 steps............................ Executing.
03/05/2007 15:10:06.570 3180 5680 PREPARE.......................................... Done.
03/05/2007 15:10:38.226 3180 4512 Initiating COMMIT of configuration changes....... Queued.
03/05/2007 15:11:09.444 3180 4512 COMMIT requesting required resources............. Obtained.
03/05/2007 15:11:11.069 3180 4984 Step 004 of 046 steps............................ Executing.
.
.

EMC CONFIDENTIAL—INTERNAL USE ONLY 98

Note the configuration change “stages” as detailed earlier. This change was invoked
via the ECC GUI, the preview stage is the “VALIDATE” step shown above. The 17
step CheckENV script was successfully run during the PREPARE stage. Then the
change is immediately committed - in this case a successful unmap (SDDR) script
of 59 steps.

98
 Advanced Configuration Training 2007

Troubleshooting SIL Configuration Changes

Example SYMAPI.LOG excerpt (cont)
And this excerpt shows the subsequent unsuccessful SIL change:
continued from last slide…..
.
03/05/2007 15:14:09.146 3180 4680 Step 050 of 059 steps............................ Executing.
03/05/2007 15:14:30.005 3180 4408 Step 059 of 059 steps............................ Executing.
03/05/2007 15:14:53.099 3180 5876 COMMIT........................................... Done.
03/05/2007 15:15:33.364 3180 4976 Terminating session with configuration server.... Done.
.
.
03/05/2007 17:36:59.355 3180 4336 EMC:ECC iCfgChgSessionStart Starting a local CfgChg session for SID
000187721531, symapi V6.2-733 (1.0) ucode 5670
03/05/2007 17:37:02.777 3180 4336 Establishing session with Local cfg srvr (000187721531)
03/05/2007 17:45:05.946 3180 6064 {
03/05/2007 17:45:05.946 3180 6064 form meta from dev 1F3 config=Striped, stripe_size=1920;
03/05/2007 17:45:05.946 3180 6064 form meta from dev 1FC config=Striped, stripe_size=1920;
.
03/05/2007 17:45:06.024 3180 6064 add dev A32 to meta A2A, protect_data=FALSE;
03/05/2007 17:45:06.024 3180 6064 }
03/05/2007 17:50:07.663 3180 6064 Submitting configuration changes................... Submitted.
03/05/2007 17:50:21.178 3180 4812 Validating configuration changes................... Validated
03/05/2007 17:50:22.616 3180 4380 VALIDATE........................................... Done.
.
.

Question: So what do we do now?

EMC CONFIDENTIAL—INTERNAL USE ONLY 99

Note that the ECC GUI implements configuration changes immediately – this is
unlike the SYMCLI (symconfigure commands) where each stage can be run
individually.

99
 Advanced Configuration Training 2007

Troubleshooting SIL Configuration Changes

Example (cont)

You can check the E7,D,A screen to see the date stamp of
the IMPL.BIN (or “System” bin file).

You can check the date of the DSSIL.DAT, IMPLSIL.BIN &
IMPL.BIN files on the Service Processor.

In this case the “System” bin and IMPL.BIN and
IMPLSIL.BIN were identical (i.e. no meta volumes created).

The time stamps appear consistent with the DSSIL.DAT file.

We have therefore confirmed that a “meta” upgrade bin file
has NOT been built. If you examine the OnlUpg.log file it will
shows a valid FBA channel un-assign step (?).

Examine the SILCONFIG.LOG (see next slide).
EMC CONFIDENTIAL—INTERNAL USE ONLY 100

As above.

100
 Advanced Configuration Training 2007

Troubleshooting SIL Configuration Changes

Example SILCONFIG.LOG excerpt
The SILCONFIG.LOG shows the last successful SIL change:
[03/05/2007 15:58:08] <<<<<======= NEW SESSION =======>>>>>
[03/05/2007 15:58:08] Start a new session by "colsan301/ECC/3180", ID=0x10BD01
[03/05/2007 16:01:17] Submit - start
[03/05/2007 16:01:17] 0x1C request(s) in submit batch
[03/05/2007 16:01:17] 01 Front_end_remove (27,19,0,1C) ==>> OK Note the execution of
[03/05/2007
[03/05/2007
16:01:17]
16:01:17]
02 Front_end_remove (1A,19,0,1B) ==>> OK
03 Front_end_remove (19,19,0,19) ==>> OK
the CheckEnv script
. at the prepare stage.
[03/05/2007 16:01:19] 1C Front_end_remove (19,7,1,19) ==>> OK
[03/05/2007 16:01:19] Submit - end successfully
[03/05/2007 16:01:29] Check Validity - start
[03/05/2007 16:01:34] Check Validity - end successfully
[03/05/2007 16:01:46] Permit - start
[03/05/2007 16:02:13] SYMPL: step=1/17, stt=8, spec=0x00, cancel=0, msg="Enforce_Regulator_Process"
[03/05/2007 16:02:20] SYMPL: step=2/17, stt=8, spec=0x00, cancel=0, msg="Init_Default_Director“
.
[03/05/2007 16:02:57] SYMPL: step=17/17, stt=0, spec=0x00, cancel=0, msg="Verify_No_Upg_InProgress"
[03/05/2007 16:02:57] Script ended successfully
[03/05/2007 16:03:29] Commit - start
[03/05/2007 16:04:07] SYMPL: step=1/46, stt=8, spec=0x00, cancel=0, msg="Enforce_Regulator_Process"
[03/05/2007 16:04:07] SYMPL: step=2/46, stt=8, spec=0x00, cancel=0, msg="Proc_starting_message“
.

Note the additional step details

(compared to the SYMAPI.LOG).
EMC CONFIDENTIAL—INTERNAL USE ONLY 101

As above.

101
 Advanced Configuration Training 2007

Troubleshooting SIL Configuration Changes

Example SILCONFIG.LOG excerpt (cont)
This log does NOT show any unsuccessful SIL changes.
Continued... The last successful activity was an SDDR change.
.
[03/05/2007 16:05:16] SYMPL: step=45/46, stt=8, spec=0x00, cancel=0, msg="Set_UserNR_Del_Volumes"
[03/05/2007 16:05:25] SYMPL: step=46/46, stt=8, spec=0x00, cancel=0, msg="OnlUpg_Add_Procs"
[03/05/2007 16:05:26] SYMPL: step=1/59, stt=8, spec=0x00, cancel=0, msg="OnlUpg_Save_Plan"
[03/05/2007 16:05:27] SYMPL: step=2/59, stt=8, spec=0x00, cancel=0, msg="Init_Two_Configs"
.
[03/05/2007 16:07:27] SYMPL: step=58/59, stt=8, spec=0x00, cancel=0, msg="OnlUpg_Mark_Proc_Done"
[03/05/2007 16:07:33] SYMPL: step=59/59, stt=0, spec=0x00, cancel=0, msg="Proc_End_Msg"
[03/05/2007 16:07:34] Script ended successfully
[03/05/2007 16:07:34] Session end
[03/05/2007 16:07:41] <<<<<======= NEW SESSION =======>>>>>
[03/05/2007 16:07:41] Start a new session by "colsan301/ECC/3180", ID=0x10BD01
[03/05/2007 16:08:31] Cancel session - start: restart=0
[03/05/2007 16:08:31] Session end
[03/05/2007 18:25:57] <<<<<======= NEW SESSION =======>>>>>
[03/05/2007 18:25:57] Start a new session by "colsan301/ECC/3180", ID=0x10D701
[03/05/2007 18:26:53] Cancel session - start: restart=0
[03/05/2007 18:26:53] Session end

This log does not show a “Submit” for a meta configuration change.

Question: So what can we conclude?

EMC CONFIDENTIAL—INTERNAL USE ONLY 102

As above.

102
 Advanced Configuration Training 2007

Troubleshooting SIL Configuration Changes

Example - Conclusions

As this point it appears that the Service Processor never
received the request to create the new meta volumes.

Question: Can you confirm whether the Service Processor
received the change request and ignored it / failed to build a
“new” bin file?

Answer: Yes, you can. To confirm what SIL changes were
last received by the Service Processor you can open the
DSSIL.DAT file.
– You can also apply these changes to a copy of the “System” bin file
to confirm that a “new” bin file can be built.
– The newly create bin can then be examined & compared or
simulated (it can even be applied via a CE implemented change).

EMC CONFIDENTIAL—INTERNAL USE ONLY 103

We have concluded that the SYMAPI log is “incorrect” however you CAN take this
troubleshooting further to see if the Service Processor received the change request
and then failed to build the “change request file” OR whether the Service Processor
simply failed to apply the change request file to the “System” bin (failed to build an
IMPLSIL.BIN).

103
 Advanced Configuration Training 2007

Troubleshooting SIL Configuration Changes

Example – Conclusions (cont)

As described earlier we can login to SymmWin as PCGroup
and go to the “QA Tools”, “Create SIL submit request”.

Click on “Load Request” and specify the DSSIL.DAT file
then click “Edit Request”. Expand the fields and they show
the requested changes:

Compare this to the

last successful SIL
change request for an
unmap (from the
SILCONFIG.LOG):

EMC CONFIDENTIAL—INTERNAL USE ONLY 104

In this case it is unnecessary to take this further.

The current DSSIL.DAT is for the last / previously successful “UNMAP” or
“Front_end_remove” request submitted to the Symmetrix. Both the SYMAPI &
SILCONFIG.LOG confirm this match.
There is no evidence of the meta change request being received by the SP.
However, if the DSSIL.DAT had contained meta volume changes (and these were
NOT evident in the IMPL.BIN & System bin) then you could have tested applying
this change request file to the System bin and seeing if the resulting “new” bin file
would have saved cleanly.
In this case you would first need to load the DSSIL.DAT (SIL request) as shown
above. Then use “Load Config” (specify System or the name of the bin file) and then
click “Save New Config” (and specify the name of the “new” bin file).

104
 Advanced Configuration Training 2007

Troubleshooting SIL Configuration Changes

Example – Conclusions (cont)

So the DSSIL.DAT file content is consistent with the last
successful change shown in the SILCONFIG.LOG.

Now we can conclude that the meta change request never
reached the Service Processor (since the “new” DSSIL.DAT
file was never created).

This was most likely a communications problem. But the
SYMAPI log IS incorrect and this was escalated to SIL QA
Engineering.

Question: So what was the actual “fix” in this case?

Answer: The customer was simply asked to resubmit the
change and this time the meta creation was successful.

EMC CONFIDENTIAL—INTERNAL USE ONLY 105

The above problem was an actual Service Request dealt with in the Solution
Support Centre. Typically the customer would have just re-tried the failed SIL
change first (when the requested meta volumes were found to be missing) BUT in
this case the SYMAPI log implied that the change had been successful and this
prompted the client to open a SEV 1 case. However, the incident is still instructive
on where to look for information on the Service Processor. Again the SSC and PSE
Lab remain expert in their respective software & hardware areas but this case had
them stumped. The DSSIL.DAT file, the generated bin files and the SymmWin step
engine, remain a “config” area of expertise.

105
 Advanced Configuration Training 2007

SIL Configuration Changes

Testing Your Knowledge

Question: What other EMC S/W product or scripts will
perform configuration changes in this way (create a
DSSIL.DAT & IMPLSIL.BIN)?

Answer: Symmetrix Optimizer uses the Service Processor
SYMAPI library and SIL to generate a DSSIL.DAT &
IMPLSIL.BIN to perform the backend mirror “swaps”.

CE initiated “Online Expand Striped Meta” script generates
a DSSIL.DAT & IMPLSIL.BIN to expand the original STD
meta volume. The DSSIL.DAT in this case will contain the
STD meta head & additional meta member information and
stripe size entered by the CE via the script.

EMC CONFIDENTIAL—INTERNAL USE ONLY 106

As above.

106
Advanced Configuration Training 2007

EMC CONFIDENTIAL—INTERNAL USE ONLY 107

107
Advanced Configuration Training 2007

Web Search Activity

Investigating the SSR Web Page

EMC CONFIDENTIAL—INTERNAL USE ONLY 108

108
 Advanced Configuration Training 2007

The SSR Web Page

The Symmetrix Service Readiness Web Page was fully

revised in September 2004.

It is constantly being updated and it is due for a further
major revision later in 2007 so there is little value in
providing static screen shots.

The point of this activity is to get you to search out some of
the “less well known” features on our web page and to
gain some familiarity with the Site Index and Search Tool.

Feel free to use any tools you are comfortable with to
answer the following questions (see next slide).

EMC CONFIDENTIAL—INTERNAL USE ONLY 109

Instructors: The purpose of this exercise is to get your students to become more
familiar with the resources on the SSR Web Page. Please ensure that you confirm
the “answers” (i.e. perform the search yourself) before giving this exercise…(“hints”
are on the last slide).

109
 Advanced Configuration Training 2007

Searching the SSR Web Page

Please locate the following items:

1. Where do I look for published “Enginuity Packs available from
SSRWeb”? Where is the “E-Pack Request Database”?
2. Where do I find the “RPQ Tips and FAQs” PDF document?
3. How do I determine the correct Edit Director Port Flags for a Unisys
A-Series fabric connected to a DMX
4. Where is the latest Fire Drill Information located?
5. Can Celerra Control Station (CS) volumes be Raid-5 protected?
Where do I confirm this?
6. Where do I find the “Configuration Road Show” presentations?
7. Where can I find the latest Regulator version for 5772?
8. How do I contact the SSR Team? What phone number should I
use? What mailbox can I use for email questions?
EMC CONFIDENTIAL—INTERNAL USE ONLY 110

Instructors: I suggest limiting the time given for this activity (say 30 minutes max
before reviewing the results). Perhaps ask half the class to start at the bottom of the
list and the other half at the top to ensure that all items are covered in the available
time…

110
 Advanced Configuration Training 2007

Searching the SSR Web Page (cont)

Please locate the following items:

9. What is the procedure for an “EMC Certified Data Erasure”?
10. Where is the Site Map hyperlink and the Site Index page?
11. Where can I find more information on SRDF/A Link Resiliency?
12. Where can I find the bin file pre-loaded by Manufacturing on a
“Pending (new) Install”?
13. What is / where do I find the “disksizer.exe” program?
14. Where do I find information on setting the correct
OPEN_VMS_BASE_ID?
15. Who is “allowed” to review 5771 Configuration Changes? Where
do I confirm the current “phase” in the “Customer Service
Procedure - EMC Products Configuration Process”?

EMC CONFIDENTIAL—INTERNAL USE ONLY 111

As above.

111
 Advanced Configuration Training 2007

Searching the SSR Web Page (cont)

Please locate the following items:

16. What minimum 5771 Enginuity level is required for the “Symmetrix
Procedure Generator”?
17. Where do I find information on “5772 SRDF InterFamily
Connectivity Support”?
18. Where are the “CS Errata Sheets”?
19. Where are the “SCSI Code Release Notes”?
20. Where is the “Config Building / Review Standards” Page?
21. Bonus question: Where is the EMC Power Calculator tool located?

EMC CONFIDENTIAL—INTERNAL USE ONLY 112

As above.

112
 Advanced Configuration Training 2007

EMC CONFIDENTIAL—INTERNAL USE ONLY 113

• NOTE that there ARE various ways of locating these pages (not all paths will be listed here). Remember
also that these links can & will change without notice…
1. Go to the “Enginuity Links” pull down (or start from the “Microcode \ Enginuity Links” page).
2. Go to the “RPQ Tips & FAQ’s” pull down.
3. Go to the “Host Information” pull down and “UNISYS” html page or download the Unisys PDF document
(provided and supported by the Unisys TS2 team).
4. SSR Home Page.
5. Download the latest “EMC Products”, “Celerra”, “Configuring the Symmetrix for the Celerra File Server”
document. The answer is NO (only 2-MIR protection is qualified by NAS Engineering).
6. Go to the “Training Home” page.
7. SSR Home Page.
8. Go to the “Contacting the SR Group” page.
9. This question is on the “Total Clear” scripted based Data Erasure Tool. Use CCA5 and make sure that
the activity type selected is a “Data Erasure” for automatically routing.
10. The “Site Map” link is on the bottom of every page.
11. Go to the “SRDF Information” and “Features” page to download the PPT.
12. This is under “Config Tips” and is explained in the “New Install Early Notification and Configuration
Process” document and the subsequent links go directly to the Manufacturing Sites.
13. This old (but useful) program converts FBA cylinder counts into MB & GB or vice versa. It is hidden under
“Config Tips” and “Calculator Tool: For Symmetrix Logical Volume Conversions”.
14. Start at the “Host Information” homepage.
15. CSP-2029 on the “Configuration and Change Control Process Information” page.
16. Noted on the “EMC Procedure Generators” page.
17. “Products”, “Symmetrix DMX-3”, “5772” OR the “5772” link from the SSR Home Page.
18. Start from the “Microcode \ Enginuity Links” page (same for Release Notes).
19. Ditto.
20. Go to the “Config Tips” page.
21. On EMC Powerlink under “Tools”.
113
 Advanced Configuration Training 2007

EMC Support Solutions

Database or “Primus”

Introduction & Searching Techniques.

EMC CONFIDENTIAL—INTERNAL USE ONLY 114

This Primus presentation assumes some degree of familiarity with the EMC Solution
Support database (or “Primus”).
That is, a CE typically knows how to enter a specific “Solution ID” and how to enter
a “search string” HOWEVER the typically inconsistent search results are a mystery
as are the possible search “techniques” necessary to improve these search results.
The examples at the end of the presentation use CE/RTS specific issues to
demonstrate the discussed search tools.

114
 Advanced Configuration Training 2007

Primus
Introduction

Primus uses an “Associative Search Algorithm”.
– This will be very different to other Search Engines (e.g. Google) or
Knowledge Base’s (e.g. Microsoft KB) that you may be familiar with.
– With these applications you focus on the individual “keywords” that
are relevant to your search and enter these in a single string.
– With Primus you will (typically) get very poor or unexpected results if
you enter individually selected “key words”.
– There are NO Boolean expressions (e.g. AND, OR, NOT) in Primus
or wildcards (e.g. “*”,“~”,“&” or “+”) in the standard search feature.
The Primus Query tool addresses some of these “issues” (this is
discussed shortly).
– There are NO “Advanced Search” options for “all words”, “exact
phrase”, “date” or “location of the text on the page”, etc.

EMC CONFIDENTIAL—INTERNAL USE ONLY 115

Instructors: This slide highlights that Primus is VERY different to any other
search engine that your students are likely to be familiar with. Do not spent
too much time on the next 2 slides. These are just an introduction to Primus
terminology AND a description of how an “Associative Search Algorithm” works (for
completeness). Don’t try to go into details – Primus training is a separate CBT & 1
to 2 day class. This should be a 30-40 minute presentation.
Concentrate on the “Searching Techniques”. The presentation is structures to show
Primus search “features” and THEN examples using these features…..

115
 Advanced Configuration Training 2007

Primus
Introduction (cont)

Primus uses terms and statements.
– Primus will evaluate each individual “term” (or word) in the search
string. It will match against the entered word AND related words
found in the “Terms Dictionary” (an EMC defined table of
synonyms). Common words such as “and” and “the” are ignored.

Each individual statement is evaluated independently.
– Primus searches all solutions for these precisely matching or closely
matching statements (as per the synonym lookup).

All statements in the solutions are evaluated.
– If you add multiple statements Primus will evaluate each statement
separately (each additional statements must be assigned a specific
“role”. Primus will weight and add the value of each individual
statement found in all the matched solutions. The highest matching
solutions are returned.
EMC CONFIDENTIAL—INTERNAL USE ONLY 116

When you enter a search string at “?..” in Primus and hit <CR> every entered word
is evaluated. The words (or “terms”) are checked for possible synonyms or “related
terms” in a Primus supplied (and EMC modified) terms dictionary (in earlier
presentations this grouping was referred to as a “concept”).. This terms dictionary or
“View Term Relationships” feature allows you to enter, for example, the word “DMX”
and still match to the alternate terms “Direct Matrix Architecture, Symmetrix 6,
symm6, RMS, Leopard, Panther, Rhino, Scarecrow, DMX1000, DMX-1000, DMX
1000”, etc. Note that while you are encouraged to express the “question” or “issue”
as a fully formed thought (i.e. NOT just selected keywords) any common or “noise”
words will be disregarded or given little weight by the search engine.
The entire search string (that is everything entered on a single line) forms a
“statement”. The content of all solutions in the Knowledgebase are searched for
these matching statements. Again this includes the precisely entered “statement”
AND all statements formed by the above synonym substitutions from the terms
dictionary.
The solutions containing the statements that most closely match the specified
statement (or its synonyms) are then returned (listed on the “Results” pane). These
solutions are listed in their match strength order. Note that the minimum match
strength returned and maximum quantity of solutions returned are user configurable
(on the “Options” screen).
If you enter multiple separate statements each statement is evaluated and matched
separately (these combined results are tallied). Note that individual statements can
be assigned a different statement “type” or “role” and different roles carry a different
“weight” in the search engine algorithm (more on this shortly).

116
 Advanced Configuration Training 2007

Primus
What is an Associative Search Engine?

This means that as more information is gathered and added
to the search request it will dynamically build relationships
(associate) between this additional information and the
solutions it finds (strengthening some matches and
weakening others).
– Enter the search string as a fully formed thought. Do not try to “pick
out” or summarise the question or issue in only a few “key” words.
– A frequently asked question is whether Primus does an “AND” or an
“OR” with the additional statements – it is effectively BOTH.
– That is, any solution matching ANY individual statement will be
returned and ANY solution matching multiple statements will also be
returned BUT with a higher strength on the “match meter”.
– If adding additional statements the correct “roles” must be assigned.

EMC CONFIDENTIAL—INTERNAL USE ONLY 117

Again, Primus uses an “associative” rather than keyword search algorithm.

Primus will evaluate individual “terms” and search for related “terms” (synonyms).
Primus does this evaluation at the individual statement level. As you add multiple
statements Primus will evaluate each statement individually and THEN combine the
search results (i.e. weighting and adding the value of each matching statement).
Note that when you enter a search string you are encouraged to express your
“question” or “issue” as a fully formed thought e.g. “What are the rules for RAID and
DMX”, “Cannot see disks after reboot of AIX host”, etc. Do not try to “pick out” or
summarise the question into a few “key” words.
Q. Why do we suggest this? A. Because this is how Solution Authors are
encouraged / required to write their solutions (i.e. all GOAL or SYMPTOM
statements must be a fully formed thought or an understandable description of the
problem). A fully formed thought is much more “unique” than a collection of random
key words…..
Note that a frequently asked question is whether Primus does an “AND” or an “OR”
with the additional statements. It effectively does BOTH. Any solution matching an
individual statement will be returned, any solution matching multiple statement lines
will also be returned BUT with a higher value on the match meter.
Remember, the available statement roles are GOAL, SYMPTOM, FACT, CHANGE,
CAUSE, FIX and NOTE.

117
 Advanced Configuration Training 2007

Primus
Searching Techniques

The default “?..” statement role is weighted the same as a
statement role of Goal or Symptom.
– We are typically looking for a solution to a problem or trying to
answer a question, for example, our search string at “?..” may be
“How do I configure….?” or “Red box during simulation….”.
– A Goal or Symptom statement role will be weighted 4 times higher on
the match meter than a matching statement with a role of Fact,
Change, Cause or Fix (a statement role of Note is not searched).
– If adding additional statements and these are a “Fact” (for example,
Enginuity: 5771) or a “Change” (for example, NDU) make sure that
the correct statement “roles” are assigned (DO NOT add as “?..”).

ALWAYS enter a unique error code or error string or error
message (if this is available). Do not paraphrase an error.
EMC CONFIDENTIAL—INTERNAL USE ONLY 118

If you initially get poor results from your first search (e.g. no solutions returned, too
many solutions returned, low or “uniform” match meter ratings, seemingly unrelated
solutions). Then it is typically LESS productive to add multiple FACT and CHANGE
statements. The best approach is to first concentrate on the quality of the
GOAL or SYMPTOM statement.
Adding more statements to the search will increase the number of potential
solutions with matching statements that can be returned by the Primus search
engine. While the desired solution MAY contain the specified GOAL or SYMPTOM
and the additional FACT and CHANGE statement (which would raise it up on the
match meter) there is also a chance that the solution author has not (at this time)
included these statements in their solution and the weighted returns will therefore be
push the desired solution down on the match meter. For example, you have
encountered an issue at 5771.94.102 during an online volume add and enter these
additional FACT & CHANGE statements, however, the problem was first seen at
5771.92.99 during a customer SIL change…..the important detail here is the failure
message, NOT necessarily the customer’s environment……
Again concentrate of the accuracy / completeness of the GOAL or SYMPTOM
statement. The most successful approach is ALWAYS to enter an exact error
code OR “cut & paste” the exact error message (including any spelling or
grammatical errors seen in the message). An error string will be a unique
statement in the Primus Solution database and will always return the best
match results.

118
 Advanced Configuration Training 2007

Primus
Searching Techniques (cont)

Use the “Matching Statements” feature (F6).
– Once you enter a search string you can use “Matching Statements”
feature to see the statements that the Primus search engine has
matched (i.e. considers synonymous) to your entered string.
– You can select any listed statement that you consider closer to the
Goal or Symptom you are searching for (and “Click to add” or “Click
to replace” your original search string).
– Be careful when deciding to do this (you may get a stronger match -
but to the wrong solution).
– Otherwise if none of the “Matching Statements” are close to your
initial search string then this may indicate a poor quality initial
statement. Remember to always use an exact unique error message
(if available). Otherwise try re-phrasing your problem or question.

EMC CONFIDENTIAL—INTERNAL USE ONLY 119

Note that selecting a “better” matching statement can be counter productive. The
“match meter” will naturally be skewed towards solutions containing the newly
selected statement (since this already exists in the database).
While you may get fewer solutions returned OR a higher match on the meter it may
be for all the “wrong” solutions.
Therefore revising the initial search string (before using “Click to replace”) may be a
better choice OR trying some of the following techniques.

119
 Advanced Configuration Training 2007

Primus
Searching Techniques (cont)

You can use the “Also Found In” feature (F10).
– If “Matching Statements” returned statements that look close to the
Goal or Symptom you are searching for you can right click on the
statement and select “Also Found In”.
– This lists ALL of the Primus Solutions that the selected statement
appears currently appears in.
– This allows you to quickly view potentially matching solutions.
– Note that this is just a raw (unweighted) list of solutions.
– NO other entered statements are considered.

EMC CONFIDENTIAL—INTERNAL USE ONLY 120

Note that “Matching Statements” combined with “Also Found In” can be a very
effective “quick search” technique.
BUT remember that you are not using the full capability of the associative
search engine.
Using this technique you are limiting the “associative” database search to the
individual statement level and finding un-weighted solutions.

120
 Advanced Configuration Training 2007

Primus
Query Tools

The “Query” tab allows you to perform a database query on
all solutions in the Knowledgebase.
– This database query tool can be used to look for specific solution
“content” such as a key word or a quoted phrase in any statement.
– You can also interrogate the database for specific solution
“attributes” such as a specific Primus Author or associated
Engineering OPT number or solution creation date, etc.
– The associative search engine is not used.

You can also use “Query Constraints” to filter the results
returned by the associative search engine.
– The results from the associative search engine can be filtered to
show ONLY solutions that also contain certain “key words” and or
solutions by specific authors and or pertaining only to certain EMC
Product Types, etc. Any solution attribute can be used as a filter.
EMC CONFIDENTIAL—INTERNAL USE ONLY 121

Please note that a “Query” is literally a database query - the associative search
engine is not used.
However, one way of effectively combining these two methods of searching the
database is to use the associative search engine to first return all solutions with
statements that match the desired search string AND to “filter” these results to
eliminate any solutions NOT also containing the desired attribute. For example,
search all solutions that refer to PowerPath, will return solutions for both the
“Symmetrix” product AND “Clariion” product ranges. A “Query Constraint” can be
set up to eliminate any solutions that do not have “Symmetrix” listed as the “Product
Type” (examples follow)….

121
 Advanced Configuration Training 2007

Searching Techniques
Testing Your Knowledge

Start with your problem statement / description.
– In this example we want to perform a 5670-to-5671 NDU. One pre-
requisite is to change the number of LRU’s (nLRU) from 16 to 1.
– You are at 5670.99.90 and the online simulation for this
configuration change fails with the following SymmWin error:

Question: What should we enter as a search string?

EMC CONFIDENTIAL—INTERNAL USE ONLY 122

Use the following slides as a guided example / exercise.

NOTE that the intent here is more to highlight the previously discussed
“features”. This is NOT to say that the above sequence or technique is the
ONLY way to find the desired solution. As highlighted earlier, your search
results always depend on the quality of the written solution.

Instructors: You can (as an alternative to this example) simply open a

Primus Search window (go to
http://csexplorer.isus.emc.com/eservice/iviewcs/ui/eserver.asp) and perform
a “live” search using examples volunteered by the class. Otherwise look at
the above screenshot – what search string should we enter?

122
 Advanced Configuration Training 2007

Primus
Testing Your Knowledge (cont)

Concentrate on accurately describing the failure or

symptom. Enter a unique error code or string.
– This can sometimes be difficult. You may need to describe the
“problem” or the circumstances of the failure instead. In this case the
error appears too generic “One or More of The Defined Changes
Cannot be Done While Online!”. For example:

Note the 50 (maximum)

returned solutions & uniform
match meter strength.

Question: These results are “poor”. So what should we try?

EMC CONFIDENTIAL—INTERNAL USE ONLY 123

Remember that Primus can display a maximum of only 50 solutions. You

may have > 50 solutions matching your search string (i.e. 51 to 5001) –
there is no indication of the total number of “matching” solutions once you
exceed 50. These matching solutions are displayed in match strength order
(as per the “Match Meter” on the RH side of the “Results” pane).
In this case the match meter strength is uniformly high – however, all this
indicates is that the entered string is a matching “Symptom” in numerous
solutions. Therefore, no one solution of the 50 returned results seems any
more “likely” than any other - we have not provided sufficient information to
uniquely identify our issue.
Obviously (in a “real” search) you would start by scanning through the 50
listed solution titles to see if any looked “likely” to answer your question / fix
your problem…..

123
 Advanced Configuration Training 2007

Primus
Testing Your Knowledge (cont)

We can try Matching Statements and Also Found In.

– Matching Statements is a quick way to see all of the statements that
Primus has “associated” to your initially entered search string.
– You can also right click on any one of the listed statements to Add
or Replace your original search string BUT I suggest doing an Also
Found In. This allows you to quickly view all of the Primus Solutions
containing the selected “matching statement”:

Right click on each “likely”

Symptom statement in turn.

EMC CONFIDENTIAL—INTERNAL USE ONLY 124

Again if you are researching an issue with a “generic” problem description

i.e. bad performance, fibre loop issues, etc, (rather than a unique error code
or error message) then “Matching Statements” can be VERY useful to see
what the Primus associative search engine considers a close match to your
currently entered statement – i.e. to quickly see if you are “on the right
track”.
Again the “Also Found In” feature (in many cases) can save you a lot of time.
Rather than trying to refine your initial search criterion to reduce the number
of solutions found OR moving the right solution up the match meter (i.e.
changing the search string or adding additional statements, etc) the “Also
Found In” feature simply lists ALL solutions containing the precise statement
you select (remember that the returned solutions are “un-weighted”).
In the above example we have only 2 solutions in the entire Primus database
containing precisely the Symptom “Error msg: One or More of The Defined
Changes Cannot be Done While Online!”. Again don’t forget that the “Also
Found In” is like a database query – we are not using any synonyms or
“association”….

124
 Advanced Configuration Training 2007

Primus
Testing Your Knowledge (cont)

If we have had poor results with these techniques try

rewording the initial search string OR adding additional
statements (remember to assign the correct “roles”).
– For example, we are performing a simulation at Enginuity
5670.99.90. So we add Fact Enginuity: 5670.99.90 and Change
Online Configuration Change. Compare this to the earlier results
without the added statements. What do you notice?

Note that 50 solutions are still returned BUT the extra statements
have changed the weighting and order on the match meter.

EMC CONFIDENTIAL—INTERNAL USE ONLY 125

Note the blue background on the Fact & Change statement – this indicates
that this precise statement already exists in the Primus database. Fact
statements especially should only be entered using the EMC “standard”
format. For example, the code level is “Enginuity: 5670.99.90” NOT
“Microcode: 56709990” or “Code 5670-99-90”.
You can use the “Matching Statements” feature to check that ANY entered
Fact or Change statement meets EMC standards OR you can use the “Topic
Tree” tab to locate EMC standard product terminology.

125
 Advanced Configuration Training 2007

Primus
Testing Your Knowledge (cont)

AND what if we tried rewording the original search string?

– Remember: Always enter (if possible) the precise error message.
– For example, if we had entered the full SymmWin error message we
would have gotten the following result for the initial search:

TIP: Go to the SymmWin logall.log file. Search on “redbox” and use

“cut & paste” to capture the exact error message with all spelling &
grammatical errors. Never paraphrase an error message.

Question: So what is the “best” search technique to use?

EMC CONFIDENTIAL—INTERNAL USE ONLY 126

Question: What is the “best” search technique to use?

Answer: There is NO “best” search technique or ideal single method for
searching Primus.
The above example DOES show us how to locate emc150089 "Online
configuration change fails when deleting multiple lru_table entries (changing
count to 1) at 5670.99.90“. However, this example and the techniques above
work well because this solution IS correctly written following EMC
Knowledgebase “Best Practice”. While all current solutions are written with
these practices in mind (and we have Primus “Coaches” and Technical
Writers to encourage / ensure compliance) there are NO guarantees that the
solution you are searching for necessarily comply OR that it will contains the
same facts and details expressed in the same way as you have encountered
(and entered as your search criterion). The benefit of the associative search
engine is that it will match against similar statements BUT again you may
encounter (for example) an issue at Enginuity 5771 that was previously only
documented at 5671 OR the returned error message / code may change
between EMC Software Product releases, etc.
Therefore we show you a number of tools / techniques of which any one (or
a combination) may work better depending on your particular
circumstances…...

126
 Advanced Configuration Training 2007

Primus
Query Tool (example)

The Query tab allows you to perform a direct database

query on the content of the Knowledgebase.
– For example, you have read about DMX-3 directors using “HANS
pipe memory architecture” and want to know if there are any special
bin file or configurations requirements. Entering “HANS pipe” as the
search string would give poor results (i.e. >50 solutions all low on
the match meter). Why should you expect poor results from this?

These results are expectedly poor because the

entered string is poorly constructed.
However, an alternative to rewording your search
string would be to perform a “Keyword” query.

EMC CONFIDENTIAL—INTERNAL USE ONLY 127

This is another guided example.

Instructors: Again you can (as an alternative to this example) simply open a
Primus Search window (go to
http://csexplorer.isus.emc.com/eservice/iviewcs/ui/eserver.asp) and perform
a “live” query using examples volunteered by the class.
In this example the desired key words are “HANS pipe memory director”.
Other than “HANS”, none of the other individual words are unique in the
Primus database. However, there may be a solution written with precisely
this phrase so entering your original question as a fully formed thought such
as “What are the configuration requirements for HANS Pipe memory director
boards?” may well get good results (i.e. if a solution had been written with
this as a Goal). You can try this, see what results you get, then try Matching
Statements, Also Found In, etc, etc.
However, as above, the search string of just “HANS pipe” is poor. Again, the
word “HANS” is probably unique but the word “pipe” is NOT. You are also
searching with the default role of “?” so the search engine emphasis will be
on Goals & Symptoms. In the solutions where this memory architecture is
discussed the words “HANS pipe” may be in Fact or Cause or Fix
statements. So, in this case, a direct database query may be the most useful
search technique…..

127
 Advanced Configuration Training 2007

Primus
Query Tool (example cont)

Open the Query tab and enter a Keyword Match. Note the
default behaviour of matching ANY entered words. You can
also use quote marks for an exact string (three words max).
Note the 210 matches
for solutions containing
the words “HANS” or
“pipe” in any statement.

Note the single

solution in the entire
Primus database
containing the specific
phrase “HANS pipe”.
EMC CONFIDENTIAL—INTERNAL USE ONLY 128

Again this example is rather artificial. However, if you know that the solution
you are searching for WILL contain a unique phrase or keyword (which is not
necessarily a Symptom or Goal) then a “Keyword Match” database query
can be VERY useful.
Remember that a database query does NOT use any synonyms or
“association” - it will only match with the precisely entered keyword or
phrase.

128
 Advanced Configuration Training 2007

Primus
Tips On Using Query

A Keyword Match query can be very useful if you are given

an Enginuity fix or SymmWin patch number and need to
locate the associated Primus Solution.

Similarly if you are given an OPT number you can enter this
as the Bug Tracking Number and locate the associated
Primus Solution.

You may want to list all of the Primus Solutions written by a
particular RTS or SSR Author.

Remember this is a “raw” query tool and it does NOT utilise
the associative search engine. If the EXACT match is not
found in the database you will get “0 solutions found”.

EMC CONFIDENTIAL—INTERNAL USE ONLY 129

As above.

129
 Advanced Configuration Training 2007

Primus
Combining Search & Query (example)

You can set up a Query Constraint and use this as a filter

on your associative Search Engine results.
– For example, the PSE Lab advise you to review the Engineering
policy on drive “low level formatting”. You are told that the solution
was written by “Michael Lee”. This is all the information you have.
– If you search on say “What is Engineering policy towards low level
formatting in the field?” you get poor results. Why?

In this case the search string IS well formed but this

particular statement is NOT part of the required solution (and
the words in the search string are not sufficiently unique).

EMC CONFIDENTIAL—INTERNAL USE ONLY 130

Again this example is rather artificial BUT the intention here is simply to
demonstrate how to combine the Primus Search and Query features.
However, for example, you may vaguely recall an email or concall update on
an issue detailed in Primus AND only remember the author’s name OR some
other attribute of the solution such as it being written in recent weeks, etc. An
associative search on vaguely remembered Symptoms or Facts such as
“Drive problems” or “Red Box” or “Raid-5” or “Enginuity: 5671” would simply
match TOO many solutions….
Equally the above also demonstrates how a well formed search string (i.e. a
well expressed complete thought) WILL “fail” to match the required solution
IF the entered statement is NOT part of the solution (i.e. a similarly
expressed “thought” in a Goal or Symptom statement).

130
 Advanced Configuration Training 2007

Primus
Combining Search & Query (example cont)

We know at least one unique attribute of the required

solution and therefore we can use a Query Constraint.
– Go to the Query tab and select the solution attribute you wish to
update. Note also that you can choose to “Include” OR “Exclude” the
chosen solution attribute.
– A raw query on “Lee, Michael” as an Author returns >180 solutions.

EMC CONFIDENTIAL—INTERNAL USE ONLY 131

As above.

131
 Advanced Configuration Training 2007

Primus
Combining Search & Query (example cont)

Go back to the My Problem tab and the associative search

engine results and search again. This time we have an
“Unsaved Query Constraint” which is active.
– When we search again (same string) we get 3 solutions returned
which match BOTH the search criterion AND the specified query.

– Also note that multiple solution attributes can be entered as query

“constraints” and useful queries can be saved for later re-use.
EMC CONFIDENTIAL—INTERNAL USE ONLY 132

Again this example is rather artificial but it leads us to our next example
(Testing Your Knowledge)…

132
 Advanced Configuration Training 2007

Primus
Testing Your Knowledge

You can set up and save a Query Constraint for re-use.

– For example, say you are planning an Enginuity 5671 drive upgrade
and remember a recent issue with Raid-5 and online VTOC? This
may have been released as an ETA, however, this is all the detail
you recall. Not surprisingly a search on Raid-5 and VTOC and 5671
returns too many / poor results:

Question: Would a “filter” be appropriate in this case? What

sort of Query “filter” should we use?

EMC CONFIDENTIAL—INTERNAL USE ONLY 133

Instructors: Briefly discuss - would a “filter” be appropriate in this case and

what sort of Query “filter” should we use (what attributes should be included
or excluded)?

133
 Advanced Configuration Training 2007

Primus
Testing Your Knowledge (cont)

In this case we can set up and save a Query Constraint for

all ETA’s published in the last 3 months (Last 90 days) that
are Product type “Symmetrix Config” related:

On the Query tab we

select the Primus
Solution attributes to
be included in the
“filter”. Then we use
Query Options… to
name (and save)
this constraint for
later re-use.

EMC CONFIDENTIAL—INTERNAL USE ONLY 134

As above.

134
 Advanced Configuration Training 2007

Primus
Testing Your Knowledge (cont)

Finally these are the same “poor” associative search results

on 5671 and Raid-5 and VTOC filtered by the saved Query
Constraint “Recent ETA’s on Config”:

Be very careful when setting up or using Query “filters”. You

do not wish to “filter” out potential matches before trying all
other search techniques.

EMC CONFIDENTIAL—INTERNAL USE ONLY 135

Be very careful when setting up or using Query “filters”. Remember – your

search results depend on how well the required solution has been written.
You do not wish to “filter” out potential matches BEFORE exhausting all
other possible search techniques. Query and Query Constraints are best
used when you are 100% certain that the desired / required solution contains
the specified attribute…..

135
 Advanced Configuration Training 2007

Primus
Miscellaneous Features

If you find a solution that you may re-use in the future you
can add it to your “Favourites”. Remember to file these
“Favourites” under categories for ease of relocation.

Note that solutions are placed in different user partitions.
We have “Customer” (external), “Support” (anyone in
Customer Support) and “Restricted” (RxS and above).

Note that customer also have direct access to Primus
“Customer” partition solutions via PowerLink.

Note that solutions can also have “Statement Level
Security”. Certain SYMPTOMS or FIX statements may be
“hidden” from customers or CE’s.

EMC CONFIDENTIAL—INTERNAL USE ONLY 136

When you find a solution that you are likely to re-use (e.g. valuable
configuration advice, a known problem with complex workaround, etc)
ALWAYS save it to your “Favourites”. This can save a LOT of frustration
when trying to re-locate the solution again at a later date.
Note that we have “Solution Level” security or different “user partitions”
which dictate solution visibility. For example, customers can only see
solutions that are placed in the “Customer” partition, but CE’s can see BOTH
“Customer” AND “Support” partition solutions AND RxS can see ALL
“Restricted” AND “Support” AND “Customer” partition solutions. NOTE that
Primus will state that the specified solution number does NOT exist if you do
not have “permission” to view it.
Similarly the “user partitions” concept extends to individual statement level.
Often a “Customer” solution with have a FIX visible only to customers and
well as a FIX visible only to CS (CE’s, RxS, Technical Support). Again you
simply won’t see (or be able to email) any individual statements set to a
partition higher than your own.

136
 Advanced Configuration Training 2007

Searching Techniques
Summary
1. Always use as exact unique error code or message.
2. Focus on an accurate description of the problem or
question (reword if necessary).
3. If this fails to produce results use “Matching Statements”
and “Also Found In” to test your original search string.
4. Consider step 1 & 2 before using “Matching Statements”
and “Click to replace” or “Click to add”.
5. Consider steps 1, 2 & 3 before adding MORE statements.
6. Consider combining an associative search with a database
query if you are getting too many results.
7. A direct query is great if you know what you are looking for.
EMC CONFIDENTIAL—INTERNAL USE ONLY 137

•ALWAYS use a precise error message or code .

•Otherwise when searching you should enter your Goal or Problem as a
“fully formed thought”. If the search results are “poor” reword your question
and try again.
•You can use “Matching Statements” to see what the Primus associative
search engine thinks is a related statement. Remember that you can
enhance this search with a quick look at the “Also Found In” solutions.
•If using “Matching Statements” and you think that another statement in the
database is more accurate / detailed (or simply a “better” way of expressing
your goal or problem) you can add to or replace your original statement.
•Adding more statements (i.e. Facts) will typically increase the number of
solutions returned BUT it should improve the match meter results. However,
this assumes a similar or identical problem “environment”. You may be too
specific if you detail all the “Facts” pertinent to your current environment and
again get too many or poor matching results.
•Remember that you can also do a database query, say to locate the Primus
that goes with a specific Enginuity fix number OR combine the database
query tool with the associative search engine, e.g. limit all your associative
searching to Symmetrix or Symmetrix Config related solutions only.

137
Advanced Configuration Training 2007

Primus Powerlink Search

Searching the Support Solution Database

via Powerlink.

EMC CONFIDENTIAL—INTERNAL USE ONLY 138

138
 Advanced Configuration Training 2007

Primus Powerlink Search

Introduction

The Primus or “Support Solutions” section of the EMC
Knowledgebase can also be searched via Powerlink.

A search via Powerlink is completely independent of the
Primus associative search and therefore you DO NOT use
any of the search techniques previously discussed.

EMC CONFIDENTIAL—INTERNAL USE ONLY 139

NOTE: The Powerlink search engine is now recommended for anyone “new” to
Primus and the concept of the “associative” search engine. This interface searches
the SAME solution Knowledgebase as the “standard” Primus interface at
http://csexplorer.isus.emc.com/eservice/iviewcs/ui/eserver.asp.
You cannot create or edit solutions via Powerlink BUT the Powerlink search engine
will be a much more familiar search engine to all users.
Note that we now generically refers to ALL sources of support information as
being part of the EMC Knowledgebase so Primus is now more commonly
referred to as the “Support Solution” or solutions database while other
“information sources” such as the Customer Support Forums and
Engineering White Papers are all part of the total knowledgebase.

139
 Advanced Configuration Training 2007

Primus Powerlink Search

Features

A Powerlink search of “Support Solutions” allows:
– A free text search.
– Allows Boolean operators AND, OR and NOT.
– Quotes are used to search for an exact string.
– Parentheses can be used to form complicated queries.
– Wildcard queries are available using an asterisk.
– Searching within results is permitted.

EMC Employees & Partners have access to all Customer &
EMC Support level solutions.

Further improvements to this search tool will be provided
with subsequent eServices releases.
EMC CONFIDENTIAL—INTERNAL USE ONLY 140

As above.

140
 Advanced Configuration Training 2007

Powerlink Knowledgebase Search

Note that you need to

select “Support Solutions”
(Primus) not the
Powerlink “Search All”.
Once on this page note
the various “pull down”
options for narrowing your
search criterion.

EMC CONFIDENTIAL—INTERNAL USE ONLY 141

As above.

141
 Advanced Configuration Training 2007

Powerlink Knowledgebase Sample Search

Note that you can use Quotes,

Wildcards and Boolean
operators in your original
search string AND then search
within your Search Results.

EMC CONFIDENTIAL—INTERNAL USE ONLY 142

As above.

142
Advanced Configuration Training 2007

EMC CONFIDENTIAL—INTERNAL USE ONLY 143

143
Advanced Configuration Training 2007

Powerlink

EMC CONFIDENTIAL—INTERNAL USE ONLY 144

144
 Advanced Configuration Training 2007

Why Use Powerlink?

Powerlink is a ‘one stop shop’ with links to numerous

documents, tools and resources.

Intuitive directory structure using cascading menus
simplifies searches. Keyword searches also possible.

Powerlink can be customised ensuring information pertinent
to the users needs is given priority.

EMC CONFIDENTIAL—INTERNAL USE ONLY 145

Whilst these notes are intended to enable the presenter to deliver the presentation
without access to Powerlink it is nonetheless recommended that the presentation
should be supported by practical demonstrations of Powerlink. The emphasis is on
how to find information not where to find information. This will enable the student to
find documents and tools in the event of links moving within Powerlink.

145
 Advanced Configuration Training 2007

Who Uses Powerlink?

Powerlink is the common site used by Customers,

Partners and Employees

Powerlink

EMC CONFIDENTIAL—INTERNAL USE ONLY 146

Whilst Customers, partners and employees use Powerlink it is important to note that
each has a unique view. Not all resources available to employees will be available
to Customers and Partners. A major advantage of Powerlink as opposed to the
dispirit locations where information was previously to be found is that CS, when
familiar with using Powerlink, can advise customer on its usage.

146
 Advanced Configuration Training 2007

How do I use Powerlink?

Overview of Homepage

Customising preferences

Document Searches

Tools

EMC CONFIDENTIAL—INTERNAL USE ONLY 147

As above.

147
Advanced Configuration Training 2007

Powerlink Homepage Quick Tour

EMC CONFIDENTIAL—INTERNAL USE ONLY 148

148
 Advanced Configuration Training 2007

S
E
A
R
C
H
EMC CONFIDENTIAL—INTERNAL USE ONLY 149

There are 3 search methods

•Search
•Navigator
•Cascading Menus

149
 Advanced Configuration Training 2007

C
O
N
T
E
N
T
EMC CONFIDENTIAL—INTERNAL USE ONLY 150

Portlet Editing will be examined in later slides.

150
 Advanced Configuration Training 2007

P
O
R
T
L
E
T
S
EMC CONFIDENTIAL—INTERNAL USE ONLY 151

Search (3 options highlighted), content and Portlets. These are the 3 main areas of
the home page.

151
 Advanced Configuration Training 2007

Using Preferences to Customise Powerlink

Views

EMC CONFIDENTIAL—INTERNAL USE ONLY 152

Customising your home page takes around 5 minutes and will help ensure that
pertinent information is given priority in the content field. In addition editing Portlets
can help de clutter the homepage ensuring useful resources are more easily
located.

152
 Advanced Configuration Training 2007

Customising Your View via Preferences

EMC CONFIDENTIAL—INTERNAL USE ONLY 153

Powerlink homepage will reflect specified preferences. Interests specified will be

prioritised in the content field. Industry and Product selections available.

153
 Advanced Configuration Training 2007

Customising Portlets via the Preference Page

EMC CONFIDENTIAL—INTERNAL USE ONLY 154

Portlets allow us to easily navigate to selected items via the portlets tab on the
toolbar or via homepage. Note (via UP and DOWN arrow) used to specify the order
items appear.

154
 Advanced Configuration Training 2007

Portlets can then be edited

on the Home page

EMC CONFIDENTIAL—INTERNAL USE ONLY 155

Each Portlet can then be edited to give only information required. Individual items can me moved
up and down within the portlet.

155
 Advanced Configuration Training 2007

Document Searches

EMC CONFIDENTIAL—INTERNAL USE ONLY 156

Formerly located in Avatar this site has been now removed. This sections
demonstrates methods of searching for information. It does not concentrate on
where documents are located since Powerlink’s directory structure is subject to
change.

156
 Advanced Configuration Training 2007

Document Searches

EMC Documents and White Papers now in Powerlink

Increased number of Documents now Customer Viewable
(i.e. White Papers)

Powerlink Document Searches Via
– Keyword Search
– Navigator
– Drop down menus
– Document Library Quicklink

EMC CONFIDENTIAL—INTERNAL USE ONLY 157

As above.

157
 Advanced Configuration Training 2007

EMC CONFIDENTIAL—INTERNAL USE ONLY 158

Powerlink Document Searches Via

Keyword Search
Navigator
Drop down menus
Document Library Quicklink

158
 Advanced Configuration Training 2007

Document Searches

•Enter Search Words

•Exact phrases should be included within quotation marks “”.
•Select appropriate Category i.e. “White Papers”.

EMC CONFIDENTIAL—INTERNAL USE ONLY 159

Use of quotation marks to link words. Wild card ‘*’ can be used at the end of complete words.

159
 Advanced Configuration Training 2007

EMC CONFIDENTIAL—INTERNAL USE ONLY 160

Note filters can be used to sort results, for example by date.

160
 Advanced Configuration Training 2007

EMC CONFIDENTIAL—INTERNAL USE ONLY 161

Advanced search is intuitive. Search Tips is a useful downloadable document which explains
how to most effectively search in Powerlink.

161
 Advanced Configuration Training 2007

EMC CONFIDENTIAL—INTERNAL USE ONLY 162

Saved searches available via homepage or via portlets. Save search name is
entered by user.

162
 Advanced Configuration Training 2007

Saved Searches via Homepage

Saved searches are available via portlets on the Homepage

and via the portlets tab on all other pages
EMC CONFIDENTIAL—INTERNAL USE ONLY 163

As above.

163
 Advanced Configuration Training 2007

Saved Searches via Portlets Tab

EMC CONFIDENTIAL—INTERNAL USE ONLY 164

Portlets tab is available on every page except the homepage and allows quick
access to portlets.

164
 Advanced Configuration Training 2007

Downloads

Adding document to “My Downloads” for subsequent downloading

EMC CONFIDENTIAL—INTERNAL USE ONLY 165

Note this page has been altered slightly in format in order to fit on one page.

165
 Advanced Configuration Training 2007

Batch Download

Selected documents can be downloaded in Batches

EMC CONFIDENTIAL—INTERNAL USE ONLY 166

Allows for efficient downloading of a of multiple publications.

166
 Advanced Configuration Training 2007

Alerts

Alerts enables automatic notification of updates to selected documents

EMC CONFIDENTIAL—INTERNAL USE ONLY 167

Alerts via e-mail or home page Portlets tab.

167
 Advanced Configuration Training 2007

Useful Documents found on Powerlink

EMC CONFIDENTIAL—INTERNAL USE ONLY 168

Limited number of tools and documents will be demonstrated. Once the student is proficient at
using Powerlink’s Search Tools he or she should be able to locate resources.

168
 Advanced Configuration Training 2007

Release and End of Life Information

EMC CONFIDENTIAL—INTERNAL USE ONLY 169

We are frequently asked for information regarding code and product EOL dates. This information
can be found in Powerlink. Demonstrated via a search using the ‘cascading menus’.

169
 Advanced Configuration Training 2007

Release and End of Life Information

EMC CONFIDENTIAL—INTERNAL USE ONLY 170

Result of the EOL dates search.

170
 Advanced Configuration Training 2007

ETAs - EMC Technical Advisories/Alerts

EMC CONFIDENTIAL—INTERNAL USE ONLY 171

Search for ETAs via cascading menus.

171
 Advanced Configuration Training 2007

EMC CONFIDENTIAL—INTERNAL USE ONLY 172

Using the navigator to locate the Technical Alerts.

172
 Advanced Configuration Training 2007

ETAs - EMC Technical Advisories/Alerts

EMC CONFIDENTIAL—INTERNAL USE ONLY 173

This same file would be found using either of the search techniques described.

173
 Advanced Configuration Training 2007

Question

Can I send this document to my customer?

EMC CONFIDENTIAL—INTERNAL USE ONLY 174

No, you cannot.

174
Advanced Configuration Training 2007

Tools on Powerlink

EMC CONFIDENTIAL—INTERNAL USE ONLY 175

175
Advanced Configuration Training 2007

EMC CONFIDENTIAL—INTERNAL USE ONLY 176

176
 Advanced Configuration Training 2007

Tools on Powerlink

EMC CONFIDENTIAL—INTERNAL USE ONLY 177

This is the full list of available tools.

177
 Advanced Configuration Training 2007

EMC CONFIDENTIAL—INTERNAL USE ONLY 178

Issue tracker can be used to list issues associated with EMC hardware and software products.
Output can be filtered to limit output to a specific environment.

178
 Advanced Configuration Training 2007

EMC CONFIDENTIAL—INTERNAL USE ONLY 179

Output from Issue Tracker which can be exported to Excel.

179
Advanced Configuration Training 2007

EMC CONFIDENTIAL—INTERNAL USE ONLY 180

180
Advanced Configuration Training 2007

E-Lab Navigator &

Interoperability

EMC CONFIDENTIAL—INTERNAL USE ONLY 181

181
 Advanced Configuration Training 2007

What is the E-Lab and Interoperability?

Interoperability in this case refers to the ability of storage

networking products to reliably work with one another.

E-Lab refers to a number of EMC Labs dedicated to ‘end to
end’ testing system interoperability. Typically this involves
servers, HBAs, third-party networking products, software,
and many other components of today’s storage network.

Results of interoperability testing are published monthly in
the EMC Support Matrix, the ESM.

E-Lab Navigator is an online tool which filters the
considerable ESM content using user defined search
criteria. This information can be used to confirm that
networked storage solutions are supported.
EMC CONFIDENTIAL—INTERNAL USE ONLY 182

As above.

182
 Advanced Configuration Training 2007

E-Lab Homepage

Homepage accessed via Powerlink

EMC CONFIDENTIAL—INTERNAL USE ONLY 183

As above.

183
 Advanced Configuration Training 2007

E-Lab Navigator Homepage

Topology Guide End of Life Help

Download

Wizards

Advanced

EMC CONFIDENTIAL—INTERNAL USE ONLY 184

As above.

184
 Advanced Configuration Training 2007

EMC Topology Guide

EMC CONFIDENTIAL—INTERNAL USE ONLY 185

EMC Topology Guide takes you to the EMC Networked Storage Topology
Guide, which contains EMC best practices and instructions about system
environments, host configurations, management, and device support.

185
 Advanced Configuration Training 2007

End of Life

EMC CONFIDENTIAL—INTERNAL USE ONLY 186

Note this lists EOL dates for non EMC products.

186
 Advanced Configuration Training 2007

Users Guide

EMC CONFIDENTIAL—INTERNAL USE ONLY 187

Comprehensive users manual available via ‘user manual’ link highlighted.

187
 Advanced Configuration Training 2007

PDFs and Guides

EMC CONFIDENTIAL—INTERNAL USE ONLY 188

As above.

188
 Advanced Configuration Training 2007

Advanced Query and Wizards

Both Advanced Query and Wizards filter the ESM content

outputting only information relevant to the configuration
defined.

The Advanced Query is used to consider complete ‘end to
end’ solutions. The entire storage network would typically
be defined using the Advanced Query.

Wizards provide answers to simple interoperability
questions. 5 wizards are currently available.

EMC CONFIDENTIAL—INTERNAL USE ONLY 189

As above.

189
 Advanced Configuration Training 2007

Advanced Query

EMC CONFIDENTIAL—INTERNAL USE ONLY 190

As above.

190
 Advanced Configuration Training 2007

Navigation Bar

EMC CONFIDENTIAL—INTERNAL USE ONLY 191

As above.

191
 Advanced Configuration Training 2007

Search Cart

EMC CONFIDENTIAL—INTERNAL USE ONLY 192

As above.

192
 Advanced Configuration Training 2007

Results

EMC CONFIDENTIAL—INTERNAL USE ONLY 193

As above.

193
 Advanced Configuration Training 2007

Search

EMC CONFIDENTIAL—INTERNAL USE ONLY 194

As above.

194
 Advanced Configuration Training 2007

Creating and Advanced Query

From navigation bar drill down to find required product and

add this to the ‘search cart’.

Add multiple items to the cart.

Where items are show in Black this indicates no further
‘drilling down’ is possible.

Blue indicates a class of items not an individual item.

‘Get Results’ creates support statement which comprise of a
subset of the ESM filtered to output only information relating
to the configuration described.

‘Support Statement Assist’ can help identify non supported
items.

Search Carts can be stored retrieved and e-mailed.
EMC CONFIDENTIAL—INTERNAL USE ONLY 195

As above.

195
 Advanced Configuration Training 2007

Adding Items to a Search Cart

Add 5771 Enginuity level via Software Operating Environment

EMC CONFIDENTIAL—INTERNAL USE ONLY 196

As above.

196
 Advanced Configuration Training 2007

Adding Items to a Search Cart

Enginuity 5771 in Blue, indicates 8795 in black, no Add selected Item to

further drilling down is possible more drilling down Search Cart
EMC CONFIDENTIAL—INTERNAL USE ONLY 197

As above.

197
 Advanced Configuration Training 2007

Finding Items via the Search Function

EMC CONFIDENTIAL—INTERNAL USE ONLY 198

As above.

198
 Advanced Configuration Training 2007

Get Results

Once Configuration has

been fully defined in the
Search Cart ‘Get Results’
will produce the support
statement.

EMC CONFIDENTIAL—INTERNAL USE ONLY 199

As above.

199
 Advanced Configuration Training 2007

EMC Support Statement

Support statements are available

for your search cart items

Refine cart items—too many
support statements are available

No support statements are
available.

No Support Statements?
Click on flag to use support
statement assist.

EMC CONFIDENTIAL—INTERNAL USE ONLY 200

This slide has been taken from the ‘EMC E-Lab Interoperability Navigator’ training presentation
available via Powerlink
The colored indicators next to the links show the availability of statements for items in the
Search Cart.
•A green indicator signifies that a valid support statement is available.
•A yellow indicator means that you haven’t narrowed down the choices to provide
enough useful information. You must add more items to the cart in order to refine the
selection.
•A red flag means that the choices you have made are not compatible and there is no
supported combination. If you click on this statement, E-Lab Interoperability Navigator
shows you the selections that are causing the problem, and allows you to change or
remove items to attain a compatible configuration.

200
 Advanced Configuration Training 2007

Support Statement Assist

Invalid

Valid

EMC CONFIDENTIAL—INTERNAL USE ONLY 201

Support Statement assist is available to assist in resolving incompatibility issues. Only valid
options are listed in the assist.

201
 Advanced Configuration Training 2007

Support Statement Example

EMC CONFIDENTIAL—INTERNAL USE ONLY 202

Ensure header information is accurate.

202
 Advanced Configuration Training 2007

Director Bit Information

EMC CONFIDENTIAL—INTERNAL USE ONLY 203

As above.

203
 Advanced Configuration Training 2007

How would we set the Director Bits for Windows

Switch?

Anything else to Consider?

EMC CONFIDENTIAL—INTERNAL USE ONLY 204

Do not forget the VCM & EAN bits.

204
 Advanced Configuration Training 2007

Search Cart

E-mail
Load
Save
Find
Delete
Reset

EMC CONFIDENTIAL—INTERNAL USE ONLY 205

As above.

205
 Advanced Configuration Training 2007

Wizards

EMC CONFIDENTIAL—INTERNAL USE ONLY 206

As above.

206
 Advanced Configuration Training 2007

Wizards

Wizards are used to perform frequently used simple searches

EMC CONFIDENTIAL—INTERNAL USE ONLY 207

5 Wizards Available

207
 Advanced Configuration Training 2007

Less Functionality then Advanced Search

Note ‘delete’ and ‘find’ option is not available in search cart

EMC CONFIDENTIAL—INTERNAL USE ONLY 208

As above.

208
 Advanced Configuration Training 2007

Storage Array Wizard

Output is filtered so that only valid options are available

EMC CONFIDENTIAL—INTERNAL USE ONLY 209

As above.

209
 Advanced Configuration Training 2007

Software Operating Environment (SOE)

EMC CONFIDENTIAL—INTERNAL USE ONLY 210

The SOE replaces the Software Support Matrix (SSM) as a method of determining Software
Interoperability. Software products are filtered such that only supported products are displayed.

210
Advanced Configuration Training 2007

EMC CONFIDENTIAL—INTERNAL USE ONLY 211

211
 Advanced Configuration Training 2007

Channel Express

EMC CONFIDENTIAL—INTERNAL USE ONLY 212

This section should appear after the Powerlink section since access is via Powerlink.

212
 Advanced Configuration Training 2007

What is Channel Express?

Channel Express is web based application used by CS and Channel

Partners to verify configuration validity. It uses the same rules as Direct
Express

Direct Express is a sales ordering tool, it includes pricing information not
available in Channel Express

Both Channel and Direct Express enforce EMCs configuration rules

Channel Express rules are also policed by Symmwin

Channel Express is typically used by CS to verify configuration validity
where discrepancies are identified between what has been sold and
what is documented as supported

Configurations highlighted as illegal in Channel Express will require an
RPQ

EMC CONFIDENTIAL—INTERNAL USE ONLY 213

Only aspects of channel express of interest to CS have been described here. Comprehensive
training is available through Knowledgelink.
Channel Express and Direct Express (used by Sales) use the same rules meaning that we can
verify if a Symmetrix configuration was valid in Direct Express when it is identified as being
illegal in Symmwin.

213
 Advanced Configuration Training 2007

Accessing Channel Express

Channel Express is accessed via Powerlink

EMC CONFIDENTIAL—INTERNAL USE ONLY 214

Channel Express is accessed via Powerlink.

214
 Advanced Configuration Training 2007

Registration and Logging into Channel Express

EMC CONFIDENTIAL—INTERNAL USE ONLY 215

Note that many in CS have never used channel express therefore it will be necessary to
demonstrate how to register.

215
 Advanced Configuration Training 2007

EMC CONFIDENTIAL—INTERNAL USE ONLY 216

Initial screen after login. Only valid region and options for logon will be displayed. The following
slides take you through the process of configuring a Symmetrix in Channel Express. Only
screens of interest to CS are shown. Minimum information to allow clarification of Cache and
Power requirement is input. Clarification of Cache an Power via Channel Express is useful
where Symmwin errors indicate not enough cache or power has been configured.

216
 Advanced Configuration Training 2007

Configuring Using Channel Express

EMC CONFIDENTIAL—INTERNAL USE ONLY 217

As above.

217
 Advanced Configuration Training 2007

Product Configuration Wizard

EMC CONFIDENTIAL—INTERNAL USE ONLY 218

Product Configuration Wizard can be used to confirm validity of both new and upgraded
configurations. Note ‘Solution Name’ is non essential.

218
 Advanced Configuration Training 2007

Entering Configuration Details

EMC CONFIDENTIAL—INTERNAL USE ONLY 219

As above.

219
 Advanced Configuration Training 2007

Entering DMX-3 Loop Configuration Details

EMC CONFIDENTIAL—INTERNAL USE ONLY 220

As above.

220
 Advanced Configuration Training 2007

Configuring Drives and Volumes

EMC CONFIDENTIAL—INTERNAL USE ONLY 221

Note the need to configure SDRF which will bring up additional pages to cater for such things as
additional cache for SRDF-A.
Loss of capacity due to Vault drives and SFS is also highlighted here.

221
 Advanced Configuration Training 2007

Configuring 120 Raid 5 Protected Drives

Split per Drive is not Hypers per Drive

Depending on protection type Splits and

Hypers man be different. The additional
overhead associated with the extra hypers
needed for Raid-5 is not taken into account
when considering Splits in Channel Express
Spare Drives are automatically configured
EMC CONFIDENTIAL—INTERNAL USE ONLY 222

The number of splits defaults to 21. Setting not enough splits can result in too little cache.
Channel Express does not cater for mixed protection types on a single drive. It assumes only
one protection type is applicable to a drive. This is a limitation of Channel Express which can
lead to illegal configurations being highlighted in Symmwin which were not picked up in Channel
express. A second problem associated with having to specify a single protection type on a group
of drives is that this concept can end up being incorrectly identified as being a customer
specification as opposed to a Channel Express limitation. Where mixed protection boxes
specifications state that only one protection type is allowed on a drive it is important to verify that
this is a genuine customer requirement not a Channel Express requirement.
The need to revalidate the configuration in the event of a changed configuration requirement is
emphasised because this a major cause of under caching typically where the customer decides
to add e-Snap or SRDF-A without purchasing the additional cache.

222
 Advanced Configuration Training 2007

Spares

Number of Spares can and in

some cases should be modified

EMC CONFIDENTIAL—INTERNAL USE ONLY 223

Channel Express, in some instances, will not allocate the appropriate number of dynamic or
permanent spares. In this case the number of spares needs to be adjusted in the Dynamic
Spares table. The issue of how many spares are required is examined elsewhere in the course.
It is recommended that CS should where possible verify the spares coverage if Permanent
Sparing is being used. This is particularly important on boxes with multiple drive types or on
small (DMX3-1500 and DMX-3 950) where raid-5 is configured. See Spares section for details
on how to calculate the spare requirements.

223
 Advanced Configuration Training 2007

Adding Front End Directors

EMC CONFIDENTIAL—INTERNAL USE ONLY 224

Front End Directors need to be configured to ensure correct power supply configuration.

224
 Advanced Configuration Training 2007

Cache

EMC CONFIDENTIAL—INTERNAL USE ONLY 225

Cache and Power rules are enforced by both Channel Express and Symmwin and should agree.

225
 Advanced Configuration Training 2007

Power

EMC CONFIDENTIAL—INTERNAL USE ONLY 226

Note it is essential that correct 3 phase wiring is selected (Delta or Wye).

226
 Advanced Configuration Training 2007

Summary

EMC CONFIDENTIAL—INTERNAL USE ONLY 227

Summary Page can be requested from Sales in instances where configuration issues are
highlighted.

227
 Advanced Configuration Training 2007

RPQ Info

EMC CONFIDENTIAL—INTERNAL USE ONLY 228

RPQ requirements are highlighted by Channel Express.

228
Advanced Configuration Training 2007

EMC CONFIDENTIAL—INTERNAL USE ONLY 229

229