VSAM Transparency In The IBM Tool Kit
By
Don
Fowler, MCE, Inc. 2004
On March 26, 2004, IBM is making available a z/OS and OS/390 VSAM migration tool of major import to the Information Technology professional dealing with CICS. Many professionals dealing in the VSE environment have had a similar tool for their systems since 1997. Many competitive database systems and third party tools, such as CA-IDMS, CA-DATACOM, and XTSOFT SQL TRANSPARENCY, have provided similar functionality.
If
you have CICS and batch applications with data in VSAM files and would like to
move that data to DB2, but have been deterred by cost, time, and lack of
resource, then you will be very interested in the new IBM CICS VSAM
Transparency for z/OS, V1.1.
To review the IBM announcement letter for the CICS VSAM Transparency for z/OS product, click here.
It is essential to
understand the nature of the problem before you can evaluate different
solutions such as CICS VT.
There are several
issues with VSAM, the main ones being:
• difficulty in sharing the data across
different processes and maintaining data integrity,
• architectural limits that impose a
maximum size on a VSAM file,
• ad hoc access to VSAM files by workers
or decision support users is difficult,
• no data mining tools for VSAM,
• real-time access to real-time data.
The CICS VT solution means that you do not
have to change any of your existing application programs, but your data is
still migrated to DB2. Because you don't change your programs, the migration
costs are reduced, but more importantly your risk is substantially reduced.
By using CICS VT, you have a single copy of
the data in DB2, without having to change any of your existing legacy
application programs. This means that the inherent problems associated with
other migration options are eliminated. The use of CICS VT ensures that:
• Only one copy of
the data exists, so synchronization is not an issue. Your legacy programs and
your end users can share the data.
• No changes to
legacy programs are required. The degree of change, and therefore the scale of
the migration risk, is reduced significantly.
• Local and
disaster recovery procedures are simplified because there is one less data
store type to support.
• You can use
standard tools, skills and resources and achieve economies of scale in your database
administration area.
• Your data
integrity is assured because it is under the control of a DBMS.
In the article by Timothy Smith,
“Of Glue and Low Hanging Fruit: Extending the Value of VSAM Data with DB2 and VSAM Transparency”
he states:
“Because the VSAM data is mapped to DB2, levels of integrity,
security, efficiency, concurrency, programmability, and administration inherent
to DB2 and previously unavailable in VSAM, become part of this new environment.
·
Integrity One of the immediate differences
between native VSAM and DB2 is the treatment of fields and data types. As VSAM
fields are mapped to DB2 columns, each field is assigned a data type, which DB2
enforces both at compile time and run time. Thus, the process of moving from a
record-based system to a tablebased system immediately adds system-level
integrity to existing application-level integrity.
·
Security DB2 includes its own security
subsystem, with user, table, column, package and action level granularity. DB2
also provides mechanisms that integrate its security with other external
security managers (ESMs), which may already exist on the platform.
Installations can tailor user authorizations to the VSAM data once it resides
in DB2.
·
Efficiency Whereas VSAM data design may have
deliberately taken a de-normalized approach for programming ease and
performance, a reasonably normalized data model may be considered as part of
the VSAM Transparency implementation without similar penalties. Concurrency
LUW size, application isolation levels, and lock granularity, represent
mechanisms in DB2 that influence concurrency across applications and users.
Age-old concerns regarding sharing and whether the file is open in CICS or not,
disappear.
·
Programmability Consider the full power of SQL with
its built-in ability to aggregate (group), sort, filter, format, evaluate and
present results to complex queries. As new applications emerge, the SQL
language itself becomes a powerful programming tool, allowing logic to be
pushed down from the application to the database server.
·
Administration Batch utilities as well as online
tools provide administrators with powerful options to manage, monitor, and
maintain a healthy DB2 system while ensuring maximum availability and
reliability. “
CICS
VT works by intercepting run-time calls to VSAM files from your application programs.
For batch programs, this is performed using an MVS subsystem. In CICS, the interception
is handled by a global user exit (GLUE). The call interception is referred to as
the “CICS VT run-time component”.
At run time, CICS VT uses two driver modules
for each migrated VSAM data set. These
are:
Data set
Information Module (DIM) This defines the relationship between the VSAM record
structure and the DB2 table it relates to.
Data set Driver
Module (DDM) This contains the necessary SQL to access the DB2 table.
These modules are generated by standard CICS
VT ISPF dialog functions. These functions are part of the “CICS VT mapping
component”.
Although there are differences between the
batch and CICS run-time components in the way that they intercept application
program VSAM calls, functionally they are equivalent. This component performs a
number of functions:
• It intercepts
the VSAM calls within your application programs.
• In CICS
programs, it analyzes the call to decide if the target file has been migrated to
DB2. If the file has been migrated, it loads and processes the appropriate CICS
VT driver modules. In batch programs, CICS VT is enabled via JCL changes.
• It accesses DB2
to process the data.
• It builds the
equivalent of the VSAM or CICS return codes that drive your program logic.
CICS VT has to understand the structure of
the records in a VSAM file and how this relates to the structure of the data in
DB2. The relationship between these structures is achieved through a CICS VT
process called mapping.
Although there are two separate mapping
methods, an automated method and a manual method, both achieve the same
purpose.
Using the re-engineering capabilities of CICS
VT enhances the business value of your data once it is migrated to DB2. This
can be especially useful for end users. However, it assumes that the VSAM data
field values are consistent with the copybook attributes. Inconsistent data
will increase the time to perform data migration.
There is a third component of CICS VT called
the “CICS VT data migration component”.
This component deals with all aspects of the
initial conversion of your VSAM files to DB2 tables.
CICS VT is a suite of tools that cover the
entire VSAM to DB2 migration process, including the key component of data
migration. To migrate data from a VSAM file to a DB2 table, you must run a
series of CICS VT batch utility jobs.
Some of these batch jobs use utilities
provided with CICS VT and some use standard DB2 utilities such as LOAD,
RUNSTATS and REORG. In addition, you can use the DSNTIAUL DB2 sample program to
generate the DB2 LOAD control cards.
When you are migrating the data, CICS VT
automatically performs data re-engineering, according to how you have mapped
the file. For example, if you have a field containing date information and you
map this field to a DB2 DATE column, the data migration process builds the
appropriate DB2 column values so that the data presented to the DB2 LOAD
utility is correct.
There are two different methods to migrate a
VSAM data set using CICS VT.
• Perform the
migration manually
• Use the
automated mapping facility
You should be able to migrate most files
using the automated mapping facility, but this depends on how the file is
defined in the copybook.
The migration methodology is the same for
each VSAM file you migrate, regardless of aspects such as the attributes of a
file or how it is used. The key steps in the migration process are summarized
below:
1. Identify which
file or files are to be migrated.
2. Analyze the
files to determine DB2 table designs.
3. Create the DB2
tablespace, table and index objects.
4. Map the file or
files.
5. Migrate the
data from VSAM to DB2.
6. Test the
mapping.
7. Test application
programs.
8. Cutover to
production.
For detail on the migration steps, see the IBM CICS VSAM Transparency Users Guide by clicking here.
When a VSAM record is updated by an
application program, the change to the data is made immediately. When you
update a DB2 row, the change is only made after a COMMIT, a CICS SYNCPOINT or
TERM. When you convert a data set to DB2 using CICS VT, you must consider how
this difference between VSAM and DB2 affects your existing operational and
recovery procedures.
You must also consider how to integrate the
DB2 elements of CICS VT migrated data sets into your existing DB2 environment.
For example, what DB2 collection will you use for your CICS VT driver packages?
How does the migration affect your backup and recovery procedures?
You must understand these implications on
your operational and recovery procedures before migrating any production data
sets from VSAM to DB2.
For each migrated VSAM data set and alternate
index path, a CICS VT DDM driver module is used to access DB2. Bind the DB2
package for the DDM into the DB2 plan that the CICS transaction uses. The
default collection for DDM packages is called VIDCOLL. If your existing CICS
DB2 programs use a single plan, add the collection VIDCOLL to the package list
for this plan.
If your CICS transactions use different
plans, bind the collection VIDCOLL into each one.
Changes to DB2 tables are accumulated until
one of the following events occur at which point the updates are committed:
• A CICS SYNCPOINT
call is processed.
• A CICS EXEC DLI
TERM call is processed.
• The application
program ends normally.
Updates to VSAM files are handled in the same
way by CICS.
Changes to DB2 tables are backed out in the
following circumstances:
• A CICS ABEND
call is issued.
• The application
program abends.
• The application
program issues a ROLLBACK TO SAVEPOINT call.
Updates to VSAM data sets are backed out in
the first two cases. If you have existing CICS programs updating DB2 and VSAM
resources in the same unit of work, a ROLLBACK TO SAVEPOINT call issued by your
application program will undo changes to CICS VT migrated DB2 tables. You
should consider how this affects your CICS application programs before
migrating your production data.
Batch programs handle updates to DB2 and VSAM
resources differently to CICS application programs. For example, there is no
logging of VSAM updates in a batch program.
To deal with this, there are two different
processing methods for batch programs in CICS VT. The default method processes
all SQL calls in the same DB2 thread. The second method, which is enabled at an
individual batch program level, uses MVS subtasking to allow you to have
multiple, independent threads to DB2.
CICS VT sub-tasking incurs a performance
overhead and should only be used when necessary.
When you migrate VSAM data sets to DB2 using
CICS VT there are no other specific application recovery implications for CICS
programs.
The use of a VSAM Transparency tool is not new technology and have been used successfully by both IBM and other Database Management System Vendors to provide an easy, cost effective, and low risk migration path for VSAM flat files to DBMS managed data entities.
The implementation of the tool is straightforward and simplistic.
Having you VSAM data available in relational format adds value for the end-user as well as making future object oriented programming much easier to accomplish than via the conventional “flat file” format.
Use your browser BACK button to return to the newsletter main page.