tcVISION is the innovative software solution to process changed data in real-time, in intervals or event based.
Transfer of mass data still makes sense for certain applications. This can be the initial load of mainframe data into an ETL or EAI tool or Data Warehouse.
Transferring mass data, however, should not be seen as the universal answer for integration of mainframe data. Mass data transfer very fast faces limitations because the available Batch windows are not large enough. Mass data transfer may not be the most efficient solution. It must be decided on an individual basis whether mass data transfer is the right way to go if only selected changed data must be provided to the remote system.
For this reason tcVISION offers technologies that allow an optimum information exchange between distributed systems.
tcVISION offers various data capture methods that can be used to support the required degrees of data latency and manage the amount of data processed. A efficient and secure data synchronisation is assured by all methods.
These methods are:
• Efficent transfer of complete databases
• Analysis for data consistency
• Perfectly suited as Initial Load prior to log-processings
• Ideal solution for periodic mass data transfer
• One-step data exchange operation
Note: BULK Transfer is also possible for ODBC, Db2/LUW and ORACLE
• Available for IMS/DB, VSAM, Db2, DL/I, ADABAS, CA-IDMS, CA-Datacom
• Transfer of changed data only for the specified time slot
• perfectly suited for nighttime processing
• perfectly suited for post log-processing
• Comparison with data snapshots
• Efficient transfer of changed data since last Batch Compare run
• Available for IMS/DB, DL/I, VSAM, Db2, ADABAS, CA-IDMS and sequential files, CA-Datacom
• Automatic creation of deltas
• Secure restart processing after error situations
• Real-Time and near Real-Time capturing of changed data
• Change data capturing directly at database level
• Available for IMS/DB, DL/I, VSAM, Db2, ADABAS, CA-IDMS, CA-Datacom
• Secure data storage, even across DBMS restart
tcVISION uses the DRDA protocol as output media if the target database is a Db2 LUW system that does not exist on a mainframe (i.e. WINDOWS, UNIX/LINUX/Linux on System z).
For synchronisation with a mainframe based Db2, a direct communication via Db2 is supported. The usage of DRDA allows for a fast and easy implementation with a Db2 LUW target-system.
• Support of the industry standard DRDA
• Simple and easy implementation
• Fast, flexible and platform independent protocol
Implementation of changed mainframe data into ORACLE via OCI, ODBC and Oracle Loader
tcVISION offers different methods to implement changed mainframe data directly into an ORACLE database:
• Propagation of changed mainframe data in SQL DML syntax format using the Oracle Call Interface (OCI)
• Using the ORACLE Loader function for the insertion of mass data into an ORACLE database
• Propagation of changed mainframe data in SQL DML syntax-format using any ODBC driver
All tools needed to create, maintain and process an ORACLE database can be used from the tcVISION Control Board.
The Change Data Capture function of ORACLE (ORACLE Stream) together with tcACCESS can be used to implement a bidirectional synchronisation with a mainframe resource.
• Synchronisation for ORACLE using OCI or ODBC
• Bidirectional and transaction oriented processing based on LUWs
• Optional synchronous or asynchronous LUW processing
• Efficient processing of large data volumes using the ORACLE LOADER function
The change data from the mainframe that has been processed can be propagated into any target system that supports ODBC. This allows for a flexible processing that is independent of the actual target system.
All tools needed to create, maintain and process an ODBC data source can be used from the tcVISION Control Board.
• Synchronisation using ODBC driver under Windows or UNIX*
• Resource access is performed with SQL-DML syntax
• LUW based processing
• Optional synchronous or asynchronous LUW processing
• Efficient processing of large data volumes using “bound” parameter
• Support of all common ODBC standards
* requires the installation of a UNIX ODBC Manager (i.e. UNIXODBC)
The powerful data mapping functions of tcVISION provide a detailed overview of the mainframe data to the tcVISION user. It is no longer necessary to always involve mainframe and database experts.
Batch processes are available to automatically create the mapping structures. For VSAM files the data mapping is being performed based upon copybook information.
For IMS/DB databases the production PSB and DBD load modules will be evaluated.
In the case of ADABAS and CA-Datacom the Data Dictionary of the database will be evaluated. For CA-IDMS, SCHEMA definitions are the source for the meta data.
The field structure definitions can be easily viewed using the tcVISION Control Board, hence allowing an easy selection of the meta data necessary for the changed production data.
tcVISION provides a very efficient LOADER function for the insertion of large data volumes into Db2-, Db2/UDB-, ORACLE-, MS-SQL Server- or MS-ACCESS-databases. All DML-statements required for the LOADER process are automatically generated by tcVISION.
These statements, together with the data that must be propagated to the target system, can be processed using a standard utility function of the target DBMS.
• Fast and efficient load of Db2, Db2/UDB, ORACLE, MS-SQL Server and MS-Access tables.
• Automatic creation of the LOADER statements based upon the metadata
• Excellent performance for the processing of large data volumes.
Typically, ETL solutions or other decentralised applications require an initial load of a database on an Open System Server or – in time intervals – request a large number of data records that must be transferred from the mainframe to an Open System Server.
tcVISION provides an efficient and comfortable solution to meet these requirements by introducing the tcVISION Bulk Processors. While other solutions only support sequential files, tcVISION can bulk transfer data directly from Db2, IMS/DB, DL/I, CA-IDMS, CA-Datacom, VSAM files and sequential files. All data protocols are directly supported no intermediate file media is necessary. The data transfer to the target system also includes all relevant transformation and conversion of the data.
Hence, transfer of mainframe data to an Open Server-System is a one step process. It is reliable and extremely time efficient. The mainframe data will be automatically converted by tcVISION in the correct format required by the target system (SQL, HTML, XML, etc.) or can be saved into a message queue for further processing.
tcVISION provides several ‘Change Data Capture’ mechanisms. They automatically detect the changes that have been applied to the operational mainframe databases and files. This completely eliminates the need to move unchanged data.
The technology that should be applied to capture the changes is based upon the type of data source. Not all file systems support date and time stamps that indicate the time when the data has been changed. Other methods must be used to detect the changes.
• Database exits
• Logfile extraction
• Post Log Processing
• Trigger based solutions
• Batch Compare
• Application based processing
Depending upon the data source and the required task multiple CDC methods can be used.
The changes can be detected on a transaction level (Logical Unit of Work). Based upon the requirements the changed data can then be immediately propagated (real time) or can be transferred to the target systems in timely intervals or based upon events.
It is the goal of the tcVISION propagation to transform the changed mainframe data from its internal structures into a format that can be used to directly implement the changes into a target DBMS or to pass the data to a target application system.
On this way the “classical” mainframe data (Packed, Binary, Hexadecimal etc.) must be transformed into a format known to and understood by the Open System world. In addition to this transformation process the changed data can be passed to the target system in different formats (SQL DML, XML, file format etc). Also the method how the data is passed to the target system may differ (ODBC, tcACCESS, DRDA, ORACLE, MQ-Series).
The processing of the changed data can be performed at any place in the tcVISION network. The processing can be done partially on the mainframe and on Windows- or UNIX/LINUX systems. As a rule of thumb, the data will only be captured on the mainframe(real time processing) and the actual processing of the data takes place on workstation-based systems. There is no demand for processing resources on the mainframe.
• Flexible implementation of the processing of changed data
• Optimum utilisation of available resources
• No additional mainframe resources required
• Processing of changed data can be easily adapted to meet the changing day to day requirements
Using the tcVISION process configurator data exchange processes can be created without any mainframe know-how. With tcVISION data exchange becomes a single-step operation. No manual intervention is required. The pass-over of the changed data can either be performed in real-time, time dependent or event based. The mainframe data will be automatically converted by tcVISION in the correct format required by the target system (SQL, HTML, XML, etc.) or can be saved into a message queue for further processing.
The state-of-the-art tcVISION process and control board makes it easy for the user to initiate and monitor data exchange processes. Integrated scheduler functions help automating these processes. No manual intervention is necessary in a production environment. The control mechanisms provided by tcVISION guarantee a high degree of transparency and auditability.