State of the Art Real-Time Mainframe Data Integration or Data Replication Across all Platforms

tcvision-graph Image

tcVISION replicates data uni and bi-directionally between the IBM Mainframe and the Open Systems world.

tcVISION is the innovative software solution to process changed data in real-time, in intervals or event based.

  • tcVISION automatically identifies the changes applied to mainframe databases and files.
  • No programming effort is necessary to obtain the changed data.
  • tcVISION continuously propagates the changed data to the target systems in real-time or can perform the process in time intervals.
  • tcVISION offers a rich set of processing and controlling mechanisms to guarantee a data exchange implementation that is fully auditable.
  • tcVISION contains powerful bulk processors that perform the initial load of mass data or the cyclic exchange of larger data volumes efficient, fast and reliable.
  • tcVISION provides modeling and mapping via a GUI modeler.
  • tcVISION enables bi-directional replication for a variety of non-mainframe sources, including Amazon Web Services (AWS), Oracle Database, Microsoft SQL Server, IBM Db2 LUW and Db2 BLU, IBM Informix and PostgreSQL.
  • tcVISION has implemented Big Data as a standard output platform.
  • tcVISION allows mainframe data to quickly and easily be integrated in near real-time into Hadoop-based operative applications or Business Intelligence and Analytics.
  • tcVISION change capture facilities identify changes in mainframe and relational data sources without programming effort.
  • tcVISION transfers the changed data to target mainframe and relational databases and applications in real time, in time intervals or in response to events.
  • tcVISION guarantees transparent and auditable data transfer between sources and targets.
  • tcVISION provides powerful processors to perform fast, efficient and reliable transfers of mass data.
  • tcVISION can deliver drastic savings for all mainframe-related data exchange operations. Change Data Capture reduces costs: less processing time, less hardware and less human resources are needed for exchange of information.
  • tcVISION reduces the amount of data that must be transferred between the different systems to an absolute minimum. Only data that has been changed is transferred–only data that is really needed will be moved
tcVISION_Connection_Overview

Focusing on change data results in cost savings in all areas:

  • The time window necessary to implement and execute the exchange processes of mainframe data will be drastically reduced.
  • The data replication and synchronisation process needs fewer resources.
  • Concentrating on change data only will ensure that only the relevant data is being moved, resulting in fewer resources for the actual transfer and the data transformation.
  • Real-time data replication will immediately free-up resources that are normally being required in the batch windows.
  • tcVISION automatically detects changed data with no need for programming efforts. Data exchange processes can be implemented in shorter period of time with less manpower.

tcVISION – Guarantee for Efficiency and Transparency

Transfer of mass data still makes sense for certain applications. This can be the initial load of mainframe data into an ETL or EAI tool or Data Warehouse.

Transferring mass data, however, should not be seen as the universal answer for integration of mainframe data. Mass data transfer very fast faces limitations because the available Batch windows are not large enough. Mass data transfer may not be the most efficient solution. It must be decided on an individual basis whether mass data transfer is the right way to go if only selected changed data must be provided to the remote system.

For this reason tcVISION offers technologies that allow an optimum information exchange between distributed systems.

tcVISION offers various data capture methods that can be used to support the required degrees of data latency and manage the amount of data processed. A efficient and secure data synchronisation is assured by all methods.

These methods are:

Bulk Transfer

• Efficent transfer of complete databases
• Analysis for data consistency
• Perfectly suited as Initial Load prior to log-processings
• Ideal solution for periodic mass data transfer
• One-step data exchange operation

Note: BULK Transfer is also possible for ODBC, Db2/LUW and ORACLE

Log Processing

• Available for IMS/DB, VSAM, Db2, DL/I, ADABAS, CA-IDMS, CA-Datacom
• Transfer of changed data only for the specified time slot
• perfectly suited for nighttime processing
• perfectly suited for post log-processing

Batch Compare

• Comparison with data snapshots
• Efficient transfer of changed data since last Batch Compare run
• Available for IMS/DB, DL/I, VSAM, Db2, ADABAS, CA-IDMS and sequential files, CA-Datacom
• Automatic creation of deltas
• Secure restart processing after error situations

DBMS Extension for Mainframe Data

• Real-Time and near Real-Time capturing of changed data
• Change data capturing directly at database level
• Available for IMS/DB, DL/I, VSAM, Db2, ADABAS, CA-IDMS, CA-Datacom
• Secure data storage, even across DBMS restart

CDC Capture Methods

DRDA for Db2 LUW in Unix, Linux, Windows

tcVISION uses the DRDA protocol as output media if the target database is a Db2 LUW system that does not exist on a mainframe (i.e. WINDOWS, UNIX/LINUX/Linux on System z).

For synchronisation with a mainframe based Db2, a direct communication via Db2 is supported. The usage of DRDA allows for a fast and easy implementation with a Db2 LUW target-system.

• Support of the industry standard DRDA
• Simple and easy implementation
• Fast, flexible and platform independent protocol

tcVISION – ORACLE

Implementation of changed mainframe data into ORACLE via OCI, ODBC and Oracle Loader

tcVISION offers different methods to implement changed mainframe data directly into an ORACLE database:

• Propagation of changed mainframe data in SQL DML syntax format using the Oracle Call Interface (OCI)
• Using the ORACLE Loader function for the insertion of mass data into an ORACLE database
• Propagation of changed mainframe data in SQL DML syntax-format using any ODBC driver

All tools needed to create, maintain and process an ORACLE database can be used from the tcVISION Control Board.

The Change Data Capture function of ORACLE (ORACLE Stream) together with tcACCESS can be used to implement a bidirectional synchronisation with a mainframe resource.

• Synchronisation for ORACLE using OCI or ODBC
• Bidirectional and transaction oriented processing based on LUWs
• Optional synchronous or asynchronous LUW processing
• Efficient processing of large data volumes using the ORACLE LOADER function

Changed Data Propagation via ODBC

The change data from the mainframe that has been processed can be propagated into any target system that supports ODBC. This allows for a flexible processing that is independent of the actual target system.

All tools needed to create, maintain and process an ODBC data source can be used from the tcVISION Control Board.

• Synchronisation using ODBC driver under Windows or UNIX*
• Resource access is performed with SQL-DML syntax
• LUW based processing
• Optional synchronous or asynchronous LUW processing
• Efficient processing of large data volumes using “bound” parameter
• Support of all common ODBC standards

* requires the installation of a UNIX ODBC Manager (i.e. UNIXODBC)

Data Mapping / Meta Data Dictionary

The powerful data mapping functions of tcVISION provide a detailed overview of the mainframe data to the tcVISION user. It is no longer necessary to always involve mainframe and database experts.

Batch processes are available to automatically create the mapping structures. For VSAM files the data mapping is being performed based upon copybook information.

For IMS/DB databases the production PSB and DBD load modules will be evaluated.

In the case of ADABAS and CA-Datacom the Data Dictionary of the database will be evaluated. For CA-IDMS, SCHEMA definitions are the source for the meta data.

The field structure definitions can be easily viewed using the tcVISION Control Board, hence allowing an easy selection of the meta data necessary for the changed production data.

Loader Format

tcVISION provides a very efficient LOADER function for the insertion of large data volumes into Db2-, Db2/UDB-, ORACLE-, MS-SQL Server- or MS-ACCESS-databases. All DML-statements required for the LOADER process are automatically generated by tcVISION.

These statements, together with the data that must be propagated to the target system, can be processed using a standard utility function of the target DBMS.

• Fast and efficient load of Db2, Db2/UDB, ORACLE, MS-SQL Server and MS-Access tables.
• Automatic creation of the LOADER statements based upon the metadata
• Excellent performance for the processing of large data volumes.

Bulk Processors

Typically, ETL solutions or other decentralised applications require an initial load of a database on an Open System Server or – in time intervals – request a large number of data records that must be transferred from the mainframe to an Open System Server.

tcVISION provides an efficient and comfortable solution to meet these requirements by introducing the tcVISION Bulk Processors. While other solutions only support sequential files, tcVISION can bulk transfer data directly from Db2, IMS/DB, DL/I, CA-IDMS, CA-Datacom, VSAM files and sequential files. All data protocols are directly supported no intermediate file media is necessary. The data transfer to the target system also includes all relevant transformation and conversion of the data.

Hence, transfer of mainframe data to an Open Server-System is a one step process. It is reliable and extremely time efficient. The mainframe data will be automatically converted by tcVISION in the correct format required by the target system (SQL, HTML, XML, etc.) or can be saved into a message queue for further processing.

The tcVISION Change Data Capture Technology

tcVISION provides several ‘Change Data Capture’ mechanisms. They automatically detect the changes that have been applied to the operational mainframe databases and files. This completely eliminates the need to move unchanged data.

The technology that should be applied to capture the changes is based upon the type of data source. Not all file systems support date and time stamps that indicate the time when the data has been changed. Other methods must be used to detect the changes.

These are:

• Database exits
• Logfile extraction
• Post Log Processing
• Trigger based solutions
• Batch Compare
• Application based processing

Depending upon the data source and the required task multiple CDC methods can be used.

The changes can be detected on a transaction level (Logical Unit of Work). Based upon the requirements the changed data can then be immediately propagated (real time) or can be transferred to the target systems in timely intervals or based upon events.

Variable Process Balancing

It is the goal of the tcVISION propagation to transform the changed mainframe data from its internal structures into a format that can be used to directly implement the changes into a target DBMS or to pass the data to a target application system.

On this way the “classical” mainframe data (Packed, Binary, Hexadecimal etc.) must be transformed into a format known to and understood by the Open System world. In addition to this transformation process the changed data can be passed to the target system in different formats (SQL DML, XML, file format etc). Also the method how the data is passed to the target system may differ (ODBC, tcACCESS, DRDA, ORACLE, MQ-Series).

staged_processing_load_balancing

The processing of the changed data can be performed at any place in the tcVISION network. The processing can be done partially on the mainframe and on Windows- or UNIX/LINUX systems. As a rule of thumb, the data will only be captured on the mainframe(real time processing) and the actual processing of the data takes place on workstation-based systems. There is no demand for processing resources on the mainframe.

• Flexible implementation of the processing of changed data
• Optimum utilisation of available resources
• No additional mainframe resources required
• Processing of changed data can be easily adapted to meet the changing day to day requirements

Propagation Engine

Using the tcVISION process configurator data exchange processes can be created without any mainframe know-how. With tcVISION data exchange becomes a single-step operation. No manual intervention is required. The pass-over of the changed data can either be performed in real-time, time dependent or event based. The mainframe data will be automatically converted by tcVISION in the correct format required by the target system (SQL, HTML, XML, etc.) or can be saved into a message queue for further processing.

The state-of-the-art tcVISION process and control board makes it easy for the user to initiate and monitor data exchange processes. Integrated scheduler functions help automating these processes. No manual intervention is necessary in a production environment. The control mechanisms provided by tcVISION guarantee a high degree of transparency and auditability.

Youtube-Channel-Picture
HOMEtcVISION ArchitecturetcVISION OverviewtcVISION Frequent QuestionstcVISION – Amazon Web ServicestcVISION Big DataReal Time Data Replication
Mainframe to CloudData SynchronisationDownloadstcVISION z/StreamtcACCESSNewsRead MoreDEMOPrivacy PolicyContact us
tcVISION is distributed in Australia and New Zealand by CCA Software Pty. Ltd. a division of FreeSoft Asia-Pacific
All Rights Reserved Copyright © 2019 Privacy Policy
error: Alert: Content is protected !!