tcVISION Extension for Kafka

A great part of the added value of modern IT systems is the latency-free data and process integration of transactional and analytical areas.

The cross-system integration platform tcVISION is unique, efficient and reliable. With tcVISION mainframe data can be fast and easily integrated into Big-Data based operative applications or Business Intelligence and Analytics in near real-time.

The tcVISION solution is practice approved and is constantly further developed to meet the requirements of the new technologies. The result is the support of Big-Data in tcVISION Version 6.

In the current version of tcVISION V6 Big-Data is a fully integrated output platform and supports the integration with Apache Kafka.

Consequently, tcVISION supports direct streaming of changed data into a Big-Data Apache Kafka environment. Apache Kafka is an Open Source data streaming platform developed by the Apache Software Foundation.

The software stands out because it is a distributed system and real-time scalable. Thus it is best suited to meet the challenges of Big-Data requirements.

As with all output platforms provided by tcVISION, the data streaming via Apache Kafka is based on official standard interfaces. The implementation of the Apache Kafka interface is fast and easy.

Data streaming to Apache Kafka as a transport layer enhances the Big-Data connectivity of tcVISION.

In addition to Apache Kafka, transport layers to Big-Data include the creation of files, direct output to a Hadoop file system (HDFS) as well as the output to MongoDB.

The currently used protocols for data transfer to Big-Data are JSON, Avro and CSV. The main focus of the tcVISION integration platform is the supply of a real-time synchronization to integrate mainframe data into Big-Data based solutions.

Features

The tcVISION integration platform consists of a variety of state-of-the-art technology components which cover far more than an ETL process.

  • Data exchange in the sense of real-time synchronization and replication turns into a “Single Step Operation” with tcVISION.
  • No additional middle-ware is required.
  • Diverse Change Data Capture technologies allow an efficient selection of the required data from the source system with focus on the changed data. The data exchange process is reduced to the necessary minimum which results in lower costs for the data exchange.
  • tcVISION can also use backup and recovery files (e.g. imagecopies, log files, UNLOADs) as a source for replication. Production data does not need to be touched.
  • tcVISION enables the fast and efficient load of large volumes of mainframe data into Big-Data (streaming). The processor costs of the mainframe are low and negligible.
  • An integrated data repository guarantees transparent data management across platforms.
  • Mainframe knowledge is not necessarily required for the replication.
  • tcVISION includes a rule engine to transform data into a target compliant format or allows user specific processing via supplied APIs.
  • The integrated staging concept supports the offload of changed data in “raw format” to less expensive processor systems. This reduces costs and mainframe processor resources to a minimum.
  • The preparation of the data for the target system can be performed on a less expensive platform (Linux, UNIX or MS-Windows).
  • The transfer to and streaming of data into Big-Data is part of the tcVISION data exchange process. No intermediate files are required.
  • The exchange of large volumes of data between a production mainframe environment and Big-Data can run in parallel processes to reduce latencies to a minimum.
  • The tcVISION integration platform contains comprehensive control mechanisms and monitoring functions for an automated data exchange.
  • tcVISION has been designed in a way that Big-Data based projects can be deployed with complete project autonomy and maximum reduction of mainframe resources.
Big Data Image 2

Benefits

The tcVISION integration platform consists of a variety of state-of-the-art technology components which cover far more than an ETL process.

  • Data exchange in the sense of real-time synchronization and replication turns into a “Single Step Operation” with tcVISION.
  • No additional middle-ware is required.
  • Diverse Change Data Capture technologies allow an efficient selection of the required data from the source system with focus on the changed data. The data exchange process is reduced to the necessary minimum which results in lower costs for the data exchange.
  • tcVISION can also use backup and recovery files (e.g. image-copies, log files, UNLOADs) as a source for replication. Production data does not need to be touched.
  • tcVISION enables the fast and efficient load of large volumes of mainframe data into Big-Data (streaming). The processor costs of the mainframe are low and negligible.
  • An integrated data repository guarantees transparent data management across platforms.
  • Mainframe knowledge is not necessarily required for the replication.
  • tcVISION includes a rule engine to transform data into a target compliant format or allows user specific processing via supplied APIs.
  • The integrated staging concept supports the offload of changed data in “raw format” to less expensive processor systems. This reduces costs and mainframe processor resources to a minimum.
  • The preparation of the data for the target system can be performed on a less expensive platform (Linux, UNIX or MS-Windows).
  • The transfer to and streaming of data into Big-Data is part of the tcVISION data exchange process. No intermediate files are required.
  • The exchange of large volumes of data between a production mainframe environment and Big-Data can run in parallel processes to reduce latencies to a minimum.
  • The tcVISION integration platform contains comprehensive control mechanisms and monitoring functions for an automated data exchange.
  • tcVISION has been designed in a way that Big-Data based projects can be deployed with complete project autonomy and maximum reduction of mainframe resources.

Our Partners

Fujitsu Partner
Oracle Partner
HP-PARTNER
IBM-Partner-Emblem-Web
BOS Logo
Microsoft Partner
Useful Links:
HOMEtcVISION ArchitecturetcVISION OverviewtcVISION Frequent QuestionstcVISION – Amazon Web ServicestcVISION Big DataReal Time Data Replication
Mainframe to CloudData SynchronisationDownloadstcVISION z/StreamtcACCESSNewsRead MoreDEMOPrivacy PolicyContact us
tcVISION is distributed in Australia and New Zealand by CCA Software Pty. Ltd. a division of FreeSoft Asia-Pacific
All Rights Reserved Copyright © 2019 Privacy Policy
error: Alert: Content is protected !!