tcVISION Data Synchronisation News

From Mainframe to Kafka

The Open Source platform Apache Kafka is a key platform for high scalable systems. Since the Open Source start in 2011 Kafka has reached a wide distribution in modern IT environments which are focusing on the processing of large data volumes. The trend to use Kafka in the area of analytics and Data Hub projects is steadily increasing.

The currency of the data is the decisive factor for the success of the project.

Low latency times during the apply of data from a variety of mainframe, Oracle, MS-SQL Server etc. data sources is a definite requirement for the real-time processing of corporate data, regardless of their origin, for analytical research, Business Intelligence or Data Warehousing.

We are happy to demonstrate how you can efficiently master the real-time integration of your corporate data into your Data Hub using our tcVISION solution, with low latency, no programming efforts and in real-time:

tcVISION supports the WADS on VSAM LDS.

Using the tcVISION Change Data Capture Technology to integrate your data sources will substantially speed up the realization of your project.

See: tcVISION Extension for Kafka

 

If you are currently developing or maintaining your Data Hub using Kafka we are looking forward to hearing from you

Implementation of new Output Targets

Db2 V12 is now supported.

Couchbase Server is planned to be available in the coming months. This also applies to Elasticsearch.

tcVISION supports the WADS on VSAM LDS.

Support of new database versions:

Db2 V12 is now supported.
Please note that Big Blue (IBM) has changed the appearance of the name to Db2 (with a small “b”).

ORACLE V19+ is supported.
That was a bit of a challenge because ORACLE changed their log-miner interface and removed the function to inform a CDC client whenever new data became part of the Oracle log, basically closing the door to use the Oracle log for external vendors. tcVISION still can be used to process the Oracle log because we have changed the log processing from passive (waiting to be informed) to active (ask for new data in intervals). This support also applies to RAC.

IMS V15 is supported.
Technically the WADS (Write Ahead Data Set) of IMS has been changed from a sequential file to a VSAM LDS (Linear Data Set).
Remember: The WADS contains a copy of committed log 
 records that are in OLDS buffers, but that have 
 not yet been written to the OLDS.
tcVISION supports the WADS on VSAM LDS.

Support of long passwords/passphrases

This support has been implemented into tcACCESS (V8 and V9) and tcVISION as of Version 6.
Passphrases can be used to sign on to tcACCESS mainframe component or VTAM application and tcVISION Managers.

tcVISION Supports Mainframe to Kafka in Real-time

Application modernisation, new requirements like Real-time Analytics and Big Data are very often the initial impulse for the renewal of IT infrastructures in the area of z mainframes. The decreasing mainframe competence is another risk factor for an enterprise critical mainframe installation.

Data exchange: A first step in the renewal.

A “High Volume“ and “High Speed“ data transmission in real-time from the mainframe to Kafka beyond traditional and batch oriented ETL-applications is a challenge!

The keys to success:

1. Parallelisation during update of data. 
Parallelisation requires parallel and coordinated receive- and update-engines. These engines are part of the tcVISION solution to synchronise data.

2. Planning for the failure of components during a cross-system data exchange in real-time. tcVISION takes that into account and offers solutions.

3. Competent and experienced partner.

Our tcVISION solution is successfully in use at a large number of z mainframe installations. (We know z mainframe and can z mainframe)

On request we are happy to create a concept for the realisation of your “Low Latency” data exchange solution from the mainframe to Kafka using our tcVISION solution.

tcVISION now supports the new version of ORACLE 18c

The last official Oracle version has been 12c and you may ask why the heck has it been going up by 6 digits? Well the truth is sometime very simple:

Quote “Oracle Database 18c is in fact “Oracle Database 12c Release 2 12.2.0.2”, the name has simply been changed to reflect the year in which the product is released.”

For our z/OS version of tcVISION we have now completed the Logreader support for IDMS.

This completes our development efforts for z/OS to access all mainframe sources using Logreader or Logstream access.

The collector/pool strategy is still part of the product but we recommend to use the new access strategy.

The VSE version of tcVISION still uses the collector/pool access because VSE does not provide these alternatives

IDMS BULK is now supported from BACKUP media.

This is supported for both operating systems. As with other BACKUP files (DB3 or IMS), the IDMS Backup can be FTP’d to a WIndows or UNIX/LINUX workstation and the BULK can performed on these less expensive platforms.

JDBC Support for sources and targets

JDBC databases can now be used as targets and sources (APPLY and BULK).

Tips and Tricks

New documents have been added to the Distributor Portal of our website. The editions from 2018 are:

  • tcVISION and MySQL/MariaDB
  • tcVISION and PostgreSQL
  • tcVISION – Implementation of a LOGSTREAM replication

Log based Change Data Capture for IBM z Db2 without mainframe agent

Flexible and efficient data exchange (synchronisation/replication/migration) with tcVISION for IBM z-, distributed- and cloud-platforms:

· Fast and efficient load of mass data
· Lower latency time because of parallelisation and real-time apply
· Continuous log based Change Data Capture data streaming for real-time data synchronisation
· Transparency because of comprehensive data mapping for structure analysis
· Comfortable data transformation and data conversion
· No programming efforts
· Integrated database specific apply mechanisms to enter the data into the targets
· Bidirectional synchronisation (Master – Master replication) with integrated Loop prevention

We would be happy to present our data integration solution tcVISION for IBM z-, distributed- and cloud-platforms and show solutions that meet your requirements.

This new functionality will be available for general download with the next refreshment of our distributor page first week of October 2018.

New and additional connection type has been added for Active Log processing in Db2 z/OS:

As you know the tcVISION solution offers connections to a variety of databases and file-systems on z/OS, z/VSE, z/VM and different operating systems in the open world (Linux, Unix, Windows). Now, a new and additional connection type has been added for Active Log processing in Db2 z/OS:

The agent-less connection to Db2 on a z/OS mainframe.

A permanent active z/OS tcVISION agent is no longer required for the processing of Db2 z/OS Active Logs. This “agent-less” connection analyses the Db2 Active log and extracts the changed data using a Db2 UDT (User Defined Table) and a connection to Db2 z/OS via DRDA or a Db2-Client.

Heinrich von Parseval, Director Research and Development:

“The installation of tcVISION in an z/OS Db2 environment has been further simplified because of this additional connectivity to Db2 z/OS. A tcVISION Host agent is not any more required but tcVISION can still monitor the Db2 Active Log and extract changed data in near real-time without using noticeable resources on the mainframe. This is an interesting perspective – may be even for future development efforts of tcVISION.”

This new functionality is in the final stage of development and testing and soon will hit the streets

B.O.S. Implements an agent-less Connection to Db2 z/OS

The tcVISION solution offers connections to a variety of databases and file systems on z/OS, z/VSE, z/VM, and different operating systems in the open world (Linux, UNIX, Windows). Detailed information can be found here.

A new connection type has been added for Active Log processing in Db2 z/OS:

A permanent active z/OS tcVISION agent is no longer required for the processing of mainframe Db2 z/OS Active Logs. This “agent-less” connection analyses the Db2 Active Log and extracts the changed data using a Db2 UDT (User Defined Table) and a connection to Db2 z/OS via DRDA or a Db2 Client.

Heinrich von Parseval, Director Research and Development: “The installation of tcVISION in a z/OS Db2 environment has been further simplified because of this additional connectivity to Db2 z/OS. A tcVISION Host-Agent is not required anymore, but tcVISION can still monitor the Db2 Active Log and extract changed data in near real-time without using noticeable resources on the mainframe. This is an interesting perspective – maybe even for future development efforts of tcVISION.“

B.O.S. Partner Conference in Haar

Partnerships are the base of a successful and world-wide representation.

B.O.S. Software Service und Vertrieb GmbH is partnering with renowned software companies in the USA, Europe, Asia, South Africa, and Australia. The existing customer base in these parts of the world is proof that the partnerships are successful.

B.O.S. invites the international partners to a conference in regular intervals. This year, the conference was conducted in Haar in April. B.O.S. presented their ongoing developments and gave insights and impressions of new releases and versions of the software solutions tcVISION and tcACCESS.

The partners also actively participated in the conference. The dynamics of the IT world, especially caused by Big Data, cloud computing and new database technologies were discussed and the partners were shown how the B.O.S. solutions cooperate with these new technologies. Different customer implementations were discussed and the strategy for a furthermore successful cooperation was consolidated.

B.O.S. is convinced that the partnerships* will continue to be the guarantee for a successful distribution of the software solutions tcVISION and tcACCESS.

* B.O.S. partners:

  • Treehouse Software – USA
  • The amod – USA
  • Bateleur – South Africa
  • CCA Software – Australia
  • Yam Pong Corporation – Taiwan
  • JVL/Syspertec – Frankreich
  • UP Consulting – Spain
  • Team Data – Skandinavia
  • B.O.S. Software S.r.l. – Italy
  • Formula Opensoft – Benelux
  • B19 Data Solutions Ltd. – United Kingdom

tcVISION supports MySQL / MariaDB

The world-wide recognition and great popularity of the relational databases MySQL and especially MariaDB is increasing. MariaDB is available as an open source database management system and is therefore a cost and licence free alternative to relational DBMS in the client server environment that require a licence.

B.O.S. Software Service und Vertrieb takes account of this trend: tcVISION supports MySQL and MariaDB as a target system for replication processes.

Both databases can also be used as the tcVISION repository.

The MySQL / MariaDB APPLY function is available for the implementation of data from other data sources in the distributed environment with MySQL / MariaDB as target, such as mainframe Db2, ADABAS, IMS/DB, VSAM, Oracle, Db2 LUW, etc.

The initial load of MySQL / MariaDB databases (BULK LOAD) can be performed using the internal MySQL / MariaDB loader. The connectivity to the two databases has been implemented using the official MySQL / MariaDB interface. The wide range of database systems and platforms supported by tcVISION is unique. This is backed up by the constant growth of world-wide installations.

B.O.S. Products support the new version of z/OS 2.3

The most recent version of the z/OS operating system (Version 2.3) is generally available since September 29, 2017. in addition to new functions and technical enhancements IBM positions the new operating system flagship as an answer to the “rapid evolution toward hybrid IT architectures”.

Both B.O.S. solutions tcVISION and tcACCESS support this new version of the z/OS operating system.

Heinrich von Parseval, CTO and Director Research & Development: “It has always been our constant effort to provide a timely support for new versions of the IBM z/OS AND z/VSE operating systems. This support is important for customers who want to perform an immediate upgrade to the new version. This B.O.S. philosophy basically applies to all operating system platforms used by our software solutions (refer to: http://www.bossoftware.de/index.php/en/solutions/data-synchronisation-replication#technologien)”.

Customers of tcACCESS and tcVISION who are already using or planning to use z/OS 2.3 can download the current product versions from http://www.bossoftware.de/index.php/en/support.

tcACCESS and tcVISION listed under “Software developers supporting z/OS V2R3”

IBM publishes a list of all software products which support the new version 2.3 of the z/OS operating system. Follow this link to the list.

Elisabeth Schwarz, Managing Director of B.O.S.: “It is our claim and ambition to provide a timely support for new releases of operating systems for our software solutions. In the case of z/OS 2.3 we have provided this support in time – like we always have in the past. Both B.O.S. solutions tcACCESS and tcVISION have been included by IBM in the directory of software developers supporting z/OS 2.3.”

Customers of tcACCESS and tcVISION who are already using or planning to use z/OS 2.3 can download the current product versions from http://www.bossoftware.de/index.php/en/support.

B.O.S. supports IBM’s new POWER8 platform

“IBM POWER Systems offer servers built with open technologies and designed for mission-critical applications. Today’s business leaders demand servers designed for Big Data that are optimised, secure, and adapt to changing workload requirements.”

The software solutions from B.O.S. help customers share their data in real-time across multiple platforms including mainframe, distributed, Hadoop and Cloud.

B.O.S. supports the new POWER8 platform with their software solutions. Heinrich von Parseval, CTO and Director Research & Development: “We are impressed by Linux on IBM POWER8 and the high performance and scalability of this trend-setting technology.”

Read the complete statement from Mr. von Parseval – concerning IBM POWER8 – here (Letter B):

* Quote from IBM

tcVISION supports SAP Hana

With tcVISION mainframe data and data from the distributed applications environment (LUW) can now be replicated or migrated easily and quickly to SAP Hana. There is no programming effort required.

  • No additional middle-ware required
  • The data volume is reduced to a minimum (compressed and transferred in “raw format”).
  • Comprehensive data mapping simplifies the selection of data and creates transparency.
  • Intuitive user interface and various wizards reduce the implementation time to a minimum.
  • Automatic character conversion between the systems and platforms.
  • The integration of data from different source data models (relational and non-relational) on different platforms to SAP Hana is fully supported.

PostgreSQL support in tcVISION is available now

tcVISION replicates data on the basis of changes in real-time and automatically implements the changes into the target systems. From now on the object-oriented, relational DBMS is also supported – it is available as Open Source and therefore a cost and licence free alternative to relational DBMS in the client server environment that require a licence.

PostgreSQL is supported as source and target of a data replication or data synchronisation with tcVISION. The PostgreSQL Change Data Capture option is an optional component of tcVISION; It automatically captures changes in the PostgreSQL DBMS and replicates them into the respective target systems.

The PostgreSQL APPLY option is available for the implementation of data from other data sources in the distributed environment with PostgreSQL as target, such as mainframe Db2, ADABAS, IMS/DB, VSAM, Oracle, Db2 LUW, etc. Therefore PostgreSQL users can exchange data in heterogeneous system environments easily and quickly without programming effort. The migration from an existing DBMS in PostgreSQL is also easily realisable. Special database know how is not required.

tcVISION now supports IAM Files – a solution of Innovation Data Processing

“The Innovation Access Method (IAM) is a transparent alternative to VSAM KSDS, ESDS and (as a cost option) AIX and RRDS files. The savings achieved over VSAM can vary, but IAM typically uses 40-80% less I/Os, 20-70% less CPU and 30-70% less DASD. Savings in batch elapsed times and online response times can range from 20-80%.“

Users of IAM files now have the possibility of replicating data into any DBMS and NoSQL DBMS systems in the distributed systems environment as well as into Cloud and Hadoop platforms in real-time.

Thus, latest technology can easily be used for Analytics and Business Intelligence. tcVISION also supports bidirectional synchronisation between IAM files and DBMS in the distributed environment. Therefore data between – for example – Oracle, SQL Server, Db2 LUW, Informix, ADABAS LUW can be synchronised in real-time without programming effort. The transparent integration of IAM files in a heterogeneous system environment can be easily and quickly implemented.

Fully Integrated Connection of Mainframe Data to Apache Kafka

tcVISION now allows streaming change data directly to Apache Kafka.
New features:

  • Propagate change data to Apache Kafka in real-time
  • The currently supported protocols are JSON, Avro and CSV
  • Connection using official interfaces ensures a fast and simple implementation
  • Any data source supported by tcVISION can be used as data source:
    • Mainframe: ADABAS, Db2, VSAM, CICS, DATACOM, IMS and others
    • Open systems: ADABAS, Oracle, Microsoft SQL Server, Db2 LUW and others

Replication of Data to MongoDB

tcVISION now fully supports MongoDB as a replication target
New features:

  • Join several tables into one MongoDB document
  • No message queueing necessary
  • No additional middleware required
  • Available for UNIX/Linux and Windows
  • Load processes can be parallelised for highly efficient data transfer
  • Integrated key management for non-indexed data
  • Detailed monitoring, logging and integrated alert notification

Direct Access to Mainframe Data from Hadoop Systems

The tcACCESS JDBC server now supports direct access to mainframe data from Hadoop systems.
New Features:

  • JDBC access to relational mainframe data e.g. Db2
  • JDBC to non-relational mainframe data such as ADABAS, VSAM, Datacom, IMS, etc.
  • Access to mainframe data is completely transparent via the JDBC driver
  • No additional middleware required
  • Hadoop developers do not need mainframe knowledge
Useful Links:
HOMEtcVISION ArchitecturetcVISION OverviewtcVISION Frequent QuestionstcVISION – Amazon Web ServicestcVISION Big DataReal Time Data Replication
Mainframe to CloudData SynchronisationDownloadstcVISION z/StreamtcACCESSNewsRead MoreDEMOPrivacy PolicyContact us
tcVISION is distributed in Australia and New Zealand by CCA Software Pty. Ltd. a division of FreeSoft Asia-Pacific
All Rights Reserved Copyright © 2019 Privacy Policy
error: Alert: Content is protected !!