Potrzebujesz pomocy?
Nasi specjaliści są do Twojej dyspozycji.
 

BAZY DANYCH
tel. +48 888 966 219

 

GOLDENGATE
tel. +48 505 549 517

 

ORACLE DATA INTEGRATOR
tel. +48 530 460 140

Oracle GoldenGate Veridata 12.2+
new enhancements

By: Nisharahmed Soneji | Passionate Senior Product Manager, Oracle Data Integration

In series of blogs around Oracle GoldenGate Foundation Suite (OGFS) products, I mentioned the new features for OGG Monitor in my previous blog. Today, I am pleased to announce that we have released the latest OGG Veridata Bundle patch(12.2.1.2.171215) end of December. In this Veridata Bundle patch, we have provided support for Hive (Big data), and automatic key column mapping.

Let me first write about automatic key column mapping.

In the absence of unique identifiers, such as the primary keys and unique indexes, which can uniquely identify each row, you need to manually define any column or columns to be used as a unique identifier. If you have hundreds of such tables, then it is an excessive work to select PKey for each table in the user interface (UI). However, there was a workaround to use the Veridata GoldenGate Parameter Processing (VGPP) and create the compare pair. It was still requires some work to be done from your side. Now with this new bundle patch, Veridata will select key columns automatically for you based on specific inputs provided while creating compare pairs or connections. If you have enabled automatic key column mapping at the Connection Level, then any group using this Connection has the feature enabled. You can override automatic key column mapping at the Compare Pair Generation stage.

 

The connection level has mainly two new options.

  • Use Source or Target Columns as Key Columns When Generating Compare Pairs: If the source table has primary keys or unique index defined, but the target does not, then Oracle GoldenGate Veridata uses the same columns (as the source) as unique identifiers for the target. Similarly, if the source does not have the primary key or index, but the target has them, then key columns of target are used for source. If either (source or target) of the columns of the Primary Key or index is not present on the target side, then the primary key or index is not considered.
  • Use All Columns as Key Columns When Generating Compare Pairs: Enables Automatic mapping for the source and target connections. If this option is disabled either at the source or the target connection, then the automatic mapping is also disabled for that group. If you enable this option to map all columns from source and target, then the mapping is considered only when both source and target table do not have primary or unique keys.

 

 

If you do not want to specify „automatic key selection” options at the connection level, you can do so while configuring compare pair as well. The Manual and Pattern compare pair configuration has following new options:-

  • Use Source Key Columns for Target: Enables automatic mapping for the target table. If you disable the option for the target side, and if there is no primary key or unique index on the target side and the source has primary key or unique index, then only the mapping of columns happen but, automatic mapping does not happen on the target table.
  • Use Target Key Columns for Source: Enables automatic mapping for the source table. If you disable the option for Source side, and if there is no primary key or unique index on the source side and target has primary key or unique index, then only the mapping of columns happen but, automatic mapping does not happen on the source table.
  • Use All Columns as Key Columns When Generating Compare Pairs: Enables automatic mapping for all columns. Ensure that this option is selected both at the source as well as the target groups. If the option is disabled either at the source or the target levels, then the automatic mapping is disabled for the corresponding groups.

 

 

Once you have provided your inputs and compare pair is created, you can see what Veridata has selected for each column in „Key Mapping Method” as mentioned below in the figure.

 

 

The other important feature was released in the bundle patch is supporting Big Data Hive. We know that you must be having your data replicated in big data targets. However, how will you make sure that the replicated data is correct? Veridata has always provided validation support for Databases. Now we have started and provided support for our first Big data target Hive. You can compare data between your source relational databases (Oracle, MSSQL, MySQL, Sybase, Informix, etc.) and Big Data hive (2.1.1+). Even you can do Hive to Hive comparison too.

The Hive comparison need not require any different options to be selected; you can make the Hive comparison just like you do Oracle to Oracle comparison. You would need to create the Hive connection (for your source or target) and create groups which will contain compare pairs from the source database and Hive database. Once all the compare pairs and options for primary key selection is selected, you can run the job. The Hive feature supports Delta comparison and raw partition. However, it does not support Hive authentication and Repair feature as of now.