In series of blogs around Oracle GoldenGate Foundation Suite (OGFS) products, I mentioned the new features for OGG Monitor in my previous blog. Today, I am pleased to announce that we have released the latest OGG Veridata Bundle patch(18.104.22.168.171215) end of December. In this Veridata Bundle patch, we have provided support for Hive (Big data), and automatic key column mapping.
Let me first write about automatic key column mapping.
In the absence of unique identifiers, such as the primary keys and unique indexes, which can uniquely identify each row, you need to manually define any column or columns to be used as a unique identifier. If you have hundreds of such tables, then it is an excessive work to select PKey for each table in the user interface (UI). However, there was a workaround to use the Veridata GoldenGate Parameter Processing (VGPP) and create the compare pair. It was still requires some work to be done from your side. Now with this new bundle patch, Veridata will select key columns automatically for you based on specific inputs provided while creating compare pairs or connections. If you have enabled automatic key column mapping at the Connection Level, then any group using this Connection has the feature enabled. You can override automatic key column mapping at the Compare Pair Generation stage.
The connection level has mainly two new options.
If you do not want to specify „automatic key selection” options at the connection level, you can do so while configuring compare pair as well. The Manual and Pattern compare pair configuration has following new options:-
Once you have provided your inputs and compare pair is created, you can see what Veridata has selected for each column in „Key Mapping Method” as mentioned below in the figure.
The other important feature was released in the bundle patch is supporting Big Data Hive. We know that you must be having your data replicated in big data targets. However, how will you make sure that the replicated data is correct? Veridata has always provided validation support for Databases. Now we have started and provided support for our first Big data target Hive. You can compare data between your source relational databases (Oracle, MSSQL, MySQL, Sybase, Informix, etc.) and Big Data hive (2.1.1+). Even you can do Hive to Hive comparison too.
The Hive comparison need not require any different options to be selected; you can make the Hive comparison just like you do Oracle to Oracle comparison. You would need to create the Hive connection (for your source or target) and create groups which will contain compare pairs from the source database and Hive database. Once all the compare pairs and options for primary key selection is selected, you can run the job. The Hive feature supports Delta comparison and raw partition. However, it does not support Hive authentication and Repair feature as of now.