[BUG]: Reconcile of type schema and source Oracle does not find differences. #1567
Description
Is there an existing issue for this?
- I have searched the existing issues
Category of Bug / Issue
ReconcileError
Current Behavior
Hi, I'm trying to run Reconcile with the schema-only option, and it does not seem to find any differences between Oracle and Databricks. I have tested with entirely different tables and another table with only datatype difference.
Expected Behavior
Output the difference / Show the mismatch of schemas.
Steps To Reproduce
- Create a Table in Oracle
- Create a Table in Databricks with different schema
- Run the Reconcile with schema only option
(This was tested running reconcile in a Notebook in Databricks with DBR 15.4 LTS, Oracle version was 21c running in docker)
Relevant log output or Exception details
14:40:35 WARN [d.l.r.reconcile.execute] Schema comparison is completed.
14:40:38 WARN [d.l.r.reconcile.recon_capture] reconciled_record_count : ReconcileRecordCount(source=0, target=0)
ReconcileOutput(recon_id='1c119c68-450b-445f-a61a-a6aca7a24d92', results=[ReconcileTableOutput(target_table_name='malcoln.BCP.table_test_1', source_table_name='BCP.table_test_1', status=StatusOutput(row=None, column=None, schema=True, aggregate=None), exception_message='')])
***************************Sample Query
-- Oracle
CREATE TABLE BCP.TABLE_TEST_1
(
id NUMBER,
name VARCHAR(255)
)
-- Databricks
create table malcoln.bcp.TABLE_TEST_1
(
id float,
name VARCHAR(255)
)
-- Reconcile config
reconcile_config = ReconcileConfig(
data_source = "oracle",
report_type = "schema", # schema,row,data or all https://github.com/databrickslabs/remorph/tree/main/docs/recon_configurations#types-of-report-supported
secret_scope = "rampim",
database_config= DatabaseConfig(#source_catalog="BCP",
source_schema="BCP",
target_catalog="malcoln",
target_schema="BCP",),
metadata_config = ReconcileMetadataConfig(
catalog = "malcoln",
schema= "reconcile"
)
)
-- table recon config
table_recon = TableRecon(
source_schema="BCP",
target_catalog="malcoln",
target_schema="BCP",
tables=[
Table(
source_name="TABLE_TEST_1",
target_name="TABLE_TEST_1",
)
],
)Operating System
macOS
Version
other