Detect Changes
    • Dark
      Light

    Detect Changes

    • Dark
      Light

    Article Summary

    This article is specific to the following platforms - Snowflake - Redshift - BigQuery - Synapse.

    Detect Changes Component

    The Detect Changes component lets users scan two separate (but similar) tables, and insert a new column detailing if data has been inserted, deleted, changed, or even if the data is unchanged.

    Any rows with key columns that contain NULL values will be ignored. NULL comparison values are considered equal.

    Properties

    PropertySettingDescription
    NameStringA human-readable name for the component.
    Master TableSelectSelect a master table from the two inputs. This table is the one treated as default in the comparison with the second table.
    Match KeysMultiple SelectSelect the key columns to join the two tables on. These columns must appear in both tables. NULL values are ignored.
    Compare ColumnsMultiple SelectSelect the columns that will be checked for changes. Just like the keys, these columns must appear in both tables; however, the two lists should not overlap.
    Output Column MappingInput ColumnSelect input columns to map to output names. Sensible defaults are provided automatically; however, these can be changed.
    Output ColumnName output columns to which selected input columns will map.
    Indicator ColumnStringInput a name for the new column in the output. By default, this column is named "Indicator". This column contains an indicator that shows the status of each record:
    C the record has been changed.
    D the record has been deleted.
    I the record is identical.
    N the record is new.
    Note: switching the master table in the Master Table property will reverse the meaning of new (N) and deleted (D).

    Strategy

    Detects changed, unchanged, added, or deleted data in a comparison table relative to the designated master table.

    Indicators

    Indicators are single-letter codes that indicate what the state of a row is with regard to Detect Changes. The table below shows all indicators and their meanings.

    IndicatorDescription
    CChanged: the record is present in both tables, with different values, but with the same ID.
    DDeleted: the record is present in the master table, but not in the second table.
    IIdentical: the same record is present in both tables with no changes.
    NNew: the record is not present in the master table, but is present in the second table.

    What's Next