Multi Table Input
    • Dark
      Light

    Multi Table Input

    • Dark
      Light

    Article Summary

    This article is specific to the following platforms - Snowflake - Redshift - Synapse - Delta Lake.

    Multi Table Input Component

    Read chosen columns from an input table into the job.

    The difference between Table Input and Multi Table Input is that Multi Table Input reads data from many input tables based on filtering all available input tables matching a pattern.

    The matching tables are expected to be very similar, e.g. Budgets_2012, Budget_2013 and so on, with a common set of columns.


    Properties

    Snowflake Properties

    PropertySettingDescription
    NameTextA human-readable name for the component.
    Pattern TypeSelectILike: The available tables are filtered using a case-insensitive SQL syntax pattern. See Snowflake ILike documentation.
    Like: The available tables are filtered using a case-sensitive SQL syntax pattern. See Snowflake Like documentation.
    Regex: The available tables are filtered using a POSIX EXE Regular Expression comparison. See Snowflake Regex documentation.
    DatabaseSelectChoose a database to create the new table in.
    SchemaSelectSelect the table schema. The special value, [Environmental Default] will use the schema defined in the environment. For more information on using multiple schemas, see this article.
    PatternTextThe pattern to match available tables to. The pattern syntax depends upon the chosen Pattern Type (see above).
    ColumnsChoiceThe available columns are generated by first scanning the available tables, and then providing all columns from any of the inputs. It is expected that the tables matching the pattern are fairly similar and share many columns. Use the Editor to select which columns to pass along.
    Cast TypesBooleanTrue: If the same-named column from multiple tables has a different data type, attempt to cast to a common type. Default is False.
    False: If the same-named column from multiple tables has a different data type, it is reported as an error. Default is False.
    Add Source Table NameTrue / FalseWhen set to True, Matillion adds a column, "source_table", containing the input table name that was matched to provide this row.

    Redshift Properties

    PropertySettingDescription
    NameTextA human-readable name for the component.
    SchemaSelectSelect the table schema. The special value, [Environmental Default] will use the schema defined in the environment. For more information on using multiple schemas, see this article.
    External schemas are not supported.
    PatternTextThe pattern to match available tables to. The pattern syntax depends upon the chosen Pattern Type (see below).
    Pattern TypeSelectLike: The available tables are filtered using an SQL Like comparison. See Redshift Like documentation.
    Similar To: The available tables are filtered using an SQL Similar To comparison. See Redshift Similar To documentation.
    Regex: The available tables are filtered using a POSIX Regular Expression comparison. See Redshift Regular Expression documentation.
    Column NamesChoiceThe available column names are generated by first scanning the available tables, and then providing all column names from any of the inputs. It is expected that the tables matching the pattern are fairly similar and share many columns.
    Cast TypesSelectYes: If the same-named column from multiple tables has a different data type, attempt to cast to a common type. Default is No.
    No: If the same-named column from multiple tables has a different data type, it is reported as an error. Default is No.
    Add Source Table NameYes / NoWhen set to Yes, adds a column, "source_table" containing the input table name that was matched to provide this row.
    Trim ColumnsSelectWraps the column names in a BTRIM function, which will strip out all the leading and trailing spaces. See the Redshift documentation for details.

    Synapse Properties

    PropertySettingDescription
    NameStringA human-readable name for the component.
    SchemaSelectSelect the table schema. The special value, [Environmental Default], will use the schema defined in the environment. For more information on using multiple schemas, see this article.
    PatternStringThe pattern to match available tables to. For more information, please refer to the Microsoft Azure documentation.
    ColumnsColumn selectSelect the one or more columns to load.
    Cast TypesBooleanA CAST command converts an expression of one data type to another. Default is true.
    For more information, please refer to the Microsoft Azure documentation.
    Add Source TableBooleanSelect whether to add the source table to the load. The default setting is false.

    Delta Lake Properties

    PropertySettingDescription
    NameTextA human-readable name for the component.
    CatalogSelectSelect a Databricks Unity Catalog. The special value, [Environment Default], will use the catalog specified in the Matillion ETL environment setup. Selecting a catalog will determine which databases are available in the next parameter.
    DatabaseSelectSelect the Delta Lake database. The special value, [Environment Default], will use the database specified in the Matillion ETL environment setup.
    PatternRegular ExpressionThe regular expression pattern used to filter out unwanted tables.
    Except for * and | characters, the pattern works like a regular expression.
    * alone matches 0 or more characters and | is used to separate multiple different regular expressions, any of which can match.
    The leading and trailing blanks are trimmed in the input pattern before processing. The pattern match is case-insensitive.
    ColumnsColumn SelectorSelect columns to include in the load. Move columns from the left list to the right list to include them.
    The available columns are generated by first scanning the available tables, and then providing all columns from any of the inputs. It is expected that the tables matching the pattern are fairly similar and share many columns.
    Cast TypesBooleanWhen True, if the same-named column from multiple tables has a different data type, attempt to cast to a common type.
    When False, if the same-named column from multiple tables has a different data type, an error is reported. The default setting is False.
    Add Source TableBooleanWhen True, Matillion ETL adds a column, "source_table", containing the input table name that was matched to provide this row. The default setting is False.

    Strategy

    Generates a set of select statements, concatenated together using UNION.



    Video


    What's Next