Edit

Share via

Iceberg format in Azure Data Factory and Azure Synapse Analytics

APPLIES TO: Azure Data Factory Azure Synapse Analytics

Tip

Try out Data Factory in Microsoft Fabric, an all-in-one analytics solution for enterprises. Microsoft Fabric covers everything from data movement to data science, real-time analytics, business intelligence, and reporting. Learn how to start a new trial for free!

Follow this article when you want to write the data into Iceberg format.

Iceberg format is supported for the following connectors:

You can use Iceberg dataset in Copy activity.

Dataset properties

For a full list of sections and properties available for defining datasets, see the Datasets article. This section provides a list of properties supported by the Iceberg format dataset.

Property Description Required
type The type property of the dataset must be set to Iceberg. Yes
___location Location settings of the file(s). Each file-based connector has its own ___location type and supported properties under ___location. Yes

Below is an example of Iceberg dataset on Azure Data Lake Storage Gen2:

{
    "name": "IcebergDataset",
    "properties": {
        "type": "Iceberg",
        "linkedServiceName": {
            "referenceName": "<Azure Data Lake Storage Gen2 linked service name>",
            "type": "LinkedServiceReference"
        },
        "schema": [ < physical schema, optional, auto retrieved during authoring >
        ],
        "typeProperties": {
            "___location": {
                "type": "AzureBlobFSLocation",
                "fileSystem": "filesystemname",
                "folderPath": "folder/subfolder",
            }
        }
    }
}

Copy activity properties

For a full list of sections and properties available for defining activities, see the Pipelines article. This section provides a list of properties supported by the Iceberg sink.

Iceberg as sink

The following properties are supported in the copy activity *sink* section.

Property Description Required
type The type property of the copy activity source must be set to IcebergSink. Yes
formatSettings A group of properties. Refer to Iceberg write settings table below. No
storeSettings A group of properties on how to write data to a data store. Each file-based connector has its own supported write settings under storeSettings. No

Supported Iceberg write settings under formatSettings:

Property Description Required
type The type of formatSettings must be set to IcebergWriteSettings. Yes

Here are some common connectors and formats related to the delimited text format:

Related content