Note
Access to this page requires authorization. You can try signing in or changing directories.
Access to this page requires authorization. You can try changing directories.
In this article, you learn how to get data from OneLake into either a new or existing table.
Prerequisites
- A workspace with a Microsoft Fabric-enabled capacity
- A Lakehouse
- A KQL database with editing permissions
Step 1: Source
Select OneLake as your data source, as follows:
On the lower ribbon of your KQL database, select Get Data to open the Source tab of the Get data window.
Select the data source. In this example, you're ingesting data from OneLake or from the list in the embedded OneLake catalog.
Note
When you select a source from the list in the embedded OneLake catalog, you can use the category buttons or filter by keyword to search for a specific source.
Step 2: Configure
Pick a destination table and configure the source, as follows:
Select a target table. If you want to ingest data into a new table, select +New table and enter a table name.
Note
Table names can be up to 1,024 characters including spaces, alphanumeric, hyphens, and underscores. Special characters aren't supported.
Select a OneLake file to ingest:
When you select OneLake as your source, you must specify the Workspace, Lakehouse, and File from the dropdowns.
When you select the embedded OneLake catalog as your source, the Workspace and Lakehouse are automatically populated. You must specify the File to ingest.
Select Next.
Step 3: Inspect
The Inspect tab opens with a preview of the data.
To complete the ingestion process, select Finish.
You can view, inspect, and configure the ingested data. The numbers in the image correspond to these options:
(1) Select Command viewer to view and copy the automatic commands generated from your inputs.
(2) Use the Schema definition file dropdown to change the file that the schema is inferred from.
(3) Change the automatically inferred data format by selecting the desired format from the dropdown. For more information, see Data formats supported by Real-Time Intelligence.
(4) Edit columns.
(5) Explore Advanced options based on data type.
Edit columns
Note
- For tabular formats (CSV, TSV, PSV), you can't map a column twice. To map to an existing column, first delete the new column.
- You can't change an existing column type. If you try to map to a column having a different format, you may end up with empty columns.
The changes you can make in a table depend on the following parameters:
- Table type is new or existing
- Mapping type is new or existing
| Table type | Mapping type | Available adjustments |
|---|---|---|
| New table | New mapping | Rename column, change data type, change data source, mapping transformation, add column, delete column |
| Existing table | New mapping | Add column (on which you can then change data type, rename, and update) |
| Existing table | Existing mapping | none |
Mapping transformations
Some data format mappings (Parquet, JSON, and Avro) support simple ingest-time transformations. To apply mapping transformations, create or update a column in the Edit columns window.
Mapping transformations can be performed on a column of type string or datetime, with the source having data type int or long. For more information, see the full list of supported mapping transformations.
Advanced options based on data type
Tabular (CSV, TSV, PSV):
If you're ingesting tabular formats in an existing table, you can select Advanced > Keep table schema. Tabular data doesn't necessarily include the column names that are used to map source data to the existing columns. When this option is checked, mapping is done by-order, and the table schema remains the same. If this option is unchecked, new columns are created for incoming data, regardless of data structure.
Tabular data doesn't necessarily include the column names that are used to map source data to the existing columns. To use the first row as column names, select First row is column header.
Step 4: Summary
In the Data preparation window, all three steps are marked with green check marks when data ingestion finishes successfully. You can select a card to query, drop the ingested data, or see a dashboard of your ingestion summary.
