Share via

Ingestion Jobs - Create

Service:
Azure AI Services
API Version:
2025-03-01-preview

Creates an ingestion job with the specified job id.

PUT {endpoint}/openai/ingestion/jobs/{job-id}?api-version=2025-03-01-preview

URI Parameters

Name In Required Type Description
endpoint
 path True

string (url)

Supported Cognitive Services endpoints (protocol and hostname, for example: https://aoairesource.openai.azure.com. Replace "aoairesource" with your Azure OpenAI account name).
job-id
 path True

string

The id of the job that will be created.
api-version
 query True

string

The requested API version.

Request Header

Name Required Type Description
mgmt-user-token

string

The token used to access the workspace (needed only for user compute jobs).
aml-user-token

string

The token used to access the resources within the job in the workspace (needed only for user compute jobs).

Request Body

The request body can be one of the following:

Name Description
IngestionJobSystemCompute
IngestionJobUserCompute

IngestionJobSystemCompute

Name Required Type Description
kind True string:

SystemCompute

IngestionJobType
The job type.
completionAction

IngestionJobCompletionAction

The completion action.
dataRefreshIntervalInHours

integer

datasource SystemComputeDatasource:

SystemComputeDatasource
jobId

string

searchServiceConnection BaseConnection:

BaseConnection
A connection to a resource.

IngestionJobUserCompute

Name Required Type Description
kind True string:

UserCompute

IngestionJobType
The job type.
workspaceId True

string

compute JobCompute:

JobCompute
The compute settings of the job.
dataRefreshIntervalInHours

integer

datasource UserComputeDatasource:

UserComputeDatasource
jobId

string

target TargetIndex:

TargetIndex
Information about the index to be created.

Responses

Name Type Description
200 OK IngestionJob:

Success
Other Status Codes

ErrorResponse

An error occurred.

Security

api-key

API key authentication

Type: apiKey
In: header

OAuth2Auth

OAuth2 authentication

Type: oauth2
Flow: implicit
Authorization URL: https://login.microsoftonline.com/common/oauth2/v2.0/authorize

Scopes

Name Description
https://cognitiveservices.azure.com/.default

Examples

Create a system-compute ingestion job
Create a user-compute ingestion job

Create a system-compute ingestion job

Sample request

PUT {endpoint}/openai/ingestion/jobs/ingestion-job?api-version=2025-03-01-preview

{
  "kind": "SystemCompute",
  "searchServiceConnection": {
    "kind": "EndpointWithManagedIdentity",
    "endpoint": "https://aykame-dev-search.search.windows.net"
  },
  "datasource": {
    "kind": "Storage",
    "connection": {
      "kind": "EndpointWithManagedIdentity",
      "endpoint": "https://mystorage.blob.core.windows.net/",
      "resourceId": "/subscriptions/1234567-abcd-1234-5678-1234abcd/resourceGroups/my-resource/providers/Microsoft.Storage/storageAccounts/mystorage"
    },
    "containerName": "container",
    "chunking": {
      "maxChunkSizeInTokens": 2048
    },
    "embeddings": [
      {
        "connection": {
          "kind": "RelativeConnection"
        },
        "deploymentName": "Ada"
      }
    ]
  },
  "dataRefreshIntervalInHours": 24,
  "completionAction": "keepAllAssets"
}

Sample response

Status code:
200 
operation-___location: https://aoairesource.openai.azure.com/openai/ingestion/jobs/ingestion-job/runs/72a2792ef7d24ba7b82c7fe4a37e379f?api-version=2025-03-01-preview
{
  "kind": "SystemCompute",
  "jobId": "ingestion-job",
  "searchServiceConnection": {
    "kind": "EndpointWithManagedIdentity",
    "endpoint": "https://aykame-dev-search.search.windows.net"
  },
  "datasource": {
    "kind": "Storage",
    "connection": {
      "kind": "EndpointWithManagedIdentity",
      "endpoint": "https://mystorage.blob.core.windows.net/",
      "resourceId": "/subscriptions/1234567-abcd-1234-5678-1234abcd/resourceGroups/my-resource/providers/Microsoft.Storage/storageAccounts/mystorage"
    },
    "containerName": "container",
    "chunking": {
      "maxChunkSizeInTokens": 2048
    },
    "embeddings": [
      {
        "connection": {
          "kind": "RelativeConnection"
        },
        "deploymentName": "Ada"
      }
    ]
  },
  "dataRefreshIntervalInHours": 24,
  "completionAction": "keepAllAssets"
}

Create a user-compute ingestion job

Sample request

PUT {endpoint}/openai/ingestion/jobs/ingestion-job?api-version=2025-03-01-preview

{
  "kind": "UserCompute",
  "workspaceId": "/subscriptions/f375b912-331c-4fc5-8e9f-2d7205e3e036/resourceGroups/adrama-copilot-demo/providers/Microsoft.MachineLearningServices/workspaces/adrama-rag-dev",
  "compute": {
    "kind": "ServerlessCompute"
  },
  "target": {
    "kind": "AzureAISearch",
    "connectionId": "/subscriptions/f375b912-331c-4fc5-8e9f-2d7205e3e036/resourceGroups/adrama-copilot-demo/providers/Microsoft.MachineLearningServices/workspaces/adrama-rag-dev/connections/search-connection"
  },
  "datasource": {
    "kind": "Dataset",
    "datasetId": "azureml://locations/centraluseuap/workspaces/83317fe6-efa6-4e4a-b020-d0edd11ec382/data/PlainText/versions/1",
    "datasetType": "uri_folder"
  }
}

Sample response

Status code:
200 
operation-___location: https://aoairesource.openai.azure.com/openai/ingestion/jobs/ingestion-job/runs/72a2792ef7d24ba7b82c7fe4a37e379f?api-version=2025-03-01-preview
{
  "kind": "UserCompute",
  "jobId": "ingestion-job",
  "workspaceId": "/subscriptions/f375b912-331c-4fc5-8e9f-2d7205e3e036/resourceGroups/adrama-copilot-demo/providers/Microsoft.MachineLearningServices/workspaces/adrama-rag-dev",
  "compute": {
    "kind": "ServerlessCompute"
  },
  "target": {
    "kind": "AzureAISearch",
    "connectionId": "/subscriptions/f375b912-331c-4fc5-8e9f-2d7205e3e036/resourceGroups/adrama-copilot-demo/providers/Microsoft.MachineLearningServices/workspaces/adrama-rag-dev/connections/search-connection"
  },
  "datasource": {
    "kind": "Dataset",
    "datasetId": "azureml://locations/centraluseuap/workspaces/83317fe6-efa6-4e4a-b020-d0edd11ec382/data/PlainText/versions/1",
    "datasetType": "uri_folder"
  }
}

Definitions

Name Description
AzureAISearchIndex

Azure AI Search Index.
ChunkingSettings

ChunkingSettings
ComputeType

The compute type.
ConnectionStringConnection

Connection string connection.
ConnectionType

The connection type.
CosmosDBIndex

CosmosDB Index.
CrawlingSettings

CrawlingSettings
CustomCompute

Custom compute.
DeploymentConnection

Relative deployment connection.
EndpointKeyConnection

Endpoint key connection.
EndpointMIConnection

Endpoint Managed Identity connection.
Error

Error
ErrorCode

ErrorCode
ErrorResponse

ErrorResponse
GenericEmbeddingSettings

ConnectionEmbeddingSettings
IngestionJobCompletionAction

The completion action.
IngestionJobSystemCompute
IngestionJobType

IngestionJobType
IngestionJobUserCompute
InnerError

InnerError
InnerErrorCode

InnerErrorCode
PineconeIndex

Pinecone Index.
ServerlessCompute

Serverless compute.
SystemComputeDatasourceType

The datasource type.
SystemComputeStorage

SystemComputeStorage
SystemComputeUrl

SystemComputeUrl
TargetType

The target type.
UserComputeDataset

UserComputeStorage
UserComputeDatasourceType

The datasource type.
UserComputeUrl

UserComputeUrl
WorkspaceConnection

AML Workspace connection.
WorkspaceConnectionEmbeddingSettings

WorkspaceConnectionEmbeddingSettings

AzureAISearchIndex

Object

Azure AI Search Index.

Name Type Description
connectionId

string

The id of the connection pointing to the Azure AI Search Index.
kind string:

AzureAISearch

The target type.

ChunkingSettings

Object

ChunkingSettings

Name Type Description
maxChunkSizeInTokens

integer

ComputeType

Enumeration

The compute type.

Value Description
ServerlessCompute

Serverless user compute.
CustomCompute

Custom user compute.

ConnectionStringConnection

Object

Connection string connection.

Name Type Description
connectionString

string

Connection string
kind string:

ConnectionString

The connection type.

ConnectionType

Enumeration

The connection type.

Value Description
EndpointWithKey

Endpoint and key connection.
ConnectionString

Connection string.
EndpointWithManagedIdentity

Endpoint and managed identity.
WorkspaceConnection

AML Workspace connection.
RelativeConnection

Relative deployment

CosmosDBIndex

Object

CosmosDB Index.

Name Type Description
collectionName

string

The name of the cosmos DB collection.
connectionId

string

The id of the connection pointing to the cosmos DB.
databaseName

string

The name of the cosmos DB database.
kind string:

CosmosDB

The target type.

CrawlingSettings

Object

CrawlingSettings

Name Type Description
maxCrawlDepth

integer

maxCrawlTimeInMins

integer

maxDownloadTimeInMins

integer

maxFileSize

integer

maxFiles

integer

maxRedirects

integer

CustomCompute

Object

Custom compute.

Name Type Description
computeId

string

Id of the custom compute
kind string:

CustomCompute

The compute type.

DeploymentConnection

Object

Relative deployment connection.

Name Type Description
kind string:

RelativeConnection

The connection type.

EndpointKeyConnection

Object

Endpoint key connection.

Name Type Description
endpoint

string

Endpoint
key

string

Key
kind string:

EndpointWithKey

The connection type.

EndpointMIConnection

Object

Endpoint Managed Identity connection.

Name Type Description
endpoint

string

Endpoint
kind string:

EndpointWithManagedIdentity

The connection type.
resourceId

string

Resource Id

Error

Object

Error

Name Type Description
code

ErrorCode

ErrorCode
Error codes as defined in the Microsoft REST guidelines (https://github.com/microsoft/api-guidelines/blob/vNext/Guidelines.md#7102-error-condition-responses).
details

Error[]

The error details if available.
innererror

InnerError

InnerError
Inner error as defined in the Microsoft REST guidelines (https://github.com/microsoft/api-guidelines/blob/vNext/Guidelines.md#7102-error-condition-responses).
message

string

minLength: 1

The message of this error.
target

string

The ___location where the error happened if available.

ErrorCode

Enumeration

ErrorCode

Value Description
conflict

The requested operation conflicts with the current resource state.
invalidPayload

The request data is invalid for this operation.
forbidden

The operation is forbidden for the current user/api key.
notFound

The resource is not found.
unexpectedEntityState

The operation cannot be executed in the current resource's state.
itemDoesAlreadyExist

The item does already exist.
serviceUnavailable

The service is currently not available.
internalFailure

Internal error. Please retry.
quotaExceeded

Quota exceeded.
jsonlValidationFailed

Validation of jsonl data failed.
fileImportFailed

Import of file failed.
tooManyRequests

Too many requests. Please retry later
unauthorized

The current user/api key is not authorized for the operation.
contentFilter

Image generation failed as a result of our safety system.

ErrorResponse

Object

ErrorResponse

Name Type Description
error

Error

Error
Error content as defined in the Microsoft REST guidelines (https://github.com/microsoft/api-guidelines/blob/vNext/Guidelines.md#7102-error-condition-responses).

GenericEmbeddingSettings

Object

ConnectionEmbeddingSettings

Name Type Description
connection BaseConnection:

BaseConnection
A connection to a resource.
deploymentName

string

modelName

string

IngestionJobCompletionAction

Enumeration

The completion action.

Value Description
cleanUpTempAssets

Will clean up intermediate assets created during the ingestion process.
keepAllAssets

Will not clean up any of the intermediate assets created during the ingestion process.

IngestionJobSystemCompute

Object
Name Type Description
completionAction

IngestionJobCompletionAction

The completion action.
dataRefreshIntervalInHours

integer

datasource SystemComputeDatasource:

SystemComputeDatasource
jobId

string

kind string:

SystemCompute

IngestionJobType
The job type.
searchServiceConnection BaseConnection:

BaseConnection
A connection to a resource.

IngestionJobType

Enumeration

IngestionJobType

Value Description
SystemCompute

Jobs that run on service owned resources.
UserCompute

Jobs that run on user owned workspace.

IngestionJobUserCompute

Object
Name Type Description
compute JobCompute:

JobCompute
The compute settings of the job.
dataRefreshIntervalInHours

integer

datasource UserComputeDatasource:

UserComputeDatasource
jobId

string

kind string:

UserCompute

IngestionJobType
The job type.
target TargetIndex:

TargetIndex
Information about the index to be created.
workspaceId

string

InnerError

Object

InnerError

Name Type Description
code

InnerErrorCode

InnerErrorCode
Inner error codes as defined in the Microsoft REST guidelines (https://github.com/microsoft/api-guidelines/blob/vNext/Guidelines.md#7102-error-condition-responses).
innererror

InnerError

InnerError
Inner error as defined in the Microsoft REST guidelines (https://github.com/microsoft/api-guidelines/blob/vNext/Guidelines.md#7102-error-condition-responses).

InnerErrorCode

Enumeration

InnerErrorCode

Value Description
invalidPayload

The request data is invalid for this operation.

PineconeIndex

Object

Pinecone Index.

Name Type Description
connectionId

string

The id of the connection pointing to the pinecone.
kind string:

Pinecone

The target type.

ServerlessCompute

Object

Serverless compute.

Name Type Description
instanceCount

integer

The count of instances to run the job on.
kind string:

ServerlessCompute

The compute type.
sku

string

SKU Level

SystemComputeDatasourceType

Enumeration

The datasource type.

Value Description
Storage

Azure Storage Account.
Urls

URLs.

SystemComputeStorage

Object

SystemComputeStorage

Name Type Description
chunking

ChunkingSettings

ChunkingSettings
Chunking settings
connection BaseConnection:

BaseConnection
A connection to a resource.
containerName

string

container name
embeddings

GenericEmbeddingSettings[]

ConnectionEmbeddingSettings
Connection Embedding Settings
kind string:

Storage

The datasource type.

SystemComputeUrl

Object

SystemComputeUrl

Name Type Description
chunking

ChunkingSettings

ChunkingSettings
Chunking settings
connection BaseConnection:

BaseConnection
A connection to a resource.
containerName

string

container name
crawling

CrawlingSettings

CrawlingSettings
Crawling settings
embeddings

GenericEmbeddingSettings[]

ConnectionEmbeddingSettings
Connection Embedding Settings
kind string:

Urls

The datasource type.
urls

string[]

TargetType

Enumeration

The target type.

Value Description
AzureAISearch

Azure AI Search Index.
CosmosDB

CosmosDB Index.
Pinecone

Pinecone Index.

UserComputeDataset

Object

UserComputeStorage

Name Type Description
chunking

ChunkingSettings

ChunkingSettings
Chunking settings
datasetId

string

datasetType

string

embeddings

WorkspaceConnectionEmbeddingSettings[]

WorkspaceConnectionEmbeddingSettings
Connection id to the embedding model
kind string:

Dataset

The datasource type.

UserComputeDatasourceType

Enumeration

The datasource type.

Value Description
Dataset

Workspace Dataset.
Urls

URLs.

UserComputeUrl

Object

UserComputeUrl

Name Type Description
chunking

ChunkingSettings

ChunkingSettings
Chunking settings
crawling

CrawlingSettings

CrawlingSettings
Crawling settings
embeddings

WorkspaceConnectionEmbeddingSettings[]

WorkspaceConnectionEmbeddingSettings
Connection id to the embedding model
kind string:

Urls

The datasource type.
urls

string[]

WorkspaceConnection

Object

AML Workspace connection.

Name Type Description
connectionId

string

ConnectionId
kind string:

WorkspaceConnection

The connection type.

WorkspaceConnectionEmbeddingSettings

Object

WorkspaceConnectionEmbeddingSettings

Name Type Description
connectionId

string

deploymentName

string

modelName

string