Start-AzureHDInsightJob
Starts an HDInsight job.
Note
The cmdlets referenced in this documentation are for managing legacy Azure resources that use Azure Service Manager (ASM) APIs. This legacy PowerShell module isn't recommended when creating new resources since ASM is scheduled for retirement. For more information, see Azure Service Manager retirement.
The Az PowerShell module is the recommended PowerShell module for managing Azure Resource Manager (ARM) resources with PowerShell.
Syntax
Start jobDetails on an HDInsight Cluster (Default)
Start-AzureHDInsightJob
-Cluster <String>
-JobDefinition <AzureHDInsightJobDefinition>
[-Credential <PSCredential>]
[-Profile <AzureSMProfile>]
[<CommonParameters>]
Start jobDetails on an HDInsight Cluster (with Specific Subscription Credential)
Start-AzureHDInsightJob
-Cluster <String>
-JobDefinition <AzureHDInsightJobDefinition>
[-Certificate <X509Certificate2>]
[-HostedService <String>]
[-Endpoint <Uri>]
[-IgnoreSslErrors <Boolean>]
[-Subscription <String>]
[-Profile <AzureSMProfile>]
[<CommonParameters>]
Description
This version of Azure PowerShell HDInsight is deprecated. These cmdlets will be removed by January 1, 2017. Please use the newer version of Azure PowerShell HDInsight.
For information about how to use the new HDInsight to create a cluster, see Create Linux-based clusters in HDInsight using Azure PowerShell (/azure/hdinsight/hdinsight-hadoop-create-linux-clusters-azure-powershell). For information about how to submit jobs by using Azure PowerShell and other approaches, see Submit Hadoop jobs in HDInsight (/azure/hdinsight/hadoop/submit-apache-hadoop-jobs-programmatically). For reference information about Azure PowerShell HDInsight, see Azure HDInsight Cmdlets.
The Start-AzureHDInsightJob cmdlet starts a defined Azure HDInsight job on a specified cluster. The job to start can be a MapReduce job, a streaming job, a Hive job, or a Pig job.
Examples
Example 1: Start an HDInsight job
PS C:\>$SubId = (Get-AzureSubscription -Current).SubscriptionId
PS C:\> $ClusterName = "Cluster01"
PS C:\> $WordCountJob = New-AzureHDInsightMapReduceJobDefinition -JarFile "/Example/Apps/Hadoop-examples.jar" -ClassName "Wordcount" -Defines @{ "mapred.map.tasks" = "3" } -Arguments "/Example/Data/Gutenberg/Davinci.txt", "/Example/Output/WordCount"
PS C:\> $WordCountJob | Start-AzureHDInsightJob -Cluster $ClusterName
| Wait-AzureHDInsightJob -Subscription $SubId -WaitTimeoutInSeconds 3600
| Get-AzureHDInsightJobOutput -Cluster $ClusterName -Subscription $SubId -StandardError
The first command gets the current subscription ID, and then stores it in the $SubId variable.
The second command assigns the name Cluster01 to the $ClusterName variable.
The third command uses the New-AzureHDInsightMapReduceJobDefinition cmdlet to create a MapReduce job definition, and then stores it in the $WordCountJob variable.
The final command uses the pipeline operator to pass the $WordCountJob to the Start-AzureHDInsightJob cmdlet to start the job. After the job starts, it is passed to the Wait-AzureHDInsightJob cmdlet, which waits for the job to complete before passing it to the Get-AzureHDInsightJobOutput cmdlet to get the job output.
Parameters
-Certificate
Specifies the management certificate for an Azure subscription.
Parameter properties
| Type: | X509Certificate2 |
| Default value: | None |
| Supports wildcards: | False |
| DontShow: | False |
| Aliases: | Cert |
Parameter sets
Start jobDetails on an HDInsight Cluster (with Specific Subscription Credential)
| Position: | Named |
| Mandatory: | False |
| Value from pipeline: | False |
| Value from pipeline by property name: | False |
| Value from remaining arguments: | False |
-Cluster
Specifies a cluster. This cmdlet starts a job on the cluster that this parameter specifies.
Parameter properties
| Type: | String |
| Default value: | None |
| Supports wildcards: | False |
| DontShow: | False |
| Aliases: | ClusterName |
Parameter sets
(All)
| Position: | Named |
| Mandatory: | True |
| Value from pipeline: | True |
| Value from pipeline by property name: | False |
| Value from remaining arguments: | False |
-Credential
Specifies cluster credentials for direct HTTP access to a cluster. You can specify this parameter instead of the Subscription parameter to authenticate access to a cluster.
Parameter properties
| Type: | PSCredential |
| Default value: | None |
| Supports wildcards: | False |
| DontShow: | False |
| Aliases: | Cred |
Parameter sets
Start jobDetails on an HDInsight Cluster
| Position: | Named |
| Mandatory: | False |
| Value from pipeline: | False |
| Value from pipeline by property name: | False |
| Value from remaining arguments: | False |
-Endpoint
Specifies the endpoint to use to connect to Azure. If you do not specify this parameter, this cmdlet uses the default endpoint.
Parameter properties
| Type: | Uri |
| Default value: | None |
| Supports wildcards: | False |
| DontShow: | False |
Parameter sets
Start jobDetails on an HDInsight Cluster (with Specific Subscription Credential)
| Position: | Named |
| Mandatory: | False |
| Value from pipeline: | False |
| Value from pipeline by property name: | False |
| Value from remaining arguments: | False |
-HostedService
Specifies the namespace of an HDInsight service if you do not want to use the default namespace.
Parameter properties
| Type: | String |
| Default value: | None |
| Supports wildcards: | False |
| DontShow: | False |
| Aliases: | CloudServiceName |
Parameter sets
Start jobDetails on an HDInsight Cluster (with Specific Subscription Credential)
| Position: | Named |
| Mandatory: | False |
| Value from pipeline: | False |
| Value from pipeline by property name: | False |
| Value from remaining arguments: | False |
-IgnoreSslErrors
Indicates whether Secure Sockets Layer (SSL) errors are ignored.
Parameter properties
| Type: | Boolean |
| Default value: | None |
| Supports wildcards: | False |
| DontShow: | False |
Parameter sets
Start jobDetails on an HDInsight Cluster (with Specific Subscription Credential)
| Position: | Named |
| Mandatory: | False |
| Value from pipeline: | False |
| Value from pipeline by property name: | False |
| Value from remaining arguments: | False |
-JobDefinition
Specifies the endpoint to use when connecting to Microsoft Azure if the endpoint is different from the default.
Parameter properties
| Type: | AzureHDInsightJobDefinition |
| Default value: | None |
| Supports wildcards: | False |
| DontShow: | False |
| Aliases: | jobDetails |
Parameter sets
(All)
| Position: | Named |
| Mandatory: | True |
| Value from pipeline: | True |
| Value from pipeline by property name: | False |
| Value from remaining arguments: | False |
-Profile
Specifies the Azure profile from which this cmdlet reads. If you do not specify a profile, this cmdlet reads from the local default profile.
Parameter properties
| Type: | AzureSMProfile |
| Default value: | None |
| Supports wildcards: | False |
| DontShow: | False |
Parameter sets
(All)
| Position: | Named |
| Mandatory: | False |
| Value from pipeline: | False |
| Value from pipeline by property name: | False |
| Value from remaining arguments: | False |
-Subscription
Specifies a subscription. This cmdlet starts a job for the subscription that this parameter specifies.
Parameter properties
| Type: | String |
| Default value: | None |
| Supports wildcards: | False |
| DontShow: | False |
| Aliases: | Sub |
Parameter sets
Start jobDetails on an HDInsight Cluster (with Specific Subscription Credential)
| Position: | Named |
| Mandatory: | False |
| Value from pipeline: | False |
| Value from pipeline by property name: | False |
| Value from remaining arguments: | False |
CommonParameters
This cmdlet supports the common parameters: -Debug, -ErrorAction, -ErrorVariable, -InformationAction, -InformationVariable, -OutBuffer, -OutVariable, -PipelineVariable, -ProgressAction, -Verbose, -WarningAction, and -WarningVariable. For more information, see about_CommonParameters.