Edit

Share via

Add-AzHDInsightMetastore

Module:
Az.HDInsight Module

Adds a SQL Database to serve as a Hive or Oozie metastore to a cluster configuration object.

Syntax

Default (Default) 

Add-AzHDInsightMetastore
    [-Config] <AzureHDInsightConfig>
    [-MetastoreType] <AzureHDInsightMetastoreType>
    [-SqlAzureServerName] <String>
    [-DatabaseName] <String>
    [-Credential] <PSCredential>
    [-DefaultProfile <IAzureContextContainer>]
    [<CommonParameters>]

Description

The Add-AzHDInsightMetastore cmdlet adds a Hive or Oozie metastore to the HDInsight configuration object created by the New-AzHDInsightClusterConfig cmdlet. A metastore is a SQL Database that can used to store metadata for Hive, Oozie, or both.

Examples

Example 1: Add a SQL database metastore to the cluster configuration object

# Primary storage account info
$storageAccountResourceGroupName = "Group"
$storageAccountResourceId = "yourstorageaccountresourceid"
$storageAccountName = "yourstorageacct001"
$storageAccountKey = (Get-AzStorageAccountKey -ResourceGroupName $storageAccountResourceGroupName -Name $storageAccountName)[0].value


$storageContainer = "container001"

# Cluster configuration info
$___location = "East US 2"
$clusterResourceGroupName = "Group"
$clusterName = "your-hadoop-001"
$clusterCreds = Get-Credential

# If the cluster's resource group doesn't exist yet, run:
#   New-AzResourceGroup -Name $clusterResourceGroupName -Location $___location

# Hive metastore info
$hiveSqlServer = "your-sqlserver-001"
$hiveDb = "your-sqldb-001"
$hiveCreds = Get-Credential

# Oozie metastore info
$oozieSqlServer = "your-sqlserver-001"
$oozieDb = "your-sqldb-002"
$oozieCreds = Get-Credential

# Create the cluster
New-AzHDInsightClusterConfig  `
            | Add-AzHDInsightMetastore `
                -SqlAzureServerName "$oozieSqlServer.database.contoso.net" `
                -DatabaseName $oozieDb `
                -Credential $oozieCreds `
                -MetastoreType OozieMetastore `
            | Add-AzHDInsightMetastore `
                -SqlAzureServerName "$hiveSqlServer.database.contoso.net" `
                -DatabaseName $hiveDb `
                -Credential $hiveCreds `
                -MetastoreType HiveMetastore `
            | New-AzHDInsightCluster `
                -ClusterType Hadoop `
                -OSType Windows `
                -ClusterSizeInNodes 4 `
                -ResourceGroupName $clusterResourceGroupName `
                -ClusterName $clusterName `
                -HttpCredential $clusterCreds `
                -Location $___location `
                -StorageAccountResourceId $storageAccountResourceId `
                -StorageAccountKey $storageAccountKey `
                -StorageContainer $storageContainer

This command adds a SQL database metastore to the cluster named your-hadoop-001.

Example 2: Add a custom Ambari database to the cluster configuration object

# Primary storage account info
$storageAccountResourceGroupName = "Group"
$storageAccountResourceId = "yourstorageaccountresourceid"
$storageAccountName = "yourstorageacct001"
$storageAccountKey = (Get-AzStorageAccountKey -ResourceGroupName $storageAccountResourceGroupName -Name $storageAccountName)[0].value


$storageContainer = "container001"

# Cluster configuration info
$___location = "East US 2"
$clusterResourceGroupName = "Group"
$clusterName = "your-hadoop-002"
$clusterCreds = Get-Credential

# If the cluster's resource group doesn't exist yet, run:
#   New-AzResourceGroup -Name $clusterResourceGroupName -Location $___location

# Custom Amari database info
$ambariSqlServer = "your-sqlserver-001"
$ambariDb = "your-sqldb-003"
$ambariCreds = Get-Credential

# Create the cluster
New-AzHDInsightClusterConfig  `
            | Add-AzHDInsightMetastore `
                -SqlAzureServerName "$ambariSqlServer.database.contoso.net" `
                -DatabaseName $ambariDb `
                -Credential $ambariCreds `
                -MetastoreType AmbariDatabase `
            | New-AzHDInsightCluster `
                -ClusterType Hadoop `
                -OSType Windows `
                -ClusterSizeInNodes 4 `
                -ResourceGroupName $clusterResourceGroupName `
                -ClusterName $clusterName `
                -HttpCredential $clusterCreds `
                -Location $___location `
                -StorageAccountResourceId $storageAccountResourceId `
                -StorageAccountKey $storageAccountKey `
                -StorageContainer $storageContainer

This command adds a custom Ambari database to the cluster named your-hadoop-002.

Parameters

-Config

Specifies the HDInsight cluster configuration object that this cmdlet modifies. This object is created by the New-AzHDInsightClusterConfig cmdlet.

Parameter properties

Type:AzureHDInsightConfig
Default value:None
Supports wildcards:False
DontShow:False

Parameter sets

(All)
Position:0
Mandatory:True
Value from pipeline:True
Value from pipeline by property name:False
Value from remaining arguments:False

-Credential

Specifies the credentials to use for the AzureSQL Server database.

Parameter properties

Type:PSCredential
Default value:None
Supports wildcards:False
DontShow:False

Parameter sets

(All)
Position:4
Mandatory:True
Value from pipeline:False
Value from pipeline by property name:False
Value from remaining arguments:False

-DatabaseName

Specifies the database on the AzureSQL Server instance to use for this metastore.

Parameter properties

Type:String
Default value:None
Supports wildcards:False
DontShow:False

Parameter sets

(All)
Position:3
Mandatory:True
Value from pipeline:False
Value from pipeline by property name:False
Value from remaining arguments:False

-DefaultProfile

The credentials, account, tenant, and subscription used for communication with azure

Parameter properties

Type:IAzureContextContainer
Default value:None
Supports wildcards:False
DontShow:False
Aliases:AzContext, AzureRmContext, AzureCredential

Parameter sets

(All)
Position:Named
Mandatory:False
Value from pipeline:False
Value from pipeline by property name:False
Value from remaining arguments:False

-MetastoreType

Specifies the type of metastore. Possible values are HiveMetastore or OozieMetastore.

Parameter properties

Type:AzureHDInsightMetastoreType
Default value:None
Accepted values:HiveMetastore, OozieMetastore, AmbariDatabase
Supports wildcards:False
DontShow:False

Parameter sets

(All)
Position:1
Mandatory:True
Value from pipeline:False
Value from pipeline by property name:False
Value from remaining arguments:False

-SqlAzureServerName

Specifies the AzureSQL Server instance to use for this metastore.

Parameter properties

Type:String
Default value:None
Supports wildcards:False
DontShow:False

Parameter sets

(All)
Position:2
Mandatory:True
Value from pipeline:False
Value from pipeline by property name:False
Value from remaining arguments:False

CommonParameters

This cmdlet supports the common parameters: -Debug, -ErrorAction, -ErrorVariable, -InformationAction, -InformationVariable, -OutBuffer, -OutVariable, -PipelineVariable, -ProgressAction, -Verbose, -WarningAction, and -WarningVariable. For more information, see about_CommonParameters.

Inputs

AzureHDInsightConfig

Outputs

AzureHDInsightConfig