Search in sources :

Example 46 with PartitionSpecProxy

use of org.apache.hadoop.hive.metastore.partition.spec.PartitionSpecProxy in project hive by apache.

the class TestAddPartitionsFromPartSpec method testAddPartitionsForViewNullPartSd.

@Test
@ConditionalIgnoreOnSessionHiveMetastoreClient
public void testAddPartitionsForViewNullPartSd() throws Exception {
    String tableName = "test_add_partition_view";
    createView(tableName);
    Partition partition = buildPartition(DB_NAME, tableName, DEFAULT_YEAR_VALUE);
    partition.setSd(null);
    PartitionSpecProxy partitionSpecProxy = buildPartitionSpec(DB_NAME, tableName, null, Lists.newArrayList(partition));
    client.add_partitions_pspec(partitionSpecProxy);
    Partition part = client.getPartition(DB_NAME, tableName, "year=2017");
    Assert.assertNull(part.getSd());
}
Also used : Partition(org.apache.hadoop.hive.metastore.api.Partition) PartitionSpecProxy(org.apache.hadoop.hive.metastore.partition.spec.PartitionSpecProxy) Test(org.junit.Test) MetastoreCheckinTest(org.apache.hadoop.hive.metastore.annotation.MetastoreCheckinTest)

Example 47 with PartitionSpecProxy

use of org.apache.hadoop.hive.metastore.partition.spec.PartitionSpecProxy in project hive by apache.

the class TestAddPartitionsFromPartSpec method testAddPartitionSpecsMultipleValues.

@Test
public void testAddPartitionSpecsMultipleValues() throws Exception {
    Table table = createTable(DB_NAME, TABLE_NAME, getYearAndMonthPartCols(), metaStore.getWarehouseRoot() + "/" + TABLE_NAME);
    Partition partition1 = buildPartition(Lists.newArrayList("2002", "march"), getYearAndMonthPartCols(), 1);
    Partition partition2 = buildPartition(Lists.newArrayList("2003", "april"), getYearAndMonthPartCols(), 2);
    PartitionWithoutSD partition3 = buildPartitionWithoutSD(Lists.newArrayList("2004", "june"), 3);
    PartitionWithoutSD partition4 = buildPartitionWithoutSD(Lists.newArrayList("2005", "may"), 4);
    List<Partition> partitions = Lists.newArrayList(partition1, partition2);
    List<PartitionWithoutSD> partitionsWithoutSD = Lists.newArrayList(partition3, partition4);
    PartitionSpecProxy partitionSpec = buildPartitionSpec(partitions, partitionsWithoutSD);
    client.add_partitions_pspec(partitionSpec);
    verifyPartition(table, "year=2002/month=march", Lists.newArrayList("2002", "march"), 1);
    verifyPartition(table, "year=2003/month=april", Lists.newArrayList("2003", "april"), 2);
    verifyPartitionSharedSD(table, "year=2004/month=june", Lists.newArrayList("2004", "june"), 3);
    verifyPartitionSharedSD(table, "year=2005/month=may", Lists.newArrayList("2005", "may"), 4);
}
Also used : Partition(org.apache.hadoop.hive.metastore.api.Partition) Table(org.apache.hadoop.hive.metastore.api.Table) PartitionWithoutSD(org.apache.hadoop.hive.metastore.api.PartitionWithoutSD) PartitionSpecProxy(org.apache.hadoop.hive.metastore.partition.spec.PartitionSpecProxy) Test(org.junit.Test) MetastoreCheckinTest(org.apache.hadoop.hive.metastore.annotation.MetastoreCheckinTest)

Example 48 with PartitionSpecProxy

use of org.apache.hadoop.hive.metastore.partition.spec.PartitionSpecProxy in project hive by apache.

the class TestAddPartitionsFromPartSpec method testAddPartitionSpecSetRootPath.

@Test
public void testAddPartitionSpecSetRootPath() throws Exception {
    Table table = createTable();
    String rootPath = table.getSd().getLocation() + "/addPartSpecRootPath/";
    String rootPath1 = table.getSd().getLocation() + "/someotherpath/";
    Partition partition = buildPartition(DB_NAME, TABLE_NAME, "2007", rootPath + "part2007/");
    PartitionSpecProxy partitionSpecProxy = buildPartitionSpec(DB_NAME, TABLE_NAME, rootPath1, Lists.newArrayList(partition));
    client.add_partitions_pspec(partitionSpecProxy);
    Partition resultPart = client.getPartition(DB_NAME, TABLE_NAME, Lists.newArrayList("2007"));
    Assert.assertEquals(rootPath + "part2007", resultPart.getSd().getLocation());
}
Also used : Partition(org.apache.hadoop.hive.metastore.api.Partition) Table(org.apache.hadoop.hive.metastore.api.Table) PartitionSpecProxy(org.apache.hadoop.hive.metastore.partition.spec.PartitionSpecProxy) Test(org.junit.Test) MetastoreCheckinTest(org.apache.hadoop.hive.metastore.annotation.MetastoreCheckinTest)

Example 49 with PartitionSpecProxy

use of org.apache.hadoop.hive.metastore.partition.spec.PartitionSpecProxy in project hive by apache.

the class TestAddPartitionsFromPartSpec method testAddPartitionSpecEmptyTable.

@Test(expected = InvalidObjectException.class)
public void testAddPartitionSpecEmptyTable() throws Exception {
    createTable();
    Partition partition = buildPartition(DB_NAME, TABLE_NAME, DEFAULT_YEAR_VALUE);
    PartitionSpecProxy partitionSpecProxy = buildPartitionSpec(DB_NAME, "", null, Lists.newArrayList(partition));
    client.add_partitions_pspec(partitionSpecProxy);
}
Also used : Partition(org.apache.hadoop.hive.metastore.api.Partition) PartitionSpecProxy(org.apache.hadoop.hive.metastore.partition.spec.PartitionSpecProxy) Test(org.junit.Test) MetastoreCheckinTest(org.apache.hadoop.hive.metastore.annotation.MetastoreCheckinTest)

Example 50 with PartitionSpecProxy

use of org.apache.hadoop.hive.metastore.partition.spec.PartitionSpecProxy in project hive by apache.

the class TestAddPartitionsFromPartSpec method testAddPartitionSpecChangeRootPath.

@Test
public void testAddPartitionSpecChangeRootPath() throws Exception {
    Table table = createTable();
    String rootPath = table.getSd().getLocation() + "/addPartSpecRootPath/";
    String rootPath1 = table.getSd().getLocation() + "/someotherpath/";
    Partition partition = buildPartition(DB_NAME, TABLE_NAME, "2007", rootPath + "part2007/");
    PartitionSpecProxy partitionSpecProxy = buildPartitionSpec(DB_NAME, TABLE_NAME, rootPath, Lists.newArrayList(partition));
    partitionSpecProxy.setRootLocation(rootPath1);
    client.add_partitions_pspec(partitionSpecProxy);
    Partition resultPart = client.getPartition(DB_NAME, TABLE_NAME, Lists.newArrayList("2007"));
    Assert.assertEquals(rootPath1 + "part2007", resultPart.getSd().getLocation());
}
Also used : Partition(org.apache.hadoop.hive.metastore.api.Partition) Table(org.apache.hadoop.hive.metastore.api.Table) PartitionSpecProxy(org.apache.hadoop.hive.metastore.partition.spec.PartitionSpecProxy) Test(org.junit.Test) MetastoreCheckinTest(org.apache.hadoop.hive.metastore.annotation.MetastoreCheckinTest)

Aggregations

PartitionSpecProxy (org.apache.hadoop.hive.metastore.partition.spec.PartitionSpecProxy)69 Test (org.junit.Test)60 MetastoreCheckinTest (org.apache.hadoop.hive.metastore.annotation.MetastoreCheckinTest)59 Partition (org.apache.hadoop.hive.metastore.api.Partition)53 Table (org.apache.hadoop.hive.metastore.api.Table)24 PartitionWithoutSD (org.apache.hadoop.hive.metastore.api.PartitionWithoutSD)15 ArrayList (java.util.ArrayList)13 Path (org.apache.hadoop.fs.Path)11 List (java.util.List)7 MetaException (org.apache.hadoop.hive.metastore.api.MetaException)7 HashMap (java.util.HashMap)4 PartitionSpec (org.apache.hadoop.hive.metastore.api.PartitionSpec)4 FieldSchema (org.apache.hadoop.hive.metastore.api.FieldSchema)3 StorageDescriptor (org.apache.hadoop.hive.metastore.api.StorageDescriptor)3 PartitionBuilder (org.apache.hadoop.hive.metastore.client.builder.PartitionBuilder)3 CompositePartitionSpecProxy (org.apache.hadoop.hive.metastore.partition.spec.CompositePartitionSpecProxy)3 PartitionSpecWithSharedSD (org.apache.hadoop.hive.metastore.api.PartitionSpecWithSharedSD)2 SQLCheckConstraint (org.apache.hadoop.hive.metastore.api.SQLCheckConstraint)2 SQLDefaultConstraint (org.apache.hadoop.hive.metastore.api.SQLDefaultConstraint)2 SQLNotNullConstraint (org.apache.hadoop.hive.metastore.api.SQLNotNullConstraint)2