Search in sources :

Example 11 with PartitionWithoutSD

use of org.apache.hadoop.hive.metastore.api.PartitionWithoutSD in project hive by apache.

the class TestAddPartitionsFromPartSpec method buildPartitionWithoutSD.

private PartitionWithoutSD buildPartitionWithoutSD(List<String> values, int index) throws MetaException {
    PartitionWithoutSD partition = new PartitionWithoutSD();
    partition.setCreateTime(DEFAULT_CREATE_TIME);
    partition.setLastAccessTime(DEFAULT_CREATE_TIME);
    partition.setValues(values);
    Map<String, String> parameters = new HashMap<>();
    parameters.put(DEFAULT_PARAM_KEY + index, DEFAULT_PARAM_VALUE + index);
    partition.setParameters(parameters);
    partition.setRelativePath("partwithoutsd" + index);
    return partition;
}
Also used : PartitionWithoutSD(org.apache.hadoop.hive.metastore.api.PartitionWithoutSD) HashMap(java.util.HashMap)

Example 12 with PartitionWithoutSD

use of org.apache.hadoop.hive.metastore.api.PartitionWithoutSD in project hive by apache.

the class TestAddPartitionsFromPartSpec method testAddPartitionSpecWithSharedSDInvalidSD.

@Test(expected = MetaException.class)
public void testAddPartitionSpecWithSharedSDInvalidSD() throws Exception {
    Table table = createTable();
    PartitionWithoutSD partition = buildPartitionWithoutSD(Lists.newArrayList("2002"), 0);
    partition.setRelativePath("year2002");
    StorageDescriptor sd = new StorageDescriptor();
    sd.setLocation(table.getSd().getLocation() + "/nullLocationTest/");
    PartitionSpecProxy partitionSpecProxy = buildPartitionSpecWithSharedSD(Lists.newArrayList(partition), sd);
    client.add_partitions_pspec(partitionSpecProxy);
}
Also used : Table(org.apache.hadoop.hive.metastore.api.Table) PartitionWithoutSD(org.apache.hadoop.hive.metastore.api.PartitionWithoutSD) StorageDescriptor(org.apache.hadoop.hive.metastore.api.StorageDescriptor) PartitionSpecProxy(org.apache.hadoop.hive.metastore.partition.spec.PartitionSpecProxy) Test(org.junit.Test) MetastoreCheckinTest(org.apache.hadoop.hive.metastore.annotation.MetastoreCheckinTest)

Example 13 with PartitionWithoutSD

use of org.apache.hadoop.hive.metastore.api.PartitionWithoutSD in project hive by apache.

the class TestAddPartitionsFromPartSpec method testAddPartitionSpecWithSharedSDWithoutRelativePath.

@Test
public void testAddPartitionSpecWithSharedSDWithoutRelativePath() throws Exception {
    Table table = createTable();
    PartitionWithoutSD partition = buildPartitionWithoutSD(Lists.newArrayList("2014"), 0);
    partition.setRelativePath(null);
    String location = table.getSd().getLocation() + "/sharedSDTest/";
    PartitionSpecProxy partitionSpecProxy = buildPartitionSpecWithSharedSD(Lists.newArrayList(partition), buildSD(location));
    client.add_partitions_pspec(partitionSpecProxy);
    Partition part = client.getPartition(DB_NAME, TABLE_NAME, "year=2014");
    Assert.assertNotNull(part);
    Assert.assertEquals(table.getSd().getLocation() + "/sharedSDTest/null", part.getSd().getLocation());
    Assert.assertTrue(metaStore.isPathExists(new Path(part.getSd().getLocation())));
}
Also used : Path(org.apache.hadoop.fs.Path) Partition(org.apache.hadoop.hive.metastore.api.Partition) Table(org.apache.hadoop.hive.metastore.api.Table) PartitionWithoutSD(org.apache.hadoop.hive.metastore.api.PartitionWithoutSD) PartitionSpecProxy(org.apache.hadoop.hive.metastore.partition.spec.PartitionSpecProxy) Test(org.junit.Test) MetastoreCheckinTest(org.apache.hadoop.hive.metastore.annotation.MetastoreCheckinTest)

Example 14 with PartitionWithoutSD

use of org.apache.hadoop.hive.metastore.api.PartitionWithoutSD in project hive by apache.

the class TestAddPartitionsFromPartSpec method testAddPartitionSpecsMultipleValues.

@Test
public void testAddPartitionSpecsMultipleValues() throws Exception {
    Table table = createTable(DB_NAME, TABLE_NAME, getYearAndMonthPartCols(), metaStore.getWarehouseRoot() + "/" + TABLE_NAME);
    Partition partition1 = buildPartition(Lists.newArrayList("2002", "march"), getYearAndMonthPartCols(), 1);
    Partition partition2 = buildPartition(Lists.newArrayList("2003", "april"), getYearAndMonthPartCols(), 2);
    PartitionWithoutSD partition3 = buildPartitionWithoutSD(Lists.newArrayList("2004", "june"), 3);
    PartitionWithoutSD partition4 = buildPartitionWithoutSD(Lists.newArrayList("2005", "may"), 4);
    List<Partition> partitions = Lists.newArrayList(partition1, partition2);
    List<PartitionWithoutSD> partitionsWithoutSD = Lists.newArrayList(partition3, partition4);
    PartitionSpecProxy partitionSpec = buildPartitionSpec(partitions, partitionsWithoutSD);
    client.add_partitions_pspec(partitionSpec);
    verifyPartition(table, "year=2002/month=march", Lists.newArrayList("2002", "march"), 1);
    verifyPartition(table, "year=2003/month=april", Lists.newArrayList("2003", "april"), 2);
    verifyPartitionSharedSD(table, "year=2004/month=june", Lists.newArrayList("2004", "june"), 3);
    verifyPartitionSharedSD(table, "year=2005/month=may", Lists.newArrayList("2005", "may"), 4);
}
Also used : Partition(org.apache.hadoop.hive.metastore.api.Partition) Table(org.apache.hadoop.hive.metastore.api.Table) PartitionWithoutSD(org.apache.hadoop.hive.metastore.api.PartitionWithoutSD) PartitionSpecProxy(org.apache.hadoop.hive.metastore.partition.spec.PartitionSpecProxy) Test(org.junit.Test) MetastoreCheckinTest(org.apache.hadoop.hive.metastore.annotation.MetastoreCheckinTest)

Example 15 with PartitionWithoutSD

use of org.apache.hadoop.hive.metastore.api.PartitionWithoutSD in project hive by apache.

the class TestAddPartitionsFromPartSpec method testAddPartitionSpecWithSharedSDEmptyValue.

@Test
public void testAddPartitionSpecWithSharedSDEmptyValue() throws Exception {
    Table table = createTable();
    PartitionWithoutSD partition = new PartitionWithoutSD();
    partition.setRelativePath("addpartspectest");
    partition.setValues(Lists.newArrayList(""));
    String location = table.getSd().getLocation() + "/nullValueTest/";
    PartitionSpecProxy partitionSpecProxy = buildPartitionSpecWithSharedSD(Lists.newArrayList(partition), buildSD(location));
    client.add_partitions_pspec(partitionSpecProxy);
    List<String> partitionNames = client.listPartitionNames(DB_NAME, TABLE_NAME, MAX);
    Assert.assertNotNull(partitionNames);
    Assert.assertTrue(partitionNames.size() == 1);
    Assert.assertEquals("year=__HIVE_DEFAULT_PARTITION__", partitionNames.get(0));
}
Also used : Table(org.apache.hadoop.hive.metastore.api.Table) PartitionWithoutSD(org.apache.hadoop.hive.metastore.api.PartitionWithoutSD) PartitionSpecProxy(org.apache.hadoop.hive.metastore.partition.spec.PartitionSpecProxy) Test(org.junit.Test) MetastoreCheckinTest(org.apache.hadoop.hive.metastore.annotation.MetastoreCheckinTest)

Aggregations

PartitionWithoutSD (org.apache.hadoop.hive.metastore.api.PartitionWithoutSD)15 MetastoreCheckinTest (org.apache.hadoop.hive.metastore.annotation.MetastoreCheckinTest)14 PartitionSpecProxy (org.apache.hadoop.hive.metastore.partition.spec.PartitionSpecProxy)14 Test (org.junit.Test)14 Table (org.apache.hadoop.hive.metastore.api.Table)9 Partition (org.apache.hadoop.hive.metastore.api.Partition)7 Path (org.apache.hadoop.fs.Path)2 StorageDescriptor (org.apache.hadoop.hive.metastore.api.StorageDescriptor)2 TTransportException (org.apache.thrift.transport.TTransportException)2 HashMap (java.util.HashMap)1 PartitionListComposingSpec (org.apache.hadoop.hive.metastore.api.PartitionListComposingSpec)1 PartitionSpec (org.apache.hadoop.hive.metastore.api.PartitionSpec)1 PartitionSpecWithSharedSD (org.apache.hadoop.hive.metastore.api.PartitionSpecWithSharedSD)1