Search in sources :

Example 1 with Namespace

use of org.apache.iceberg.catalog.Namespace in project hive by apache.

the class HiveTableTest method testNonDefaultDatabaseLocation.

@Test
public void testNonDefaultDatabaseLocation() throws IOException, TException {
    Namespace namespace = Namespace.of(NON_DEFAULT_DATABASE);
    // Create a new location and a non-default database / namespace for it
    File nonDefaultLocation = createTempDirectory(NON_DEFAULT_DATABASE, asFileAttribute(fromString("rwxrwxrwx"))).toFile();
    catalog.createNamespace(namespace, Collections.singletonMap("location", nonDefaultLocation.getPath()));
    Map<String, String> namespaceMeta = catalog.loadNamespaceMetadata(namespace);
    // Make sure that we are testing a namespace with a non default location :)
    Assert.assertEquals(namespaceMeta.get("location"), "file:" + nonDefaultLocation.getPath());
    TableIdentifier tableIdentifier = TableIdentifier.of(namespace, TABLE_NAME);
    catalog.createTable(tableIdentifier, schema);
    // Let's check the location loaded through the catalog
    Table table = catalog.loadTable(tableIdentifier);
    Assert.assertEquals(namespaceMeta.get("location") + "/" + TABLE_NAME, table.location());
    // Drop the database and purge the files
    metastoreClient.dropDatabase(NON_DEFAULT_DATABASE, true, true, true);
}
Also used : TableIdentifier(org.apache.iceberg.catalog.TableIdentifier) Table(org.apache.iceberg.Table) PosixFilePermissions.fromString(java.nio.file.attribute.PosixFilePermissions.fromString) DataFile(org.apache.iceberg.DataFile) ManifestFile(org.apache.iceberg.ManifestFile) File(java.io.File) Namespace(org.apache.iceberg.catalog.Namespace) Test(org.junit.Test)

Example 2 with Namespace

use of org.apache.iceberg.catalog.Namespace in project hive by apache.

the class TestHiveCatalog method testDropNamespace.

@Test
public void testDropNamespace() throws TException {
    Namespace namespace = Namespace.of("dbname_drop");
    TableIdentifier identifier = TableIdentifier.of(namespace, "table");
    Schema schema = new Schema(Types.StructType.of(required(1, "id", Types.LongType.get())).fields());
    catalog.createNamespace(namespace, meta);
    catalog.createTable(identifier, schema);
    Map<String, String> nameMata = catalog.loadNamespaceMetadata(namespace);
    Assert.assertTrue(nameMata.get("owner").equals("apache"));
    Assert.assertTrue(nameMata.get("group").equals("iceberg"));
    AssertHelpers.assertThrows("Should fail to drop namespace is not empty" + namespace, NamespaceNotEmptyException.class, "Namespace dbname_drop is not empty. One or more tables exist.", () -> {
        catalog.dropNamespace(namespace);
    });
    Assert.assertTrue(catalog.dropTable(identifier, true));
    Assert.assertTrue("Should fail to drop namespace if it is not empty", catalog.dropNamespace(namespace));
    Assert.assertFalse("Should fail to drop when namespace doesn't exist", catalog.dropNamespace(Namespace.of("db.ns1")));
    AssertHelpers.assertThrows("Should fail to drop namespace exist" + namespace, NoSuchNamespaceException.class, "Namespace does not exist: ", () -> {
        catalog.loadNamespaceMetadata(namespace);
    });
}
Also used : TableIdentifier(org.apache.iceberg.catalog.TableIdentifier) Schema(org.apache.iceberg.Schema) Namespace(org.apache.iceberg.catalog.Namespace) Test(org.junit.Test)

Example 3 with Namespace

use of org.apache.iceberg.catalog.Namespace in project hive by apache.

the class TestHiveCatalog method testLoadNamespaceMeta.

@Test
public void testLoadNamespaceMeta() throws TException {
    Namespace namespace = Namespace.of("dbname_load");
    catalog.createNamespace(namespace, meta);
    Map<String, String> nameMata = catalog.loadNamespaceMetadata(namespace);
    Assert.assertTrue(nameMata.get("owner").equals("apache"));
    Assert.assertTrue(nameMata.get("group").equals("iceberg"));
    Assert.assertEquals("There no same location for db and namespace", nameMata.get("location"), catalog.convertToDatabase(namespace, meta).getLocationUri());
}
Also used : Namespace(org.apache.iceberg.catalog.Namespace) Test(org.junit.Test)

Example 4 with Namespace

use of org.apache.iceberg.catalog.Namespace in project hive by apache.

the class TestHiveCatalog method testListNamespace.

@Test
public void testListNamespace() throws TException {
    List<Namespace> namespaces;
    Namespace namespace1 = Namespace.of("dbname1");
    catalog.createNamespace(namespace1, meta);
    namespaces = catalog.listNamespaces(namespace1);
    Assert.assertTrue("Hive db not hive the namespace 'dbname1'", namespaces.isEmpty());
    Namespace namespace2 = Namespace.of("dbname2");
    catalog.createNamespace(namespace2, meta);
    namespaces = catalog.listNamespaces();
    Assert.assertTrue("Hive db not hive the namespace 'dbname2'", namespaces.contains(namespace2));
}
Also used : Namespace(org.apache.iceberg.catalog.Namespace) Test(org.junit.Test)

Example 5 with Namespace

use of org.apache.iceberg.catalog.Namespace in project hive by apache.

the class TestHiveCatalog method testRemoveNamespaceProperties.

@Test
public void testRemoveNamespaceProperties() throws TException {
    Namespace namespace = Namespace.of("dbname_remove");
    catalog.createNamespace(namespace, meta);
    catalog.removeProperties(namespace, ImmutableSet.of("comment", "owner"));
    Database database = metastoreClient.getDatabase(namespace.level(0));
    Assert.assertEquals(database.getParameters().get("owner"), null);
    Assert.assertEquals(database.getParameters().get("group"), "iceberg");
    AssertHelpers.assertThrows("Should fail to namespace not exist" + namespace, NoSuchNamespaceException.class, "Namespace does not exist: ", () -> {
        catalog.removeProperties(Namespace.of("db2", "db2", "ns2"), ImmutableSet.of("comment", "owner"));
    });
}
Also used : Database(org.apache.hadoop.hive.metastore.api.Database) Namespace(org.apache.iceberg.catalog.Namespace) Test(org.junit.Test)

Aggregations

Namespace (org.apache.iceberg.catalog.Namespace)9 Test (org.junit.Test)8 Database (org.apache.hadoop.hive.metastore.api.Database)4 TableIdentifier (org.apache.iceberg.catalog.TableIdentifier)3 ImmutableMap (org.apache.iceberg.relocated.com.google.common.collect.ImmutableMap)2 File (java.io.File)1 PosixFilePermissions.fromString (java.nio.file.attribute.PosixFilePermissions.fromString)1 List (java.util.List)1 Map (java.util.Map)1 Set (java.util.Set)1 Collectors (java.util.stream.Collectors)1 Configurable (org.apache.hadoop.conf.Configurable)1 Configuration (org.apache.hadoop.conf.Configuration)1 Path (org.apache.hadoop.fs.Path)1 HiveConf (org.apache.hadoop.hive.conf.HiveConf)1 IMetaStoreClient (org.apache.hadoop.hive.metastore.IMetaStoreClient)1 AlreadyExistsException (org.apache.hadoop.hive.metastore.api.AlreadyExistsException)1 InvalidOperationException (org.apache.hadoop.hive.metastore.api.InvalidOperationException)1 NoSuchObjectException (org.apache.hadoop.hive.metastore.api.NoSuchObjectException)1 Table (org.apache.hadoop.hive.metastore.api.Table)1