Search in sources :

Example 11 with Transaction

use of org.apache.iceberg.Transaction in project hive by apache.

the class HiveCreateReplaceTableTest method testCreateOrReplaceTableTxnTableNotExists.

@Test
public void testCreateOrReplaceTableTxnTableNotExists() {
    Assert.assertFalse("Table should not exist", catalog.tableExists(TABLE_IDENTIFIER));
    Transaction txn = catalog.newReplaceTableTransaction(TABLE_IDENTIFIER, SCHEMA, SPEC, true);
    txn.updateProperties().set("prop", "value").commit();
    txn.commitTransaction();
    Table table = catalog.loadTable(TABLE_IDENTIFIER);
    Assert.assertEquals("Table props should match", "value", table.properties().get("prop"));
}
Also used : Table(org.apache.iceberg.Table) Transaction(org.apache.iceberg.Transaction) Test(org.junit.Test)

Example 12 with Transaction

use of org.apache.iceberg.Transaction in project hive by apache.

the class HiveCreateReplaceTableTest method testReplaceTableTxnTableDeletedConcurrently.

@Test
public void testReplaceTableTxnTableDeletedConcurrently() {
    catalog.createTable(TABLE_IDENTIFIER, SCHEMA, SPEC, tableLocation, Maps.newHashMap());
    Assert.assertTrue("Table should exist", catalog.tableExists(TABLE_IDENTIFIER));
    Transaction txn = catalog.newReplaceTableTransaction(TABLE_IDENTIFIER, SCHEMA, SPEC, false);
    catalog.dropTable(TABLE_IDENTIFIER);
    txn.updateProperties().set("prop", "value").commit();
    AssertHelpers.assertThrows("Replace table txn should fail", NoSuchTableException.class, "No such table: hivedb.tbl", txn::commitTransaction);
}
Also used : Transaction(org.apache.iceberg.Transaction) Test(org.junit.Test)

Example 13 with Transaction

use of org.apache.iceberg.Transaction in project hive by apache.

the class HiveCreateReplaceTableTest method testCreateTableTxn.

@Test
public void testCreateTableTxn() {
    Assert.assertFalse("Table should not exist", catalog.tableExists(TABLE_IDENTIFIER));
    Transaction txn = catalog.newCreateTableTransaction(TABLE_IDENTIFIER, SCHEMA, SPEC, tableLocation, Maps.newHashMap());
    txn.updateProperties().set("prop", "value").commit();
    // verify the table is still not visible before the transaction is committed
    Assert.assertFalse(catalog.tableExists(TABLE_IDENTIFIER));
    txn.commitTransaction();
    Table table = catalog.loadTable(TABLE_IDENTIFIER);
    Assert.assertEquals("Table props should match", "value", table.properties().get("prop"));
}
Also used : Table(org.apache.iceberg.Table) Transaction(org.apache.iceberg.Transaction) Test(org.junit.Test)

Example 14 with Transaction

use of org.apache.iceberg.Transaction in project hive by apache.

the class HiveCreateReplaceTableTest method testCreateOrReplaceTableTxnTableDeletedConcurrently.

@Test
public void testCreateOrReplaceTableTxnTableDeletedConcurrently() {
    Assert.assertFalse("Table should not exist", catalog.tableExists(TABLE_IDENTIFIER));
    catalog.createTable(TABLE_IDENTIFIER, SCHEMA, SPEC);
    Assert.assertTrue("Table should be created", catalog.tableExists(TABLE_IDENTIFIER));
    Transaction txn = catalog.newReplaceTableTransaction(TABLE_IDENTIFIER, SCHEMA, PartitionSpec.unpartitioned(), tableLocation, Maps.newHashMap(), true);
    txn.updateProperties().set("prop", "value").commit();
    // drop the table concurrently
    catalog.dropTable(TABLE_IDENTIFIER);
    // expect the transaction to succeed anyway
    txn.commitTransaction();
    Table table = catalog.loadTable(TABLE_IDENTIFIER);
    Assert.assertEquals("Table props should match", "value", table.properties().get("prop"));
}
Also used : Table(org.apache.iceberg.Table) Transaction(org.apache.iceberg.Transaction) Test(org.junit.Test)

Example 15 with Transaction

use of org.apache.iceberg.Transaction in project drill by apache.

the class IcebergModify method executeOperations.

private void executeOperations(List<IcebergOperation> operations) {
    Transaction transaction = context.table().newTransaction();
    operations.forEach(op -> op.add(transaction));
    transaction.commitTransaction();
    // expiration process should not intervene with data modification operations
    // if expiration fails, will attempt to expire the next time
    context.expirationHandler().expireQuietly();
}
Also used : Transaction(org.apache.iceberg.Transaction)

Aggregations

Transaction (org.apache.iceberg.Transaction)16 Test (org.junit.Test)13 Table (org.apache.iceberg.Table)11 DataFile (org.apache.iceberg.DataFile)3 PartitionSpec (org.apache.iceberg.PartitionSpec)3 TableIdentifier (org.apache.iceberg.catalog.TableIdentifier)3 Schema (org.apache.iceberg.Schema)2 Snapshot (org.apache.iceberg.Snapshot)2 IOException (java.io.IOException)1 Lock (java.util.concurrent.locks.Lock)1 ReadWriteLock (java.util.concurrent.locks.ReadWriteLock)1 ReentrantReadWriteLock (java.util.concurrent.locks.ReentrantReadWriteLock)1 AutoCloseableHiveLock (org.apache.gobblin.hive.AutoCloseableHiveLock)1 HiveLock (org.apache.gobblin.hive.HiveLock)1 SchemaRegistryException (org.apache.gobblin.metrics.kafka.SchemaRegistryException)1 Path (org.apache.hadoop.fs.Path)1 AppendFiles (org.apache.iceberg.AppendFiles)1 UpdateProperties (org.apache.iceberg.UpdateProperties)1 GenericAppenderFactory (org.apache.iceberg.data.GenericAppenderFactory)1 GenericRecord (org.apache.iceberg.data.GenericRecord)1