Search in sources :

Example 1 with Reference

use of org.projectnessie.model.Reference in project iceberg by apache.

the class NessieCatalog method renameTable.

@Override
public void renameTable(TableIdentifier from, TableIdentifier toOriginal) {
    reference.checkMutable();
    TableIdentifier to = NessieUtil.removeCatalogName(toOriginal, name());
    IcebergTable existingFromTable = table(from);
    if (existingFromTable == null) {
        throw new NoSuchTableException("table %s doesn't exists", from.name());
    }
    IcebergTable existingToTable = table(to);
    if (existingToTable != null) {
        throw new AlreadyExistsException("table %s already exists", to.name());
    }
    CommitMultipleOperationsBuilder operations = api.commitMultipleOperations().commitMeta(NessieUtil.buildCommitMetadata(String.format("Iceberg rename table from '%s' to '%s'", from, to), catalogOptions)).operation(Operation.Put.of(NessieUtil.toKey(to), existingFromTable, existingFromTable)).operation(Operation.Delete.of(NessieUtil.toKey(from)));
    try {
        Tasks.foreach(operations).retry(5).stopRetryOn(NessieNotFoundException.class).throwFailureWhenFinished().onFailure((o, exception) -> refresh()).run(ops -> {
            Branch branch = ops.branch(reference.getAsBranch()).commit();
            reference.updateReference(branch);
        }, BaseNessieClientServerException.class);
    } catch (NessieNotFoundException e) {
        // and removed by another.
        throw new RuntimeException("Failed to drop table as ref is no longer valid.", e);
    } catch (BaseNessieClientServerException e) {
        throw new CommitFailedException(e, "Failed to rename table: the current reference is not up to date.");
    } catch (HttpClientException ex) {
        // safe than sorry.
        throw new CommitStateUnknownException(ex);
    }
// Intentionally just "throw through" Nessie's HttpClientException here and do not "special case"
// just the "timeout" variant to propagate all kinds of network errors (e.g. connection reset).
// Network code implementation details and all kinds of network devices can induce unexpected
// behavior. So better be safe than sorry.
}
Also used : TableIdentifier(org.apache.iceberg.catalog.TableIdentifier) AlreadyExistsException(org.apache.iceberg.exceptions.AlreadyExistsException) HttpClientBuilder(org.projectnessie.client.http.HttpClientBuilder) CatalogUtil(org.apache.iceberg.CatalogUtil) ImmutableMap(org.apache.iceberg.relocated.com.google.common.collect.ImmutableMap) CommitStateUnknownException(org.apache.iceberg.exceptions.CommitStateUnknownException) LoggerFactory(org.slf4j.LoggerFactory) HadoopFileIO(org.apache.iceberg.hadoop.HadoopFileIO) Function(java.util.function.Function) Reference(org.projectnessie.model.Reference) NessieClientBuilder(org.projectnessie.client.NessieClientBuilder) HttpClientException(org.projectnessie.client.http.HttpClientException) NessieConflictException(org.projectnessie.error.NessieConflictException) CatalogProperties(org.apache.iceberg.CatalogProperties) TableOperations(org.apache.iceberg.TableOperations) NoSuchNamespaceException(org.apache.iceberg.exceptions.NoSuchNamespaceException) Map(java.util.Map) Configuration(org.apache.hadoop.conf.Configuration) BaseMetastoreCatalog(org.apache.iceberg.BaseMetastoreCatalog) NoSuchTableException(org.apache.iceberg.exceptions.NoSuchTableException) Namespace(org.apache.iceberg.catalog.Namespace) Content(org.projectnessie.model.Content) Configurable(org.apache.hadoop.conf.Configurable) SupportsNamespaces(org.apache.iceberg.catalog.SupportsNamespaces) CommitFailedException(org.apache.iceberg.exceptions.CommitFailedException) Operation(org.projectnessie.model.Operation) Logger(org.slf4j.Logger) TableIdentifier(org.apache.iceberg.catalog.TableIdentifier) Branch(org.projectnessie.model.Branch) Set(java.util.Set) Collectors(java.util.stream.Collectors) Joiner(org.apache.iceberg.relocated.com.google.common.base.Joiner) NessieApiV1(org.projectnessie.client.api.NessieApiV1) List(java.util.List) Stream(java.util.stream.Stream) NessieConfigConstants(org.projectnessie.client.NessieConfigConstants) IcebergTable(org.projectnessie.model.IcebergTable) Tasks(org.apache.iceberg.util.Tasks) DynMethods(org.apache.iceberg.common.DynMethods) Preconditions(org.apache.iceberg.relocated.com.google.common.base.Preconditions) Tag(org.projectnessie.model.Tag) BaseNessieClientServerException(org.projectnessie.error.BaseNessieClientServerException) ContentKey(org.projectnessie.model.ContentKey) FileIO(org.apache.iceberg.io.FileIO) CommitMultipleOperationsBuilder(org.projectnessie.client.api.CommitMultipleOperationsBuilder) NessieNotFoundException(org.projectnessie.error.NessieNotFoundException) VisibleForTesting(org.apache.iceberg.relocated.com.google.common.annotations.VisibleForTesting) TableReference(org.projectnessie.model.TableReference) CommitMultipleOperationsBuilder(org.projectnessie.client.api.CommitMultipleOperationsBuilder) AlreadyExistsException(org.apache.iceberg.exceptions.AlreadyExistsException) NoSuchTableException(org.apache.iceberg.exceptions.NoSuchTableException) CommitStateUnknownException(org.apache.iceberg.exceptions.CommitStateUnknownException) NessieNotFoundException(org.projectnessie.error.NessieNotFoundException) HttpClientException(org.projectnessie.client.http.HttpClientException) Branch(org.projectnessie.model.Branch) IcebergTable(org.projectnessie.model.IcebergTable) CommitFailedException(org.apache.iceberg.exceptions.CommitFailedException) BaseNessieClientServerException(org.projectnessie.error.BaseNessieClientServerException)

Example 2 with Reference

use of org.projectnessie.model.Reference in project iceberg by apache.

the class UpdateableReference method refresh.

public boolean refresh(NessieApiV1 api) throws NessieNotFoundException {
    if (!mutable) {
        return false;
    }
    Reference oldReference = reference;
    reference = api.getReference().refName(reference.getName()).get();
    return !oldReference.equals(reference);
}
Also used : Reference(org.projectnessie.model.Reference)

Example 3 with Reference

use of org.projectnessie.model.Reference in project iceberg by apache.

the class TestBranchVisibility method after.

@AfterEach
public void after() throws NessieNotFoundException, NessieConflictException {
    catalog.dropTable(tableIdentifier1);
    catalog.dropTable(tableIdentifier2);
    catalog.refresh();
    for (Reference reference : api.getAllReferences().get().getReferences()) {
        if (!reference.getName().equals("main")) {
            api.deleteBranch().branch((Branch) reference).delete();
        }
    }
    testCatalog = null;
}
Also used : Reference(org.projectnessie.model.Reference) Branch(org.projectnessie.model.Branch) AfterEach(org.junit.jupiter.api.AfterEach)

Example 4 with Reference

use of org.projectnessie.model.Reference in project nessie by projectnessie.

the class AbstractSparkSqlTest method testAssignBranchTo.

@Test
void testAssignBranchTo() throws NessieConflictException, NessieNotFoundException {
    String random = "randomBranch";
    assertThat(sql("CREATE BRANCH %s IN nessie", random)).containsExactly(row("Branch", random, hash));
    commitAndReturnLog(refName);
    sql("USE REFERENCE %s IN nessie", refName);
    sql("MERGE BRANCH %s INTO main IN nessie", refName);
    Reference main = api.getReference().refName("main").get();
    assertThat(sql("ASSIGN BRANCH %s TO main IN nessie", random)).containsExactly(row("Branch", random, main.getHash()));
    for (Object[] commit : fetchLog("main")) {
        String currentHash = (String) commit[2];
        assertThat(sql("ASSIGN BRANCH %s TO main AT %s IN nessie", random, currentHash)).containsExactly(row("Branch", random, currentHash));
    }
    String invalidHash = "abc";
    String unknownHash = "dd8d46a3dd5478ce69749a5455dba29d74f6d1171188f4c21d0e15ff4a0a9a9c";
    String invalidBranch = "invalidBranch";
    assertThatThrownBy(() -> sql("ASSIGN BRANCH %s TO main AT %s IN nessie", random, invalidHash)).isInstanceOf(IllegalArgumentException.class).hasMessage(Validation.HASH_MESSAGE + " - but was: " + invalidHash);
    assertThatThrownBy(() -> sql("ASSIGN BRANCH %s TO main AT %s IN nessie", random, unknownHash)).isInstanceOf(NessieNotFoundException.class).hasMessage(String.format("Could not find commit '%s' in reference '%s'.", unknownHash, "main"));
    assertThatThrownBy(() -> sql("ASSIGN BRANCH %s TO %s AT %s IN nessie", random, invalidBranch, hash)).isInstanceOf(NessieNotFoundException.class).hasMessage(String.format("Named reference '%s' not found", invalidBranch));
}
Also used : Reference(org.projectnessie.model.Reference) NessieNotFoundException(org.projectnessie.error.NessieNotFoundException) Test(org.junit.jupiter.api.Test)

Example 5 with Reference

use of org.projectnessie.model.Reference in project nessie by projectnessie.

the class AbstractSparkSqlTest method testAssignBranch.

@Test
void testAssignBranch() throws NessieConflictException, NessieNotFoundException {
    String random = "randomBranch";
    assertThat(sql("CREATE BRANCH %s IN nessie", random)).containsExactly(row("Branch", random, hash));
    commitAndReturnLog(refName);
    sql("USE REFERENCE %s IN nessie", refName);
    sql("MERGE BRANCH %s INTO main IN nessie", refName);
    Reference main = api.getReference().refName("main").get();
    assertThat(sql("ASSIGN BRANCH %s IN nessie", random)).containsExactly(row("Branch", random, main.getHash()));
}
Also used : Reference(org.projectnessie.model.Reference) Test(org.junit.jupiter.api.Test)

Aggregations

Reference (org.projectnessie.model.Reference)38 Branch (org.projectnessie.model.Branch)19 Test (org.junit.jupiter.api.Test)17 ContentKey (org.projectnessie.model.ContentKey)11 ParameterizedTest (org.junit.jupiter.params.ParameterizedTest)10 NessieNotFoundException (org.projectnessie.error.NessieNotFoundException)10 CommitMeta (org.projectnessie.model.CommitMeta)10 IcebergTable (org.projectnessie.model.IcebergTable)10 List (java.util.List)9 Collectors (java.util.stream.Collectors)9 LogResponse (org.projectnessie.model.LogResponse)9 Tag (org.projectnessie.model.Tag)9 Put (org.projectnessie.model.Operation.Put)8 Map (java.util.Map)7 Stream (java.util.stream.Stream)7 Assertions.assertThat (org.assertj.core.api.Assertions.assertThat)7 BaseNessieClientServerException (org.projectnessie.error.BaseNessieClientServerException)7 Content (org.projectnessie.model.Content)7 NessieApiV1 (org.projectnessie.client.api.NessieApiV1)6 Operation (org.projectnessie.model.Operation)6