Search in sources :

Example 6 with CommitMultipleOperationsBuilder

use of org.projectnessie.client.api.CommitMultipleOperationsBuilder in project nessie by projectnessie.

the class AbstractRestContents method verifyAllContentAndOperationTypes.

@Test
public void verifyAllContentAndOperationTypes() throws BaseNessieClientServerException {
    Branch branch = createBranch("contentAndOperationAll");
    CommitMultipleOperationsBuilder commit = getApi().commitMultipleOperations().branch(branch).commitMeta(CommitMeta.fromMessage("verifyAllContentAndOperationTypes"));
    contentAndOperationTypes().flatMap(c -> c.globalOperation == null ? Stream.of(c.operation) : Stream.of(c.operation, c.globalOperation)).forEach(commit::operation);
    commit.commit();
    List<Entry> entries = getApi().getEntries().refName(branch.getName()).get().getEntries();
    List<Entry> expect = contentAndOperationTypes().filter(c -> c.operation instanceof Put).map(c -> Entry.builder().type(c.type).name(c.operation.getKey()).build()).collect(Collectors.toList());
    assertThat(entries).containsExactlyInAnyOrderElementsOf(expect);
}
Also used : Put(org.projectnessie.model.Operation.Put) Unchanged(org.projectnessie.model.Operation.Unchanged) Assertions.assertThat(org.assertj.core.api.Assertions.assertThat) Entry(org.projectnessie.model.EntriesResponse.Entry) Collections.singletonList(java.util.Collections.singletonList) Type(org.projectnessie.model.Content.Type) Map(java.util.Map) Content(org.projectnessie.model.Content) CommitMeta(org.projectnessie.model.CommitMeta) MethodSource(org.junit.jupiter.params.provider.MethodSource) Operation(org.projectnessie.model.Operation) Collections.emptyList(java.util.Collections.emptyList) Branch(org.projectnessie.model.Branch) ImmutableDeltaLakeTable(org.projectnessie.model.ImmutableDeltaLakeTable) Collectors(java.util.stream.Collectors) Test(org.junit.jupiter.api.Test) List(java.util.List) ParameterizedTest(org.junit.jupiter.params.ParameterizedTest) Stream(java.util.stream.Stream) IcebergView(org.projectnessie.model.IcebergView) Delete(org.projectnessie.model.Operation.Delete) IcebergTable(org.projectnessie.model.IcebergTable) BaseNessieClientServerException(org.projectnessie.error.BaseNessieClientServerException) ContentKey(org.projectnessie.model.ContentKey) CommitMultipleOperationsBuilder(org.projectnessie.client.api.CommitMultipleOperationsBuilder) Entry(org.projectnessie.model.EntriesResponse.Entry) CommitMultipleOperationsBuilder(org.projectnessie.client.api.CommitMultipleOperationsBuilder) Branch(org.projectnessie.model.Branch) Put(org.projectnessie.model.Operation.Put) Test(org.junit.jupiter.api.Test) ParameterizedTest(org.junit.jupiter.params.ParameterizedTest)

Example 7 with CommitMultipleOperationsBuilder

use of org.projectnessie.client.api.CommitMultipleOperationsBuilder in project nessie by projectnessie.

the class AbstractRestContents method verifyContentAndOperationTypesIndividually.

@ParameterizedTest
@MethodSource("contentAndOperationTypes")
public void verifyContentAndOperationTypesIndividually(ContentAndOperationType contentAndOperationType) throws BaseNessieClientServerException {
    Branch branch = createBranch("contentAndOperation_" + contentAndOperationType);
    CommitMultipleOperationsBuilder commit = getApi().commitMultipleOperations().branch(branch).commitMeta(CommitMeta.fromMessage("commit " + contentAndOperationType)).operation(contentAndOperationType.operation);
    if (contentAndOperationType.globalOperation != null) {
        commit.operation(contentAndOperationType.globalOperation);
    }
    commit.commit();
    List<Entry> entries = getApi().getEntries().refName(branch.getName()).get().getEntries();
    // Oh, yea - this is weird. The property ContentAndOperationType.operation.key.namespace is null
    // (!!!)
    // here, because somehow JUnit @MethodSource implementation re-constructs the objects returned
    // from
    // the source-method contentAndOperationTypes.
    ContentKey fixedContentKey = ContentKey.of(contentAndOperationType.operation.getKey().getElements());
    List<Entry> expect = contentAndOperationType.operation instanceof Put ? singletonList(Entry.builder().name(fixedContentKey).type(contentAndOperationType.type).build()) : emptyList();
    assertThat(entries).containsExactlyInAnyOrderElementsOf(expect);
}
Also used : ContentKey(org.projectnessie.model.ContentKey) Entry(org.projectnessie.model.EntriesResponse.Entry) CommitMultipleOperationsBuilder(org.projectnessie.client.api.CommitMultipleOperationsBuilder) Branch(org.projectnessie.model.Branch) Put(org.projectnessie.model.Operation.Put) ParameterizedTest(org.junit.jupiter.params.ParameterizedTest) MethodSource(org.junit.jupiter.params.provider.MethodSource)

Example 8 with CommitMultipleOperationsBuilder

use of org.projectnessie.client.api.CommitMultipleOperationsBuilder in project nessie by projectnessie.

the class NessieViewOperations method drop.

@Override
public void drop(String viewIdentifier) {
    reference.checkMutable();
    IcebergView existingView = view(toCatalogTableIdentifier(viewIdentifier));
    if (existingView == null) {
        return;
    }
    CommitMultipleOperationsBuilder commitBuilderBase = api.commitMultipleOperations().commitMeta(NessieUtil.buildCommitMetadata(String.format("Iceberg delete view %s", viewIdentifier), catalogOptions)).operation(Operation.Delete.of(NessieUtil.toKey(toCatalogTableIdentifier(viewIdentifier))));
    // We try to drop the view. Simple retry after ref update.
    try {
        Tasks.foreach(commitBuilderBase).retry(5).stopRetryOn(NessieNotFoundException.class).throwFailureWhenFinished().onFailure((o, exception) -> refresh()).run(commitBuilder -> {
            Branch branch = commitBuilder.branch(reference.getAsBranch()).commit();
            reference.updateReference(branch);
        }, BaseNessieClientServerException.class);
    } catch (NessieConflictException e) {
        LOG.error("Cannot drop view: failed after retry (update ref and retry)", e);
    } catch (NessieNotFoundException e) {
        LOG.error("Cannot drop view: ref is no longer valid.", e);
    } catch (BaseNessieClientServerException e) {
        LOG.error("Cannot drop view: unknown error", e);
    }
}
Also used : ImmutableIcebergView(org.projectnessie.model.ImmutableIcebergView) CommitStateUnknownException(org.apache.iceberg.exceptions.CommitStateUnknownException) LoggerFactory(org.slf4j.LoggerFactory) HttpClientException(org.projectnessie.client.http.HttpClientException) NessieConflictException(org.projectnessie.error.NessieConflictException) ViewVersionMetadata(org.apache.iceberg.view.ViewVersionMetadata) ImmutableCommitMeta(org.projectnessie.model.ImmutableCommitMeta) Map(java.util.Map) NoSuchTableException(org.apache.iceberg.exceptions.NoSuchTableException) Content(org.projectnessie.model.Content) CommitFailedException(org.apache.iceberg.exceptions.CommitFailedException) Operation(org.projectnessie.model.Operation) Logger(org.slf4j.Logger) TableIdentifier(org.apache.iceberg.catalog.TableIdentifier) Predicate(java.util.function.Predicate) Branch(org.projectnessie.model.Branch) NessieApiV1(org.projectnessie.client.api.NessieApiV1) IcebergView(org.projectnessie.model.IcebergView) Tasks(org.apache.iceberg.util.Tasks) BaseNessieClientServerException(org.projectnessie.error.BaseNessieClientServerException) ContentKey(org.projectnessie.model.ContentKey) ViewUtils.toCatalogTableIdentifier(org.apache.iceberg.view.ViewUtils.toCatalogTableIdentifier) FileIO(org.apache.iceberg.io.FileIO) CommitMultipleOperationsBuilder(org.projectnessie.client.api.CommitMultipleOperationsBuilder) NessieNotFoundException(org.projectnessie.error.NessieNotFoundException) BaseMetastoreViewOperations(org.apache.iceberg.view.BaseMetastoreViewOperations) CommitMultipleOperationsBuilder(org.projectnessie.client.api.CommitMultipleOperationsBuilder) Branch(org.projectnessie.model.Branch) ImmutableIcebergView(org.projectnessie.model.ImmutableIcebergView) IcebergView(org.projectnessie.model.IcebergView) NessieConflictException(org.projectnessie.error.NessieConflictException) NessieNotFoundException(org.projectnessie.error.NessieNotFoundException) BaseNessieClientServerException(org.projectnessie.error.BaseNessieClientServerException)

Aggregations

CommitMultipleOperationsBuilder (org.projectnessie.client.api.CommitMultipleOperationsBuilder)8 Branch (org.projectnessie.model.Branch)7 ContentKey (org.projectnessie.model.ContentKey)7 BaseNessieClientServerException (org.projectnessie.error.BaseNessieClientServerException)6 IcebergTable (org.projectnessie.model.IcebergTable)6 List (java.util.List)5 Collectors (java.util.stream.Collectors)5 Content (org.projectnessie.model.Content)5 Map (java.util.Map)4 Stream (java.util.stream.Stream)4 NessieApiV1 (org.projectnessie.client.api.NessieApiV1)4 TableIdentifier (org.apache.iceberg.catalog.TableIdentifier)3 CommitFailedException (org.apache.iceberg.exceptions.CommitFailedException)3 CommitStateUnknownException (org.apache.iceberg.exceptions.CommitStateUnknownException)3 NoSuchTableException (org.apache.iceberg.exceptions.NoSuchTableException)3 FileIO (org.apache.iceberg.io.FileIO)3 Tasks (org.apache.iceberg.util.Tasks)3 HttpClientException (org.projectnessie.client.http.HttpClientException)3 Entry (org.projectnessie.model.EntriesResponse.Entry)3 Put (org.projectnessie.model.Operation.Put)3