Search in sources :

Example 16 with NessieNotFoundException

use of org.projectnessie.error.NessieNotFoundException in project nessie by projectnessie.

the class NessieViewOperations method refresh.

@Override
public ViewVersionMetadata refresh() {
    try {
        reference.refresh(api);
    } catch (NessieNotFoundException e) {
        throw new RuntimeException("Failed to refresh as ref is no longer valid.", e);
    }
    String metadataLocation = null;
    try {
        Content content = api.getContent().key(key).reference(reference.getReference()).get().get(key);
        LOG.debug("Content '{}' at '{}': {}", key, reference.getReference(), content);
        if (content == null) {
            if (currentMetadataLocation() != null) {
                throw new NoSuchTableException("No such view %s in %s", key, reference.getReference());
            }
        } else {
            this.icebergView = content.unwrap(IcebergView.class).orElseThrow(() -> new IllegalStateException("Cannot refresh iceberg view: " + String.format("Nessie points to a non-Iceberg object for path: %s.", key)));
            metadataLocation = icebergView.getMetadataLocation();
        }
    } catch (NessieNotFoundException ex) {
        if (currentMetadataLocation() != null) {
            throw new NoSuchTableException(ex, "No such view %s", key);
        }
    }
    refreshFromMetadataLocation(metadataLocation, RETRY_IF, 2);
    return current();
}
Also used : Content(org.projectnessie.model.Content) NoSuchTableException(org.apache.iceberg.exceptions.NoSuchTableException) NessieNotFoundException(org.projectnessie.error.NessieNotFoundException)

Example 17 with NessieNotFoundException

use of org.projectnessie.error.NessieNotFoundException in project nessie by projectnessie.

the class NessieViewOperations method drop.

@Override
public void drop(String viewIdentifier) {
    reference.checkMutable();
    IcebergView existingView = view(toCatalogTableIdentifier(viewIdentifier));
    if (existingView == null) {
        return;
    }
    CommitMultipleOperationsBuilder commitBuilderBase = api.commitMultipleOperations().commitMeta(NessieUtil.buildCommitMetadata(String.format("Iceberg delete view %s", viewIdentifier), catalogOptions)).operation(Operation.Delete.of(NessieUtil.toKey(toCatalogTableIdentifier(viewIdentifier))));
    // We try to drop the view. Simple retry after ref update.
    try {
        Tasks.foreach(commitBuilderBase).retry(5).stopRetryOn(NessieNotFoundException.class).throwFailureWhenFinished().onFailure((o, exception) -> refresh()).run(commitBuilder -> {
            Branch branch = commitBuilder.branch(reference.getAsBranch()).commit();
            reference.updateReference(branch);
        }, BaseNessieClientServerException.class);
    } catch (NessieConflictException e) {
        LOG.error("Cannot drop view: failed after retry (update ref and retry)", e);
    } catch (NessieNotFoundException e) {
        LOG.error("Cannot drop view: ref is no longer valid.", e);
    } catch (BaseNessieClientServerException e) {
        LOG.error("Cannot drop view: unknown error", e);
    }
}
Also used : ImmutableIcebergView(org.projectnessie.model.ImmutableIcebergView) CommitStateUnknownException(org.apache.iceberg.exceptions.CommitStateUnknownException) LoggerFactory(org.slf4j.LoggerFactory) HttpClientException(org.projectnessie.client.http.HttpClientException) NessieConflictException(org.projectnessie.error.NessieConflictException) ViewVersionMetadata(org.apache.iceberg.view.ViewVersionMetadata) ImmutableCommitMeta(org.projectnessie.model.ImmutableCommitMeta) Map(java.util.Map) NoSuchTableException(org.apache.iceberg.exceptions.NoSuchTableException) Content(org.projectnessie.model.Content) CommitFailedException(org.apache.iceberg.exceptions.CommitFailedException) Operation(org.projectnessie.model.Operation) Logger(org.slf4j.Logger) TableIdentifier(org.apache.iceberg.catalog.TableIdentifier) Predicate(java.util.function.Predicate) Branch(org.projectnessie.model.Branch) NessieApiV1(org.projectnessie.client.api.NessieApiV1) IcebergView(org.projectnessie.model.IcebergView) Tasks(org.apache.iceberg.util.Tasks) BaseNessieClientServerException(org.projectnessie.error.BaseNessieClientServerException) ContentKey(org.projectnessie.model.ContentKey) ViewUtils.toCatalogTableIdentifier(org.apache.iceberg.view.ViewUtils.toCatalogTableIdentifier) FileIO(org.apache.iceberg.io.FileIO) CommitMultipleOperationsBuilder(org.projectnessie.client.api.CommitMultipleOperationsBuilder) NessieNotFoundException(org.projectnessie.error.NessieNotFoundException) BaseMetastoreViewOperations(org.apache.iceberg.view.BaseMetastoreViewOperations) CommitMultipleOperationsBuilder(org.projectnessie.client.api.CommitMultipleOperationsBuilder) Branch(org.projectnessie.model.Branch) ImmutableIcebergView(org.projectnessie.model.ImmutableIcebergView) IcebergView(org.projectnessie.model.IcebergView) NessieConflictException(org.projectnessie.error.NessieConflictException) NessieNotFoundException(org.projectnessie.error.NessieNotFoundException) BaseNessieClientServerException(org.projectnessie.error.BaseNessieClientServerException)

Example 18 with NessieNotFoundException

use of org.projectnessie.error.NessieNotFoundException in project nessie by projectnessie.

the class NessieViewOperations method commit.

@Override
public void commit(ViewVersionMetadata base, ViewVersionMetadata metadata, Map<String, String> properties) {
    reference.checkMutable();
    String newMetadataLocation = writeNewMetadata(metadata, currentVersion() + 1);
    boolean delete = true;
    try {
        ImmutableIcebergView.Builder viewBuilder = ImmutableIcebergView.builder();
        if (icebergView != null) {
            viewBuilder.id(icebergView.getId());
        }
        IcebergView newView = viewBuilder.metadataLocation(newMetadataLocation).versionId(metadata.currentVersionId()).schemaId(metadata.definition().schema().schemaId()).dialect("TODO: needs to be defined in Iceberg ViewDefinition").sqlText(metadata.definition().sql()).build();
        LOG.debug("Committing '{}' against '{}': {}", key, reference.getReference(), newView);
        ImmutableCommitMeta.Builder builder = ImmutableCommitMeta.builder();
        builder.message(buildCommitMsg(base, metadata) + " " + key.getName());
        Branch branch = api.commitMultipleOperations().operation(Operation.Put.of(key, newView, icebergView)).commitMeta(NessieUtil.catalogOptions(builder, catalogOptions).build()).branch(reference.getAsBranch()).commit();
        reference.updateReference(branch);
        delete = false;
    } catch (NessieConflictException ex) {
        throw new CommitFailedException(ex, "Commit failed: Reference hash is out of date. " + "Update the reference %s and try again", reference.getName());
    } catch (HttpClientException ex) {
        // Intentionally catch all nessie-client-exceptions here and not just the "timeout" variant
        // to catch all kinds of network errors (e.g. connection reset). Network code implementation
        // details and all kinds of network devices can induce unexpected behavior. So better be
        // safe than sorry.
        delete = false;
        throw new CommitStateUnknownException(ex);
    } catch (NessieNotFoundException ex) {
        throw new RuntimeException(String.format("Commit failed: Reference %s no longer exist", reference.getName()), ex);
    } finally {
        if (delete) {
            io().deleteFile(newMetadataLocation);
        }
    }
}
Also used : ImmutableCommitMeta(org.projectnessie.model.ImmutableCommitMeta) ImmutableIcebergView(org.projectnessie.model.ImmutableIcebergView) IcebergView(org.projectnessie.model.IcebergView) CommitStateUnknownException(org.apache.iceberg.exceptions.CommitStateUnknownException) NessieNotFoundException(org.projectnessie.error.NessieNotFoundException) ImmutableIcebergView(org.projectnessie.model.ImmutableIcebergView) HttpClientException(org.projectnessie.client.http.HttpClientException) Branch(org.projectnessie.model.Branch) NessieConflictException(org.projectnessie.error.NessieConflictException) CommitFailedException(org.apache.iceberg.exceptions.CommitFailedException)

Example 19 with NessieNotFoundException

use of org.projectnessie.error.NessieNotFoundException in project nessie by projectnessie.

the class NessieViewOperations method view.

private IcebergView view(TableIdentifier viewIdentifier) {
    try {
        ContentKey key = NessieUtil.toKey(viewIdentifier);
        Content view = api.getContent().key(key).reference(reference.getReference()).get().get(key);
        return view != null ? view.unwrap(IcebergView.class).orElse(null) : null;
    } catch (NessieNotFoundException e) {
        return null;
    }
}
Also used : ContentKey(org.projectnessie.model.ContentKey) Content(org.projectnessie.model.Content) NessieNotFoundException(org.projectnessie.error.NessieNotFoundException)

Aggregations

NessieNotFoundException (org.projectnessie.error.NessieNotFoundException)19 Content (org.projectnessie.model.Content)12 ContentKey (org.projectnessie.model.ContentKey)10 Map (java.util.Map)8 Branch (org.projectnessie.model.Branch)8 Reference (org.projectnessie.model.Reference)8 NessieConflictException (org.projectnessie.error.NessieConflictException)7 List (java.util.List)5 Stream (java.util.stream.Stream)5 CommitFailedException (org.apache.iceberg.exceptions.CommitFailedException)5 CommitStateUnknownException (org.apache.iceberg.exceptions.CommitStateUnknownException)5 NoSuchTableException (org.apache.iceberg.exceptions.NoSuchTableException)5 HttpClientException (org.projectnessie.client.http.HttpClientException)5 Operation (org.projectnessie.model.Operation)5 Instant (java.time.Instant)4 Set (java.util.Set)4 FetchOption (org.projectnessie.api.params.FetchOption)4 CommitMultipleOperationsBuilder (org.projectnessie.client.api.CommitMultipleOperationsBuilder)4 CommitMeta (org.projectnessie.model.CommitMeta)4 HashMap (java.util.HashMap)3