Search in sources :

Example 1 with ProvisionResponse

use of org.eclipse.dataspaceconnector.spi.types.domain.transfer.ProvisionResponse in project DataSpaceConnector by eclipse-dataspaceconnector.

the class ObjectStorageProvisioner method provision.

@Override
public CompletableFuture<ProvisionResponse> provision(ObjectStorageResourceDefinition resourceDefinition) {
    String containerName = resourceDefinition.getContainerName();
    String accountName = resourceDefinition.getAccountName();
    monitor.info("Azure Storage Container request submitted: " + containerName);
    OffsetDateTime expiryTime = OffsetDateTime.now().plusHours(1);
    return with(retryPolicy).getAsync(() -> blobStoreApi.exists(accountName, containerName)).thenCompose(exists -> {
        if (exists) {
            return reusingExistingContainer(containerName);
        } else {
            return createContainer(containerName, accountName);
        }
    }).thenCompose(empty -> createContainerSasToken(containerName, accountName, expiryTime)).thenApply(writeOnlySas -> {
        var resource = ObjectContainerProvisionedResource.Builder.newInstance().id(containerName).accountName(accountName).containerName(containerName).resourceDefinitionId(resourceDefinition.getId()).transferProcessId(resourceDefinition.getTransferProcessId()).build();
        var secretToken = new AzureSasToken("?" + writeOnlySas, expiryTime.toInstant().toEpochMilli());
        return ProvisionResponse.Builder.newInstance().resource(resource).secretToken(secretToken).build();
    });
}
Also used : BlobStoreApi(org.eclipse.dataspaceconnector.azure.blob.core.api.BlobStoreApi) ProvisionedResource(org.eclipse.dataspaceconnector.spi.types.domain.transfer.ProvisionedResource) Failsafe.with(net.jodah.failsafe.Failsafe.with) ResourceDefinition(org.eclipse.dataspaceconnector.spi.types.domain.transfer.ResourceDefinition) RetryPolicy(net.jodah.failsafe.RetryPolicy) CompletableFuture(java.util.concurrent.CompletableFuture) Provisioner(org.eclipse.dataspaceconnector.spi.transfer.provision.Provisioner) OffsetDateTime(java.time.OffsetDateTime) AzureSasToken(org.eclipse.dataspaceconnector.azure.blob.core.AzureSasToken) ProvisionResponse(org.eclipse.dataspaceconnector.spi.types.domain.transfer.ProvisionResponse) DeprovisionResponse(org.eclipse.dataspaceconnector.spi.types.domain.transfer.DeprovisionResponse) NotNull(org.jetbrains.annotations.NotNull) Monitor(org.eclipse.dataspaceconnector.spi.monitor.Monitor) OffsetDateTime(java.time.OffsetDateTime) AzureSasToken(org.eclipse.dataspaceconnector.azure.blob.core.AzureSasToken)

Example 2 with ProvisionResponse

use of org.eclipse.dataspaceconnector.spi.types.domain.transfer.ProvisionResponse in project DataSpaceConnector by eclipse-dataspaceconnector.

the class ProvisionManagerImplTest method provision_should_provision_all_the_transfer_process_definitions.

@Test
void provision_should_provision_all_the_transfer_process_definitions() {
    when(provisioner.canProvision(isA(TestResourceDefinition.class))).thenReturn(true);
    var provisionResult = StatusResult.success(ProvisionResponse.Builder.newInstance().resource(new TestProvisionedDataDestinationResource("test-resource", "1")).build());
    when(provisioner.provision(isA(TestResourceDefinition.class), isA(Policy.class))).thenReturn(completedFuture(provisionResult));
    var result = provisionManager.provision(List.of(new TestResourceDefinition()), policy);
    assertThat(result).succeedsWithin(1, SECONDS).extracting(responses -> responses.get(0)).extracting(StatusResult::getContent).extracting(ProvisionResponse::getResource).extracting(ProvisionedDataDestinationResource.class::cast).extracting(ProvisionedDataDestinationResource::getResourceName).isEqualTo("test-resource");
}
Also used : Policy(org.eclipse.dataspaceconnector.policy.model.Policy) TestResourceDefinition(org.eclipse.dataspaceconnector.transfer.core.TestResourceDefinition) BeforeEach(org.junit.jupiter.api.BeforeEach) Policy(org.eclipse.dataspaceconnector.policy.model.Policy) TestProvisionedDataDestinationResource(org.eclipse.dataspaceconnector.transfer.core.TestProvisionedDataDestinationResource) ProvisionedResource(org.eclipse.dataspaceconnector.spi.types.domain.transfer.ProvisionedResource) CompletableFuture.completedFuture(java.util.concurrent.CompletableFuture.completedFuture) Assertions.assertThat(org.assertj.core.api.Assertions.assertThat) EdcException(org.eclipse.dataspaceconnector.spi.EdcException) Mockito.when(org.mockito.Mockito.when) StatusResult(org.eclipse.dataspaceconnector.spi.response.StatusResult) Test(org.junit.jupiter.api.Test) ExecutionException(java.util.concurrent.ExecutionException) DeprovisionedResource(org.eclipse.dataspaceconnector.spi.types.domain.transfer.DeprovisionedResource) CompletableFuture.failedFuture(java.util.concurrent.CompletableFuture.failedFuture) Provisioner(org.eclipse.dataspaceconnector.spi.transfer.provision.Provisioner) List(java.util.List) ProvisionResponse(org.eclipse.dataspaceconnector.spi.types.domain.transfer.ProvisionResponse) ProvisionedDataDestinationResource(org.eclipse.dataspaceconnector.spi.types.domain.transfer.ProvisionedDataDestinationResource) Monitor(org.eclipse.dataspaceconnector.spi.monitor.Monitor) SECONDS(java.util.concurrent.TimeUnit.SECONDS) ArgumentMatchers.isA(org.mockito.ArgumentMatchers.isA) Mockito.mock(org.mockito.Mockito.mock) TestProvisionedDataDestinationResource(org.eclipse.dataspaceconnector.transfer.core.TestProvisionedDataDestinationResource) ProvisionedDataDestinationResource(org.eclipse.dataspaceconnector.spi.types.domain.transfer.ProvisionedDataDestinationResource) StatusResult(org.eclipse.dataspaceconnector.spi.response.StatusResult) TestResourceDefinition(org.eclipse.dataspaceconnector.transfer.core.TestResourceDefinition) TestProvisionedDataDestinationResource(org.eclipse.dataspaceconnector.transfer.core.TestProvisionedDataDestinationResource) Test(org.junit.jupiter.api.Test)

Example 3 with ProvisionResponse

use of org.eclipse.dataspaceconnector.spi.types.domain.transfer.ProvisionResponse in project DataSpaceConnector by eclipse-dataspaceconnector.

the class TransferProcessManagerImplTest method provisioning_shouldContinueOnRetryProvisionError.

@Test
void provisioning_shouldContinueOnRetryProvisionError() throws InterruptedException {
    var process = createTransferProcess(PROVISIONING).toBuilder().resourceManifest(ResourceManifest.Builder.newInstance().definitions(List.of(new TestResourceDefinition())).build()).build();
    var provisionResult = StatusResult.<ProvisionResponse>failure(ResponseStatus.ERROR_RETRY, "test error");
    when(policyArchive.findPolicyForContract(anyString())).thenReturn(Policy.Builder.newInstance().build());
    when(provisionManager.provision(any(), isA(Policy.class))).thenReturn(completedFuture(List.of(provisionResult)));
    when(transferProcessStore.nextForState(eq(PROVISIONING.code()), anyInt())).thenReturn(List.of(process)).thenReturn(emptyList());
    when(transferProcessStore.find(process.getId())).thenReturn(process);
    var latch = countDownOnUpdateLatch();
    manager.start();
    assertThat(latch.await(TIMEOUT, TimeUnit.SECONDS)).isTrue();
    verify(policyArchive, atLeastOnce()).findPolicyForContract(anyString());
    verify(transferProcessStore).update(argThat(p -> p.getState() == PROVISIONING.code()));
}
Also used : Policy(org.eclipse.dataspaceconnector.policy.model.Policy) BeforeEach(org.junit.jupiter.api.BeforeEach) PROVISIONING(org.eclipse.dataspaceconnector.spi.types.domain.transfer.TransferProcessStates.PROVISIONING) TestProvisionedDataDestinationResource(org.eclipse.dataspaceconnector.transfer.core.TestProvisionedDataDestinationResource) ArgumentMatchers.argThat(org.mockito.ArgumentMatchers.argThat) TypeManager(org.eclipse.dataspaceconnector.spi.types.TypeManager) ArgumentMatchers.eq(org.mockito.ArgumentMatchers.eq) DataAddressResolver(org.eclipse.dataspaceconnector.spi.asset.DataAddressResolver) CompletableFuture.completedFuture(java.util.concurrent.CompletableFuture.completedFuture) Assertions.assertThat(org.assertj.core.api.Assertions.assertThat) ExponentialWaitStrategy(org.eclipse.dataspaceconnector.spi.retry.ExponentialWaitStrategy) TransferProcess(org.eclipse.dataspaceconnector.spi.types.domain.transfer.TransferProcess) PROVISIONED(org.eclipse.dataspaceconnector.spi.types.domain.transfer.TransferProcessStates.PROVISIONED) Mockito.verifyNoInteractions(org.mockito.Mockito.verifyNoInteractions) ProvisionManager(org.eclipse.dataspaceconnector.spi.transfer.provision.ProvisionManager) Mockito.doThrow(org.mockito.Mockito.doThrow) DataFlowManager(org.eclipse.dataspaceconnector.spi.transfer.flow.DataFlowManager) Map(java.util.Map) Mockito.doAnswer(org.mockito.Mockito.doAnswer) SecretToken(org.eclipse.dataspaceconnector.spi.types.domain.transfer.SecretToken) StatusCheckerRegistry(org.eclipse.dataspaceconnector.spi.types.domain.transfer.StatusCheckerRegistry) PolicyArchive(org.eclipse.dataspaceconnector.spi.policy.store.PolicyArchive) TestResourceDefinition(org.eclipse.dataspaceconnector.transfer.core.TestResourceDefinition) ERROR(org.eclipse.dataspaceconnector.spi.types.domain.transfer.TransferProcessStates.ERROR) Collections.emptyList(java.util.Collections.emptyList) ENDED(org.eclipse.dataspaceconnector.spi.types.domain.transfer.TransferProcessStates.ENDED) Mockito.atLeastOnce(org.mockito.Mockito.atLeastOnce) EdcException(org.eclipse.dataspaceconnector.spi.EdcException) PROVIDER(org.eclipse.dataspaceconnector.spi.types.domain.transfer.TransferProcess.Type.PROVIDER) DEPROVISIONED(org.eclipse.dataspaceconnector.spi.types.domain.transfer.TransferProcessStates.DEPROVISIONED) UUID(java.util.UUID) JsonPOJOBuilder(com.fasterxml.jackson.databind.annotation.JsonPOJOBuilder) StatusResult(org.eclipse.dataspaceconnector.spi.response.StatusResult) Instant(java.time.Instant) TransferProcessStore(org.eclipse.dataspaceconnector.spi.transfer.store.TransferProcessStore) DataAddress(org.eclipse.dataspaceconnector.spi.types.domain.DataAddress) Test(org.junit.jupiter.api.Test) DeprovisionedResource(org.eclipse.dataspaceconnector.spi.types.domain.transfer.DeprovisionedResource) CountDownLatch(java.util.concurrent.CountDownLatch) CompletableFuture.failedFuture(java.util.concurrent.CompletableFuture.failedFuture) List(java.util.List) TransferType(org.eclipse.dataspaceconnector.spi.types.domain.transfer.TransferType) ProvisionResponse(org.eclipse.dataspaceconnector.spi.types.domain.transfer.ProvisionResponse) UTC(java.time.ZoneOffset.UTC) CommandQueue(org.eclipse.dataspaceconnector.spi.command.CommandQueue) ProvisionedDataDestinationResource(org.eclipse.dataspaceconnector.spi.types.domain.transfer.ProvisionedDataDestinationResource) JsonDeserialize(com.fasterxml.jackson.databind.annotation.JsonDeserialize) REQUESTING(org.eclipse.dataspaceconnector.spi.types.domain.transfer.TransferProcessStates.REQUESTING) ProvisionedContentResource(org.eclipse.dataspaceconnector.spi.types.domain.transfer.ProvisionedContentResource) Mockito.mock(org.mockito.Mockito.mock) ArgumentMatchers.any(org.mockito.ArgumentMatchers.any) Result(org.eclipse.dataspaceconnector.spi.result.Result) CommandRunner(org.eclipse.dataspaceconnector.spi.command.CommandRunner) JsonTypeName(com.fasterxml.jackson.annotation.JsonTypeName) DEPROVISIONING(org.eclipse.dataspaceconnector.spi.types.domain.transfer.TransferProcessStates.DEPROVISIONING) TransferProcessObservable(org.eclipse.dataspaceconnector.spi.transfer.observe.TransferProcessObservable) REQUESTED(org.eclipse.dataspaceconnector.spi.types.domain.transfer.TransferProcessStates.REQUESTED) STREAMING(org.eclipse.dataspaceconnector.spi.types.domain.transfer.TransferProcessStates.STREAMING) IN_PROGRESS(org.eclipse.dataspaceconnector.spi.types.domain.transfer.TransferProcessStates.IN_PROGRESS) ArgumentMatchers.anyInt(org.mockito.ArgumentMatchers.anyInt) CONSUMER(org.eclipse.dataspaceconnector.spi.types.domain.transfer.TransferProcess.Type.CONSUMER) ArgumentMatchers.isA(org.mockito.ArgumentMatchers.isA) ResponseStatus(org.eclipse.dataspaceconnector.spi.response.ResponseStatus) ResourceManifest(org.eclipse.dataspaceconnector.spi.types.domain.transfer.ResourceManifest) Policy(org.eclipse.dataspaceconnector.policy.model.Policy) ResourceManifestGenerator(org.eclipse.dataspaceconnector.spi.transfer.provision.ResourceManifestGenerator) ProvisionedResourceSet(org.eclipse.dataspaceconnector.spi.types.domain.transfer.ProvisionedResourceSet) DataRequest(org.eclipse.dataspaceconnector.spi.types.domain.transfer.DataRequest) Mockito.times(org.mockito.Mockito.times) Vault(org.eclipse.dataspaceconnector.spi.security.Vault) Mockito.when(org.mockito.Mockito.when) DisplayName(org.junit.jupiter.api.DisplayName) Mockito.verify(org.mockito.Mockito.verify) TimeUnit(java.util.concurrent.TimeUnit) COMPLETED(org.eclipse.dataspaceconnector.spi.types.domain.transfer.TransferProcessStates.COMPLETED) Mockito.never(org.mockito.Mockito.never) JsonCreator(com.fasterxml.jackson.annotation.JsonCreator) RemoteMessageDispatcherRegistry(org.eclipse.dataspaceconnector.spi.message.RemoteMessageDispatcherRegistry) Clock(java.time.Clock) INITIAL(org.eclipse.dataspaceconnector.spi.types.domain.transfer.TransferProcessStates.INITIAL) TransferProcessStates(org.eclipse.dataspaceconnector.spi.types.domain.transfer.TransferProcessStates) Monitor(org.eclipse.dataspaceconnector.spi.monitor.Monitor) ArgumentMatchers.anyString(org.mockito.ArgumentMatchers.anyString) ProvisionResponse(org.eclipse.dataspaceconnector.spi.types.domain.transfer.ProvisionResponse) TestResourceDefinition(org.eclipse.dataspaceconnector.transfer.core.TestResourceDefinition) Test(org.junit.jupiter.api.Test)

Example 4 with ProvisionResponse

use of org.eclipse.dataspaceconnector.spi.types.domain.transfer.ProvisionResponse in project DataSpaceConnector by eclipse-dataspaceconnector.

the class HttpProviderProvisioner method provision.

@Override
public CompletableFuture<StatusResult<ProvisionResponse>> provision(HttpProviderResourceDefinition resourceDefinition, Policy policy) {
    var scopedPolicy = policyEngine.filter(policy, policyScope);
    Request request;
    try {
        request = createRequest(PROVISION, resourceDefinition.getTransferProcessId(), resourceDefinition.getAssetId(), scopedPolicy);
    } catch (JsonProcessingException e) {
        monitor.severe("Error serializing provision request for provisioner: " + name, e);
        return completedFuture(StatusResult.failure(ResponseStatus.FATAL_ERROR, "Fatal error serializing request: " + e.getMessage()));
    }
    try (var response = httpClient.newCall(request).execute()) {
        if (response.isSuccessful()) {
            // in-process
            return completedFuture(StatusResult.success(new ProvisionResponse()));
        } else if (response.code() >= 500 && response.code() <= 504) {
            // retry
            return completedFuture(StatusResult.failure(ResponseStatus.ERROR_RETRY, "Received error code: " + response.code()));
        } else {
            // fatal error
            return completedFuture(StatusResult.failure(ResponseStatus.FATAL_ERROR, "Received fatal error code: " + response.code()));
        }
    } catch (IOException e) {
        monitor.severe("Error invoking provisioner: " + name, e);
        return completedFuture(StatusResult.failure(ResponseStatus.ERROR_RETRY, "Received error: " + e.getMessage()));
    }
}
Also used : ProvisionResponse(org.eclipse.dataspaceconnector.spi.types.domain.transfer.ProvisionResponse) Request(okhttp3.Request) IOException(java.io.IOException) JsonProcessingException(com.fasterxml.jackson.core.JsonProcessingException)

Example 5 with ProvisionResponse

use of org.eclipse.dataspaceconnector.spi.types.domain.transfer.ProvisionResponse in project DataSpaceConnector by eclipse-dataspaceconnector.

the class ObjectStorageProvisioner method provision.

@Override
public CompletableFuture<StatusResult<ProvisionResponse>> provision(ObjectStorageResourceDefinition resourceDefinition, Policy policy) {
    String containerName = resourceDefinition.getContainerName();
    String accountName = resourceDefinition.getAccountName();
    monitor.info("Azure Storage Container request submitted: " + containerName);
    OffsetDateTime expiryTime = OffsetDateTime.now().plusHours(1);
    return with(retryPolicy).getAsync(() -> blobStoreApi.exists(accountName, containerName)).thenCompose(exists -> {
        if (exists) {
            return reusingExistingContainer(containerName);
        } else {
            return createContainer(containerName, accountName);
        }
    }).thenCompose(empty -> createContainerSasToken(containerName, accountName, expiryTime)).thenApply(writeOnlySas -> {
        // Ensure resource name is unique to avoid key collisions in local and remote vaults
        String resourceName = resourceDefinition.getId() + "-container";
        var resource = ObjectContainerProvisionedResource.Builder.newInstance().id(containerName).accountName(accountName).containerName(containerName).resourceDefinitionId(resourceDefinition.getId()).transferProcessId(resourceDefinition.getTransferProcessId()).resourceName(resourceName).hasToken(true).build();
        var secretToken = new AzureSasToken("?" + writeOnlySas, expiryTime.toInstant().toEpochMilli());
        var response = ProvisionResponse.Builder.newInstance().resource(resource).secretToken(secretToken).build();
        return StatusResult.success(response);
    });
}
Also used : BlobStoreApi(org.eclipse.dataspaceconnector.azure.blob.core.api.BlobStoreApi) Policy(org.eclipse.dataspaceconnector.policy.model.Policy) ProvisionedResource(org.eclipse.dataspaceconnector.spi.types.domain.transfer.ProvisionedResource) Failsafe.with(net.jodah.failsafe.Failsafe.with) ResourceDefinition(org.eclipse.dataspaceconnector.spi.types.domain.transfer.ResourceDefinition) RetryPolicy(net.jodah.failsafe.RetryPolicy) CompletableFuture(java.util.concurrent.CompletableFuture) StatusResult(org.eclipse.dataspaceconnector.spi.response.StatusResult) DeprovisionedResource(org.eclipse.dataspaceconnector.spi.types.domain.transfer.DeprovisionedResource) Provisioner(org.eclipse.dataspaceconnector.spi.transfer.provision.Provisioner) OffsetDateTime(java.time.OffsetDateTime) AzureSasToken(org.eclipse.dataspaceconnector.azure.blob.core.AzureSasToken) ProvisionResponse(org.eclipse.dataspaceconnector.spi.types.domain.transfer.ProvisionResponse) NotNull(org.jetbrains.annotations.NotNull) Monitor(org.eclipse.dataspaceconnector.spi.monitor.Monitor) OffsetDateTime(java.time.OffsetDateTime) AzureSasToken(org.eclipse.dataspaceconnector.azure.blob.core.AzureSasToken)

Aggregations

ProvisionResponse (org.eclipse.dataspaceconnector.spi.types.domain.transfer.ProvisionResponse)6 Monitor (org.eclipse.dataspaceconnector.spi.monitor.Monitor)5 Policy (org.eclipse.dataspaceconnector.policy.model.Policy)4 StatusResult (org.eclipse.dataspaceconnector.spi.response.StatusResult)4 DeprovisionedResource (org.eclipse.dataspaceconnector.spi.types.domain.transfer.DeprovisionedResource)4 List (java.util.List)3 CompletableFuture.completedFuture (java.util.concurrent.CompletableFuture.completedFuture)3 CompletableFuture.failedFuture (java.util.concurrent.CompletableFuture.failedFuture)3 Assertions.assertThat (org.assertj.core.api.Assertions.assertThat)3 EdcException (org.eclipse.dataspaceconnector.spi.EdcException)3 Provisioner (org.eclipse.dataspaceconnector.spi.transfer.provision.Provisioner)3 ProvisionedResource (org.eclipse.dataspaceconnector.spi.types.domain.transfer.ProvisionedResource)3 JsonCreator (com.fasterxml.jackson.annotation.JsonCreator)2 JsonTypeName (com.fasterxml.jackson.annotation.JsonTypeName)2 JsonDeserialize (com.fasterxml.jackson.databind.annotation.JsonDeserialize)2 JsonPOJOBuilder (com.fasterxml.jackson.databind.annotation.JsonPOJOBuilder)2 Clock (java.time.Clock)2 Instant (java.time.Instant)2 OffsetDateTime (java.time.OffsetDateTime)2 UTC (java.time.ZoneOffset.UTC)2