Search in sources :

Example 6 with BusinessObjectData

use of org.finra.herd.model.api.xml.BusinessObjectData in project herd by FINRAOS.

the class DownloaderWebClientTest method toBusinessObjectData.

private BusinessObjectData toBusinessObjectData(final UploaderInputManifestDto uploaderInputManifestDto) {
    BusinessObjectData businessObjectData = new BusinessObjectData();
    businessObjectData.setNamespace(uploaderInputManifestDto.getNamespace());
    businessObjectData.setBusinessObjectDefinitionName(uploaderInputManifestDto.getBusinessObjectDefinitionName());
    businessObjectData.setBusinessObjectFormatUsage(uploaderInputManifestDto.getBusinessObjectFormatUsage());
    businessObjectData.setBusinessObjectFormatFileType(uploaderInputManifestDto.getBusinessObjectFormatFileType());
    businessObjectData.setBusinessObjectFormatVersion(Integer.valueOf(uploaderInputManifestDto.getBusinessObjectFormatVersion()));
    businessObjectData.setPartitionKey(uploaderInputManifestDto.getPartitionKey());
    businessObjectData.setPartitionValue(uploaderInputManifestDto.getPartitionValue());
    businessObjectData.setSubPartitionValues(uploaderInputManifestDto.getSubPartitionValues());
    businessObjectData.setVersion(TEST_DATA_VERSION_V0);
    businessObjectData.setStorageUnits(Arrays.asList(new StorageUnit(new Storage(StorageEntity.MANAGED_STORAGE, null, null), null, null, null, null, null, null)));
    return businessObjectData;
}
Also used : Storage(org.finra.herd.model.api.xml.Storage) BusinessObjectData(org.finra.herd.model.api.xml.BusinessObjectData) StorageUnit(org.finra.herd.model.api.xml.StorageUnit)

Example 7 with BusinessObjectData

use of org.finra.herd.model.api.xml.BusinessObjectData in project herd by FINRAOS.

the class DownloaderWebClientTest method testGetBusinessObjectDataAssertNoAuthorizationHeaderWhenNoSsl.

@Test
public void testGetBusinessObjectDataAssertNoAuthorizationHeaderWhenNoSsl() throws Exception {
    HttpClientOperations mockHttpClientOperations = mock(HttpClientOperations.class);
    HttpClientOperations originalHttpClientOperations = (HttpClientOperations) ReflectionTestUtils.getField(downloaderWebClient, "httpClientOperations");
    ReflectionTestUtils.setField(downloaderWebClient, "httpClientOperations", mockHttpClientOperations);
    try {
        CloseableHttpResponse closeableHttpResponse = mock(CloseableHttpResponse.class);
        when(mockHttpClientOperations.execute(any(), any())).thenReturn(closeableHttpResponse);
        when(closeableHttpResponse.getStatusLine()).thenReturn(new BasicStatusLine(HttpVersion.HTTP_1_1, 200, "OK"));
        when(closeableHttpResponse.getEntity()).thenReturn(new StringEntity(xmlHelper.objectToXml(new BusinessObjectData())));
        DownloaderInputManifestDto manifest = new DownloaderInputManifestDto();
        downloaderWebClient.getRegServerAccessParamsDto().setUseSsl(false);
        downloaderWebClient.getBusinessObjectData(manifest);
        verify(mockHttpClientOperations).execute(any(), argThat(httpUriRequest -> httpUriRequest.getFirstHeader("Authorization") == null));
    } finally {
        ReflectionTestUtils.setField(downloaderWebClient, "httpClientOperations", originalHttpClientOperations);
    }
}
Also used : HttpVersion(org.apache.http.HttpVersion) Arrays(java.util.Arrays) BasicStatusLine(org.apache.http.message.BasicStatusLine) StorageUnit(org.finra.herd.model.api.xml.StorageUnit) Mockito.doThrow(org.mockito.Mockito.doThrow) BusinessObjectDataKey(org.finra.herd.model.api.xml.BusinessObjectDataKey) UploaderInputManifestDto(org.finra.herd.model.dto.UploaderInputManifestDto) Command(org.finra.herd.core.Command) CloseableHttpResponse(org.apache.http.client.methods.CloseableHttpResponse) S3KeyPrefixInformation(org.finra.herd.model.api.xml.S3KeyPrefixInformation) Storage(org.finra.herd.model.api.xml.Storage) DataBridgeWebClient(org.finra.herd.tools.common.databridge.DataBridgeWebClient) Assert.fail(org.junit.Assert.fail) Objects(com.google.common.base.Objects) DownloaderInputManifestDto(org.finra.herd.model.dto.DownloaderInputManifestDto) BusinessObjectData(org.finra.herd.model.api.xml.BusinessObjectData) StorageUnitDownloadCredential(org.finra.herd.model.api.xml.StorageUnitDownloadCredential) CloseableHttpClient(org.apache.http.impl.client.CloseableHttpClient) Assert.assertNotNull(org.junit.Assert.assertNotNull) Assert.assertTrue(org.junit.Assert.assertTrue) StringEntity(org.apache.http.entity.StringEntity) ReflectionTestUtils(org.springframework.test.util.ReflectionTestUtils) IOException(java.io.IOException) Test(org.junit.Test) Mockito.when(org.mockito.Mockito.when) RegServerAccessParamsDto(org.finra.herd.model.dto.RegServerAccessParamsDto) Mockito.verify(org.mockito.Mockito.verify) Matchers.any(org.mockito.Matchers.any) HttpClientOperations(org.finra.herd.dao.HttpClientOperations) Matchers.argThat(org.mockito.Matchers.argThat) Assert(org.junit.Assert) AwsCredential(org.finra.herd.model.api.xml.AwsCredential) Assert.assertEquals(org.junit.Assert.assertEquals) StorageEntity(org.finra.herd.model.jpa.StorageEntity) Mockito.mock(org.mockito.Mockito.mock) BusinessObjectDataStatusEntity(org.finra.herd.model.jpa.BusinessObjectDataStatusEntity) StringEntity(org.apache.http.entity.StringEntity) BusinessObjectData(org.finra.herd.model.api.xml.BusinessObjectData) DownloaderInputManifestDto(org.finra.herd.model.dto.DownloaderInputManifestDto) CloseableHttpResponse(org.apache.http.client.methods.CloseableHttpResponse) HttpClientOperations(org.finra.herd.dao.HttpClientOperations) BasicStatusLine(org.apache.http.message.BasicStatusLine) Test(org.junit.Test)

Example 8 with BusinessObjectData

use of org.finra.herd.model.api.xml.BusinessObjectData in project herd by FINRAOS.

the class DownloaderWebClientTest method testGetS3KeyPrefix.

@Test
public void testGetS3KeyPrefix() throws Exception {
    // Upload and register business object data parents.
    uploadAndRegisterTestDataParents(downloaderWebClient);
    // Upload and register the initial version if of the test business object data.
    uploadTestDataFilesToS3(S3_TEST_PATH_V0);
    final UploaderInputManifestDto uploaderInputManifestDto = getTestUploaderInputManifestDto();
    executeWithoutLogging(DataBridgeWebClient.class, new Command() {

        @Override
        public void execute() throws Exception {
            BusinessObjectData businessObjectData = downloaderWebClient.preRegisterBusinessObjectData(uploaderInputManifestDto, StorageEntity.MANAGED_STORAGE, false);
            BusinessObjectDataKey businessObjectDataKey = businessObjectDataHelper.getBusinessObjectDataKey(businessObjectData);
            downloaderWebClient.addStorageFiles(businessObjectDataKey, uploaderInputManifestDto, getTestS3FileTransferRequestParamsDto(S3_TEST_PATH_V0 + "/"), StorageEntity.MANAGED_STORAGE);
            downloaderWebClient.updateBusinessObjectDataStatus(businessObjectDataKey, BusinessObjectDataStatusEntity.VALID);
        }
    });
    // Get S3 key prefix.
    BusinessObjectData businessObjectData = toBusinessObjectData(uploaderInputManifestDto);
    S3KeyPrefixInformation resultS3KeyPrefixInformation = downloaderWebClient.getS3KeyPrefix(businessObjectData);
    // Validate the results.
    assertNotNull(resultS3KeyPrefixInformation);
    assertEquals(S3_SIMPLE_TEST_PATH, resultS3KeyPrefixInformation.getS3KeyPrefix());
}
Also used : UploaderInputManifestDto(org.finra.herd.model.dto.UploaderInputManifestDto) Command(org.finra.herd.core.Command) BusinessObjectData(org.finra.herd.model.api.xml.BusinessObjectData) S3KeyPrefixInformation(org.finra.herd.model.api.xml.S3KeyPrefixInformation) BusinessObjectDataKey(org.finra.herd.model.api.xml.BusinessObjectDataKey) IOException(java.io.IOException) Test(org.junit.Test)

Example 9 with BusinessObjectData

use of org.finra.herd.model.api.xml.BusinessObjectData in project herd by FINRAOS.

the class DownloaderWebClientTest method testGetBusinessObjectDataAssertNamespaceOptional.

@Test
public void testGetBusinessObjectDataAssertNamespaceOptional() throws Exception {
    HttpClientOperations mockHttpClientOperations = mock(HttpClientOperations.class);
    HttpClientOperations originalHttpClientOperations = (HttpClientOperations) ReflectionTestUtils.getField(downloaderWebClient, "httpClientOperations");
    ReflectionTestUtils.setField(downloaderWebClient, "httpClientOperations", mockHttpClientOperations);
    try {
        String expectedHttpMethod = "GET";
        String expectedUri = "https://testWebServiceHostname:1234/herd-app/rest/businessObjectData" + "/businessObjectDefinitionNames/businessObjectDefinitionName/businessObjectFormatUsages/businessObjectFormatUsage" + "/businessObjectFormatFileTypes/businessObjectFormatFileType?partitionKey=partitionKey&partitionValue=partitionValue&" + "businessObjectFormatVersion=0&businessObjectDataVersion=1";
        CloseableHttpResponse closeableHttpResponse = mock(CloseableHttpResponse.class);
        when(mockHttpClientOperations.execute(any(), any())).thenReturn(closeableHttpResponse);
        when(closeableHttpResponse.getStatusLine()).thenReturn(new BasicStatusLine(HttpVersion.HTTP_1_1, 200, "OK"));
        BusinessObjectData expectedBusinessObjectData = new BusinessObjectData();
        expectedBusinessObjectData.setId(1234);
        StringEntity httpEntity = new StringEntity(xmlHelper.objectToXml(expectedBusinessObjectData));
        when(closeableHttpResponse.getEntity()).thenReturn(httpEntity);
        DownloaderInputManifestDto manifest = new DownloaderInputManifestDto();
        manifest.setBusinessObjectDefinitionName("businessObjectDefinitionName");
        manifest.setBusinessObjectFormatUsage("businessObjectFormatUsage");
        manifest.setBusinessObjectFormatFileType("businessObjectFormatFileType");
        manifest.setBusinessObjectFormatVersion("0");
        manifest.setPartitionKey("partitionKey");
        manifest.setPartitionValue("partitionValue");
        manifest.setBusinessObjectDataVersion("1");
        assertEquals(expectedBusinessObjectData.getId(), downloaderWebClient.getBusinessObjectData(manifest).getId());
        verify(mockHttpClientOperations).execute(any(), argThat(httpUriRequest -> Objects.equal(expectedHttpMethod, httpUriRequest.getMethod()) && Objects.equal(expectedUri, httpUriRequest.getURI().toString())));
    } finally {
        ReflectionTestUtils.setField(downloaderWebClient, "httpClientOperations", originalHttpClientOperations);
    }
}
Also used : HttpVersion(org.apache.http.HttpVersion) Arrays(java.util.Arrays) BasicStatusLine(org.apache.http.message.BasicStatusLine) StorageUnit(org.finra.herd.model.api.xml.StorageUnit) Mockito.doThrow(org.mockito.Mockito.doThrow) BusinessObjectDataKey(org.finra.herd.model.api.xml.BusinessObjectDataKey) UploaderInputManifestDto(org.finra.herd.model.dto.UploaderInputManifestDto) Command(org.finra.herd.core.Command) CloseableHttpResponse(org.apache.http.client.methods.CloseableHttpResponse) S3KeyPrefixInformation(org.finra.herd.model.api.xml.S3KeyPrefixInformation) Storage(org.finra.herd.model.api.xml.Storage) DataBridgeWebClient(org.finra.herd.tools.common.databridge.DataBridgeWebClient) Assert.fail(org.junit.Assert.fail) Objects(com.google.common.base.Objects) DownloaderInputManifestDto(org.finra.herd.model.dto.DownloaderInputManifestDto) BusinessObjectData(org.finra.herd.model.api.xml.BusinessObjectData) StorageUnitDownloadCredential(org.finra.herd.model.api.xml.StorageUnitDownloadCredential) CloseableHttpClient(org.apache.http.impl.client.CloseableHttpClient) Assert.assertNotNull(org.junit.Assert.assertNotNull) Assert.assertTrue(org.junit.Assert.assertTrue) StringEntity(org.apache.http.entity.StringEntity) ReflectionTestUtils(org.springframework.test.util.ReflectionTestUtils) IOException(java.io.IOException) Test(org.junit.Test) Mockito.when(org.mockito.Mockito.when) RegServerAccessParamsDto(org.finra.herd.model.dto.RegServerAccessParamsDto) Mockito.verify(org.mockito.Mockito.verify) Matchers.any(org.mockito.Matchers.any) HttpClientOperations(org.finra.herd.dao.HttpClientOperations) Matchers.argThat(org.mockito.Matchers.argThat) Assert(org.junit.Assert) AwsCredential(org.finra.herd.model.api.xml.AwsCredential) Assert.assertEquals(org.junit.Assert.assertEquals) StorageEntity(org.finra.herd.model.jpa.StorageEntity) Mockito.mock(org.mockito.Mockito.mock) BusinessObjectDataStatusEntity(org.finra.herd.model.jpa.BusinessObjectDataStatusEntity) StringEntity(org.apache.http.entity.StringEntity) BusinessObjectData(org.finra.herd.model.api.xml.BusinessObjectData) DownloaderInputManifestDto(org.finra.herd.model.dto.DownloaderInputManifestDto) CloseableHttpResponse(org.apache.http.client.methods.CloseableHttpResponse) HttpClientOperations(org.finra.herd.dao.HttpClientOperations) BasicStatusLine(org.apache.http.message.BasicStatusLine) Test(org.junit.Test)

Example 10 with BusinessObjectData

use of org.finra.herd.model.api.xml.BusinessObjectData in project herd by FINRAOS.

the class AbstractDownloaderTest method uploadAndRegisterTestData.

/**
 * Uploads and registers a version if of the test business object data with the specified data files.
 *
 * @param s3KeyPrefix the destination S3 key prefix that must comply with the S3 naming conventions including the expected data version value
 * @param manifestFiles the test data files to be uploaded to S3 and registered
 * @param directoryPaths the list of directory paths to be created in S3 relative to the S3 key prefix
 */
protected void uploadAndRegisterTestData(String s3KeyPrefix, List<ManifestFile> manifestFiles, List<String> directoryPaths) throws Exception {
    uploadTestDataFilesToS3(s3KeyPrefix, manifestFiles, directoryPaths);
    UploaderInputManifestDto uploaderInputManifestDto = getTestUploaderInputManifestDto();
    uploaderInputManifestDto.setManifestFiles(manifestFiles);
    S3FileTransferRequestParamsDto s3FileTransferRequestParamsDto = getTestS3FileTransferRequestParamsDto();
    s3FileTransferRequestParamsDto.setS3KeyPrefix(s3KeyPrefix + "/");
    BusinessObjectData businessObjectData = downloaderWebClient.preRegisterBusinessObjectData(uploaderInputManifestDto, StorageEntity.MANAGED_STORAGE, false);
    BusinessObjectDataKey businessObjectDataKey = businessObjectDataHelper.getBusinessObjectDataKey(businessObjectData);
    downloaderWebClient.addStorageFiles(businessObjectDataKey, uploaderInputManifestDto, s3FileTransferRequestParamsDto, StorageEntity.MANAGED_STORAGE);
    downloaderWebClient.updateBusinessObjectDataStatus(businessObjectDataKey, BusinessObjectDataStatusEntity.VALID);
    // Clean up the local input directory used for the test data files upload.
    FileUtils.cleanDirectory(LOCAL_TEMP_PATH_INPUT.toFile());
}
Also used : S3FileTransferRequestParamsDto(org.finra.herd.model.dto.S3FileTransferRequestParamsDto) UploaderInputManifestDto(org.finra.herd.model.dto.UploaderInputManifestDto) BusinessObjectData(org.finra.herd.model.api.xml.BusinessObjectData) BusinessObjectDataKey(org.finra.herd.model.api.xml.BusinessObjectDataKey)

Aggregations

BusinessObjectData (org.finra.herd.model.api.xml.BusinessObjectData)150 Test (org.junit.Test)121 BusinessObjectDataKey (org.finra.herd.model.api.xml.BusinessObjectDataKey)64 BusinessObjectDataEntity (org.finra.herd.model.jpa.BusinessObjectDataEntity)57 ArrayList (java.util.ArrayList)39 BusinessObjectDataSearchKey (org.finra.herd.model.api.xml.BusinessObjectDataSearchKey)29 StorageUnit (org.finra.herd.model.api.xml.StorageUnit)25 BusinessObjectDataCreateRequest (org.finra.herd.model.api.xml.BusinessObjectDataCreateRequest)19 Storage (org.finra.herd.model.api.xml.Storage)19 AbstractServiceTest (org.finra.herd.service.AbstractServiceTest)16 Attribute (org.finra.herd.model.api.xml.Attribute)15 BusinessObjectFormatEntity (org.finra.herd.model.jpa.BusinessObjectFormatEntity)14 S3FileTransferRequestParamsDto (org.finra.herd.model.dto.S3FileTransferRequestParamsDto)12 StorageUnitEntity (org.finra.herd.model.jpa.StorageUnitEntity)11 ObjectNotFoundException (org.finra.herd.model.ObjectNotFoundException)9 AttributeValueFilter (org.finra.herd.model.api.xml.AttributeValueFilter)8 UploaderInputManifestDto (org.finra.herd.model.dto.UploaderInputManifestDto)8 IOException (java.io.IOException)7 BusinessObjectDataRetryStoragePolicyTransitionRequest (org.finra.herd.model.api.xml.BusinessObjectDataRetryStoragePolicyTransitionRequest)7 BusinessObjectDataSearchRequest (org.finra.herd.model.api.xml.BusinessObjectDataSearchRequest)7