Search in sources :

Example 26 with ExportParameters

use of org.talend.dataprep.api.export.ExportParameters in project data-prep by Talend.

the class OptimizedExportStrategyTest method testAcceptKO_preparationNotExist.

@Ignore
@Test(expected = TDPException.class)
public void testAcceptKO_preparationNotExist() throws Exception {
    // Given
    ExportParameters exportParameters = new ExportParameters();
    exportParameters.setPreparationId("1234");
    // Then
    assertFalse(optimizedExportStrategy.accept(exportParameters));
}
Also used : ExportParameters(org.talend.dataprep.api.export.ExportParameters) Ignore(org.junit.Ignore) Test(org.junit.Test) TransformationServiceBaseTest(org.talend.dataprep.transformation.service.TransformationServiceBaseTest)

Example 27 with ExportParameters

use of org.talend.dataprep.api.export.ExportParameters in project data-prep by Talend.

the class OptimizedExportStrategyTest method testAcceptKO_noMetadataCache.

@Test
public void testAcceptKO_noMetadataCache() throws Exception {
    // Given
    final String preparation = createEmptyPreparationFromDataset("1234", "test");
    applyAction(preparation, "[{}]");
    applyAction(preparation, "[{}]");
    ExportParameters exportParameters = new ExportParameters();
    exportParameters.setPreparationId(preparation);
    // Then
    assertFalse(optimizedExportStrategy.accept(exportParameters));
}
Also used : ExportParameters(org.talend.dataprep.api.export.ExportParameters) Test(org.junit.Test) TransformationServiceBaseTest(org.talend.dataprep.transformation.service.TransformationServiceBaseTest)

Example 28 with ExportParameters

use of org.talend.dataprep.api.export.ExportParameters in project data-prep by Talend.

the class OptimizedExportStrategyTest method testAcceptKO_withMetadataCacheNoContentCache.

@Test
public void testAcceptKO_withMetadataCacheNoContentCache() throws Exception {
    // Given
    final String preparation = createEmptyPreparationFromDataset("1234", "test");
    applyAction(preparation, "[{}]");
    applyAction(preparation, "[{}]");
    final Preparation preparationDetails = getPreparation(preparation);
    for (Step step : preparationDetails.getSteps()) {
        try (OutputStream content = contentCache.put(cacheKeyGenerator.generateMetadataKey(preparation, step.id(), HEAD), ContentCache.TimeToLive.DEFAULT)) {
            content.write("{}".getBytes());
            content.flush();
        }
    }
    ExportParameters exportParameters = new ExportParameters();
    exportParameters.setPreparationId(preparation);
    exportParameters.setFrom(HEAD);
    // Then
    assertFalse(optimizedExportStrategy.accept(exportParameters));
}
Also used : Preparation(org.talend.dataprep.api.preparation.Preparation) ExportParameters(org.talend.dataprep.api.export.ExportParameters) OutputStream(java.io.OutputStream) Step(org.talend.dataprep.api.preparation.Step) Test(org.junit.Test) TransformationServiceBaseTest(org.talend.dataprep.transformation.service.TransformationServiceBaseTest)

Example 29 with ExportParameters

use of org.talend.dataprep.api.export.ExportParameters in project data-prep by Talend.

the class OptimizedExportStrategyTest method testExecute.

@Test
public void testExecute() throws Exception {
    // Given
    final String datasetId = "1234";
    final String format = "JSON";
    final String preparation = createEmptyPreparationFromDataset(datasetId, "test");
    applyAction(preparation, "[{}]");
    applyAction(preparation, "[{}]");
    final Preparation preparationDetails = getPreparation(preparation);
    for (Step step : preparationDetails.getSteps()) {
        try (OutputStream content = contentCache.put(cacheKeyGenerator.generateMetadataKey(preparation, step.id(), HEAD), ContentCache.TimeToLive.DEFAULT)) {
            content.write("{}".getBytes());
            content.flush();
        }
        final TransformationCacheKey key = // 
        cacheKeyGenerator.generateContentKey(// 
        datasetId, // 
        preparation, // 
        step.id(), // 
        format, // 
        HEAD, // no filter
        "");
        try (OutputStream content = contentCache.put(key, ContentCache.TimeToLive.DEFAULT)) {
            content.write("{\"records\": [{\"0000\": \"a\"}]}".getBytes());
            content.flush();
        }
    }
    ExportParameters exportParameters = new ExportParameters();
    exportParameters.setPreparationId(preparation);
    exportParameters.setDatasetId(datasetId);
    exportParameters.setExportType(format);
    exportParameters.setFrom(HEAD);
    // Then
    optimizedExportStrategy.execute(exportParameters);
}
Also used : TransformationCacheKey(org.talend.dataprep.cache.TransformationCacheKey) Preparation(org.talend.dataprep.api.preparation.Preparation) ExportParameters(org.talend.dataprep.api.export.ExportParameters) OutputStream(java.io.OutputStream) Step(org.talend.dataprep.api.preparation.Step) Test(org.junit.Test) TransformationServiceBaseTest(org.talend.dataprep.transformation.service.TransformationServiceBaseTest)

Example 30 with ExportParameters

use of org.talend.dataprep.api.export.ExportParameters in project data-prep by Talend.

the class PreparationExportStrategyTest method shouldUsedVersionedPreparation.

@Test
public void shouldUsedVersionedPreparation() throws IOException {
    // Given
    final ExportParameters parameters = new ExportParameters();
    parameters.setExportType("JSON");
    parameters.setPreparationId("prep-1234");
    parameters.setStepId("step-1234");
    final Preparation preparation = new Preparation();
    preparation.setId("prep-1234");
    preparation.setHeadId("step-1234");
    configurePreparation(preparation, "prep-1234", "step-1234");
    // When
    final StreamingResponseBody body = strategy.execute(parameters);
    body.writeTo(new NullOutputStream());
    // Then
    final ArgumentCaptor<Configuration> captor = ArgumentCaptor.forClass(Configuration.class);
    verify(transformer).buildExecutable(any(), captor.capture());
    assertEquals("prep-1234", captor.getValue().getPreparationId());
    assertEquals("step-1234", captor.getValue().getPreparation().getHeadId());
}
Also used : StreamingResponseBody(org.springframework.web.servlet.mvc.method.annotation.StreamingResponseBody) Configuration(org.talend.dataprep.transformation.api.transformer.configuration.Configuration) ExportParameters(org.talend.dataprep.api.export.ExportParameters) Preparation(org.talend.dataprep.api.preparation.Preparation) NullOutputStream(org.apache.commons.io.output.NullOutputStream) Test(org.junit.Test)

Aggregations

ExportParameters (org.talend.dataprep.api.export.ExportParameters)40 Test (org.junit.Test)23 StreamingResponseBody (org.springframework.web.servlet.mvc.method.annotation.StreamingResponseBody)11 TDPException (org.talend.dataprep.exception.TDPException)11 Preparation (org.talend.dataprep.api.preparation.Preparation)10 DataSet (org.talend.dataprep.api.dataset.DataSet)9 TransformationCacheKey (org.talend.dataprep.cache.TransformationCacheKey)9 OutputStream (java.io.OutputStream)8 Configuration (org.talend.dataprep.transformation.api.transformer.configuration.Configuration)8 TransformationServiceBaseTest (org.talend.dataprep.transformation.service.TransformationServiceBaseTest)8 JsonParser (com.fasterxml.jackson.core.JsonParser)7 InputStream (java.io.InputStream)7 StringUtils (org.apache.commons.lang.StringUtils)7 Component (org.springframework.stereotype.Component)7 ServiceBaseTest (org.talend.ServiceBaseTest)7 ExportFormat (org.talend.dataprep.format.export.ExportFormat)7 CSVFormat (org.talend.dataprep.transformation.format.CSVFormat)7 BaseExportStrategy (org.talend.dataprep.transformation.service.BaseExportStrategy)7 ExportUtils (org.talend.dataprep.transformation.service.ExportUtils)7 InputStreamReader (java.io.InputStreamReader)6