Search in sources :

Example 16 with StreamingResponseBody

use of org.springframework.web.servlet.mvc.method.annotation.StreamingResponseBody in project data-prep by Talend.

the class PreparationExportStrategyTest method shouldUsedVersionedPreparation.

@Test
public void shouldUsedVersionedPreparation() throws IOException {
    // Given
    final ExportParameters parameters = new ExportParameters();
    parameters.setExportType("JSON");
    parameters.setPreparationId("prep-1234");
    parameters.setStepId("step-1234");
    final Preparation preparation = new Preparation();
    preparation.setId("prep-1234");
    preparation.setHeadId("step-1234");
    configurePreparation(preparation, "prep-1234", "step-1234");
    // When
    final StreamingResponseBody body = strategy.execute(parameters);
    body.writeTo(new NullOutputStream());
    // Then
    final ArgumentCaptor<Configuration> captor = ArgumentCaptor.forClass(Configuration.class);
    verify(transformer).buildExecutable(any(), captor.capture());
    assertEquals("prep-1234", captor.getValue().getPreparationId());
    assertEquals("step-1234", captor.getValue().getPreparation().getHeadId());
}
Also used : StreamingResponseBody(org.springframework.web.servlet.mvc.method.annotation.StreamingResponseBody) Configuration(org.talend.dataprep.transformation.api.transformer.configuration.Configuration) ExportParameters(org.talend.dataprep.api.export.ExportParameters) Preparation(org.talend.dataprep.api.preparation.Preparation) NullOutputStream(org.apache.commons.io.output.NullOutputStream) Test(org.junit.Test)

Example 17 with StreamingResponseBody

use of org.springframework.web.servlet.mvc.method.annotation.StreamingResponseBody in project data-prep by Talend.

the class OptimizedExportStrategy method execute.

@Override
public StreamingResponseBody execute(ExportParameters parameters) {
    final String formatName = parameters.getExportType();
    final ExportFormat format = getFormat(formatName);
    // 
    ExportUtils.setExportHeaders(// 
    parameters.getExportName(), // 
    parameters.getArguments().get(ExportFormat.PREFIX + CSVFormat.ParametersCSV.ENCODING), format);
    return outputStream -> performOptimizedTransform(parameters, outputStream);
}
Also used : ExportFormat(org.talend.dataprep.format.export.ExportFormat) StringUtils(org.apache.commons.lang.StringUtils) TDPException(org.talend.dataprep.exception.TDPException) TransformationErrorCodes(org.talend.dataprep.exception.error.TransformationErrorCodes) LoggerFactory(org.slf4j.LoggerFactory) Autowired(org.springframework.beans.factory.annotation.Autowired) Configuration(org.talend.dataprep.transformation.api.transformer.configuration.Configuration) PreparationMessage(org.talend.dataprep.api.preparation.PreparationMessage) ArrayList(java.util.ArrayList) CacheKeyGenerator(org.talend.dataprep.cache.CacheKeyGenerator) TransformationMetadataCacheKey(org.talend.dataprep.cache.TransformationMetadataCacheKey) DataSetMetadata(org.talend.dataprep.api.dataset.DataSetMetadata) TeeOutputStream(org.apache.commons.io.output.TeeOutputStream) ExportUtils(org.talend.dataprep.transformation.service.ExportUtils) DataSet(org.talend.dataprep.api.dataset.DataSet) OutputStream(java.io.OutputStream) Preparation(org.talend.dataprep.api.preparation.Preparation) ExportParameters(org.talend.dataprep.api.export.ExportParameters) Logger(org.slf4j.Logger) BaseExportStrategy(org.talend.dataprep.transformation.service.BaseExportStrategy) JsonParser(com.fasterxml.jackson.core.JsonParser) UTF_8(java.nio.charset.StandardCharsets.UTF_8) StreamingResponseBody(org.springframework.web.servlet.mvc.method.annotation.StreamingResponseBody) IOException(java.io.IOException) Step(org.talend.dataprep.api.preparation.Step) InputStreamReader(java.io.InputStreamReader) Collectors(java.util.stream.Collectors) ContentCache(org.talend.dataprep.cache.ContentCache) CSVFormat(org.talend.dataprep.transformation.format.CSVFormat) List(java.util.List) Component(org.springframework.stereotype.Component) TransformationCacheKey(org.talend.dataprep.cache.TransformationCacheKey) InputStream(java.io.InputStream) ExportFormat(org.talend.dataprep.format.export.ExportFormat)

Example 18 with StreamingResponseBody

use of org.springframework.web.servlet.mvc.method.annotation.StreamingResponseBody in project data-prep by Talend.

the class PreparationExportStrategyTest method shouldUsedHeadPreparation.

@Test
public void shouldUsedHeadPreparation() throws IOException {
    // Given
    final ExportParameters parameters = new ExportParameters();
    parameters.setExportType("JSON");
    parameters.setPreparationId("prep-1234");
    parameters.setStepId("head");
    final Preparation preparation = new Preparation();
    preparation.setId("prep-1234");
    preparation.setHeadId("head");
    configurePreparation(preparation, "prep-1234", "head");
    // When
    final StreamingResponseBody body = strategy.execute(parameters);
    body.writeTo(new NullOutputStream());
    // Then
    final ArgumentCaptor<Configuration> captor = ArgumentCaptor.forClass(Configuration.class);
    verify(transformer).buildExecutable(any(), captor.capture());
    assertEquals("prep-1234", captor.getValue().getPreparationId());
    assertEquals("head", captor.getValue().getPreparation().getHeadId());
}
Also used : StreamingResponseBody(org.springframework.web.servlet.mvc.method.annotation.StreamingResponseBody) Configuration(org.talend.dataprep.transformation.api.transformer.configuration.Configuration) ExportParameters(org.talend.dataprep.api.export.ExportParameters) Preparation(org.talend.dataprep.api.preparation.Preparation) NullOutputStream(org.apache.commons.io.output.NullOutputStream) Test(org.junit.Test)

Example 19 with StreamingResponseBody

use of org.springframework.web.servlet.mvc.method.annotation.StreamingResponseBody in project data-prep by Talend.

the class TransformationService method addPreparationInCache.

/**
 * Add the following preparation in cache.
 *
 * @param preparation the preparation to cache.
 * @param stepId the preparation step id.
 */
private void addPreparationInCache(Preparation preparation, String stepId) {
    final ExportParameters exportParameters = new ExportParameters();
    exportParameters.setPreparationId(preparation.getId());
    exportParameters.setExportType("JSON");
    exportParameters.setStepId(stepId);
    exportParameters.setDatasetId(preparation.getDataSetId());
    final StreamingResponseBody streamingResponseBody = executeSampleExportStrategy(exportParameters);
    try {
        // the result is not important here as it will be cached !
        streamingResponseBody.writeTo(new NullOutputStream());
    } catch (IOException e) {
        throw new TDPException(UNEXPECTED_EXCEPTION, e);
    }
}
Also used : TDPException(org.talend.dataprep.exception.TDPException) StreamingResponseBody(org.springframework.web.servlet.mvc.method.annotation.StreamingResponseBody) ExportParameters(org.talend.dataprep.api.export.ExportParameters) NullOutputStream(org.apache.commons.io.output.NullOutputStream)

Example 20 with StreamingResponseBody

use of org.springframework.web.servlet.mvc.method.annotation.StreamingResponseBody in project data-prep by Talend.

the class ApplyPreparationExportStrategy method execute.

@Override
public StreamingResponseBody execute(ExportParameters parameters) {
    final String formatName = parameters.getExportType();
    final ExportFormat format = getFormat(formatName);
    // 
    ExportUtils.setExportHeaders(// 
    parameters.getExportName(), // 
    parameters.getArguments().get(ExportFormat.PREFIX + CSVFormat.ParametersCSV.ENCODING), format);
    return outputStream -> executeApplyPreparation(parameters, outputStream);
}
Also used : ExportFormat(org.talend.dataprep.format.export.ExportFormat) StringUtils(org.apache.commons.lang.StringUtils) DataSetGet(org.talend.dataprep.command.dataset.DataSetGet) TDPException(org.talend.dataprep.exception.TDPException) TransformationErrorCodes(org.talend.dataprep.exception.error.TransformationErrorCodes) SMALL(org.talend.dataprep.transformation.api.transformer.configuration.Configuration.Volume.SMALL) LoggerFactory(org.slf4j.LoggerFactory) Autowired(org.springframework.beans.factory.annotation.Autowired) Configuration(org.talend.dataprep.transformation.api.transformer.configuration.Configuration) CacheKeyGenerator(org.talend.dataprep.cache.CacheKeyGenerator) TeeOutputStream(org.apache.commons.io.output.TeeOutputStream) ExportUtils(org.talend.dataprep.transformation.service.ExportUtils) DataSet(org.talend.dataprep.api.dataset.DataSet) OutputStream(java.io.OutputStream) Preparation(org.talend.dataprep.api.preparation.Preparation) ExportParameters(org.talend.dataprep.api.export.ExportParameters) Logger(org.slf4j.Logger) BaseExportStrategy(org.talend.dataprep.transformation.service.BaseExportStrategy) JsonParser(com.fasterxml.jackson.core.JsonParser) UTF_8(java.nio.charset.StandardCharsets.UTF_8) StreamingResponseBody(org.springframework.web.servlet.mvc.method.annotation.StreamingResponseBody) JSON(org.talend.dataprep.transformation.format.JsonFormat.JSON) InputStreamReader(java.io.InputStreamReader) ContentCache(org.talend.dataprep.cache.ContentCache) Objects(java.util.Objects) CSVFormat(org.talend.dataprep.transformation.format.CSVFormat) Component(org.springframework.stereotype.Component) TransformationCacheKey(org.talend.dataprep.cache.TransformationCacheKey) InputStream(java.io.InputStream) ExportFormat(org.talend.dataprep.format.export.ExportFormat)

Aggregations

StreamingResponseBody (org.springframework.web.servlet.mvc.method.annotation.StreamingResponseBody)26 Test (org.junit.Test)16 Problem (org.zalando.problem.Problem)10 ExportParameters (org.talend.dataprep.api.export.ExportParameters)9 InputStream (java.io.InputStream)8 TDPException (org.talend.dataprep.exception.TDPException)8 ByteArrayOutputStream (java.io.ByteArrayOutputStream)6 OutputStream (java.io.OutputStream)6 Configuration (org.talend.dataprep.transformation.api.transformer.configuration.Configuration)6 JsonParser (com.fasterxml.jackson.core.JsonParser)5 StringUtils (org.apache.commons.lang.StringUtils)5 Logger (org.slf4j.Logger)5 LoggerFactory (org.slf4j.LoggerFactory)5 Autowired (org.springframework.beans.factory.annotation.Autowired)5 Component (org.springframework.stereotype.Component)5 DataSet (org.talend.dataprep.api.dataset.DataSet)5 ExportFormat (org.talend.dataprep.format.export.ExportFormat)5 CSVFormat (org.talend.dataprep.transformation.format.CSVFormat)5 BaseExportStrategy (org.talend.dataprep.transformation.service.BaseExportStrategy)5 ExportUtils (org.talend.dataprep.transformation.service.ExportUtils)5