Search in sources :

Example 6 with VolumeMetered

use of org.talend.dataprep.metrics.VolumeMetered in project data-prep by Talend.

the class TransformationService method executeMetadata.

@RequestMapping(value = "/apply/preparation/{preparationId}/{stepId}/metadata", method = GET)
@ApiOperation(value = "Run the transformation given the provided export parameters", notes = "This operation transforms the dataset or preparation using parameters in export parameters.")
@VolumeMetered
@AsyncOperation(// 
conditionalClass = GetPrepMetadataAsyncCondition.class, // 
resultUrlGenerator = PrepMetadataGetContentUrlGenerator.class, executionIdGeneratorClass = PrepMetadataExecutionIdGenerator.class)
public DataSetMetadata executeMetadata(@PathVariable("preparationId") @AsyncParameter String preparationId, @PathVariable("stepId") @AsyncParameter String stepId) {
    LOG.debug("getting preparation metadata for #{}, step {}", preparationId, stepId);
    final Preparation preparation = getPreparation(preparationId);
    if (preparation.getSteps().size() > 1) {
        String headId = "head".equalsIgnoreCase(stepId) ? preparation.getHeadId() : stepId;
        final TransformationMetadataCacheKey cacheKey = cacheKeyGenerator.generateMetadataKey(preparationId, headId, HEAD);
        // No metadata in cache, recompute it
        if (!contentCache.has(cacheKey)) {
            try {
                LOG.debug("Metadata not available for preparation '{}' at step '{}'", preparationId, headId);
                ExportParameters parameters = new ExportParameters();
                parameters.setPreparationId(preparationId);
                parameters.setExportType("JSON");
                parameters.setStepId(headId);
                parameters.setFrom(HEAD);
                // we regenerate cache
                parameters = exportParametersUtil.populateFromPreparationExportParameter(parameters);
                preparationExportStrategy.performPreparation(parameters, new NullOutputStream());
            } catch (Exception e) {
                throw new TDPException(TransformationErrorCodes.METADATA_NOT_FOUND, e);
            }
        }
        // }
        if (contentCache.has(cacheKey)) {
            try (InputStream stream = contentCache.get(cacheKey)) {
                return mapper.readerFor(DataSetMetadata.class).readValue(stream);
            } catch (IOException e) {
                throw new TDPException(CommonErrorCodes.UNEXPECTED_EXCEPTION, e);
            }
        }
    } else {
        LOG.debug("No step in preparation '{}', falls back to get dataset metadata (id: {})", preparationId, preparation.getDataSetId());
        DataSetGetMetadata getMetadata = context.getBean(DataSetGetMetadata.class, preparation.getDataSetId());
        return getMetadata.execute();
    }
    return null;
}
Also used : TDPException(org.talend.dataprep.exception.TDPException) Preparation(org.talend.dataprep.api.preparation.Preparation) ExportParameters(org.talend.dataprep.api.export.ExportParameters) TransformationMetadataCacheKey(org.talend.dataprep.cache.TransformationMetadataCacheKey) TDPException(org.talend.dataprep.exception.TDPException) DataSetMetadata(org.talend.dataprep.api.dataset.DataSetMetadata) DataSetGetMetadata(org.talend.dataprep.command.dataset.DataSetGetMetadata) NullOutputStream(org.apache.commons.io.output.NullOutputStream) VolumeMetered(org.talend.dataprep.metrics.VolumeMetered) ApiOperation(io.swagger.annotations.ApiOperation)

Aggregations

ApiOperation (io.swagger.annotations.ApiOperation)6 VolumeMetered (org.talend.dataprep.metrics.VolumeMetered)6 ExportParameters (org.talend.dataprep.api.export.ExportParameters)4 TDPException (org.talend.dataprep.exception.TDPException)4 NullOutputStream (org.apache.commons.io.output.NullOutputStream)3 DataSetMetadata (org.talend.dataprep.api.dataset.DataSetMetadata)3 IOException (java.io.IOException)2 RequestMapping (org.springframework.web.bind.annotation.RequestMapping)2 StrictlyBoundedInputStream (org.talend.dataprep.dataset.store.content.StrictlyBoundedInputStream)2 Timed (org.talend.dataprep.metrics.Timed)2 JsonParser (com.fasterxml.jackson.core.JsonParser)1 InputStream (java.io.InputStream)1 OutputStream (java.io.OutputStream)1 PipedInputStream (java.io.PipedInputStream)1 PipedOutputStream (java.io.PipedOutputStream)1 Marker (org.slf4j.Marker)1 StreamingResponseBody (org.springframework.web.servlet.mvc.method.annotation.StreamingResponseBody)1 DataSet (org.talend.dataprep.api.dataset.DataSet)1 DataSetLocation (org.talend.dataprep.api.dataset.DataSetLocation)1 Preparation (org.talend.dataprep.api.preparation.Preparation)1