Search in sources :

Example 1 with EuropeanaGeneratedIdsMap

use of eu.europeana.metis.transformation.service.EuropeanaGeneratedIdsMap in project Europeana-Cloud by europeana.

the class IdentifierSupplier method prepareIdentifiers.

public void prepareIdentifiers(StormTaskTuple tuple) throws EuropeanaIdException {
    final boolean useDefaultIdentifiers = useDefaultIdentifier(tuple);
    String metisDatasetId = null;
    if (!useDefaultIdentifiers) {
        metisDatasetId = tuple.getParameter(PluginParameterKeys.METIS_DATASET_ID);
        if (StringUtils.isEmpty(metisDatasetId)) {
            throw new EuropeanaIdException("Could not create identifier - parameter " + PluginParameterKeys.METIS_DATASET_ID + " is empty!");
        }
    }
    String localId;
    if (useDefaultIdentifiers) {
        localId = formulateLocalId(evaluateFileRelativePath(tuple));
    } else {
        EuropeanaGeneratedIdsMap europeanaIdentifier = getEuropeanaIdentifier(tuple, metisDatasetId);
        localId = europeanaIdentifier.getEuropeanaGeneratedId();
        tuple.addParameter(PluginParameterKeys.ADDITIONAL_LOCAL_IDENTIFIER, europeanaIdentifier.getSourceProvidedChoAbout());
    }
    tuple.addParameter(PluginParameterKeys.CLOUD_LOCAL_IDENTIFIER, localId);
}
Also used : EuropeanaIdException(eu.europeana.metis.transformation.service.EuropeanaIdException) EuropeanaGeneratedIdsMap(eu.europeana.metis.transformation.service.EuropeanaGeneratedIdsMap)

Example 2 with EuropeanaGeneratedIdsMap

use of eu.europeana.metis.transformation.service.EuropeanaGeneratedIdsMap in project metis-framework by europeana.

the class DatasetService method transformRecords.

private List<Record> transformRecords(Dataset dataset, Collection<Record> records, String xsltUrl) throws XsltSetupException {
    // Set up transformer.
    final XsltTransformer transformer;
    final EuropeanaIdCreator europeanIdCreator;
    try {
        final TransformationParameters transformationParameters = new TransformationParameters(dataset);
        transformer = new XsltTransformer(xsltUrl, transformationParameters.getDatasetName(), transformationParameters.getEdmCountry(), transformationParameters.getEdmLanguage());
        europeanIdCreator = new EuropeanaIdCreator();
    } catch (TransformationException e) {
        throw new XsltSetupException("Could not setup XSL transformation.", e);
    } catch (EuropeanaIdException e) {
        throw new XsltSetupException(CommonStringValues.EUROPEANA_ID_CREATOR_INITIALIZATION_FAILED, e);
    }
    // Transform the records.
    return records.stream().map(record -> {
        try {
            EuropeanaGeneratedIdsMap europeanaGeneratedIdsMap = europeanIdCreator.constructEuropeanaId(record.getXmlRecord(), dataset.getDatasetId());
            return new Record(record.getEcloudId(), transformer.transform(record.getXmlRecord().getBytes(StandardCharsets.UTF_8), europeanaGeneratedIdsMap).toString());
        } catch (TransformationException e) {
            LOGGER.info("Record from list failed transformation", e);
            return new Record(record.getEcloudId(), e.getMessage());
        } catch (EuropeanaIdException e) {
            LOGGER.info(CommonStringValues.EUROPEANA_ID_CREATOR_INITIALIZATION_FAILED, e);
            return new Record(record.getEcloudId(), e.getMessage());
        }
    }).collect(Collectors.toList());
}
Also used : Arrays(java.util.Arrays) EuropeanaGeneratedIdsMap(eu.europeana.metis.transformation.service.EuropeanaGeneratedIdsMap) Date(java.util.Date) LoggerFactory(org.slf4j.LoggerFactory) Autowired(org.springframework.beans.factory.annotation.Autowired) ScheduledWorkflowDao(eu.europeana.metis.core.dao.ScheduledWorkflowDao) StringUtils(org.apache.commons.lang3.StringUtils) RestEndpoints(eu.europeana.metis.utils.RestEndpoints) XsltSetupException(eu.europeana.metis.core.exceptions.XsltSetupException) EuropeanaIdCreator(eu.europeana.metis.transformation.service.EuropeanaIdCreator) DatasetAlreadyExistsException(eu.europeana.metis.core.exceptions.DatasetAlreadyExistsException) ArrayList(java.util.ArrayList) GenericMetisException(eu.europeana.metis.exception.GenericMetisException) EuropeanaIdException(eu.europeana.metis.transformation.service.EuropeanaIdException) PluginWithExecutionId(eu.europeana.metis.core.dao.PluginWithExecutionId) RLock(org.redisson.api.RLock) Service(org.springframework.stereotype.Service) RedissonClient(org.redisson.api.RedissonClient) CommonStringValues(eu.europeana.metis.utils.CommonStringValues) Dataset(eu.europeana.metis.core.dataset.Dataset) EnumSet(java.util.EnumSet) WorkflowDao(eu.europeana.metis.core.dao.WorkflowDao) UserUnauthorizedException(eu.europeana.metis.exception.UserUnauthorizedException) Logger(org.slf4j.Logger) DatasetSearchView(eu.europeana.metis.core.dataset.DatasetSearchView) DatasetXslt(eu.europeana.metis.core.dataset.DatasetXslt) ExecutablePlugin(eu.europeana.metis.core.workflow.plugins.ExecutablePlugin) XsltTransformer(eu.europeana.metis.transformation.service.XsltTransformer) Collection(java.util.Collection) WorkflowExecutionDao(eu.europeana.metis.core.dao.WorkflowExecutionDao) WorkflowExecution(eu.europeana.metis.core.workflow.WorkflowExecution) ExecutablePluginType(eu.europeana.metis.core.workflow.plugins.ExecutablePluginType) TransformationException(eu.europeana.metis.transformation.service.TransformationException) UUID(java.util.UUID) Collectors(java.util.stream.Collectors) MetisUserView(eu.europeana.metis.authentication.user.MetisUserView) DatasetDao(eu.europeana.metis.core.dao.DatasetDao) StandardCharsets(java.nio.charset.StandardCharsets) TransformationParameters(eu.europeana.metis.core.common.TransformationParameters) NoDatasetFoundException(eu.europeana.metis.core.exceptions.NoDatasetFoundException) List(java.util.List) NoXsltFoundException(eu.europeana.metis.core.exceptions.NoXsltFoundException) Record(eu.europeana.metis.core.rest.Record) ObjectId(org.bson.types.ObjectId) DatasetXsltDao(eu.europeana.metis.core.dao.DatasetXsltDao) TransformationPlugin(eu.europeana.metis.core.workflow.plugins.TransformationPlugin) BadContentException(eu.europeana.metis.exception.BadContentException) Collections(java.util.Collections) EuropeanaIdCreator(eu.europeana.metis.transformation.service.EuropeanaIdCreator) TransformationException(eu.europeana.metis.transformation.service.TransformationException) XsltTransformer(eu.europeana.metis.transformation.service.XsltTransformer) TransformationParameters(eu.europeana.metis.core.common.TransformationParameters) XsltSetupException(eu.europeana.metis.core.exceptions.XsltSetupException) Record(eu.europeana.metis.core.rest.Record) EuropeanaIdException(eu.europeana.metis.transformation.service.EuropeanaIdException) EuropeanaGeneratedIdsMap(eu.europeana.metis.transformation.service.EuropeanaGeneratedIdsMap)

Example 3 with EuropeanaGeneratedIdsMap

use of eu.europeana.metis.transformation.service.EuropeanaGeneratedIdsMap in project metis-sandbox by europeana.

the class TransformationServiceImpl method transformToEdmInternal.

@Override
public RecordInfo transformToEdmInternal(Record record) {
    requireNonNull(record, "Record must not be null");
    final byte[] recordTransformed;
    try {
        final EuropeanaGeneratedIdsMap europeanaGeneratedIdsMap = new EuropeanaIdCreator().constructEuropeanaId(record.getContentInputStream(), record.getDatasetId());
        XsltTransformer transformer = getTransformer(getJoinDatasetIdDatasetName(record), record.getCountry().xmlValue(), record.getLanguage().name().toLowerCase());
        recordTransformed = transformer.transformToBytes(record.getContent(), europeanaGeneratedIdsMap);
    } catch (TransformationException | EuropeanaIdException e) {
        throw new RecordProcessingException(record.getProviderId(), e);
    }
    return new RecordInfo(Record.from(record, recordTransformed));
}
Also used : EuropeanaIdCreator(eu.europeana.metis.transformation.service.EuropeanaIdCreator) TransformationException(eu.europeana.metis.transformation.service.TransformationException) RecordProcessingException(eu.europeana.metis.sandbox.common.exception.RecordProcessingException) RecordInfo(eu.europeana.metis.sandbox.domain.RecordInfo) XsltTransformer(eu.europeana.metis.transformation.service.XsltTransformer) EuropeanaGeneratedIdsMap(eu.europeana.metis.transformation.service.EuropeanaGeneratedIdsMap) EuropeanaIdException(eu.europeana.metis.transformation.service.EuropeanaIdException)

Aggregations

EuropeanaGeneratedIdsMap (eu.europeana.metis.transformation.service.EuropeanaGeneratedIdsMap)3 EuropeanaIdException (eu.europeana.metis.transformation.service.EuropeanaIdException)3 EuropeanaIdCreator (eu.europeana.metis.transformation.service.EuropeanaIdCreator)2 TransformationException (eu.europeana.metis.transformation.service.TransformationException)2 XsltTransformer (eu.europeana.metis.transformation.service.XsltTransformer)2 MetisUserView (eu.europeana.metis.authentication.user.MetisUserView)1 TransformationParameters (eu.europeana.metis.core.common.TransformationParameters)1 DatasetDao (eu.europeana.metis.core.dao.DatasetDao)1 DatasetXsltDao (eu.europeana.metis.core.dao.DatasetXsltDao)1 PluginWithExecutionId (eu.europeana.metis.core.dao.PluginWithExecutionId)1 ScheduledWorkflowDao (eu.europeana.metis.core.dao.ScheduledWorkflowDao)1 WorkflowDao (eu.europeana.metis.core.dao.WorkflowDao)1 WorkflowExecutionDao (eu.europeana.metis.core.dao.WorkflowExecutionDao)1 Dataset (eu.europeana.metis.core.dataset.Dataset)1 DatasetSearchView (eu.europeana.metis.core.dataset.DatasetSearchView)1 DatasetXslt (eu.europeana.metis.core.dataset.DatasetXslt)1 DatasetAlreadyExistsException (eu.europeana.metis.core.exceptions.DatasetAlreadyExistsException)1 NoDatasetFoundException (eu.europeana.metis.core.exceptions.NoDatasetFoundException)1 NoXsltFoundException (eu.europeana.metis.core.exceptions.NoXsltFoundException)1 XsltSetupException (eu.europeana.metis.core.exceptions.XsltSetupException)1