Search in sources :

Example 1 with ActionDefinition

use of org.talend.dataprep.api.action.ActionDefinition in project data-prep by Talend.

the class SerializableTest method testSerializableClass.

@Test
public void testSerializableClass() throws Exception {
    LOGGER.info("{} actions for testing...", actions.length);
    int okActions = 0;
    for (ActionDefinition action : actions) {
        Map<String, String> parameters = new HashMap<>();
        if (action.acceptScope(ScopeCategory.COLUMN)) {
            parameters.put(ImplicitParameters.SCOPE.getKey(), "COLUMN");
        } else if (action.acceptScope(ScopeCategory.CELL)) {
            parameters.put(ImplicitParameters.SCOPE.getKey(), "CELL");
        } else if (action.acceptScope(ScopeCategory.LINE)) {
            parameters.put(ImplicitParameters.SCOPE.getKey(), "LINE");
        } else if (action.acceptScope(ScopeCategory.DATASET)) {
            parameters.put(ImplicitParameters.SCOPE.getKey(), "DATASET");
        }
        parameters.put(ImplicitParameters.COLUMN_ID.getKey(), "0000");
        parameters.put(ImplicitParameters.ROW_ID.getKey(), "0");
        final Action actionInstance = factory.create(action, parameters);
        try {
            try (ObjectOutputStream oos = new ObjectOutputStream(new NullOutputStream())) {
                oos.writeObject(actionInstance);
                oos.flush();
            }
            okActions++;
        } catch (IOException e) {
            LOGGER.error("Unable to serialize {}", action.getClass());
        }
    }
    LOGGER.info("{} actions for tested ({}/{} ok).", actions.length, okActions, actions.length);
}
Also used : Action(org.talend.dataprep.api.preparation.Action) HashMap(java.util.HashMap) IOException(java.io.IOException) ObjectOutputStream(java.io.ObjectOutputStream) ActionDefinition(org.talend.dataprep.api.action.ActionDefinition) NullOutputStream(org.apache.commons.io.output.NullOutputStream) Test(org.junit.Test)

Example 2 with ActionDefinition

use of org.talend.dataprep.api.action.ActionDefinition in project data-prep by Talend.

the class ActionsStaticProfiler method getActionMetadataByAction.

/**
 * Get the actions metadata by actions
 */
private Map<Action, ActionDefinition> getActionMetadataByAction(final List<RunnableAction> actions) {
    final Map<Action, ActionDefinition> actionToMetadata = new HashMap<>(actions.size());
    for (final Action action : actions) {
        final ActionDefinition actionMetadata = // 
        actionRegistry.get(action.getName()).adapt(ScopeCategory.from(action.getParameters().get(ImplicitParameters.SCOPE.getKey())));
        actionToMetadata.put(action, actionMetadata);
    }
    return actionToMetadata;
}
Also used : Action(org.talend.dataprep.api.preparation.Action) RunnableAction(org.talend.dataprep.transformation.actions.common.RunnableAction) ActionDefinition(org.talend.dataprep.api.action.ActionDefinition)

Example 3 with ActionDefinition

use of org.talend.dataprep.api.action.ActionDefinition in project data-prep by Talend.

the class PreparationConversions method toStudioPreparation.

private PreparationSummary toStudioPreparation(Preparation source, PreparationSummary target, ApplicationContext applicationContext) {
    if (target.getOwner() == null) {
        final Security security = applicationContext.getBean(Security.class);
        Owner owner = new Owner(security.getUserId(), security.getUserDisplayName(), StringUtils.EMPTY);
        target.setOwner(owner);
    }
    final PreparationRepository preparationRepository = applicationContext.getBean(PreparationRepository.class);
    final ActionRegistry actionRegistry = applicationContext.getBean(ActionRegistry.class);
    // Get preparation actions
    PreparationActions prepActions = preparationRepository.get(source.getHeadId(), PreparationActions.class);
    if (prepActions != null) {
        List<Action> actions = prepActions.getActions();
        boolean allowDistributedRun = true;
        for (Action action : actions) {
            final ActionDefinition actionDefinition = actionRegistry.get(action.getName());
            if (actionDefinition.getBehavior().contains(ActionDefinition.Behavior.FORBID_DISTRIBUTED)) {
                // Disallow distributed run
                allowDistributedRun = false;
                break;
            }
        }
        target.setAllowDistributedRun(allowDistributedRun);
    }
    return target;
}
Also used : PreparationRepository(org.talend.dataprep.preparation.store.PreparationRepository) Owner(org.talend.dataprep.api.share.Owner) Security(org.talend.dataprep.security.Security) ActionRegistry(org.talend.dataprep.transformation.pipeline.ActionRegistry) ActionDefinition(org.talend.dataprep.api.action.ActionDefinition)

Example 4 with ActionDefinition

use of org.talend.dataprep.api.action.ActionDefinition in project data-prep by Talend.

the class PreparationConversions method toPreparationMessage.

private PreparationMessage toPreparationMessage(Preparation source, PreparationMessage target, ApplicationContext applicationContext) {
    final PreparationRepository preparationRepository = applicationContext.getBean(PreparationRepository.class);
    final ActionRegistry actionRegistry = applicationContext.getBean(ActionRegistry.class);
    // Steps diff metadata
    final List<StepDiff> diffs = // 
    source.getSteps().stream().filter(// 
    step -> !Step.ROOT_STEP.id().equals(step.id())).map(// 
    Step::getDiff).collect(toList());
    target.setDiff(diffs);
    // Actions
    if (source.getHeadId() != null) {
        // Get preparation actions
        final String headId = source.getHeadId();
        final Step head = preparationRepository.get(headId, Step.class);
        if (head != null) {
            final PreparationActions prepActions = preparationRepository.get(head.getContent(), PreparationActions.class);
            if (prepActions != null) {
                final List<Action> actions = prepActions.getActions();
                target.setActions(prepActions.getActions());
                // Allow distributed run
                boolean allowDistributedRun = true;
                for (Action action : actions) {
                    final ActionDefinition actionDefinition = actionRegistry.get(action.getName());
                    if (actionDefinition.getBehavior().contains(ActionDefinition.Behavior.FORBID_DISTRIBUTED)) {
                        allowDistributedRun = false;
                        break;
                    }
                }
                target.setAllowDistributedRun(allowDistributedRun);
                // no need to have lock information (and may also break StandAlonePreparationParser...)
                target.setLock(null);
                // Actions metadata
                if (actionRegistry == null) {
                    LOGGER.debug("No action metadata available, unable to serialize action metadata for preparation {}.", source.id());
                } else {
                    List<ActionForm> actionDefinitions = // 
                    actions.stream().map(a -> // 
                    actionRegistry.get(a.getName()).adapt(// 
                    ScopeCategory.from(a.getParameters().get(ImplicitParameters.SCOPE.getKey())))).map(// 
                    a -> a.getActionForm(getLocale())).map(// 
                    PreparationConversions::disallowColumnCreationChange).collect(Collectors.toList());
                    target.setMetadata(actionDefinitions);
                }
            }
        } else {
            LOGGER.warn("Head step #{} for preparation #{} does not exist.", headId, source.id());
            target.setActions(Collections.emptyList());
            target.setSteps(Collections.singletonList(Step.ROOT_STEP));
            target.setMetadata(Collections.emptyList());
        }
    } else {
        target.setActions(Collections.emptyList());
        target.setSteps(Collections.singletonList(Step.ROOT_STEP));
        target.setMetadata(Collections.emptyList());
    }
    return target;
}
Also used : ImplicitParameters(org.talend.dataprep.transformation.actions.common.ImplicitParameters) StringUtils(org.apache.commons.lang.StringUtils) CREATE_NEW_COLUMN(org.talend.dataprep.transformation.actions.common.ActionsUtils.CREATE_NEW_COLUMN) BeanConversionService.fromBean(org.talend.dataprep.conversions.BeanConversionService.fromBean) LoggerFactory(org.slf4j.LoggerFactory) PreparationRepository(org.talend.dataprep.preparation.store.PreparationRepository) org.talend.dataprep.api.preparation(org.talend.dataprep.api.preparation) PersistentPreparation(org.talend.dataprep.preparation.store.PersistentPreparation) ScopeCategory(org.talend.dataprep.transformation.actions.category.ScopeCategory) BeanConversionService(org.talend.dataprep.conversions.BeanConversionService) Logger(org.slf4j.Logger) LocaleContextHolder.getLocale(org.springframework.context.i18n.LocaleContextHolder.getLocale) ActionRegistry(org.talend.dataprep.transformation.pipeline.ActionRegistry) Owner(org.talend.dataprep.api.share.Owner) Collectors(java.util.stream.Collectors) ApplicationContext(org.springframework.context.ApplicationContext) ActionForm(org.talend.dataprep.api.action.ActionForm) Collectors.toList(java.util.stream.Collectors.toList) List(java.util.List) Component(org.springframework.stereotype.Component) ActionDefinition(org.talend.dataprep.api.action.ActionDefinition) UserPreparation(org.talend.dataprep.preparation.service.UserPreparation) Security(org.talend.dataprep.security.Security) Collections(java.util.Collections) BeanConversionServiceWrapper(org.talend.dataprep.processor.BeanConversionServiceWrapper) PreparationRepository(org.talend.dataprep.preparation.store.PreparationRepository) ActionRegistry(org.talend.dataprep.transformation.pipeline.ActionRegistry) ActionDefinition(org.talend.dataprep.api.action.ActionDefinition) ActionForm(org.talend.dataprep.api.action.ActionForm)

Example 5 with ActionDefinition

use of org.talend.dataprep.api.action.ActionDefinition in project data-prep by Talend.

the class ActionMetadataValidationTest method checkScopeConsistency_should_pass.

@Test
public void checkScopeConsistency_should_pass() throws Exception {
    // given
    final Map<String, String> parameters = new HashMap<>();
    parameters.put("scope", "column");
    parameters.put("column_id", "0001");
    ActionDefinition actionMock = new ActionMetadataExtendingColumn();
    // when
    validator.checkScopeConsistency(actionMock, parameters);
// then : should not throw exception
}
Also used : HashMap(java.util.HashMap) ActionDefinition(org.talend.dataprep.api.action.ActionDefinition) Test(org.junit.Test) ServiceBaseTest(org.talend.ServiceBaseTest)

Aggregations

ActionDefinition (org.talend.dataprep.api.action.ActionDefinition)13 Test (org.junit.Test)7 Action (org.talend.dataprep.api.preparation.Action)4 RunnableAction (org.talend.dataprep.transformation.actions.common.RunnableAction)4 ActionRegistry (org.talend.dataprep.transformation.pipeline.ActionRegistry)4 HashMap (java.util.HashMap)3 ObjectMapper (com.fasterxml.jackson.databind.ObjectMapper)2 ArrayList (java.util.ArrayList)2 Collections (java.util.Collections)2 List (java.util.List)2 Collectors (java.util.stream.Collectors)2 StringUtils (org.apache.commons.lang.StringUtils)2 ServiceBaseTest (org.talend.ServiceBaseTest)2 TalendRuntimeException (org.talend.daikon.exception.TalendRuntimeException)2 ColumnMetadata (org.talend.dataprep.api.dataset.ColumnMetadata)2 Owner (org.talend.dataprep.api.share.Owner)2 Parameter (org.talend.dataprep.parameters.Parameter)2 PreparationRepository (org.talend.dataprep.preparation.store.PreparationRepository)2 Security (org.talend.dataprep.security.Security)2 AbstractMetadataBaseTest (org.talend.dataprep.transformation.actions.AbstractMetadataBaseTest)2