Search in sources :

Example 31 with Collections

use of java.util.Collections in project component-runtime by Talend.

the class ConfigurationMapper method map.

private Map<String, String> map(final List<ParameterMeta> nestedParameters, final Object instance, final Map<Integer, Integer> indexes) {
    if (nestedParameters == null) {
        return emptyMap();
    }
    return nestedParameters.stream().map(param -> {
        final Object value = getValue(instance, param.getName());
        if (value == null) {
            return Collections.<String, String>emptyMap();
        }
        switch(param.getType()) {
            case OBJECT:
                return map(param.getNestedParameters(), value, indexes);
            case ARRAY:
                final Collection<Object> values = Collection.class.isInstance(value) ? Collection.class.cast(value) : /* array */
                asList(Object[].class.cast(value));
                final int arrayIndex = indexes.keySet().size();
                final AtomicInteger valuesIndex = new AtomicInteger(0);
                final Map<String, String> config = values.stream().map((Object item) -> {
                    indexes.put(arrayIndex, valuesIndex.getAndIncrement());
                    final Map<String, String> res = param.getNestedParameters().stream().filter(ConfigurationMapper::isPrimitive).collect(toMap(p -> evaluateIndexes(p.getPath(), indexes), p -> getValue(item, p.getName()).toString()));
                    res.putAll(map(param.getNestedParameters().stream().filter(p -> !isPrimitive(p)).collect(toList()), item, indexes));
                    return res;
                }).flatMap(m -> m.entrySet().stream()).collect(toMap(Map.Entry::getKey, Map.Entry::getValue));
                // clear index after the end of array handling
                indexes.clear();
                return config;
            default:
                // primitives
                return singletonMap(evaluateIndexes(param.getPath(), indexes), value.toString());
        }
    }).flatMap(m -> m.entrySet().stream()).collect(toMap(Map.Entry::getKey, Map.Entry::getValue));
}
Also used : Collections.emptyMap(java.util.Collections.emptyMap) Collection(java.util.Collection) HashMap(java.util.HashMap) Field(java.lang.reflect.Field) Collectors.toList(java.util.stream.Collectors.toList) List(java.util.List) Stream(java.util.stream.Stream) Collectors.toMap(java.util.stream.Collectors.toMap) AtomicInteger(java.util.concurrent.atomic.AtomicInteger) Arrays.asList(java.util.Arrays.asList) Map(java.util.Map) Collections.singletonMap(java.util.Collections.singletonMap) Collections(java.util.Collections) ParameterMeta(org.talend.sdk.component.runtime.manager.ParameterMeta) AtomicInteger(java.util.concurrent.atomic.AtomicInteger) Collection(java.util.Collection) Collections.emptyMap(java.util.Collections.emptyMap) HashMap(java.util.HashMap) Collectors.toMap(java.util.stream.Collectors.toMap) Map(java.util.Map) Collections.singletonMap(java.util.Collections.singletonMap)

Example 32 with Collections

use of java.util.Collections in project neo4j by neo4j.

the class JoltNodeSerializer method serialize.

@Override
public void serialize(Node node, JsonGenerator generator, SerializerProvider provider) throws IOException {
    generator.writeStartObject(node);
    generator.writeFieldName(Sigil.NODE.getValue());
    generator.writeStartArray();
    generator.writeNumber(node.getId());
    generator.writeStartArray();
    for (Label label : node.getLabels()) {
        generator.writeString(label.name());
    }
    generator.writeEndArray();
    var properties = Optional.ofNullable(node.getAllProperties()).orElseGet(Collections::emptyMap);
    generator.writeStartObject();
    for (var entry : properties.entrySet()) {
        generator.writeFieldName(entry.getKey());
        generator.writeObject(entry.getValue());
    }
    generator.writeEndObject();
    generator.writeEndArray();
    generator.writeEndObject();
}
Also used : Label(org.neo4j.graphdb.Label) Collections(java.util.Collections)

Example 33 with Collections

use of java.util.Collections in project flink by apache.

the class TaskAsyncCallTest method createTask.

private Task createTask(Class<? extends AbstractInvokable> invokableClass) throws Exception {
    final TestingClassLoaderLease classLoaderHandle = TestingClassLoaderLease.newBuilder().setGetOrResolveClassLoaderFunction((permanentBlobKeys, urls) -> TestingUserCodeClassLoader.newBuilder().setClassLoader(new TestUserCodeClassLoader()).build()).build();
    ResultPartitionConsumableNotifier consumableNotifier = new NoOpResultPartitionConsumableNotifier();
    PartitionProducerStateChecker partitionProducerStateChecker = mock(PartitionProducerStateChecker.class);
    Executor executor = mock(Executor.class);
    TaskMetricGroup taskMetricGroup = UnregisteredMetricGroups.createUnregisteredTaskMetricGroup();
    JobInformation jobInformation = new JobInformation(new JobID(), "Job Name", new SerializedValue<>(new ExecutionConfig()), new Configuration(), Collections.emptyList(), Collections.emptyList());
    TaskInformation taskInformation = new TaskInformation(new JobVertexID(), "Test Task", 1, 1, invokableClass.getName(), new Configuration());
    return new Task(jobInformation, taskInformation, new ExecutionAttemptID(), new AllocationID(), 0, 0, Collections.<ResultPartitionDeploymentDescriptor>emptyList(), Collections.<InputGateDeploymentDescriptor>emptyList(), mock(MemoryManager.class), mock(IOManager.class), shuffleEnvironment, new KvStateService(new KvStateRegistry(), null, null), mock(BroadcastVariableManager.class), new TaskEventDispatcher(), ExternalResourceInfoProvider.NO_EXTERNAL_RESOURCES, new TestTaskStateManager(), mock(TaskManagerActions.class), mock(InputSplitProvider.class), mock(CheckpointResponder.class), new NoOpTaskOperatorEventGateway(), new TestGlobalAggregateManager(), classLoaderHandle, mock(FileCache.class), new TestingTaskManagerRuntimeInfo(), taskMetricGroup, consumableNotifier, partitionProducerStateChecker, executor);
}
Also used : CheckpointMetricsBuilder(org.apache.flink.runtime.checkpoint.CheckpointMetricsBuilder) ShuffleEnvironment(org.apache.flink.runtime.shuffle.ShuffleEnvironment) IOManager(org.apache.flink.runtime.io.disk.iomanager.IOManager) InputSplitProvider(org.apache.flink.runtime.jobgraph.tasks.InputSplitProvider) Assert.assertThat(org.junit.Assert.assertThat) TestTaskStateManager(org.apache.flink.runtime.state.TestTaskStateManager) Future(java.util.concurrent.Future) CheckpointException(org.apache.flink.runtime.checkpoint.CheckpointException) JobInformation(org.apache.flink.runtime.executiongraph.JobInformation) KvStateService(org.apache.flink.runtime.taskexecutor.KvStateService) After(org.junit.After) TestGlobalAggregateManager(org.apache.flink.runtime.taskexecutor.TestGlobalAggregateManager) TestLogger(org.apache.flink.util.TestLogger) TaskEventDispatcher(org.apache.flink.runtime.io.network.TaskEventDispatcher) ResultPartitionConsumableNotifier(org.apache.flink.runtime.io.network.partition.ResultPartitionConsumableNotifier) Matchers.isOneOf(org.hamcrest.Matchers.isOneOf) AbstractInvokable(org.apache.flink.runtime.jobgraph.tasks.AbstractInvokable) KvStateRegistry(org.apache.flink.runtime.query.KvStateRegistry) CheckpointOptions(org.apache.flink.runtime.checkpoint.CheckpointOptions) BroadcastVariableManager(org.apache.flink.runtime.broadcast.BroadcastVariableManager) SerializedValue(org.apache.flink.util.SerializedValue) ResultPartitionDeploymentDescriptor(org.apache.flink.runtime.deployment.ResultPartitionDeploymentDescriptor) Assert.assertFalse(org.junit.Assert.assertFalse) ExecutionConfig(org.apache.flink.api.common.ExecutionConfig) AllocationID(org.apache.flink.runtime.clusterframework.types.AllocationID) InputGateDeploymentDescriptor(org.apache.flink.runtime.deployment.InputGateDeploymentDescriptor) Environment(org.apache.flink.runtime.execution.Environment) Mockito.mock(org.mockito.Mockito.mock) OneShotLatch(org.apache.flink.core.testutils.OneShotLatch) MemoryManager(org.apache.flink.runtime.memory.MemoryManager) CheckpointMetaData(org.apache.flink.runtime.checkpoint.CheckpointMetaData) CompletableFuture(java.util.concurrent.CompletableFuture) JobVertexID(org.apache.flink.runtime.jobgraph.JobVertexID) TestingTaskManagerRuntimeInfo(org.apache.flink.runtime.util.TestingTaskManagerRuntimeInfo) TaskMetricGroup(org.apache.flink.runtime.metrics.groups.TaskMetricGroup) TestingClassLoaderLease(org.apache.flink.runtime.execution.librarycache.TestingClassLoaderLease) ExternalResourceInfoProvider(org.apache.flink.runtime.externalresource.ExternalResourceInfoProvider) Before(org.junit.Before) NettyShuffleEnvironmentBuilder(org.apache.flink.runtime.io.network.NettyShuffleEnvironmentBuilder) Executor(java.util.concurrent.Executor) Configuration(org.apache.flink.configuration.Configuration) ExecutionState(org.apache.flink.runtime.execution.ExecutionState) FileCache(org.apache.flink.runtime.filecache.FileCache) Test(org.junit.Test) PartitionProducerStateChecker(org.apache.flink.runtime.taskexecutor.PartitionProducerStateChecker) ExecutionAttemptID(org.apache.flink.runtime.executiongraph.ExecutionAttemptID) JobID(org.apache.flink.api.common.JobID) NoOpResultPartitionConsumableNotifier(org.apache.flink.runtime.io.network.partition.NoOpResultPartitionConsumableNotifier) UnregisteredMetricGroups(org.apache.flink.runtime.metrics.groups.UnregisteredMetricGroups) TestingUserCodeClassLoader(org.apache.flink.runtime.util.TestingUserCodeClassLoader) TaskInformation(org.apache.flink.runtime.executiongraph.TaskInformation) Collections(java.util.Collections) KvStateRegistry(org.apache.flink.runtime.query.KvStateRegistry) Configuration(org.apache.flink.configuration.Configuration) JobVertexID(org.apache.flink.runtime.jobgraph.JobVertexID) ExecutionConfig(org.apache.flink.api.common.ExecutionConfig) KvStateService(org.apache.flink.runtime.taskexecutor.KvStateService) Executor(java.util.concurrent.Executor) TestingTaskManagerRuntimeInfo(org.apache.flink.runtime.util.TestingTaskManagerRuntimeInfo) BroadcastVariableManager(org.apache.flink.runtime.broadcast.BroadcastVariableManager) PartitionProducerStateChecker(org.apache.flink.runtime.taskexecutor.PartitionProducerStateChecker) ResultPartitionConsumableNotifier(org.apache.flink.runtime.io.network.partition.ResultPartitionConsumableNotifier) NoOpResultPartitionConsumableNotifier(org.apache.flink.runtime.io.network.partition.NoOpResultPartitionConsumableNotifier) InputSplitProvider(org.apache.flink.runtime.jobgraph.tasks.InputSplitProvider) JobInformation(org.apache.flink.runtime.executiongraph.JobInformation) TaskInformation(org.apache.flink.runtime.executiongraph.TaskInformation) ExecutionAttemptID(org.apache.flink.runtime.executiongraph.ExecutionAttemptID) IOManager(org.apache.flink.runtime.io.disk.iomanager.IOManager) TaskMetricGroup(org.apache.flink.runtime.metrics.groups.TaskMetricGroup) AllocationID(org.apache.flink.runtime.clusterframework.types.AllocationID) NoOpResultPartitionConsumableNotifier(org.apache.flink.runtime.io.network.partition.NoOpResultPartitionConsumableNotifier) MemoryManager(org.apache.flink.runtime.memory.MemoryManager) TestGlobalAggregateManager(org.apache.flink.runtime.taskexecutor.TestGlobalAggregateManager) FileCache(org.apache.flink.runtime.filecache.FileCache) TestTaskStateManager(org.apache.flink.runtime.state.TestTaskStateManager) TestingClassLoaderLease(org.apache.flink.runtime.execution.librarycache.TestingClassLoaderLease) TaskEventDispatcher(org.apache.flink.runtime.io.network.TaskEventDispatcher) JobID(org.apache.flink.api.common.JobID)

Example 34 with Collections

use of java.util.Collections in project flink by apache.

the class ExecutingTest method testFalseReportsViaUpdateTaskExecutionStateAreIgnored.

@Test
public void testFalseReportsViaUpdateTaskExecutionStateAreIgnored() throws Exception {
    try (MockExecutingContext ctx = new MockExecutingContext()) {
        MockExecutionGraph returnsFailedStateExecutionGraph = new MockExecutionGraph(false, Collections::emptyList);
        Executing exec = new ExecutingStateBuilder().setExecutionGraph(returnsFailedStateExecutionGraph).build(ctx);
        Exception exception = new RuntimeException();
        TestingAccessExecution execution = TestingAccessExecution.newBuilder().withExecutionState(ExecutionState.FAILED).withErrorInfo(new ErrorInfo(exception, System.currentTimeMillis())).build();
        returnsFailedStateExecutionGraph.registerExecution(execution);
        TaskExecutionStateTransition taskExecutionStateTransition = createFailingStateTransition(execution.getAttemptId(), exception);
        exec.updateTaskExecutionState(taskExecutionStateTransition);
        ctx.assertNoStateTransition();
    }
}
Also used : TestingAccessExecution(org.apache.flink.runtime.scheduler.exceptionhistory.TestingAccessExecution) TaskExecutionStateTransition(org.apache.flink.runtime.executiongraph.TaskExecutionStateTransition) ErrorInfo(org.apache.flink.runtime.executiongraph.ErrorInfo) Collections(java.util.Collections) JobException(org.apache.flink.runtime.JobException) JobExecutionException(org.apache.flink.runtime.client.JobExecutionException) Test(org.junit.Test)

Example 35 with Collections

use of java.util.Collections in project druid by druid-io.

the class LookupSegmentWrangler method getSegmentsForIntervals.

@Override
public Iterable<Segment> getSegmentsForIntervals(final DataSource dataSource, final Iterable<Interval> intervals) {
    final LookupDataSource lookupDataSource = (LookupDataSource) dataSource;
    final Optional<LookupExtractorFactoryContainer> maybeContainer = lookupProvider.get(lookupDataSource.getLookupName());
    return maybeContainer.map(container -> Collections.<Segment>singletonList(new LookupSegment(lookupDataSource.getLookupName(), container.getLookupExtractorFactory()))).orElse(Collections.emptyList());
}
Also used : Interval(org.joda.time.Interval) LookupDataSource(org.apache.druid.query.LookupDataSource) Inject(com.google.inject.Inject) DataSource(org.apache.druid.query.DataSource) LookupExtractorFactoryContainerProvider(org.apache.druid.query.lookup.LookupExtractorFactoryContainerProvider) Optional(java.util.Optional) JoinableFactory(org.apache.druid.segment.join.JoinableFactory) Collections(java.util.Collections) LookupSegment(org.apache.druid.query.lookup.LookupSegment) LookupExtractorFactoryContainer(org.apache.druid.query.lookup.LookupExtractorFactoryContainer) LookupDataSource(org.apache.druid.query.LookupDataSource) LookupSegment(org.apache.druid.query.lookup.LookupSegment) LookupExtractorFactoryContainer(org.apache.druid.query.lookup.LookupExtractorFactoryContainer) LookupSegment(org.apache.druid.query.lookup.LookupSegment)

Aggregations

Collections (java.util.Collections)116 List (java.util.List)60 ArrayList (java.util.ArrayList)41 Test (org.junit.Test)39 Map (java.util.Map)38 Collectors (java.util.stream.Collectors)35 Arrays (java.util.Arrays)28 HashMap (java.util.HashMap)27 Set (java.util.Set)25 HashSet (java.util.HashSet)23 IOException (java.io.IOException)19 Collection (java.util.Collection)19 Optional (java.util.Optional)19 TimeUnit (java.util.concurrent.TimeUnit)16 URI (java.net.URI)13 Assert (org.junit.Assert)13 Function (java.util.function.Function)12 Stream (java.util.stream.Stream)12 Before (org.junit.Before)12 Logger (org.slf4j.Logger)12