Search in sources :

Example 21 with Stream

use of java.util.stream.Stream in project neo4j by neo4j.

the class BuiltInProcedures method listConstraints.

@Description("List all constraints in the database.")
@Procedure(name = "db.constraints", mode = READ)
public Stream<ConstraintResult> listConstraints() {
    Statement statement = tx.acquireStatement();
    ReadOperations operations = statement.readOperations();
    TokenNameLookup tokens = new StatementTokenNameLookup(operations);
    return asList(operations.constraintsGetAll()).stream().map((constraint) -> constraint.prettyPrint(tokens)).sorted().map(ConstraintResult::new).onClose(statement::close);
}
Also used : IndexingService(org.neo4j.kernel.impl.api.index.IndexingService) ProcedureException(org.neo4j.kernel.api.exceptions.ProcedureException) Label(org.neo4j.graphdb.Label) Context(org.neo4j.procedure.Context) Status(org.neo4j.kernel.api.exceptions.Status) Iterators.asSet(org.neo4j.helpers.collection.Iterators.asSet) StatementTokenNameLookup(org.neo4j.kernel.api.StatementTokenNameLookup) Statement(org.neo4j.kernel.api.Statement) IndexNotFoundKernelException(org.neo4j.kernel.api.exceptions.index.IndexNotFoundKernelException) ArrayList(java.util.ArrayList) TokenNameLookup(org.neo4j.kernel.api.TokenNameLookup) Procedure(org.neo4j.procedure.Procedure) TokenAccess(org.neo4j.kernel.impl.api.TokenAccess) ReadOperations(org.neo4j.kernel.api.ReadOperations) Set(java.util.Set) READ(org.neo4j.procedure.Mode.READ) Description(org.neo4j.procedure.Description) NewIndexDescriptor(org.neo4j.kernel.api.schema_new.index.NewIndexDescriptor) GraphDatabaseAPI(org.neo4j.kernel.internal.GraphDatabaseAPI) TimeUnit(java.util.concurrent.TimeUnit) List(java.util.List) Stream(java.util.stream.Stream) KernelTransaction(org.neo4j.kernel.api.KernelTransaction) Iterators.asList(org.neo4j.helpers.collection.Iterators.asList) DependencyResolver(org.neo4j.graphdb.DependencyResolver) Name(org.neo4j.procedure.Name) RelationshipType(org.neo4j.graphdb.RelationshipType) Comparator(java.util.Comparator) ReadOperations(org.neo4j.kernel.api.ReadOperations) StatementTokenNameLookup(org.neo4j.kernel.api.StatementTokenNameLookup) TokenNameLookup(org.neo4j.kernel.api.TokenNameLookup) StatementTokenNameLookup(org.neo4j.kernel.api.StatementTokenNameLookup) Statement(org.neo4j.kernel.api.Statement) Description(org.neo4j.procedure.Description) Procedure(org.neo4j.procedure.Procedure)

Example 22 with Stream

use of java.util.stream.Stream in project neo4j by neo4j.

the class ProcedureIT method shouldBeAbleToUseCallYieldWithPeriodicCommit.

@Test
public void shouldBeAbleToUseCallYieldWithPeriodicCommit() throws IOException {
    // GIVEN
    String[] lines = IntStream.rangeClosed(1, 100).boxed().map(i -> Integer.toString(i)).toArray(String[]::new);
    String url = createCsvFile(lines);
    //WHEN
    Result result = db.execute("USING PERIODIC COMMIT 1 " + "LOAD CSV FROM '" + url + "' AS line " + "CALL org.neo4j.procedure.createNode(line[0]) YIELD node as n " + "RETURN n.prop");
    // THEN
    for (int i = 1; i <= 100; i++) {
        assertThat(result.next().get("n.prop"), equalTo(Integer.toString(i)));
    }
    //Make sure all the lines has been properly commited to the database.
    String[] dbContents = db.execute("MATCH (n) return n.prop").stream().map(m -> (String) m.get("n.prop")).toArray(String[]::new);
    assertThat(dbContents, equalTo(lines));
}
Also used : Arrays(java.util.Arrays) AnonymousContext(org.neo4j.kernel.api.security.AnonymousContext) Log(org.neo4j.logging.Log) SCHEMA(org.neo4j.procedure.Mode.SCHEMA) Iterators(org.neo4j.helpers.collection.Iterators) IsEqual.equalTo(org.hamcrest.core.IsEqual.equalTo) TransactionFailureException(org.neo4j.graphdb.TransactionFailureException) TestGraphDatabaseFactory(org.neo4j.test.TestGraphDatabaseFactory) Collections.singletonList(java.util.Collections.singletonList) AssertableLogProvider(org.neo4j.logging.AssertableLogProvider) After(org.junit.After) Map(java.util.Map) StringMatcherIgnoresNewlines.containsStringIgnoreNewlines(org.neo4j.procedure.StringMatcherIgnoresNewlines.containsStringIgnoreNewlines) Transaction(org.neo4j.graphdb.Transaction) QueryExecutionException(org.neo4j.graphdb.QueryExecutionException) PrintWriter(java.io.PrintWriter) Result(org.neo4j.graphdb.Result) FileUtils(org.neo4j.io.fs.FileUtils) AssertableLogProvider.inLog(org.neo4j.logging.AssertableLogProvider.inLog) Collectors(java.util.stream.Collectors) StandardCharsets(java.nio.charset.StandardCharsets) Executors(java.util.concurrent.Executors) GraphDatabaseAPI(org.neo4j.kernel.internal.GraphDatabaseAPI) Path(org.neo4j.graphdb.Path) List(java.util.List) Stream(java.util.stream.Stream) JarBuilder(org.neo4j.kernel.impl.proc.JarBuilder) Assert.assertFalse(org.junit.Assert.assertFalse) RelationshipType(org.neo4j.graphdb.RelationshipType) MapUtil.map(org.neo4j.helpers.collection.MapUtil.map) GraphDatabaseSettings(org.neo4j.graphdb.factory.GraphDatabaseSettings) GraphDatabaseSettings.procedure_unrestricted(org.neo4j.graphdb.factory.GraphDatabaseSettings.procedure_unrestricted) IntStream(java.util.stream.IntStream) Label(org.neo4j.graphdb.Label) Node(org.neo4j.graphdb.Node) ArrayList(java.util.ArrayList) WRITE(org.neo4j.procedure.Mode.WRITE) GraphDatabaseService(org.neo4j.graphdb.GraphDatabaseService) ScheduledExecutorService(java.util.concurrent.ScheduledExecutorService) MatcherAssert.assertThat(org.hamcrest.MatcherAssert.assertThat) ExpectedException(org.junit.rules.ExpectedException) Before(org.junit.Before) Assert.assertTrue(org.junit.Assert.assertTrue) Test(org.junit.Test) IOException(java.io.IOException) Label.label(org.neo4j.graphdb.Label.label) Iterables.asList(org.neo4j.helpers.collection.Iterables.asList) File(java.io.File) ExecutionException(java.util.concurrent.ExecutionException) Procedures(org.neo4j.kernel.impl.proc.Procedures) Rule(org.junit.Rule) Relationship(org.neo4j.graphdb.Relationship) AuthorizationViolationException(org.neo4j.graphdb.security.AuthorizationViolationException) KernelTransaction(org.neo4j.kernel.api.KernelTransaction) GraphDatabaseSettings.plugin_dir(org.neo4j.graphdb.factory.GraphDatabaseSettings.plugin_dir) Collections(java.util.Collections) TemporaryFolder(org.junit.rules.TemporaryFolder) Assert.assertEquals(org.junit.Assert.assertEquals) Result(org.neo4j.graphdb.Result) Test(org.junit.Test)

Example 23 with Stream

use of java.util.stream.Stream in project traffic-engine by opentraffic.

the class VehicleStates method processLocationUpdates.

public void processLocationUpdates() {
    Map<Fun.Tuple2<Integer, Integer>, AtomicInteger> sortedMap = sortByValue(tileCount);
    for (Fun.Tuple2<Integer, Integer> tile : sortedMap.keySet()) {
        if (sortedMap.get(tile).get() >= MINIMUM_VEHICLE_COUNT) {
            if (!osmData.isLoadingOSM() || osmData.osmAreas.containsKey(tile)) {
                Map<Long, Boolean> vehicles = tileVehicleMap.get(tile);
                for (Long vehicleId : vehicles.keySet()) {
                    Vehicle vehicle = getVehicle(vehicleId, false);
                    if (vehicle != null && vehicle.tryLock()) {
                        try {
                            long processedLocations = vehicle.processVehicle();
                            synchronized (lastEmptyVehicleUpdateMap) {
                                if (processedLocations == 0 || vehicle.queueSize.get() == 0) {
                                    if (!lastEmptyVehicleUpdateMap.containsKey(vehicleId)) {
                                        lastEmptyVehicleUpdateMap.put(vehicleId, System.currentTimeMillis());
                                    }
                                } else {
                                    lastEmptyVehicleUpdateMap.remove(vehicleId);
                                }
                            }
                            updateProcessingRate();
                        } finally {
                            vehicle.unlock();
                        }
                    }
                }
            }
        } else {
            Map<Long, Boolean> vehicles = tileVehicleMap.get(tile);
            for (Long vehicleId : vehicles.keySet()) {
                if (!lastEmptyVehicleUpdateMap.containsKey(vehicleId)) {
                    lastEmptyVehicleUpdateMap.put(vehicleId, System.currentTimeMillis());
                }
            }
        }
        updateProcessingRate();
    }
    synchronized (lastEmptyVehicleUpdateMap) {
        lastEmptyVehicleUpdateMap.keySet().stream().filter(vehicleId -> lastEmptyVehicleUpdateMap.containsKey(vehicleId)).forEach(vehicleId -> {
            long lastEmptyUpdate = lastEmptyVehicleUpdateMap.get(vehicleId);
            Vehicle vehicle = getVehicle(vehicleId, false);
            if (vehicle != null && System.currentTimeMillis() - lastEmptyUpdate > VEHICLE_INVALIDATION_TIME) {
                if (vehicle.tile != null && tileCount.containsKey(vehicle.tile) && tileCount.get(vehicle.tile).get() < MINIMUM_VEHICLE_COUNT) {
                    removeVehicle(vehicleId);
                } else if (vehicle.queueSize.get() == 0) {
                    removeVehicle(vehicleId);
                }
            }
        });
    }
}
Also used : AtomicInteger(java.util.concurrent.atomic.AtomicInteger) Fun(org.mapdb.Fun) AtomicLong(java.util.concurrent.atomic.AtomicLong) java.util(java.util) Stream(java.util.stream.Stream) GPSPoint(io.opentraffic.engine.geom.GPSPoint) AtomicInteger(java.util.concurrent.atomic.AtomicInteger) ConcurrentHashMap(java.util.concurrent.ConcurrentHashMap) Crossing(io.opentraffic.engine.geom.Crossing) OSMDataStore(io.opentraffic.engine.osm.OSMDataStore) GPSSegment(io.opentraffic.engine.geom.GPSSegment) TripLine(io.opentraffic.engine.geom.TripLine) AtomicInteger(java.util.concurrent.atomic.AtomicInteger) AtomicLong(java.util.concurrent.atomic.AtomicLong) Fun(org.mapdb.Fun)

Example 24 with Stream

use of java.util.stream.Stream in project presto by prestodb.

the class PlanPrinter method textDistributedPlan.

public static String textDistributedPlan(List<StageInfo> stages, Metadata metadata, Session session) {
    StringBuilder builder = new StringBuilder();
    List<StageInfo> allStages = stages.stream().flatMap(stage -> getAllStages(Optional.of(stage)).stream()).collect(toImmutableList());
    for (StageInfo stageInfo : allStages) {
        Map<PlanNodeId, PlanNodeStats> aggregatedStats = new HashMap<>();
        List<PlanNodeStats> planNodeStats = stageInfo.getTasks().stream().map(TaskInfo::getStats).flatMap(taskStats -> getPlanNodeStats(taskStats).stream()).collect(toList());
        for (PlanNodeStats stats : planNodeStats) {
            aggregatedStats.merge(stats.getPlanNodeId(), stats, PlanNodeStats::merge);
        }
        builder.append(formatFragment(metadata, session, stageInfo.getPlan(), Optional.of(stageInfo.getStageStats()), Optional.of(aggregatedStats)));
    }
    return builder.toString();
}
Also used : ComparisonExpressionType(com.facebook.presto.sql.tree.ComparisonExpressionType) Arrays(java.util.Arrays) FrameBound(com.facebook.presto.sql.tree.FrameBound) DistinctLimitNode(com.facebook.presto.sql.planner.plan.DistinctLimitNode) OperatorStats(com.facebook.presto.operator.OperatorStats) StageInfo(com.facebook.presto.execution.StageInfo) MarkDistinctNode(com.facebook.presto.sql.planner.plan.MarkDistinctNode) Map(java.util.Map) IndexJoinNode(com.facebook.presto.sql.planner.plan.IndexJoinNode) ImmutableCollectors.toImmutableList(com.facebook.presto.util.ImmutableCollectors.toImmutableList) Double.max(java.lang.Double.max) StageInfo.getAllStages(com.facebook.presto.execution.StageInfo.getAllStages) StageStats(com.facebook.presto.execution.StageStats) RowNumberNode(com.facebook.presto.sql.planner.plan.RowNumberNode) Set(java.util.Set) VARCHAR(com.facebook.presto.spi.type.VarcharType.VARCHAR) MILLISECONDS(java.util.concurrent.TimeUnit.MILLISECONDS) ExceptNode(com.facebook.presto.sql.planner.plan.ExceptNode) Domain(com.facebook.presto.spi.predicate.Domain) Stream(java.util.stream.Stream) OperatorNotFoundException(com.facebook.presto.metadata.OperatorNotFoundException) ApplyNode(com.facebook.presto.sql.planner.plan.ApplyNode) AssignUniqueId(com.facebook.presto.sql.planner.plan.AssignUniqueId) LimitNode(com.facebook.presto.sql.planner.plan.LimitNode) Lists.reverse(com.google.common.collect.Lists.reverse) Joiner(com.google.common.base.Joiner) Iterables(com.google.common.collect.Iterables) SINGLE_DISTRIBUTION(com.facebook.presto.sql.planner.SystemPartitioningHandle.SINGLE_DISTRIBUTION) Slice(io.airlift.slice.Slice) GroupIdNode(com.facebook.presto.sql.planner.plan.GroupIdNode) DomainUtils.simplifyDomain(com.facebook.presto.sql.planner.DomainUtils.simplifyDomain) DataSize.succinctBytes(io.airlift.units.DataSize.succinctBytes) ArrayList(java.util.ArrayList) TableScanNode(com.facebook.presto.sql.planner.plan.TableScanNode) Strings(com.google.common.base.Strings) Lists(com.google.common.collect.Lists) Type(com.facebook.presto.spi.type.Type) PlanNode(com.facebook.presto.sql.planner.plan.PlanNode) PlanFragmentId(com.facebook.presto.sql.planner.plan.PlanFragmentId) PlanNodeId(com.facebook.presto.sql.planner.plan.PlanNodeId) Functions(com.google.common.base.Functions) PipelineStats(com.facebook.presto.operator.PipelineStats) WindowNode(com.facebook.presto.sql.planner.plan.WindowNode) Session(com.facebook.presto.Session) Throwables(com.google.common.base.Throwables) Iterables.getOnlyElement(com.google.common.collect.Iterables.getOnlyElement) Range(com.facebook.presto.spi.predicate.Range) Math.sqrt(java.lang.Math.sqrt) ColumnHandle(com.facebook.presto.spi.ColumnHandle) FunctionInvoker(com.facebook.presto.sql.FunctionInvoker) SemiJoinNode(com.facebook.presto.sql.planner.plan.SemiJoinNode) FilterNode(com.facebook.presto.sql.planner.plan.FilterNode) TaskInfo(com.facebook.presto.execution.TaskInfo) Metadata(com.facebook.presto.metadata.Metadata) TaskStats(com.facebook.presto.operator.TaskStats) ExplainAnalyzeNode(com.facebook.presto.sql.planner.plan.ExplainAnalyzeNode) RemoteSourceNode(com.facebook.presto.sql.planner.plan.RemoteSourceNode) SortNode(com.facebook.presto.sql.planner.plan.SortNode) MetadataDeleteNode(com.facebook.presto.sql.planner.plan.MetadataDeleteNode) TableWriterNode(com.facebook.presto.sql.planner.plan.TableWriterNode) ValuesNode(com.facebook.presto.sql.planner.plan.ValuesNode) Duration(io.airlift.units.Duration) GraphvizPrinter(com.facebook.presto.util.GraphvizPrinter) Preconditions.checkArgument(com.google.common.base.Preconditions.checkArgument) Collectors.toMap(java.util.stream.Collectors.toMap) Locale(java.util.Locale) TopNRowNumberNode(com.facebook.presto.sql.planner.plan.TopNRowNumberNode) IndexSourceNode(com.facebook.presto.sql.planner.plan.IndexSourceNode) TopNNode(com.facebook.presto.sql.planner.plan.TopNNode) Assignments(com.facebook.presto.sql.planner.plan.Assignments) NullableValue(com.facebook.presto.spi.predicate.NullableValue) DataSize.succinctDataSize(io.airlift.units.DataSize.succinctDataSize) FunctionCall(com.facebook.presto.sql.tree.FunctionCall) OutputNode(com.facebook.presto.sql.planner.plan.OutputNode) SymbolReference(com.facebook.presto.sql.tree.SymbolReference) ImmutableSet(com.google.common.collect.ImmutableSet) Scope(com.facebook.presto.sql.planner.plan.ExchangeNode.Scope) ImmutableMap(com.google.common.collect.ImmutableMap) Collection(java.util.Collection) Iterables.getLast(com.google.common.collect.Iterables.getLast) Marker(com.facebook.presto.spi.predicate.Marker) Collectors(java.util.stream.Collectors) BinaryOperator(java.util.function.BinaryOperator) String.format(java.lang.String.format) Preconditions.checkState(com.google.common.base.Preconditions.checkState) TupleDomain(com.facebook.presto.spi.predicate.TupleDomain) DataSize(io.airlift.units.DataSize) List(java.util.List) Window(com.facebook.presto.sql.tree.Window) SampleNode(com.facebook.presto.sql.planner.plan.SampleNode) Optional(java.util.Optional) ConnectorTableLayoutHandle(com.facebook.presto.spi.ConnectorTableLayoutHandle) HashMap(java.util.HashMap) HashSet(java.util.HashSet) WindowFrame(com.facebook.presto.sql.tree.WindowFrame) PlanVisitor(com.facebook.presto.sql.planner.plan.PlanVisitor) ImmutableList(com.google.common.collect.ImmutableList) Objects.requireNonNull(java.util.Objects.requireNonNull) Double.isFinite(java.lang.Double.isFinite) LinkedList(java.util.LinkedList) AggregationNode(com.facebook.presto.sql.planner.plan.AggregationNode) UnnestNode(com.facebook.presto.sql.planner.plan.UnnestNode) JoinNode(com.facebook.presto.sql.planner.plan.JoinNode) Collections.emptyMap(java.util.Collections.emptyMap) TableLayout(com.facebook.presto.metadata.TableLayout) CaseFormat(com.google.common.base.CaseFormat) Signature(com.facebook.presto.metadata.Signature) HashCollisionsInfo(com.facebook.presto.operator.HashCollisionsInfo) ComparisonExpression(com.facebook.presto.sql.tree.ComparisonExpression) DeleteNode(com.facebook.presto.sql.planner.plan.DeleteNode) ProjectNode(com.facebook.presto.sql.planner.plan.ProjectNode) UPPER_UNDERSCORE(com.google.common.base.CaseFormat.UPPER_UNDERSCORE) Collectors.toList(java.util.stream.Collectors.toList) UnionNode(com.facebook.presto.sql.planner.plan.UnionNode) Expression(com.facebook.presto.sql.tree.Expression) TableHandle(com.facebook.presto.metadata.TableHandle) TableFinishNode(com.facebook.presto.sql.planner.plan.TableFinishNode) BYTE(io.airlift.units.DataSize.Unit.BYTE) IntersectNode(com.facebook.presto.sql.planner.plan.IntersectNode) EnforceSingleRowNode(com.facebook.presto.sql.planner.plan.EnforceSingleRowNode) ExchangeNode(com.facebook.presto.sql.planner.plan.ExchangeNode) PlanNodeId(com.facebook.presto.sql.planner.plan.PlanNodeId) TaskInfo(com.facebook.presto.execution.TaskInfo) HashMap(java.util.HashMap) StageInfo(com.facebook.presto.execution.StageInfo)

Example 25 with Stream

use of java.util.stream.Stream in project ratpack by ratpack.

the class AbstractPropertiesConfigSource method loadConfigData.

@Override
public ObjectNode loadConfigData(ObjectMapper objectMapper, FileSystemBinding fileSystemBinding) throws Exception {
    ObjectNode rootNode = objectMapper.createObjectNode();
    Properties properties = loadProperties();
    Stream<Pair<String, String>> pairs = properties.stringPropertyNames().stream().map(key -> Pair.of(key, properties.getProperty(key)));
    if (prefix.isPresent()) {
        pairs = pairs.filter(p -> p.left.startsWith(prefix.get())).map(((Function<Pair<String, String>, Pair<String, String>>) p -> p.mapLeft(s -> s.substring(prefix.get().length()))).toFunction());
    }
    pairs.forEach(p -> populate(rootNode, p.left, p.right));
    return rootNode;
}
Also used : ArrayNode(com.fasterxml.jackson.databind.node.ArrayNode) Function(ratpack.func.Function) Properties(java.util.Properties) Stream(java.util.stream.Stream) FileSystemBinding(ratpack.file.FileSystemBinding) ConfigSource(ratpack.config.ConfigSource) ObjectMapper(com.fasterxml.jackson.databind.ObjectMapper) Optional(java.util.Optional) Pair(ratpack.func.Pair) ObjectNode(com.fasterxml.jackson.databind.node.ObjectNode) TextNode(com.fasterxml.jackson.databind.node.TextNode) Function(ratpack.func.Function) ObjectNode(com.fasterxml.jackson.databind.node.ObjectNode) Properties(java.util.Properties) Pair(ratpack.func.Pair)

Aggregations

Stream (java.util.stream.Stream)161 Collectors (java.util.stream.Collectors)98 List (java.util.List)89 ArrayList (java.util.ArrayList)66 Map (java.util.Map)66 Set (java.util.Set)59 IOException (java.io.IOException)58 Optional (java.util.Optional)45 Collections (java.util.Collections)43 HashMap (java.util.HashMap)43 Arrays (java.util.Arrays)33 HashSet (java.util.HashSet)33 File (java.io.File)32 Path (java.nio.file.Path)32 Function (java.util.function.Function)28 Logger (org.slf4j.Logger)26 LoggerFactory (org.slf4j.LoggerFactory)26 java.util (java.util)25 Predicate (java.util.function.Predicate)23 Objects (java.util.Objects)22