Search in sources :

Example 1 with DynamicScoreVisitor

use of com.amazon.randomcutforest.anomalydetection.DynamicScoreVisitor in project random-cut-forest-by-aws by aws.

the class RandomCutForest method getDynamicScore.

/**
 * Score a point using the given scoring functions.
 *
 * @param point                   input point being scored
 * @param ignoreLeafMassThreshold said threshold
 * @param seen                    the function that applies if input is equal to
 *                                a previously seen sample in a leaf
 * @param unseen                  if the input does not have a match in the
 *                                leaves
 * @param damp                    damping function based on the duplicity of the
 *                                previously seen samples
 * @return anomaly score
 */
public double getDynamicScore(float[] point, int ignoreLeafMassThreshold, BiFunction<Double, Double, Double> seen, BiFunction<Double, Double, Double> unseen, BiFunction<Double, Double, Double> damp) {
    checkArgument(ignoreLeafMassThreshold >= 0, "ignoreLeafMassThreshold should be greater than or equal to 0");
    if (!isOutputReady()) {
        return 0.0;
    }
    VisitorFactory<Double> visitorFactory = new VisitorFactory<>((tree, y) -> new DynamicScoreVisitor(tree.projectToTree(y), tree.getMass(), ignoreLeafMassThreshold, seen, unseen, damp));
    BinaryOperator<Double> accumulator = Double::sum;
    Function<Double, Double> finisher = sum -> sum / numberOfTrees;
    return traverseForest(transformToShingledPoint(point), visitorFactory, accumulator, finisher);
}
Also used : CommonUtils.checkNotNull(com.amazon.randomcutforest.CommonUtils.checkNotNull) Arrays(java.util.Arrays) BiFunction(java.util.function.BiFunction) ParallelForestTraversalExecutor(com.amazon.randomcutforest.executor.ParallelForestTraversalExecutor) Random(java.util.Random) ParallelForestUpdateExecutor(com.amazon.randomcutforest.executor.ParallelForestUpdateExecutor) AbstractForestUpdateExecutor(com.amazon.randomcutforest.executor.AbstractForestUpdateExecutor) IStateCoordinator(com.amazon.randomcutforest.executor.IStateCoordinator) RandomCutTree(com.amazon.randomcutforest.tree.RandomCutTree) CommonUtils.toFloatArray(com.amazon.randomcutforest.CommonUtils.toFloatArray) Neighbor(com.amazon.randomcutforest.returntypes.Neighbor) ConditionalSampleSummarizer(com.amazon.randomcutforest.imputation.ConditionalSampleSummarizer) ImputeVisitor(com.amazon.randomcutforest.imputation.ImputeVisitor) NearNeighborVisitor(com.amazon.randomcutforest.inspect.NearNeighborVisitor) IBoundingBoxView(com.amazon.randomcutforest.tree.IBoundingBoxView) Collector(java.util.stream.Collector) PointStoreCoordinator(com.amazon.randomcutforest.executor.PointStoreCoordinator) AnomalyAttributionVisitor(com.amazon.randomcutforest.anomalydetection.AnomalyAttributionVisitor) OneSidedConvergingDoubleAccumulator(com.amazon.randomcutforest.returntypes.OneSidedConvergingDoubleAccumulator) AnomalyScoreVisitor(com.amazon.randomcutforest.anomalydetection.AnomalyScoreVisitor) AbstractForestTraversalExecutor(com.amazon.randomcutforest.executor.AbstractForestTraversalExecutor) BinaryOperator(java.util.function.BinaryOperator) SequentialForestTraversalExecutor(com.amazon.randomcutforest.executor.SequentialForestTraversalExecutor) List(java.util.List) Math.max(java.lang.Math.max) Optional(java.util.Optional) DensityOutput(com.amazon.randomcutforest.returntypes.DensityOutput) CommonUtils.toDoubleArray(com.amazon.randomcutforest.CommonUtils.toDoubleArray) Precision(com.amazon.randomcutforest.config.Precision) CompactSampler(com.amazon.randomcutforest.sampler.CompactSampler) SamplerPlusTree(com.amazon.randomcutforest.executor.SamplerPlusTree) ShingleBuilder(com.amazon.randomcutforest.util.ShingleBuilder) Function(java.util.function.Function) ArrayList(java.util.ArrayList) SimulatedTransductiveScalarScoreVisitor(com.amazon.randomcutforest.anomalydetection.SimulatedTransductiveScalarScoreVisitor) PointStore(com.amazon.randomcutforest.store.PointStore) DynamicAttributionVisitor(com.amazon.randomcutforest.anomalydetection.DynamicAttributionVisitor) ConvergingAccumulator(com.amazon.randomcutforest.returntypes.ConvergingAccumulator) Config(com.amazon.randomcutforest.config.Config) SimpleInterpolationVisitor(com.amazon.randomcutforest.interpolation.SimpleInterpolationVisitor) InterpolationMeasure(com.amazon.randomcutforest.returntypes.InterpolationMeasure) IPointStore(com.amazon.randomcutforest.store.IPointStore) SequentialForestUpdateExecutor(com.amazon.randomcutforest.executor.SequentialForestUpdateExecutor) ArrayUtils(com.amazon.randomcutforest.util.ArrayUtils) OneSidedConvergingDiVectorAccumulator(com.amazon.randomcutforest.returntypes.OneSidedConvergingDiVectorAccumulator) CommonUtils.checkArgument(com.amazon.randomcutforest.CommonUtils.checkArgument) DynamicScoreVisitor(com.amazon.randomcutforest.anomalydetection.DynamicScoreVisitor) DiVector(com.amazon.randomcutforest.returntypes.DiVector) ITree(com.amazon.randomcutforest.tree.ITree) ConditionalTreeSample(com.amazon.randomcutforest.returntypes.ConditionalTreeSample) ConditionalSampleSummary(com.amazon.randomcutforest.returntypes.ConditionalSampleSummary) Collections(java.util.Collections) IStreamSampler(com.amazon.randomcutforest.sampler.IStreamSampler) DynamicScoreVisitor(com.amazon.randomcutforest.anomalydetection.DynamicScoreVisitor)

Example 2 with DynamicScoreVisitor

use of com.amazon.randomcutforest.anomalydetection.DynamicScoreVisitor in project random-cut-forest-by-aws by aws.

the class RandomCutForest method getApproximateDynamicScore.

/**
 * Score a point using the given scoring functions. This method will
 * short-circuit before visiting all trees if the scores that are returned from
 * a subset of trees appears to be converging to a given value. See
 * {@link OneSidedConvergingDoubleAccumulator} for more about convergence.
 *
 * @param point                   input point
 * @param precision               controls early convergence
 * @param highIsCritical          this is true for the default scoring function.
 *                                If the user wishes to use a different scoring
 *                                function where anomaly scores are low values
 *                                (for example, height in tree) then this should
 *                                be set to false.
 * @param ignoreLeafMassThreshold said threshold
 * @param seen                    scoring function when the input matches some
 *                                tuple in the leaves
 * @param unseen                  scoring function when the input is not found
 * @param damp                    dampening function for duplicates which are
 *                                same as input (applies with seen)
 * @return the dynamic score under sequential early stopping
 */
public double getApproximateDynamicScore(float[] point, double precision, boolean highIsCritical, int ignoreLeafMassThreshold, BiFunction<Double, Double, Double> seen, BiFunction<Double, Double, Double> unseen, BiFunction<Double, Double, Double> damp) {
    checkArgument(ignoreLeafMassThreshold >= 0, "ignoreLeafMassThreshold should be greater than or equal to 0");
    if (!isOutputReady()) {
        return 0.0;
    }
    VisitorFactory<Double> visitorFactory = new VisitorFactory<>((tree, y) -> new DynamicScoreVisitor(tree.projectToTree(y), tree.getMass(), ignoreLeafMassThreshold, seen, unseen, damp));
    ConvergingAccumulator<Double> accumulator = new OneSidedConvergingDoubleAccumulator(highIsCritical, precision, DEFAULT_APPROXIMATE_DYNAMIC_SCORE_MIN_VALUES_ACCEPTED, numberOfTrees);
    Function<Double, Double> finisher = x -> x / accumulator.getValuesAccepted();
    return traverseForest(transformToShingledPoint(point), visitorFactory, accumulator, finisher);
}
Also used : OneSidedConvergingDoubleAccumulator(com.amazon.randomcutforest.returntypes.OneSidedConvergingDoubleAccumulator) CommonUtils.checkNotNull(com.amazon.randomcutforest.CommonUtils.checkNotNull) Arrays(java.util.Arrays) BiFunction(java.util.function.BiFunction) ParallelForestTraversalExecutor(com.amazon.randomcutforest.executor.ParallelForestTraversalExecutor) Random(java.util.Random) ParallelForestUpdateExecutor(com.amazon.randomcutforest.executor.ParallelForestUpdateExecutor) AbstractForestUpdateExecutor(com.amazon.randomcutforest.executor.AbstractForestUpdateExecutor) IStateCoordinator(com.amazon.randomcutforest.executor.IStateCoordinator) RandomCutTree(com.amazon.randomcutforest.tree.RandomCutTree) CommonUtils.toFloatArray(com.amazon.randomcutforest.CommonUtils.toFloatArray) Neighbor(com.amazon.randomcutforest.returntypes.Neighbor) ConditionalSampleSummarizer(com.amazon.randomcutforest.imputation.ConditionalSampleSummarizer) ImputeVisitor(com.amazon.randomcutforest.imputation.ImputeVisitor) NearNeighborVisitor(com.amazon.randomcutforest.inspect.NearNeighborVisitor) IBoundingBoxView(com.amazon.randomcutforest.tree.IBoundingBoxView) Collector(java.util.stream.Collector) PointStoreCoordinator(com.amazon.randomcutforest.executor.PointStoreCoordinator) AnomalyAttributionVisitor(com.amazon.randomcutforest.anomalydetection.AnomalyAttributionVisitor) OneSidedConvergingDoubleAccumulator(com.amazon.randomcutforest.returntypes.OneSidedConvergingDoubleAccumulator) AnomalyScoreVisitor(com.amazon.randomcutforest.anomalydetection.AnomalyScoreVisitor) AbstractForestTraversalExecutor(com.amazon.randomcutforest.executor.AbstractForestTraversalExecutor) BinaryOperator(java.util.function.BinaryOperator) SequentialForestTraversalExecutor(com.amazon.randomcutforest.executor.SequentialForestTraversalExecutor) List(java.util.List) Math.max(java.lang.Math.max) Optional(java.util.Optional) DensityOutput(com.amazon.randomcutforest.returntypes.DensityOutput) CommonUtils.toDoubleArray(com.amazon.randomcutforest.CommonUtils.toDoubleArray) Precision(com.amazon.randomcutforest.config.Precision) CompactSampler(com.amazon.randomcutforest.sampler.CompactSampler) SamplerPlusTree(com.amazon.randomcutforest.executor.SamplerPlusTree) ShingleBuilder(com.amazon.randomcutforest.util.ShingleBuilder) Function(java.util.function.Function) ArrayList(java.util.ArrayList) SimulatedTransductiveScalarScoreVisitor(com.amazon.randomcutforest.anomalydetection.SimulatedTransductiveScalarScoreVisitor) PointStore(com.amazon.randomcutforest.store.PointStore) DynamicAttributionVisitor(com.amazon.randomcutforest.anomalydetection.DynamicAttributionVisitor) ConvergingAccumulator(com.amazon.randomcutforest.returntypes.ConvergingAccumulator) Config(com.amazon.randomcutforest.config.Config) SimpleInterpolationVisitor(com.amazon.randomcutforest.interpolation.SimpleInterpolationVisitor) InterpolationMeasure(com.amazon.randomcutforest.returntypes.InterpolationMeasure) IPointStore(com.amazon.randomcutforest.store.IPointStore) SequentialForestUpdateExecutor(com.amazon.randomcutforest.executor.SequentialForestUpdateExecutor) ArrayUtils(com.amazon.randomcutforest.util.ArrayUtils) OneSidedConvergingDiVectorAccumulator(com.amazon.randomcutforest.returntypes.OneSidedConvergingDiVectorAccumulator) CommonUtils.checkArgument(com.amazon.randomcutforest.CommonUtils.checkArgument) DynamicScoreVisitor(com.amazon.randomcutforest.anomalydetection.DynamicScoreVisitor) DiVector(com.amazon.randomcutforest.returntypes.DiVector) ITree(com.amazon.randomcutforest.tree.ITree) ConditionalTreeSample(com.amazon.randomcutforest.returntypes.ConditionalTreeSample) ConditionalSampleSummary(com.amazon.randomcutforest.returntypes.ConditionalSampleSummary) Collections(java.util.Collections) IStreamSampler(com.amazon.randomcutforest.sampler.IStreamSampler) DynamicScoreVisitor(com.amazon.randomcutforest.anomalydetection.DynamicScoreVisitor)

Aggregations

CommonUtils.checkArgument (com.amazon.randomcutforest.CommonUtils.checkArgument)2 CommonUtils.checkNotNull (com.amazon.randomcutforest.CommonUtils.checkNotNull)2 CommonUtils.toDoubleArray (com.amazon.randomcutforest.CommonUtils.toDoubleArray)2 CommonUtils.toFloatArray (com.amazon.randomcutforest.CommonUtils.toFloatArray)2 AnomalyAttributionVisitor (com.amazon.randomcutforest.anomalydetection.AnomalyAttributionVisitor)2 AnomalyScoreVisitor (com.amazon.randomcutforest.anomalydetection.AnomalyScoreVisitor)2 DynamicAttributionVisitor (com.amazon.randomcutforest.anomalydetection.DynamicAttributionVisitor)2 DynamicScoreVisitor (com.amazon.randomcutforest.anomalydetection.DynamicScoreVisitor)2 SimulatedTransductiveScalarScoreVisitor (com.amazon.randomcutforest.anomalydetection.SimulatedTransductiveScalarScoreVisitor)2 Config (com.amazon.randomcutforest.config.Config)2 Precision (com.amazon.randomcutforest.config.Precision)2 AbstractForestTraversalExecutor (com.amazon.randomcutforest.executor.AbstractForestTraversalExecutor)2 AbstractForestUpdateExecutor (com.amazon.randomcutforest.executor.AbstractForestUpdateExecutor)2 IStateCoordinator (com.amazon.randomcutforest.executor.IStateCoordinator)2 ParallelForestTraversalExecutor (com.amazon.randomcutforest.executor.ParallelForestTraversalExecutor)2 ParallelForestUpdateExecutor (com.amazon.randomcutforest.executor.ParallelForestUpdateExecutor)2 PointStoreCoordinator (com.amazon.randomcutforest.executor.PointStoreCoordinator)2 SamplerPlusTree (com.amazon.randomcutforest.executor.SamplerPlusTree)2 SequentialForestTraversalExecutor (com.amazon.randomcutforest.executor.SequentialForestTraversalExecutor)2 SequentialForestUpdateExecutor (com.amazon.randomcutforest.executor.SequentialForestUpdateExecutor)2