Search in sources :

Example 1 with OneSidedConvergingDiVectorAccumulator

use of com.amazon.randomcutforest.returntypes.OneSidedConvergingDiVectorAccumulator in project random-cut-forest-by-aws by aws.

the class RandomCutForestTest method testGetApproximateAnomalyAttribution.

@Test
public void testGetApproximateAnomalyAttribution() {
    float[] point = { 1.2f, -3.4f };
    DiVector zero = new DiVector(dimensions);
    DiVector result = forest.getApproximateAnomalyAttribution(point);
    assertFalse(forest.isOutputReady());
    assertArrayEquals(zero.high, result.high, EPSILON);
    assertArrayEquals(zero.low, result.low, EPSILON);
    doReturn(true).when(forest).isOutputReady();
    ConvergingAccumulator<DiVector> accumulator = new OneSidedConvergingDiVectorAccumulator(dimensions, RandomCutForest.DEFAULT_APPROXIMATE_ANOMALY_SCORE_HIGH_IS_CRITICAL, RandomCutForest.DEFAULT_APPROXIMATE_DYNAMIC_SCORE_PRECISION, RandomCutForest.DEFAULT_APPROXIMATE_DYNAMIC_SCORE_MIN_VALUES_ACCEPTED, numberOfTrees);
    for (int i = 0; i < numberOfTrees; i++) {
        SamplerPlusTree<Integer, float[]> component = (SamplerPlusTree<Integer, float[]>) components.get(i);
        ITree<Integer, float[]> tree = component.getTree();
        DiVector treeResult = new DiVector(dimensions);
        for (int j = 0; j < dimensions; j++) {
            treeResult.high[j] = Math.random();
            treeResult.low[j] = Math.random();
        }
        when(tree.traverse(aryEq(point), any(VisitorFactory.class))).thenReturn(treeResult);
        when(tree.getMass()).thenReturn(256);
        if (!accumulator.isConverged()) {
            accumulator.accept(treeResult);
        }
    }
    DiVector expectedResult = accumulator.getAccumulatedValue().scale(1.0 / accumulator.getValuesAccepted());
    result = forest.getApproximateAnomalyAttribution(point);
    assertArrayEquals(expectedResult.high, result.high, EPSILON);
    assertArrayEquals(expectedResult.low, result.low, EPSILON);
}
Also used : DiVector(com.amazon.randomcutforest.returntypes.DiVector) OneSidedConvergingDiVectorAccumulator(com.amazon.randomcutforest.returntypes.OneSidedConvergingDiVectorAccumulator) SamplerPlusTree(com.amazon.randomcutforest.executor.SamplerPlusTree) Test(org.junit.jupiter.api.Test)

Example 2 with OneSidedConvergingDiVectorAccumulator

use of com.amazon.randomcutforest.returntypes.OneSidedConvergingDiVectorAccumulator in project random-cut-forest-by-aws by aws.

the class RandomCutForest method getApproximateAnomalyAttribution.

public DiVector getApproximateAnomalyAttribution(float[] point) {
    if (!isOutputReady()) {
        return new DiVector(dimensions);
    }
    IVisitorFactory<DiVector> visitorFactory = new VisitorFactory<>((tree, y) -> new AnomalyAttributionVisitor(tree.projectToTree(y), tree.getMass()), (tree, x) -> x.lift(tree::liftFromTree));
    ConvergingAccumulator<DiVector> accumulator = new OneSidedConvergingDiVectorAccumulator(dimensions, DEFAULT_APPROXIMATE_ANOMALY_SCORE_HIGH_IS_CRITICAL, DEFAULT_APPROXIMATE_DYNAMIC_SCORE_PRECISION, DEFAULT_APPROXIMATE_DYNAMIC_SCORE_MIN_VALUES_ACCEPTED, numberOfTrees);
    Function<DiVector, DiVector> finisher = x -> x.scale(1.0 / accumulator.getValuesAccepted());
    return traverseForest(transformToShingledPoint(point), visitorFactory, accumulator, finisher);
}
Also used : CommonUtils.checkNotNull(com.amazon.randomcutforest.CommonUtils.checkNotNull) Arrays(java.util.Arrays) BiFunction(java.util.function.BiFunction) ParallelForestTraversalExecutor(com.amazon.randomcutforest.executor.ParallelForestTraversalExecutor) Random(java.util.Random) ParallelForestUpdateExecutor(com.amazon.randomcutforest.executor.ParallelForestUpdateExecutor) AbstractForestUpdateExecutor(com.amazon.randomcutforest.executor.AbstractForestUpdateExecutor) IStateCoordinator(com.amazon.randomcutforest.executor.IStateCoordinator) RandomCutTree(com.amazon.randomcutforest.tree.RandomCutTree) CommonUtils.toFloatArray(com.amazon.randomcutforest.CommonUtils.toFloatArray) Neighbor(com.amazon.randomcutforest.returntypes.Neighbor) ConditionalSampleSummarizer(com.amazon.randomcutforest.imputation.ConditionalSampleSummarizer) ImputeVisitor(com.amazon.randomcutforest.imputation.ImputeVisitor) NearNeighborVisitor(com.amazon.randomcutforest.inspect.NearNeighborVisitor) IBoundingBoxView(com.amazon.randomcutforest.tree.IBoundingBoxView) Collector(java.util.stream.Collector) PointStoreCoordinator(com.amazon.randomcutforest.executor.PointStoreCoordinator) AnomalyAttributionVisitor(com.amazon.randomcutforest.anomalydetection.AnomalyAttributionVisitor) OneSidedConvergingDoubleAccumulator(com.amazon.randomcutforest.returntypes.OneSidedConvergingDoubleAccumulator) AnomalyScoreVisitor(com.amazon.randomcutforest.anomalydetection.AnomalyScoreVisitor) AbstractForestTraversalExecutor(com.amazon.randomcutforest.executor.AbstractForestTraversalExecutor) BinaryOperator(java.util.function.BinaryOperator) SequentialForestTraversalExecutor(com.amazon.randomcutforest.executor.SequentialForestTraversalExecutor) List(java.util.List) Math.max(java.lang.Math.max) Optional(java.util.Optional) DensityOutput(com.amazon.randomcutforest.returntypes.DensityOutput) CommonUtils.toDoubleArray(com.amazon.randomcutforest.CommonUtils.toDoubleArray) Precision(com.amazon.randomcutforest.config.Precision) CompactSampler(com.amazon.randomcutforest.sampler.CompactSampler) SamplerPlusTree(com.amazon.randomcutforest.executor.SamplerPlusTree) ShingleBuilder(com.amazon.randomcutforest.util.ShingleBuilder) Function(java.util.function.Function) ArrayList(java.util.ArrayList) SimulatedTransductiveScalarScoreVisitor(com.amazon.randomcutforest.anomalydetection.SimulatedTransductiveScalarScoreVisitor) PointStore(com.amazon.randomcutforest.store.PointStore) DynamicAttributionVisitor(com.amazon.randomcutforest.anomalydetection.DynamicAttributionVisitor) ConvergingAccumulator(com.amazon.randomcutforest.returntypes.ConvergingAccumulator) Config(com.amazon.randomcutforest.config.Config) SimpleInterpolationVisitor(com.amazon.randomcutforest.interpolation.SimpleInterpolationVisitor) InterpolationMeasure(com.amazon.randomcutforest.returntypes.InterpolationMeasure) IPointStore(com.amazon.randomcutforest.store.IPointStore) SequentialForestUpdateExecutor(com.amazon.randomcutforest.executor.SequentialForestUpdateExecutor) ArrayUtils(com.amazon.randomcutforest.util.ArrayUtils) OneSidedConvergingDiVectorAccumulator(com.amazon.randomcutforest.returntypes.OneSidedConvergingDiVectorAccumulator) CommonUtils.checkArgument(com.amazon.randomcutforest.CommonUtils.checkArgument) DynamicScoreVisitor(com.amazon.randomcutforest.anomalydetection.DynamicScoreVisitor) DiVector(com.amazon.randomcutforest.returntypes.DiVector) ITree(com.amazon.randomcutforest.tree.ITree) ConditionalTreeSample(com.amazon.randomcutforest.returntypes.ConditionalTreeSample) ConditionalSampleSummary(com.amazon.randomcutforest.returntypes.ConditionalSampleSummary) Collections(java.util.Collections) IStreamSampler(com.amazon.randomcutforest.sampler.IStreamSampler) DiVector(com.amazon.randomcutforest.returntypes.DiVector) AnomalyAttributionVisitor(com.amazon.randomcutforest.anomalydetection.AnomalyAttributionVisitor) OneSidedConvergingDiVectorAccumulator(com.amazon.randomcutforest.returntypes.OneSidedConvergingDiVectorAccumulator)

Example 3 with OneSidedConvergingDiVectorAccumulator

use of com.amazon.randomcutforest.returntypes.OneSidedConvergingDiVectorAccumulator in project random-cut-forest-by-aws by aws.

the class RandomCutForest method getApproximateDynamicAttribution.

/**
 * Atrribution for dynamic sequential scoring; getL1Norm() should agree with
 * getDynamicScoringSequential
 *
 * @param point                   input
 * @param precision               parameter to stop early stopping
 * @param highIsCritical          are high values anomalous (otherwise low
 *                                values are anomalous)
 * @param ignoreLeafMassThreshold we ignore leaves with mass equal/below *
 *                                threshold
 * @param seen                    function for scoring points that have been
 *                                seen before
 * @param unseen                  function for scoring points not seen in tree
 * @param newDamp                 dampening function based on duplicates
 * @return attribution DiVector of the score
 */
public DiVector getApproximateDynamicAttribution(float[] point, double precision, boolean highIsCritical, int ignoreLeafMassThreshold, BiFunction<Double, Double, Double> seen, BiFunction<Double, Double, Double> unseen, BiFunction<Double, Double, Double> newDamp) {
    if (!isOutputReady()) {
        return new DiVector(dimensions);
    }
    VisitorFactory<DiVector> visitorFactory = new VisitorFactory<>((tree, y) -> new DynamicAttributionVisitor(y, tree.getMass(), ignoreLeafMassThreshold, seen, unseen, newDamp), (tree, x) -> x.lift(tree::liftFromTree));
    ConvergingAccumulator<DiVector> accumulator = new OneSidedConvergingDiVectorAccumulator(dimensions, highIsCritical, precision, DEFAULT_APPROXIMATE_DYNAMIC_SCORE_MIN_VALUES_ACCEPTED, numberOfTrees);
    Function<DiVector, DiVector> finisher = vector -> vector.scale(1.0 / accumulator.getValuesAccepted());
    return traverseForest(transformToShingledPoint(point), visitorFactory, accumulator, finisher);
}
Also used : CommonUtils.checkNotNull(com.amazon.randomcutforest.CommonUtils.checkNotNull) Arrays(java.util.Arrays) BiFunction(java.util.function.BiFunction) ParallelForestTraversalExecutor(com.amazon.randomcutforest.executor.ParallelForestTraversalExecutor) Random(java.util.Random) ParallelForestUpdateExecutor(com.amazon.randomcutforest.executor.ParallelForestUpdateExecutor) AbstractForestUpdateExecutor(com.amazon.randomcutforest.executor.AbstractForestUpdateExecutor) IStateCoordinator(com.amazon.randomcutforest.executor.IStateCoordinator) RandomCutTree(com.amazon.randomcutforest.tree.RandomCutTree) CommonUtils.toFloatArray(com.amazon.randomcutforest.CommonUtils.toFloatArray) Neighbor(com.amazon.randomcutforest.returntypes.Neighbor) ConditionalSampleSummarizer(com.amazon.randomcutforest.imputation.ConditionalSampleSummarizer) ImputeVisitor(com.amazon.randomcutforest.imputation.ImputeVisitor) NearNeighborVisitor(com.amazon.randomcutforest.inspect.NearNeighborVisitor) IBoundingBoxView(com.amazon.randomcutforest.tree.IBoundingBoxView) Collector(java.util.stream.Collector) PointStoreCoordinator(com.amazon.randomcutforest.executor.PointStoreCoordinator) AnomalyAttributionVisitor(com.amazon.randomcutforest.anomalydetection.AnomalyAttributionVisitor) OneSidedConvergingDoubleAccumulator(com.amazon.randomcutforest.returntypes.OneSidedConvergingDoubleAccumulator) AnomalyScoreVisitor(com.amazon.randomcutforest.anomalydetection.AnomalyScoreVisitor) AbstractForestTraversalExecutor(com.amazon.randomcutforest.executor.AbstractForestTraversalExecutor) BinaryOperator(java.util.function.BinaryOperator) SequentialForestTraversalExecutor(com.amazon.randomcutforest.executor.SequentialForestTraversalExecutor) List(java.util.List) Math.max(java.lang.Math.max) Optional(java.util.Optional) DensityOutput(com.amazon.randomcutforest.returntypes.DensityOutput) CommonUtils.toDoubleArray(com.amazon.randomcutforest.CommonUtils.toDoubleArray) Precision(com.amazon.randomcutforest.config.Precision) CompactSampler(com.amazon.randomcutforest.sampler.CompactSampler) SamplerPlusTree(com.amazon.randomcutforest.executor.SamplerPlusTree) ShingleBuilder(com.amazon.randomcutforest.util.ShingleBuilder) Function(java.util.function.Function) ArrayList(java.util.ArrayList) SimulatedTransductiveScalarScoreVisitor(com.amazon.randomcutforest.anomalydetection.SimulatedTransductiveScalarScoreVisitor) PointStore(com.amazon.randomcutforest.store.PointStore) DynamicAttributionVisitor(com.amazon.randomcutforest.anomalydetection.DynamicAttributionVisitor) ConvergingAccumulator(com.amazon.randomcutforest.returntypes.ConvergingAccumulator) Config(com.amazon.randomcutforest.config.Config) SimpleInterpolationVisitor(com.amazon.randomcutforest.interpolation.SimpleInterpolationVisitor) InterpolationMeasure(com.amazon.randomcutforest.returntypes.InterpolationMeasure) IPointStore(com.amazon.randomcutforest.store.IPointStore) SequentialForestUpdateExecutor(com.amazon.randomcutforest.executor.SequentialForestUpdateExecutor) ArrayUtils(com.amazon.randomcutforest.util.ArrayUtils) OneSidedConvergingDiVectorAccumulator(com.amazon.randomcutforest.returntypes.OneSidedConvergingDiVectorAccumulator) CommonUtils.checkArgument(com.amazon.randomcutforest.CommonUtils.checkArgument) DynamicScoreVisitor(com.amazon.randomcutforest.anomalydetection.DynamicScoreVisitor) DiVector(com.amazon.randomcutforest.returntypes.DiVector) ITree(com.amazon.randomcutforest.tree.ITree) ConditionalTreeSample(com.amazon.randomcutforest.returntypes.ConditionalTreeSample) ConditionalSampleSummary(com.amazon.randomcutforest.returntypes.ConditionalSampleSummary) Collections(java.util.Collections) IStreamSampler(com.amazon.randomcutforest.sampler.IStreamSampler) DynamicAttributionVisitor(com.amazon.randomcutforest.anomalydetection.DynamicAttributionVisitor) DiVector(com.amazon.randomcutforest.returntypes.DiVector) OneSidedConvergingDiVectorAccumulator(com.amazon.randomcutforest.returntypes.OneSidedConvergingDiVectorAccumulator)

Aggregations

SamplerPlusTree (com.amazon.randomcutforest.executor.SamplerPlusTree)3 DiVector (com.amazon.randomcutforest.returntypes.DiVector)3 OneSidedConvergingDiVectorAccumulator (com.amazon.randomcutforest.returntypes.OneSidedConvergingDiVectorAccumulator)3 CommonUtils.checkArgument (com.amazon.randomcutforest.CommonUtils.checkArgument)2 CommonUtils.checkNotNull (com.amazon.randomcutforest.CommonUtils.checkNotNull)2 CommonUtils.toDoubleArray (com.amazon.randomcutforest.CommonUtils.toDoubleArray)2 CommonUtils.toFloatArray (com.amazon.randomcutforest.CommonUtils.toFloatArray)2 AnomalyAttributionVisitor (com.amazon.randomcutforest.anomalydetection.AnomalyAttributionVisitor)2 AnomalyScoreVisitor (com.amazon.randomcutforest.anomalydetection.AnomalyScoreVisitor)2 DynamicAttributionVisitor (com.amazon.randomcutforest.anomalydetection.DynamicAttributionVisitor)2 DynamicScoreVisitor (com.amazon.randomcutforest.anomalydetection.DynamicScoreVisitor)2 SimulatedTransductiveScalarScoreVisitor (com.amazon.randomcutforest.anomalydetection.SimulatedTransductiveScalarScoreVisitor)2 Config (com.amazon.randomcutforest.config.Config)2 Precision (com.amazon.randomcutforest.config.Precision)2 AbstractForestTraversalExecutor (com.amazon.randomcutforest.executor.AbstractForestTraversalExecutor)2 AbstractForestUpdateExecutor (com.amazon.randomcutforest.executor.AbstractForestUpdateExecutor)2 IStateCoordinator (com.amazon.randomcutforest.executor.IStateCoordinator)2 ParallelForestTraversalExecutor (com.amazon.randomcutforest.executor.ParallelForestTraversalExecutor)2 ParallelForestUpdateExecutor (com.amazon.randomcutforest.executor.ParallelForestUpdateExecutor)2 PointStoreCoordinator (com.amazon.randomcutforest.executor.PointStoreCoordinator)2