Search in sources :

Example 1 with LabelPartitionDataOnHeap

use of org.apache.ignite.ml.structures.partition.LabelPartitionDataOnHeap in project ignite by apache.

the class OneVsRestTrainer method extractClassLabels.

/**
 * Iterates among dataset and collects class labels.
 */
private <K, V> List<Double> extractClassLabels(DatasetBuilder<K, V> datasetBuilder, Preprocessor<K, V> preprocessor) {
    assert datasetBuilder != null;
    PartitionDataBuilder<K, V, EmptyContext, LabelPartitionDataOnHeap> partDataBuilder = new LabelPartitionDataBuilderOnHeap<>(preprocessor);
    List<Double> res = new ArrayList<>();
    try (Dataset<EmptyContext, LabelPartitionDataOnHeap> dataset = datasetBuilder.build(envBuilder, (env, upstream, upstreamSize) -> new EmptyContext(), partDataBuilder, learningEnvironment())) {
        final Set<Double> clsLabels = dataset.compute(data -> {
            final Set<Double> locClsLabels = new HashSet<>();
            final double[] lbs = data.getY();
            for (double lb : lbs) locClsLabels.add(lb);
            return locClsLabels;
        }, (a, b) -> {
            if (a == null)
                return b == null ? new HashSet<>() : b;
            if (b == null)
                return a;
            return Stream.of(a, b).flatMap(Collection::stream).collect(Collectors.toSet());
        });
        if (clsLabels != null)
            res.addAll(clsLabels);
    } catch (Exception e) {
        throw new RuntimeException(e);
    }
    return res;
}
Also used : EmptyContext(org.apache.ignite.ml.dataset.primitive.context.EmptyContext) ArrayList(java.util.ArrayList) LabelPartitionDataBuilderOnHeap(org.apache.ignite.ml.structures.partition.LabelPartitionDataBuilderOnHeap) LabelPartitionDataOnHeap(org.apache.ignite.ml.structures.partition.LabelPartitionDataOnHeap) HashSet(java.util.HashSet)

Aggregations

ArrayList (java.util.ArrayList)1 HashSet (java.util.HashSet)1 EmptyContext (org.apache.ignite.ml.dataset.primitive.context.EmptyContext)1 LabelPartitionDataBuilderOnHeap (org.apache.ignite.ml.structures.partition.LabelPartitionDataBuilderOnHeap)1 LabelPartitionDataOnHeap (org.apache.ignite.ml.structures.partition.LabelPartitionDataOnHeap)1