Search in sources :

Example 16 with LearningEnvironment

use of org.apache.ignite.ml.environment.LearningEnvironment in project ignite by apache.

the class ConvergenceChecker method isConverged.

/**
 * Checks convergency on dataset.
 *
 * @param envBuilder Learning environment builder.
 * @param currMdl Current model.
 * @return True if GDB is converged.
 */
public boolean isConverged(LearningEnvironmentBuilder envBuilder, DatasetBuilder<K, V> datasetBuilder, ModelsComposition currMdl) {
    LearningEnvironment environment = envBuilder.buildForTrainer();
    environment.initDeployingContext(preprocessor);
    try (Dataset<EmptyContext, FeatureMatrixWithLabelsOnHeapData> dataset = datasetBuilder.build(envBuilder, new EmptyContextBuilder<>(), new FeatureMatrixWithLabelsOnHeapDataBuilder<>(preprocessor), environment)) {
        return isConverged(dataset, currMdl);
    } catch (Exception e) {
        throw new RuntimeException(e);
    }
}
Also used : LearningEnvironment(org.apache.ignite.ml.environment.LearningEnvironment) EmptyContext(org.apache.ignite.ml.dataset.primitive.context.EmptyContext) FeatureMatrixWithLabelsOnHeapData(org.apache.ignite.ml.dataset.primitive.FeatureMatrixWithLabelsOnHeapData)

Example 17 with LearningEnvironment

use of org.apache.ignite.ml.environment.LearningEnvironment in project ignite by apache.

the class Pipeline method fit.

/**
 * Fits the pipeline to the input dataset builder.
 */
public PipelineMdl<K, V> fit(DatasetBuilder datasetBuilder) {
    if (finalStage == null)
        throw new IllegalStateException("The Pipeline should be finished with the Training Stage.");
    // Reload for new fit
    finalPreprocessor = vectorizer;
    preprocessingTrainers.forEach(e -> {
        finalPreprocessor = e.fit(envBuilder, datasetBuilder, finalPreprocessor);
    });
    LearningEnvironment env = LearningEnvironmentBuilder.defaultBuilder().buildForTrainer();
    env.initDeployingContext(finalPreprocessor);
    IgniteModel<Vector, Double> internalMdl = finalStage.fit(datasetBuilder, finalPreprocessor, env);
    return new PipelineMdl<K, V>().withPreprocessor(finalPreprocessor).withInternalMdl(internalMdl);
}
Also used : LearningEnvironment(org.apache.ignite.ml.environment.LearningEnvironment) Vector(org.apache.ignite.ml.math.primitives.vector.Vector)

Example 18 with LearningEnvironment

use of org.apache.ignite.ml.environment.LearningEnvironment in project ignite by apache.

the class PreprocessingTrainer method learningEnvironment.

/**
 * Returns local learning environment with initialized deploying context by base preprocessor.
 *
 * @param basePreprocessor Preprocessor.
 * @return Learning environment.
 */
public default LearningEnvironment learningEnvironment(Preprocessor<K, V> basePreprocessor) {
    LearningEnvironment env = LearningEnvironmentBuilder.defaultBuilder().buildForTrainer();
    env.initDeployingContext(basePreprocessor);
    return env;
}
Also used : LearningEnvironment(org.apache.ignite.ml.environment.LearningEnvironment)

Aggregations

LearningEnvironment (org.apache.ignite.ml.environment.LearningEnvironment)18 LearningEnvironmentBuilder (org.apache.ignite.ml.environment.LearningEnvironmentBuilder)6 UUID (java.util.UUID)5 Serializable (java.io.Serializable)4 PartitionDataBuilder (org.apache.ignite.ml.dataset.PartitionDataBuilder)4 IgniteFunction (org.apache.ignite.ml.math.functions.IgniteFunction)4 ArrayList (java.util.ArrayList)3 Arrays (java.util.Arrays)3 Iterator (java.util.Iterator)3 Map (java.util.Map)3 Ignite (org.apache.ignite.Ignite)3 IgniteCache (org.apache.ignite.IgniteCache)3 Ignition (org.apache.ignite.Ignition)3 IgniteBiPredicate (org.apache.ignite.lang.IgniteBiPredicate)3 PartitionContextBuilder (org.apache.ignite.ml.dataset.PartitionContextBuilder)3 UpstreamTransformer (org.apache.ignite.ml.dataset.UpstreamTransformer)3 UpstreamTransformerBuilder (org.apache.ignite.ml.dataset.UpstreamTransformerBuilder)3 EmptyContext (org.apache.ignite.ml.dataset.primitive.context.EmptyContext)3 BitSet (java.util.BitSet)2 Collection (java.util.Collection)2