Search in sources :

Example 6 with UnknownCategorialValueException

use of org.apache.ignite.ml.math.exceptions.preprocessing.UnknownCategorialValueException in project ignite by apache.

the class OneHotEncoderPreprocessorTest method testApplyWithUnknownCategorialValues.

/**
 * The {@code apply()} method is failed with UnknownCategorialFeatureValue exception.
 *
 * The reason is missed information in encodingValues.
 *
 * @see UnknownCategorialValueException
 */
@Test
public void testApplyWithUnknownCategorialValues() {
    Vector[] data = new Vector[] { new DenseVector(new Serializable[] { "1", "Moscow", "A" }), new DenseVector(new Serializable[] { "2", "Moscow", "A" }), new DenseVector(new Serializable[] { "2", "Moscow", "B" }) };
    Vectorizer<Integer, Vector, Integer, Double> vectorizer = new DummyVectorizer<>(0, 1, 2);
    OneHotEncoderPreprocessor<Integer, Vector> preprocessor = new OneHotEncoderPreprocessor<Integer, Vector>(new HashMap[] { new HashMap() {

        {
            put("2", 0);
        }
    }, new HashMap() {

        {
            put("Moscow", 0);
        }
    }, new HashMap() {

        {
            put("A", 0);
            put("B", 1);
        }
    } }, vectorizer, new HashSet() {

        {
            add(0);
            add(1);
            add(2);
        }
    });
    double[][] postProcessedData = new double[][] { { 0.0, 1.0, 1.0, 1.0, 0.0 }, { 1.0, 0.0, 1.0, 1.0, 0.0 }, { 1.0, 0.0, 1.0, 0.0, 1.0 } };
    try {
        for (int i = 0; i < data.length; i++) assertArrayEquals(postProcessedData[i], preprocessor.apply(i, data[i]).features().asArray(), 1e-8);
        fail("UnknownCategorialFeatureValue");
    } catch (UnknownCategorialValueException e) {
        return;
    }
    fail("UnknownCategorialFeatureValue");
}
Also used : HashMap(java.util.HashMap) DummyVectorizer(org.apache.ignite.ml.dataset.feature.extractor.impl.DummyVectorizer) OneHotEncoderPreprocessor(org.apache.ignite.ml.preprocessing.encoding.onehotencoder.OneHotEncoderPreprocessor) UnknownCategorialValueException(org.apache.ignite.ml.math.exceptions.preprocessing.UnknownCategorialValueException) Vector(org.apache.ignite.ml.math.primitives.vector.Vector) DenseVector(org.apache.ignite.ml.math.primitives.vector.impl.DenseVector) DenseVector(org.apache.ignite.ml.math.primitives.vector.impl.DenseVector) HashSet(java.util.HashSet) Test(org.junit.Test)

Aggregations

UnknownCategorialValueException (org.apache.ignite.ml.math.exceptions.preprocessing.UnknownCategorialValueException)6 LabeledVector (org.apache.ignite.ml.structures.LabeledVector)4 DeployableObject (org.apache.ignite.ml.environment.deploy.DeployableObject)3 HashMap (java.util.HashMap)2 DummyVectorizer (org.apache.ignite.ml.dataset.feature.extractor.impl.DummyVectorizer)2 Vector (org.apache.ignite.ml.math.primitives.vector.Vector)2 DenseVector (org.apache.ignite.ml.math.primitives.vector.impl.DenseVector)2 Test (org.junit.Test)2 Serializable (java.io.Serializable)1 HashSet (java.util.HashSet)1 TrainerTest (org.apache.ignite.ml.common.TrainerTest)1 LocalDatasetBuilder (org.apache.ignite.ml.dataset.impl.local.LocalDatasetBuilder)1 IllegalFeatureTypeException (org.apache.ignite.ml.math.exceptions.preprocessing.IllegalFeatureTypeException)1 OneHotEncoderPreprocessor (org.apache.ignite.ml.preprocessing.encoding.onehotencoder.OneHotEncoderPreprocessor)1