Search in sources :

Example 1 with TrainingDatasetFilter

use of io.hops.hopsworks.persistence.entity.featurestore.trainingdataset.TrainingDatasetFilter in project hopsworks by logicalclocks.

the class TrainingDatasetControllerTest method testconvertToFilterEntities_bothLogic.

@Test
public void testconvertToFilterEntities_bothLogic() throws Exception {
    // fg.feature > 1 and (fg.feature > 2 OR fg.feature > 3)
    // "fg.feature > 1" stores as filter logic
    TrainingDataset trainingDataset = new TrainingDataset();
    Feature f1 = new Feature("test_f", "fg0");
    FilterLogic head = new FilterLogic();
    head.setType(AND);
    FilterLogic left = new FilterLogic();
    Filter left_left = new Filter(f1, GREATER_THAN, "1");
    left.setType(SINGLE);
    left.setLeftFilter(left_left);
    head.setLeftLogic(left);
    FilterLogic right = new FilterLogic();
    right.setType(OR);
    Filter right_left = new Filter(f1, GREATER_THAN, "2");
    Filter right_right = new Filter(f1, GREATER_THAN, "3");
    right.setLeftFilter(right_left);
    right.setRightFilter(right_right);
    head.setRightLogic(right);
    List<TrainingDatasetFilter> actual = target.convertToFilterEntities(head, trainingDataset, "L");
    List<TrainingDatasetFilter> expected = new ArrayList<>();
    expected.add(createTrainingDatasetFilter(null, AND, "L"));
    expected.add(createTrainingDatasetFilter(createTrainingDatasetFilterCondition("test_f", GREATER_THAN, "1"), SINGLE, "L.L"));
    expected.add(createTrainingDatasetFilter(null, OR, "L.R"));
    expected.add(createTrainingDatasetFilter(createTrainingDatasetFilterCondition("test_f", GREATER_THAN, "2"), SINGLE, "L.R.L"));
    expected.add(createTrainingDatasetFilter(createTrainingDatasetFilterCondition("test_f", GREATER_THAN, "3"), SINGLE, "L.R.R"));
    Assert.assertEquals(expected.size(), actual.size());
    Assert.assertTrue(expected.containsAll(actual));
    Assert.assertTrue(actual.containsAll(expected));
}
Also used : TrainingDatasetFilter(io.hops.hopsworks.persistence.entity.featurestore.trainingdataset.TrainingDatasetFilter) Filter(io.hops.hopsworks.common.featurestore.query.filter.Filter) TrainingDatasetFilter(io.hops.hopsworks.persistence.entity.featurestore.trainingdataset.TrainingDatasetFilter) TrainingDataset(io.hops.hopsworks.persistence.entity.featurestore.trainingdataset.TrainingDataset) SqlFilterLogic(io.hops.hopsworks.persistence.entity.featurestore.trainingdataset.SqlFilterLogic) FilterLogic(io.hops.hopsworks.common.featurestore.query.filter.FilterLogic) ArrayList(java.util.ArrayList) Feature(io.hops.hopsworks.common.featurestore.query.Feature) Test(org.junit.Test)

Example 2 with TrainingDatasetFilter

use of io.hops.hopsworks.persistence.entity.featurestore.trainingdataset.TrainingDatasetFilter in project hopsworks by logicalclocks.

the class TrainingDatasetControllerTest method createTrainingDatasetFilter.

public TrainingDatasetFilter createTrainingDatasetFilter(TrainingDatasetFilterCondition condition, SqlFilterLogic type, String path) {
    TrainingDatasetFilter filter = new TrainingDatasetFilter();
    filter.setCondition(condition);
    filter.setType(type);
    filter.setPath(path);
    return filter;
}
Also used : TrainingDatasetFilter(io.hops.hopsworks.persistence.entity.featurestore.trainingdataset.TrainingDatasetFilter)

Example 3 with TrainingDatasetFilter

use of io.hops.hopsworks.persistence.entity.featurestore.trainingdataset.TrainingDatasetFilter in project hopsworks by logicalclocks.

the class TrainingDatasetControllerTest method testconvertToFilterEntities_leftOnlyFilter.

@Test
public void testconvertToFilterEntities_leftOnlyFilter() throws Exception {
    // fg.feature > 2
    // "fg.feature > 2" stores in the left-hand side
    TrainingDataset trainingDataset = new TrainingDataset();
    Feature f1 = new Feature("test_f", "fg0");
    FilterLogic head = new FilterLogic();
    head.setType(SINGLE);
    Filter right = new Filter(f1, GREATER_THAN, "2");
    head.setLeftFilter(right);
    List<TrainingDatasetFilter> actual = target.convertToFilterEntities(head, trainingDataset, "L");
    List<TrainingDatasetFilter> expected = new ArrayList<>();
    expected.add(createTrainingDatasetFilter(createTrainingDatasetFilterCondition("test_f", GREATER_THAN, "2"), SINGLE, "L"));
    Assert.assertEquals(expected.size(), actual.size());
    Assert.assertTrue(expected.containsAll(actual));
    Assert.assertTrue(actual.containsAll(expected));
}
Also used : TrainingDatasetFilter(io.hops.hopsworks.persistence.entity.featurestore.trainingdataset.TrainingDatasetFilter) Filter(io.hops.hopsworks.common.featurestore.query.filter.Filter) TrainingDatasetFilter(io.hops.hopsworks.persistence.entity.featurestore.trainingdataset.TrainingDatasetFilter) TrainingDataset(io.hops.hopsworks.persistence.entity.featurestore.trainingdataset.TrainingDataset) SqlFilterLogic(io.hops.hopsworks.persistence.entity.featurestore.trainingdataset.SqlFilterLogic) FilterLogic(io.hops.hopsworks.common.featurestore.query.filter.FilterLogic) ArrayList(java.util.ArrayList) Feature(io.hops.hopsworks.common.featurestore.query.Feature) Test(org.junit.Test)

Example 4 with TrainingDatasetFilter

use of io.hops.hopsworks.persistence.entity.featurestore.trainingdataset.TrainingDatasetFilter in project hopsworks by logicalclocks.

the class TrainingDatasetControllerTest method testConvertToFilterLogic_multipleConditions.

public void testConvertToFilterLogic_multipleConditions() throws Exception {
    // fg.feature > 1 and fg.feature > 2
    List<TrainingDatasetFilter> filters = new ArrayList<>();
    filters.add(createTrainingDatasetFilter(null, AND, "L"));
    filters.add(createTrainingDatasetFilter(createTrainingDatasetFilterCondition("test_f", GREATER_THAN, "2", 0), SINGLE, "L.R"));
    filters.add(createTrainingDatasetFilter(createTrainingDatasetFilterCondition("test_f", GREATER_THAN, "1", 0), SINGLE, "L.L"));
    Map<String, Feature> featureLookup = Maps.newHashMap();
    Feature feature = new Feature("test_f", "fg0");
    Featuregroup featuregroup = new Featuregroup();
    featuregroup.setId(0);
    feature.setFeatureGroup(featuregroup);
    featureLookup.put("0.test_f", feature);
    FilterLogic actual = target.convertToFilterLogic(filters, featureLookup, "L");
    Feature f1 = new Feature("test_f", "fg0", null, false, null, null);
    f1.setFeatureGroup(featuregroup);
    Filter left = new Filter(f1, GREATER_THAN, "1");
    Filter right = new Filter(f1, GREATER_THAN, "2");
    Assert.assertEquals(actual.getType(), AND);
    Assert.assertEquals(actual.getLeftFilter(), left);
    Assert.assertNull(actual.getLeftLogic());
    Assert.assertEquals(actual.getRightFilter(), right);
    Assert.assertNull(actual.getRightLogic());
}
Also used : TrainingDatasetFilter(io.hops.hopsworks.persistence.entity.featurestore.trainingdataset.TrainingDatasetFilter) TrainingDatasetFilter(io.hops.hopsworks.persistence.entity.featurestore.trainingdataset.TrainingDatasetFilter) Filter(io.hops.hopsworks.common.featurestore.query.filter.Filter) SqlFilterLogic(io.hops.hopsworks.persistence.entity.featurestore.trainingdataset.SqlFilterLogic) FilterLogic(io.hops.hopsworks.common.featurestore.query.filter.FilterLogic) Featuregroup(io.hops.hopsworks.persistence.entity.featurestore.featuregroup.Featuregroup) ArrayList(java.util.ArrayList) Feature(io.hops.hopsworks.common.featurestore.query.Feature)

Example 5 with TrainingDatasetFilter

use of io.hops.hopsworks.persistence.entity.featurestore.trainingdataset.TrainingDatasetFilter in project hopsworks by logicalclocks.

the class TrainingDatasetControllerTest method testconvertToFilterEntities_bothFilter.

@Test
public void testconvertToFilterEntities_bothFilter() throws Exception {
    // fg.feature > 1 and fg.feature > 2
    // "fg.feature > 1" and "fg.feature > 2" stores as filter
    TrainingDataset trainingDataset = new TrainingDataset();
    Feature f1 = new Feature("test_f", "fg0");
    FilterLogic head = new FilterLogic();
    head.setType(AND);
    Filter left = new Filter(f1, GREATER_THAN, "1");
    head.setLeftFilter(left);
    Filter right = new Filter(f1, GREATER_THAN, "2");
    head.setRightFilter(right);
    List<TrainingDatasetFilter> actual = target.convertToFilterEntities(head, trainingDataset, "L");
    List<TrainingDatasetFilter> expected = new ArrayList<>();
    expected.add(createTrainingDatasetFilter(null, AND, "L"));
    expected.add(createTrainingDatasetFilter(createTrainingDatasetFilterCondition("test_f", GREATER_THAN, "2"), SINGLE, "L.R"));
    expected.add(createTrainingDatasetFilter(createTrainingDatasetFilterCondition("test_f", GREATER_THAN, "1"), SINGLE, "L.L"));
    Assert.assertEquals(expected.size(), actual.size());
    Assert.assertTrue(expected.containsAll(actual));
    Assert.assertTrue(actual.containsAll(expected));
}
Also used : TrainingDatasetFilter(io.hops.hopsworks.persistence.entity.featurestore.trainingdataset.TrainingDatasetFilter) Filter(io.hops.hopsworks.common.featurestore.query.filter.Filter) TrainingDatasetFilter(io.hops.hopsworks.persistence.entity.featurestore.trainingdataset.TrainingDatasetFilter) TrainingDataset(io.hops.hopsworks.persistence.entity.featurestore.trainingdataset.TrainingDataset) SqlFilterLogic(io.hops.hopsworks.persistence.entity.featurestore.trainingdataset.SqlFilterLogic) FilterLogic(io.hops.hopsworks.common.featurestore.query.filter.FilterLogic) ArrayList(java.util.ArrayList) Feature(io.hops.hopsworks.common.featurestore.query.Feature) Test(org.junit.Test)

Aggregations

TrainingDatasetFilter (io.hops.hopsworks.persistence.entity.featurestore.trainingdataset.TrainingDatasetFilter)14 Feature (io.hops.hopsworks.common.featurestore.query.Feature)12 Filter (io.hops.hopsworks.common.featurestore.query.filter.Filter)12 FilterLogic (io.hops.hopsworks.common.featurestore.query.filter.FilterLogic)12 SqlFilterLogic (io.hops.hopsworks.persistence.entity.featurestore.trainingdataset.SqlFilterLogic)12 ArrayList (java.util.ArrayList)12 TrainingDataset (io.hops.hopsworks.persistence.entity.featurestore.trainingdataset.TrainingDataset)8 Test (org.junit.Test)7 Featuregroup (io.hops.hopsworks.persistence.entity.featurestore.featuregroup.Featuregroup)5 FilterValue (io.hops.hopsworks.common.featurestore.query.filter.FilterValue)4 TrainingDatasetFilterCondition (io.hops.hopsworks.persistence.entity.featurestore.trainingdataset.TrainingDatasetFilterCondition)2 Streams (com.logicalclocks.shaded.com.google.common.collect.Streams)1 FeaturestoreFacade (io.hops.hopsworks.common.featurestore.FeaturestoreFacade)1 FeaturestoreActivityFacade (io.hops.hopsworks.common.featurestore.activity.FeaturestoreActivityFacade)1 TrainingDatasetFeatureDTO (io.hops.hopsworks.common.featurestore.feature.TrainingDatasetFeatureDTO)1 FeaturegroupController (io.hops.hopsworks.common.featurestore.featuregroup.FeaturegroupController)1 FeaturegroupDTO (io.hops.hopsworks.common.featurestore.featuregroup.FeaturegroupDTO)1 OnlineFeaturegroupController (io.hops.hopsworks.common.featurestore.featuregroup.online.OnlineFeaturegroupController)1 OnlineFeaturestoreController (io.hops.hopsworks.common.featurestore.online.OnlineFeaturestoreController)1 Query (io.hops.hopsworks.common.featurestore.query.Query)1