Search in sources :

Example 1 with RangeValidator

use of com.thinkbiganalytics.policy.validation.RangeValidator in project kylo by Teradata.

the class CleanseAndValidateRowTest method rangeValidate.

private ValidationResult rangeValidate(Number min, Number max, String dataType, String value) {
    RangeValidator validatorPolicy = new RangeValidator(min, max);
    List<BaseFieldPolicy> policies = new ArrayList<>();
    policies.add(validatorPolicy);
    FieldPolicy fieldPolicy = FieldPolicyBuilder.newBuilder().addPolicies(policies).tableName("emp").fieldName("field1").feedFieldName("field1").addPolicies(policies).build();
    StandardizationAndValidationResult result = validator.standardizeAndValidateField(fieldPolicy, value, HCatDataType.createFromDataType("field1", dataType), new HashMap<Class, Class>());
    return result.getFinalValidationResult();
}
Also used : FieldPolicy(com.thinkbiganalytics.policy.FieldPolicy) BaseFieldPolicy(com.thinkbiganalytics.policy.BaseFieldPolicy) ArrayList(java.util.ArrayList) RangeValidator(com.thinkbiganalytics.policy.validation.RangeValidator) BaseFieldPolicy(com.thinkbiganalytics.policy.BaseFieldPolicy) StandardizationAndValidationResult(com.thinkbiganalytics.spark.datavalidator.StandardizationAndValidationResult)

Example 2 with RangeValidator

use of com.thinkbiganalytics.policy.validation.RangeValidator in project kylo by Teradata.

the class TestJsonPolicies method testJson.

@Test
public void testJson() throws IOException {
    List<com.thinkbiganalytics.policy.rest.model.FieldPolicy> fieldPolicies = new ArrayList<>();
    List<FieldStandardizationRule> standardizationPolicyList = new ArrayList<>();
    List<FieldValidationRule> validationRules = new ArrayList<>();
    DefaultValueStandardizer defaultValueStandardizer = new DefaultValueStandardizer("My Default");
    standardizationPolicyList.add(StandardizationAnnotationTransformer.instance().toUIModel(defaultValueStandardizer));
    DateTimeStandardizer dateTimeStandardizer = new DateTimeStandardizer("MM/DD/YYYY", DateTimeStandardizer.OutputFormats.DATETIME_NOMILLIS);
    standardizationPolicyList.add(StandardizationAnnotationTransformer.instance().toUIModel(dateTimeStandardizer));
    RangeValidator validator = new RangeValidator(10, 20);
    validationRules.add(ValidatorAnnotationTransformer.instance().toUIModel(validator));
    fieldPolicies.add(new FieldPolicyBuilder("field1").addStandardization(standardizationPolicyList).addValidations(validationRules).build());
    ObjectMapper mapper = new ObjectMapper();
    String json = mapper.writeValueAsString(fieldPolicies);
    FieldPoliciesJsonTransformer fieldPolicyTransformer = new FieldPoliciesJsonTransformer(json);
    Map<String, com.thinkbiganalytics.policy.FieldPolicy> policyMap = fieldPolicyTransformer.buildPolicies();
    com.thinkbiganalytics.policy.FieldPolicy field1Policy = policyMap.get("field1");
    Assert.assertEquals(2, field1Policy.getStandardizationPolicies().size());
    Assert.assertEquals(1, field1Policy.getValidators().size());
}
Also used : FieldValidationRule(com.thinkbiganalytics.policy.rest.model.FieldValidationRule) ArrayList(java.util.ArrayList) DefaultValueStandardizer(com.thinkbiganalytics.policy.standardization.DefaultValueStandardizer) FieldPolicyBuilder(com.thinkbiganalytics.policy.rest.model.FieldPolicyBuilder) RangeValidator(com.thinkbiganalytics.policy.validation.RangeValidator) ObjectMapper(com.fasterxml.jackson.databind.ObjectMapper) FieldStandardizationRule(com.thinkbiganalytics.policy.rest.model.FieldStandardizationRule) DateTimeStandardizer(com.thinkbiganalytics.policy.standardization.DateTimeStandardizer) Test(org.junit.Test)

Example 3 with RangeValidator

use of com.thinkbiganalytics.policy.validation.RangeValidator in project kylo by Teradata.

the class TestValidationTransformation method testRangeValidator.

@Test
@Ignore
public void testRangeValidator() {
    Integer min = 10;
    Integer max = 20;
    RangeValidator validator = new RangeValidator(min, max);
    FieldValidationRule uiModel = ValidatorAnnotationTransformer.instance().toUIModel(validator);
    RangeValidator convertedValidator = fromUI(uiModel, RangeValidator.class);
    Assert.assertEquals(min, convertedValidator.getMin());
    Assert.assertEquals(max, convertedValidator.getMin());
}
Also used : FieldValidationRule(com.thinkbiganalytics.policy.rest.model.FieldValidationRule) RangeValidator(com.thinkbiganalytics.policy.validation.RangeValidator) Ignore(org.junit.Ignore) Test(org.junit.Test)

Aggregations

RangeValidator (com.thinkbiganalytics.policy.validation.RangeValidator)3 FieldValidationRule (com.thinkbiganalytics.policy.rest.model.FieldValidationRule)2 ArrayList (java.util.ArrayList)2 Test (org.junit.Test)2 ObjectMapper (com.fasterxml.jackson.databind.ObjectMapper)1 BaseFieldPolicy (com.thinkbiganalytics.policy.BaseFieldPolicy)1 FieldPolicy (com.thinkbiganalytics.policy.FieldPolicy)1 FieldPolicyBuilder (com.thinkbiganalytics.policy.rest.model.FieldPolicyBuilder)1 FieldStandardizationRule (com.thinkbiganalytics.policy.rest.model.FieldStandardizationRule)1 DateTimeStandardizer (com.thinkbiganalytics.policy.standardization.DateTimeStandardizer)1 DefaultValueStandardizer (com.thinkbiganalytics.policy.standardization.DefaultValueStandardizer)1 StandardizationAndValidationResult (com.thinkbiganalytics.spark.datavalidator.StandardizationAndValidationResult)1 Ignore (org.junit.Ignore)1