Search in sources :

Example 1 with SplunkJSONEventField

use of org.talend.components.splunk.objects.SplunkJSONEventField in project components by Talend.

the class TSplunkEventCollectorTestIT method testDefaultSchema.

@Test
public void testDefaultSchema() {
    TSplunkEventCollectorProperties props = (TSplunkEventCollectorProperties) new TSplunkEventCollectorDefinition().createProperties();
    Schema s = props.getSchema();
    List<Field> fields = s.getFields();
    List<SplunkJSONEventField> schemaEventFields = new ArrayList<>();
    for (Field f : fields) {
        schemaEventFields.add(SplunkJSONEventField.getByName(f.name()));
    }
    assertTrue("Default schema is wrong.", schemaEventFields.size() == SplunkJSONEventField.getMetadataFields().size());
    assertTrue("Default schema doesn't contain all metadata fields.", schemaEventFields.containsAll(SplunkJSONEventField.getMetadataFields()));
}
Also used : Field(org.apache.avro.Schema.Field) SplunkJSONEventField(org.talend.components.splunk.objects.SplunkJSONEventField) Schema(org.apache.avro.Schema) ArrayList(java.util.ArrayList) SplunkJSONEventField(org.talend.components.splunk.objects.SplunkJSONEventField) Test(org.junit.Test) SpringBootTest(org.springframework.boot.test.context.SpringBootTest)

Example 2 with SplunkJSONEventField

use of org.talend.components.splunk.objects.SplunkJSONEventField in project components by Talend.

the class TSplunkEventCollectorWriter method initDefaultSchema.

private Schema initDefaultSchema(Schema designSchema) {
    AvroRegistry avroReg = new AvroRegistry();
    FieldAssembler<Schema> record = SchemaBuilder.record("Main").fields();
    for (SplunkJSONEventField metadataField : SplunkJSONEventField.getMetadataFields()) {
        Schema base = avroReg.getConverter(metadataField.getDataType()).getSchema();
        FieldBuilder<Schema> fieldBuilder = record.name(metadataField.getName());
        if (metadataField.getName().equals(SplunkJSONEventField.TIME.getName())) {
            String datePattern;
            Field designField = designSchema.getField(metadataField.getName());
            if (designField != null) {
                datePattern = designField.getProp(SchemaConstants.TALEND_COLUMN_PATTERN);
            } else {
                datePattern = designSchema.getProp(SchemaConstants.TALEND_COLUMN_PATTERN);
            }
            if (datePattern == null || datePattern.isEmpty()) {
                datePattern = "dd-MM-yyyy";
            }
            fieldBuilder.prop(SchemaConstants.TALEND_COLUMN_PATTERN, datePattern);
        }
        fieldBuilder.type(AvroUtils.wrapAsNullable(base)).noDefault();
    }
    Schema defaultSchema = record.endRecord();
    return defaultSchema;
}
Also used : SplunkJSONEventField(org.talend.components.splunk.objects.SplunkJSONEventField) Field(org.apache.avro.Schema.Field) GenericAvroRegistry(org.talend.components.common.runtime.GenericAvroRegistry) AvroRegistry(org.talend.daikon.avro.AvroRegistry) Schema(org.apache.avro.Schema) SplunkJSONEventField(org.talend.components.splunk.objects.SplunkJSONEventField)

Example 3 with SplunkJSONEventField

use of org.talend.components.splunk.objects.SplunkJSONEventField in project components by Talend.

the class TSplunkEventCollectorProperties method setupDefaultSchema.

private void setupDefaultSchema() {
    AvroRegistry avroReg = new AvroRegistry();
    FieldAssembler<Schema> record = SchemaBuilder.record("Main").fields();
    for (SplunkJSONEventField metadataField : SplunkJSONEventField.getMetadataFields()) {
        Schema base = avroReg.getConverter(metadataField.getDataType()).getSchema();
        FieldBuilder<Schema> fieldBuilder = record.name(metadataField.getName());
        if (metadataField.getName().equals(SplunkJSONEventField.TIME.getName())) {
            fieldBuilder.prop(SchemaConstants.TALEND_COLUMN_PATTERN, "dd-MM-yyyy");
        }
        fieldBuilder.type(AvroUtils.wrapAsNullable(base)).noDefault();
    }
    Schema defaultSchema = record.endRecord();
    schema.schema.setValue(defaultSchema);
}
Also used : AvroRegistry(org.talend.daikon.avro.AvroRegistry) Schema(org.apache.avro.Schema) SplunkJSONEventField(org.talend.components.splunk.objects.SplunkJSONEventField)

Aggregations

Schema (org.apache.avro.Schema)3 SplunkJSONEventField (org.talend.components.splunk.objects.SplunkJSONEventField)3 Field (org.apache.avro.Schema.Field)2 AvroRegistry (org.talend.daikon.avro.AvroRegistry)2 ArrayList (java.util.ArrayList)1 Test (org.junit.Test)1 SpringBootTest (org.springframework.boot.test.context.SpringBootTest)1 GenericAvroRegistry (org.talend.components.common.runtime.GenericAvroRegistry)1