Examples with Metadata org.apache.gobblin.metadata.types.Metadata used on opensource projects

Example 11 with Metadata

use of org.apache.gobblin.metadata.types.Metadata in project incubator-gobblin by apache.

the class EnvelopedRecordWithMetadataToRecordWithMetadataTest method testSuccessWithRecord.

@Test
public void testSuccessWithRecord() throws DataConversionException, IOException {
    ObjectMapper objectMapper = new ObjectMapper();
    String innerRecord = "abracadabra";
    // Build the input record
    HashMap<String, Object> map = new HashMap<>();
    map.put("r", innerRecord);
    Metadata md = new Metadata();
    md.getRecordMetadata().put("test1", "test2");
    map.put("rMd", md);
    JsonNode jsonNode = objectMapper.valueToTree(map);
    RecordWithMetadata<byte[]> inputRecord = new RecordWithMetadata<>(jsonNode.toString().getBytes(), null);
    EnvelopedRecordWithMetadataToRecordWithMetadata converter = new EnvelopedRecordWithMetadataToRecordWithMetadata();
    Iterator<RecordWithMetadata<?>> iterator = converter.convertRecord(null, inputRecord, null).iterator();
    Assert.assertTrue(iterator.hasNext());
    RecordWithMetadata<?> outputRecord = iterator.next();
    Assert.assertEquals(outputRecord.getRecord(), innerRecord);
    Assert.assertEquals(outputRecord.getMetadata().getRecordMetadata().get("test1"), "test2");
}

Example 12 with Metadata

use of org.apache.gobblin.metadata.types.Metadata in project incubator-gobblin by apache.

the class EnvelopedRecordWithMetadataToRecordWithMetadataTest method testFailureWithoutRecord.

@Test(expectedExceptions = DataConversionException.class, expectedExceptionsMessageRegExp = "Input data does not have record.")
public void testFailureWithoutRecord() throws DataConversionException, IOException {
    ObjectMapper objectMapper = new ObjectMapper();
    // Build the input record without data
    HashMap<String, Object> map = new HashMap<>();
    Metadata md = new Metadata();
    md.getRecordMetadata().put("test1", "test2");
    map.put("rMd", md);
    JsonNode jsonNode = objectMapper.valueToTree(map);
    RecordWithMetadata<byte[]> inputRecord = new RecordWithMetadata<>(jsonNode.toString().getBytes(), null);
    EnvelopedRecordWithMetadataToRecordWithMetadata converter = new EnvelopedRecordWithMetadataToRecordWithMetadata();
    converter.convertRecord(null, inputRecord, null);
}

Example 13 with Metadata

use of org.apache.gobblin.metadata.types.Metadata in project incubator-gobblin by apache.

the class RecordWithMetadataSchemaRegistrationConverter method convertRecord.

@Override
public Iterable<RecordWithMetadata<?>> convertRecord(String outputSchema, RecordWithMetadata<?> inputRecord, WorkUnitState workUnit) throws DataConversionException {
    Preconditions.checkNotNull(schemaId);
    Metadata metadata = inputRecord.getMetadata();
    metadata.getGlobalMetadata().setContentType(CONTENT_TYPE);
    metadata.getRecordMetadata().put(SCHEMA_ID_KEY, schemaId);
    return Collections.singleton(new RecordWithMetadata<>(inputRecord.getRecord(), metadata));
}

Example 14 with Metadata

use of org.apache.gobblin.metadata.types.Metadata in project incubator-gobblin by apache.

the class AvroToJsonRecordWithMetadataConverter method convertSchema.

@Override
public String convertSchema(final Schema inputSchema, WorkUnitState workUnit) throws SchemaConversionException {
    this.defaultMetadata = new Metadata();
    defaultMetadata.getGlobalMetadata().setContentType(inputSchema.getFullName() + "+json");
    return innerConverter.convertSchema(inputSchema, workUnit);
}

Example 15 with Metadata

use of org.apache.gobblin.metadata.types.Metadata in project incubator-gobblin by apache.

the class SerializedRecordToEncryptedSerializedRecordConverterTest method setUp.

@BeforeTest
public void setUp() {
    workUnitState = new WorkUnitState();
    converter = new SerializedRecordToEncryptedSerializedRecordConverter();
    sampleRecord = new RecordWithMetadata<>(new byte[] { 'a', 'b', 'c', 'd' }, new Metadata());
    shiftedValue = new byte[] { 'b', 'c', 'd', 'e' };
    insecureShiftTag = InsecureShiftCodec.TAG;
}