Examples with RecordWithMetadata org.apache.gobblin.type.RecordWithMetadata used on opensource projects

Example 16 with RecordWithMetadata

use of org.apache.gobblin.type.RecordWithMetadata in project incubator-gobblin by apache.

the class RecordWithMetadataSchemaRegistrationConverter method convertRecord.

@Override
public Iterable<RecordWithMetadata<?>> convertRecord(String outputSchema, RecordWithMetadata<?> inputRecord, WorkUnitState workUnit) throws DataConversionException {
    Preconditions.checkNotNull(schemaId);
    Metadata metadata = inputRecord.getMetadata();
    metadata.getGlobalMetadata().setContentType(CONTENT_TYPE);
    metadata.getRecordMetadata().put(SCHEMA_ID_KEY, schemaId);
    return Collections.singleton(new RecordWithMetadata<>(inputRecord.getRecord(), metadata));
}

Example 17 with RecordWithMetadata

use of org.apache.gobblin.type.RecordWithMetadata in project incubator-gobblin by apache.

the class SerializedRecordToEncryptedSerializedRecordConverterTest method setUp.

@BeforeTest
public void setUp() {
    workUnitState = new WorkUnitState();
    converter = new SerializedRecordToEncryptedSerializedRecordConverter();
    sampleRecord = new RecordWithMetadata<>(new byte[] { 'a', 'b', 'c', 'd' }, new Metadata());
    shiftedValue = new byte[] { 'b', 'c', 'd', 'e' };
    insecureShiftTag = InsecureShiftCodec.TAG;
}

Example 18 with RecordWithMetadata

use of org.apache.gobblin.type.RecordWithMetadata in project incubator-gobblin by apache.

the class EncryptedSerializedRecordToSerializedRecordConverterBase method convertRecord.

@Override
public Iterable<RecordWithMetadata<byte[]>> convertRecord(String outputSchema, RecordWithMetadata<byte[]> inputRecord, WorkUnitState workUnit) throws DataConversionException {
    try {
        ByteArrayInputStream inputStream = new ByteArrayInputStream(inputRecord.getRecord());
        byte[] decryptedBytes;
        try (InputStream decryptedInputStream = decryptor.decodeInputStream(inputStream)) {
            decryptedBytes = IOUtils.toByteArray(decryptedInputStream);
        }
        inputRecord.getMetadata().getGlobalMetadata().addTransferEncoding(decryptor.getTag());
        RecordWithMetadata<byte[]> serializedRecord = new RecordWithMetadata<byte[]>(decryptedBytes, inputRecord.getMetadata());
        return Collections.singleton(serializedRecord);
    } catch (Exception e) {
        throw new DataConversionException(e);
    }
}

Example 19 with RecordWithMetadata

use of org.apache.gobblin.type.RecordWithMetadata in project incubator-gobblin by apache.

the class SerializedRecordToEncryptedSerializedRecordConverterBase method convertRecord.

@Override
public Iterable<RecordWithMetadata<byte[]>> convertRecord(String outputSchema, RecordWithMetadata<byte[]> inputRecord, WorkUnitState workUnit) throws DataConversionException {
    try {
        ByteArrayOutputStream bOs = new ByteArrayOutputStream();
        try (OutputStream encryptedStream = encryptor.encodeOutputStream(bOs)) {
            encryptedStream.write(inputRecord.getRecord());
        }
        inputRecord.getMetadata().getGlobalMetadata().addTransferEncoding(encryptor.getTag());
        RecordWithMetadata<byte[]> serializedRecord = new RecordWithMetadata<byte[]>(bOs.toByteArray(), inputRecord.getMetadata());
        return Collections.singleton(serializedRecord);
    } catch (Exception e) {
        throw new DataConversionException(e);
    }
}