Examples with SamzaSqlRelMessage org.apache.samza.sql.data.SamzaSqlRelMessage used on opensource projects

Example 21 with SamzaSqlRelMessage

use of org.apache.samza.sql.data.SamzaSqlRelMessage in project samza by apache.

the class ProjectTranslator method translateFlatten.

private MessageStream<SamzaSqlRelMessage> translateFlatten(Integer flattenIndex, MessageStream<SamzaSqlRelMessage> inputStream) {
    return inputStream.flatMap(message -> {
        Object targetFlattenColumn = message.getSamzaSqlRelRecord().getFieldValues().get(flattenIndex);
        final List<SamzaSqlRelMessage> outMessages = new ArrayList<>();
        if (targetFlattenColumn != null && targetFlattenColumn instanceof List) {
            List<Object> objectList = (List<Object>) targetFlattenColumn;
            SamzaSqlRelMsgMetadata messageMetadata = message.getSamzaSqlRelMsgMetadata();
            SamzaSqlRelMsgMetadata newMetadata = new SamzaSqlRelMsgMetadata(messageMetadata.getEventTime(), messageMetadata.getArrivalTime(), messageMetadata.getScanTimeNanos(), messageMetadata.getScanTimeMillis());
            for (Object fieldValue : objectList) {
                List<Object> newValues = new ArrayList<>(message.getSamzaSqlRelRecord().getFieldValues());
                newValues.set(flattenIndex, fieldValue);
                outMessages.add(new SamzaSqlRelMessage(message.getSamzaSqlRelRecord().getFieldNames(), newValues, newMetadata));
                newMetadata = new SamzaSqlRelMsgMetadata(newMetadata.getEventTime(), newMetadata.getArrivalTime(), newMetadata.getScanTimeNanos(), newMetadata.getScanTimeMillis());
            }
        } else {
            message.getSamzaSqlRelMsgMetadata().isNewInputMessage = true;
            outMessages.add(message);
        }
        return outMessages;
    });
}

Example 22 with SamzaSqlRelMessage

use of org.apache.samza.sql.data.SamzaSqlRelMessage in project samza by apache.

the class TestAvroRelConversion method testEmptyRecordConversion.

@Test
public void testEmptyRecordConversion() {
    GenericData.Record record = new GenericData.Record(SimpleRecord.SCHEMA$);
    SamzaSqlRelMessage message = simpleRecordAvroRelConverter.convertToRelMessage(new KV<>("key", record));
    Assert.assertEquals(message.getSamzaSqlRelRecord().getFieldNames().size(), message.getSamzaSqlRelRecord().getFieldValues().size());
}

Example 23 with SamzaSqlRelMessage

use of org.apache.samza.sql.data.SamzaSqlRelMessage in project samza by apache.

the class TestAvroRelConversion method testNullRecordConversion.

@Test
public void testNullRecordConversion() {
    SamzaSqlRelMessage message = simpleRecordAvroRelConverter.convertToRelMessage(new KV<>("key", null));
    Assert.assertEquals(message.getSamzaSqlRelRecord().getFieldNames().size(), message.getSamzaSqlRelRecord().getFieldValues().size());
}

Example 24 with SamzaSqlRelMessage

use of org.apache.samza.sql.data.SamzaSqlRelMessage in project samza by apache.

the class TestAvroRelConversion method testRecordConversionWithNullPayload.

// SAMZA-2110 We need to enable this when we have a true support for Null records
@Ignore
@Test
public void testRecordConversionWithNullPayload() throws IOException {
    GenericData.Record record = null;
    SamzaSqlRelMessage relMessage = nestedRecordAvroRelConverter.convertToRelMessage(new KV<>("key", record));
    LOG.info(relMessage.toString());
    KV<Object, Object> samzaMessage = nestedRecordAvroRelConverter.convertToSamzaMessage(relMessage);
    GenericRecord recordPostConversion = (GenericRecord) samzaMessage.getValue();
    Assert.assertTrue(recordPostConversion == null);
}

Example 25 with SamzaSqlRelMessage

use of org.apache.samza.sql.data.SamzaSqlRelMessage in project samza by apache.

the class TestAvroRelConversion method testSimpleRecordConversion.

@Test
public void testSimpleRecordConversion() {
    GenericData.Record record = new GenericData.Record(SimpleRecord.SCHEMA$);
    record.put("id", 1);
    record.put("name", "name1");
    SamzaSqlRelMessage message = simpleRecordAvroRelConverter.convertToRelMessage(new KV<>("key", record));
    LOG.info(message.toString());
}