Search in sources :

Example 31 with Put

use of co.cask.cdap.api.dataset.table.Put in project cdap by caskdata.

the class RecordPutTransformerTest method testNullableFields.

@Test
public void testNullableFields() throws Exception {
    Schema schema = Schema.recordOf("record", Schema.Field.of("key", Schema.of(Schema.Type.INT)), Schema.Field.of("nullable", Schema.nullableOf(Schema.of(Schema.Type.STRING))), Schema.Field.of("non_nullable", Schema.of(Schema.Type.STRING)));
    RecordPutTransformer transformer = new RecordPutTransformer("key", schema);
    // valid record
    StructuredRecord record = StructuredRecord.builder(schema).set("key", 1).set("non_nullable", "foo").build();
    Put transformed = transformer.toPut(record);
    Assert.assertEquals(1, Bytes.toInt(transformed.getRow()));
    // expect a null value for the nullable field
    Assert.assertEquals(2, transformed.getValues().size());
    Assert.assertEquals("foo", Bytes.toString(transformed.getValues().get(Bytes.toBytes("non_nullable"))));
    Assert.assertNull(transformed.getValues().get(Bytes.toBytes("nullable")));
}
Also used : Schema(co.cask.cdap.api.data.schema.Schema) StructuredRecord(co.cask.cdap.api.data.format.StructuredRecord) Put(co.cask.cdap.api.dataset.table.Put) Test(org.junit.Test)

Example 32 with Put

use of co.cask.cdap.api.dataset.table.Put in project cdap by caskdata.

the class RecordPutTransformerTest method testTransform.

@Test
public void testTransform() throws Exception {
    Schema schema = Schema.recordOf("record", Schema.Field.of("boolField", Schema.nullableOf(Schema.of(Schema.Type.BOOLEAN))), Schema.Field.of("intField", Schema.nullableOf(Schema.of(Schema.Type.INT))), Schema.Field.of("longField", Schema.nullableOf(Schema.of(Schema.Type.LONG))), Schema.Field.of("floatField", Schema.nullableOf(Schema.of(Schema.Type.FLOAT))), Schema.Field.of("doubleField", Schema.nullableOf(Schema.of(Schema.Type.DOUBLE))), Schema.Field.of("bytesField", Schema.nullableOf(Schema.of(Schema.Type.BYTES))), Schema.Field.of("stringField", Schema.of(Schema.Type.STRING)));
    RecordPutTransformer transformer = new RecordPutTransformer("stringField", schema);
    StructuredRecord record = StructuredRecord.builder(schema).set("boolField", true).set("intField", 5).set("longField", 10L).set("floatField", 3.14f).set("doubleField", 3.14).set("bytesField", Bytes.toBytes("foo")).set("stringField", "key").build();
    Put transformed = transformer.toPut(record);
    Assert.assertEquals("key", Bytes.toString(transformed.getRow()));
    Map<byte[], byte[]> values = transformed.getValues();
    Assert.assertTrue(Bytes.toBoolean(values.get(Bytes.toBytes("boolField"))));
    Assert.assertEquals(5, Bytes.toInt(values.get(Bytes.toBytes("intField"))));
    Assert.assertEquals(10L, Bytes.toLong(values.get(Bytes.toBytes("longField"))));
    Assert.assertTrue(Math.abs(3.14f - Bytes.toFloat(values.get(Bytes.toBytes("floatField")))) < 0.000001);
    Assert.assertTrue(Math.abs(3.14 - Bytes.toDouble(values.get(Bytes.toBytes("doubleField")))) < 0.000001);
    Assert.assertArrayEquals(Bytes.toBytes("foo"), values.get(Bytes.toBytes("bytesField")));
}
Also used : Schema(co.cask.cdap.api.data.schema.Schema) StructuredRecord(co.cask.cdap.api.data.format.StructuredRecord) Put(co.cask.cdap.api.dataset.table.Put) Test(org.junit.Test)

Example 33 with Put

use of co.cask.cdap.api.dataset.table.Put in project cdap by caskdata.

the class MockRuntimeDatasetSink method transform.

@Override
public void transform(StructuredRecord input, Emitter<KeyValue<byte[], Put>> emitter) throws Exception {
    byte[] rowkey = Bytes.toBytes(UUID.randomUUID());
    Put put = new Put(rowkey);
    put.add(SCHEMA_COL, input.getSchema().toString());
    put.add(RECORD_COL, StructuredRecordStringConverter.toJsonString(input));
    emitter.emit(new KeyValue<>(rowkey, put));
}
Also used : Put(co.cask.cdap.api.dataset.table.Put)

Example 34 with Put

use of co.cask.cdap.api.dataset.table.Put in project cdap by caskdata.

the class MockAction method run.

@Override
public void run(ActionContext context) throws Exception {
    context.execute(new TxRunnable() {

        @Override
        public void run(DatasetContext context) throws Exception {
            Table table = context.getDataset(config.tableName);
            Put put = new Put(config.rowKey);
            put.add(config.columnKey, config.value);
            table.put(put);
        }
    });
    // Set the same value in the arguments as well.
    context.getArguments().set(config.rowKey + config.columnKey, config.value);
}
Also used : Table(co.cask.cdap.api.dataset.table.Table) TxRunnable(co.cask.cdap.api.TxRunnable) DatasetContext(co.cask.cdap.api.data.DatasetContext) Put(co.cask.cdap.api.dataset.table.Put)

Example 35 with Put

use of co.cask.cdap.api.dataset.table.Put in project cdap by caskdata.

the class NodeStatesAction method run.

@Override
public void run(BatchActionContext context) throws Exception {
    Table table = context.getDataset(conf.tableName);
    for (Map.Entry<String, WorkflowNodeState> entry : context.getNodeStates().entrySet()) {
        Put put = new Put(entry.getKey());
        WorkflowNodeState nodeState = entry.getValue();
        put.add("runid", nodeState.getRunId());
        put.add("nodeid", nodeState.getNodeId());
        put.add("status", nodeState.getNodeStatus().name());
        table.put(put);
    }
}
Also used : WorkflowNodeState(co.cask.cdap.api.workflow.WorkflowNodeState) Table(co.cask.cdap.api.dataset.table.Table) ImmutableMap(com.google.common.collect.ImmutableMap) Map(java.util.Map) Put(co.cask.cdap.api.dataset.table.Put)

Aggregations

Put (co.cask.cdap.api.dataset.table.Put)52 Test (org.junit.Test)21 Table (co.cask.cdap.api.dataset.table.Table)20 Row (co.cask.cdap.api.dataset.table.Row)14 Get (co.cask.cdap.api.dataset.table.Get)13 TransactionExecutor (org.apache.tephra.TransactionExecutor)10 Schema (co.cask.cdap.api.data.schema.Schema)9 TransactionAware (org.apache.tephra.TransactionAware)9 Map (java.util.Map)8 Transaction (org.apache.tephra.Transaction)8 StructuredRecord (co.cask.cdap.api.data.format.StructuredRecord)7 IOException (java.io.IOException)7 DatasetAdmin (co.cask.cdap.api.dataset.DatasetAdmin)6 HBaseTable (co.cask.cdap.data2.dataset2.lib.table.hbase.HBaseTable)6 WriteOnly (co.cask.cdap.api.annotation.WriteOnly)5 DataSetException (co.cask.cdap.api.dataset.DataSetException)5 Scanner (co.cask.cdap.api.dataset.table.Scanner)5 KeyValueTable (co.cask.cdap.api.dataset.lib.KeyValueTable)4 ReflectionPutWriter (co.cask.cdap.internal.io.ReflectionPutWriter)4 ImmutableMap (com.google.common.collect.ImmutableMap)4