Search in sources :

Example 1 with ReflectionSchemaGenerator

use of co.cask.cdap.internal.io.ReflectionSchemaGenerator in project cdap by caskdata.

the class ScheduleSpecificationCodecTest method testAppConfigurerRoute.

@Test
public void testAppConfigurerRoute() throws Exception {
    Application app = new AbstractApplication() {

        @Override
        public void configure() {
            // intentionally use the deprecated scheduleWorkflow method to for timeSchedule
            // to test TimeSchedule deserialization
            scheduleWorkflow(Schedules.builder("timeSchedule").createTimeSchedule("0 * * * *"), "workflow");
            scheduleWorkflow(Schedules.builder("streamSizeSchedule").createDataSchedule(Schedules.Source.STREAM, "stream", 1), "workflow");
        }
    };
    ApplicationSpecification specification = Specifications.from(app);
    ApplicationSpecificationAdapter gsonAdapater = ApplicationSpecificationAdapter.create(new ReflectionSchemaGenerator());
    String jsonStr = gsonAdapater.toJson(specification);
    ApplicationSpecification deserializedSpec = gsonAdapater.fromJson(jsonStr);
    Assert.assertEquals(new TimeSchedule("timeSchedule", "", "0 * * * *"), deserializedSpec.getSchedules().get("timeSchedule").getSchedule());
    Assert.assertEquals(new StreamSizeSchedule("streamSizeSchedule", "", "stream", 1), deserializedSpec.getSchedules().get("streamSizeSchedule").getSchedule());
}
Also used : ApplicationSpecification(co.cask.cdap.api.app.ApplicationSpecification) AbstractApplication(co.cask.cdap.api.app.AbstractApplication) TimeSchedule(co.cask.cdap.internal.schedule.TimeSchedule) ReflectionSchemaGenerator(co.cask.cdap.internal.io.ReflectionSchemaGenerator) StreamSizeSchedule(co.cask.cdap.internal.schedule.StreamSizeSchedule) Application(co.cask.cdap.api.app.Application) AbstractApplication(co.cask.cdap.api.app.AbstractApplication) Test(org.junit.Test)

Example 2 with ReflectionSchemaGenerator

use of co.cask.cdap.internal.io.ReflectionSchemaGenerator in project cdap by caskdata.

the class RecordWithString method testEmptyValue.

// this tests that the datum reader treats empty fields correctly. It reproduces the issue in ENG-2404.
@Test
public void testEmptyValue() throws UnsupportedTypeException, IOException {
    Schema schema = new ReflectionSchemaGenerator().generate(RecordWithString.class);
    TypeRepresentation typeRep = new TypeRepresentation(RecordWithString.class);
    DatumWriter<RecordWithString> datumWriter = new ReflectionDatumWriter<>(schema);
    @SuppressWarnings("unchecked") ReflectionDatumReader<RecordWithString> datumReader = new ReflectionDatumReader<>(schema, (TypeToken<RecordWithString>) TypeToken.of(typeRep.toType()));
    RecordWithString record = new RecordWithString();
    record.setA(42);
    record.setTheString("");
    ByteArrayOutputStream bos = new ByteArrayOutputStream();
    BinaryEncoder encoder = new BinaryEncoder(bos);
    datumWriter.encode(record, encoder);
    byte[] bytes = bos.toByteArray();
    ByteArrayInputStream bis = new ByteArrayInputStream(bytes);
    BinaryDecoder decoder = new BinaryDecoder(bis);
    RecordWithString rec = datumReader.read(decoder, schema);
    Assert.assertEquals(record.getA(), rec.getA());
    Assert.assertEquals(record.getTheString(), rec.getTheString());
}
Also used : Schema(co.cask.cdap.api.data.schema.Schema) ReflectionDatumWriter(co.cask.cdap.internal.io.ReflectionDatumWriter) ReflectionDatumReader(co.cask.cdap.internal.io.ReflectionDatumReader) ByteArrayOutputStream(java.io.ByteArrayOutputStream) ReflectionSchemaGenerator(co.cask.cdap.internal.io.ReflectionSchemaGenerator) BinaryDecoder(co.cask.cdap.common.io.BinaryDecoder) BinaryEncoder(co.cask.cdap.common.io.BinaryEncoder) ByteArrayInputStream(java.io.ByteArrayInputStream) TypeRepresentation(co.cask.cdap.internal.io.TypeRepresentation) Test(org.junit.Test)

Example 3 with ReflectionSchemaGenerator

use of co.cask.cdap.internal.io.ReflectionSchemaGenerator in project cdap by caskdata.

the class RecordWithString method testReduceProjection.

@Test
public void testReduceProjection() throws IOException, UnsupportedTypeException {
    PipedOutputStream output = new PipedOutputStream();
    PipedInputStream input = new PipedInputStream(output);
    Schema sourceSchema = new ReflectionSchemaGenerator().generate(MoreFields.class);
    Schema targetSchema = new ReflectionSchemaGenerator().generate(LessFields.class);
    MoreFields moreFields = new MoreFields(10, 20.2, "30", ImmutableList.of("1", "2"));
    new ReflectionDatumWriter<MoreFields>(sourceSchema).encode(moreFields, new BinaryEncoder(output));
    LessFields lessFields = new ReflectionDatumReader<>(targetSchema, TypeToken.of(LessFields.class)).read(new BinaryDecoder(input), sourceSchema);
    Assert.assertEquals("30", lessFields.k);
    Assert.assertEquals(moreFields.inner.b, lessFields.inner.b);
}
Also used : BinaryEncoder(co.cask.cdap.common.io.BinaryEncoder) Schema(co.cask.cdap.api.data.schema.Schema) PipedOutputStream(java.io.PipedOutputStream) PipedInputStream(java.io.PipedInputStream) ReflectionSchemaGenerator(co.cask.cdap.internal.io.ReflectionSchemaGenerator) BinaryDecoder(co.cask.cdap.common.io.BinaryDecoder) Test(org.junit.Test)

Example 4 with ReflectionSchemaGenerator

use of co.cask.cdap.internal.io.ReflectionSchemaGenerator in project cdap by caskdata.

the class SchemaTest method testGenerateSchema.

@Test
public void testGenerateSchema() throws UnsupportedTypeException {
    Schema schema = (new ReflectionSchemaGenerator()).generate((new TypeToken<Child<Node>>() {
    }).getType());
    Gson gson = new GsonBuilder().registerTypeAdapter(Schema.class, new SchemaTypeAdapter()).create();
    Assert.assertEquals(schema, gson.fromJson(gson.toJson(schema), Schema.class));
}
Also used : SchemaTypeAdapter(co.cask.cdap.internal.io.SchemaTypeAdapter) GsonBuilder(com.google.gson.GsonBuilder) Schema(co.cask.cdap.api.data.schema.Schema) Gson(com.google.gson.Gson) ReflectionSchemaGenerator(co.cask.cdap.internal.io.ReflectionSchemaGenerator) Test(org.junit.Test)

Example 5 with ReflectionSchemaGenerator

use of co.cask.cdap.internal.io.ReflectionSchemaGenerator in project cdap by caskdata.

the class SchemaTest method testParseJson.

@Test
public void testParseJson() throws IOException, UnsupportedTypeException {
    Schema schema = new ReflectionSchemaGenerator().generate(Node.class);
    Assert.assertEquals(schema, Schema.parseJson(schema.toString()));
}
Also used : Schema(co.cask.cdap.api.data.schema.Schema) ReflectionSchemaGenerator(co.cask.cdap.internal.io.ReflectionSchemaGenerator) Test(org.junit.Test)

Aggregations

ReflectionSchemaGenerator (co.cask.cdap.internal.io.ReflectionSchemaGenerator)42 Test (org.junit.Test)37 Schema (co.cask.cdap.api.data.schema.Schema)23 ApplicationSpecification (co.cask.cdap.api.app.ApplicationSpecification)12 ApplicationSpecificationAdapter (co.cask.cdap.internal.app.ApplicationSpecificationAdapter)10 Id (co.cask.cdap.common.id.Id)6 BinaryDecoder (co.cask.cdap.common.io.BinaryDecoder)6 BinaryEncoder (co.cask.cdap.common.io.BinaryEncoder)6 ApplicationClass (co.cask.cdap.api.artifact.ApplicationClass)5 Location (org.apache.twill.filesystem.Location)5 Table (co.cask.cdap.api.dataset.table.Table)4 PluginClass (co.cask.cdap.api.plugin.PluginClass)4 InspectionApp (co.cask.cdap.internal.app.runtime.artifact.app.inspection.InspectionApp)4 Gson (com.google.gson.Gson)4 TransactionExecutor (org.apache.tephra.TransactionExecutor)4 WordCountApp (co.cask.cdap.WordCountApp)3 CloseableClassLoader (co.cask.cdap.api.artifact.CloseableClassLoader)3 StructuredRecord (co.cask.cdap.api.data.format.StructuredRecord)3 VerifyResult (co.cask.cdap.app.verification.VerifyResult)3 ApplicationId (co.cask.cdap.proto.id.ApplicationId)3