Search in sources :

Example 11 with ReflectionSchemaGenerator

use of co.cask.cdap.internal.io.ReflectionSchemaGenerator in project cdap by caskdata.

the class ReflectionTableTest method testPutAndGet.

@Test
public void testPutAndGet() throws Exception {
    dsFrameworkUtil.createInstance("table", users, DatasetProperties.builder().build());
    try {
        final Table usersTable = dsFrameworkUtil.getInstance(users);
        final byte[] rowKey = Bytes.toBytes(123);
        final Schema schema = new ReflectionSchemaGenerator().generate(User.class);
        assertGetAndPut(usersTable, rowKey, SAMUEL, schema);
    } finally {
        dsFrameworkUtil.deleteInstance(users);
    }
}
Also used : Table(co.cask.cdap.api.dataset.table.Table) Schema(co.cask.cdap.api.data.schema.Schema) ReflectionSchemaGenerator(co.cask.cdap.internal.io.ReflectionSchemaGenerator) Test(org.junit.Test)

Example 12 with ReflectionSchemaGenerator

use of co.cask.cdap.internal.io.ReflectionSchemaGenerator in project cdap by caskdata.

the class ReflectionTableTest method testStructuredRecordProjection.

@Test
public void testStructuredRecordProjection() throws Exception {
    dsFrameworkUtil.createInstance("table", users, DatasetProperties.builder().build());
    try {
        final Table usersTable = dsFrameworkUtil.getInstance(users);
        final byte[] rowKey = Bytes.toBytes(123);
        final User2 projected = new User2("Samuel L.", 123L, ((Float) 50000000.02f).doubleValue(), Double.MAX_VALUE, ByteBuffer.wrap(new byte[] { 0, 1, 2 }));
        final Schema fullSchema = new ReflectionSchemaGenerator().generate(User.class);
        final Schema projSchema = new ReflectionSchemaGenerator().generate(User2.class);
        // TableDataset is not accessible here, but we know that's the underlying implementation...
        TransactionExecutor tx = dsFrameworkUtil.newTransactionExecutor((TransactionAware) usersTable);
        tx.execute(new TransactionExecutor.Subroutine() {

            @Override
            public void apply() throws Exception {
                Put put = new Put(rowKey);
                ReflectionPutWriter<User> putWriter = new ReflectionPutWriter<>(fullSchema);
                putWriter.write(SAMUEL, put);
                usersTable.put(put);
                Row row = usersTable.get(rowKey);
                ReflectionRowRecordReader rowReader = new ReflectionRowRecordReader(projSchema, null);
                StructuredRecord actual = rowReader.read(row, fullSchema);
                assertRecordEqualsUser(projected, actual);
            }
        });
    } finally {
        dsFrameworkUtil.deleteInstance(users);
    }
}
Also used : Table(co.cask.cdap.api.dataset.table.Table) Schema(co.cask.cdap.api.data.schema.Schema) TransactionExecutor(org.apache.tephra.TransactionExecutor) ReflectionSchemaGenerator(co.cask.cdap.internal.io.ReflectionSchemaGenerator) Put(co.cask.cdap.api.dataset.table.Put) StructuredRecord(co.cask.cdap.api.data.format.StructuredRecord) ReflectionPutWriter(co.cask.cdap.internal.io.ReflectionPutWriter) Row(co.cask.cdap.api.dataset.table.Row) ReflectionRowRecordReader(co.cask.cdap.internal.io.ReflectionRowRecordReader) Test(org.junit.Test)

Example 13 with ReflectionSchemaGenerator

use of co.cask.cdap.internal.io.ReflectionSchemaGenerator in project cdap by caskdata.

the class ApplicationVerificationTest method testGoodApplication.

/**
 * Good test
 */
@Test
public void testGoodApplication() throws Exception {
    ApplicationSpecification appSpec = Specifications.from(new WebCrawlApp());
    ApplicationSpecificationAdapter adapter = ApplicationSpecificationAdapter.create(new ReflectionSchemaGenerator());
    ApplicationSpecification newSpec = adapter.fromJson(adapter.toJson(appSpec));
    ApplicationVerification app = new ApplicationVerification();
    VerifyResult result = app.verify(new ApplicationId("test", newSpec.getName()), newSpec);
    Assert.assertTrue(result.getMessage(), result.getStatus() == VerifyResult.Status.SUCCESS);
}
Also used : ApplicationSpecification(co.cask.cdap.api.app.ApplicationSpecification) WebCrawlApp(co.cask.cdap.WebCrawlApp) ApplicationSpecificationAdapter(co.cask.cdap.internal.app.ApplicationSpecificationAdapter) ReflectionSchemaGenerator(co.cask.cdap.internal.io.ReflectionSchemaGenerator) VerifyResult(co.cask.cdap.app.verification.VerifyResult) ApplicationId(co.cask.cdap.proto.id.ApplicationId) Test(org.junit.Test)

Example 14 with ReflectionSchemaGenerator

use of co.cask.cdap.internal.io.ReflectionSchemaGenerator in project cdap by caskdata.

the class FlowVerificationTest method testFlowMissingConnection.

/**
 * This test that verification of flow connections
 */
@Test
public void testFlowMissingConnection() throws Exception {
    ApplicationSpecification appSpec = Specifications.from(new NoConsumerApp());
    ApplicationSpecificationAdapter adapter = ApplicationSpecificationAdapter.create(new ReflectionSchemaGenerator());
    ApplicationSpecification newSpec = adapter.fromJson(adapter.toJson(appSpec));
    FlowVerification flowVerifier = new FlowVerification();
    for (FlowSpecification flowSpec : appSpec.getFlows().values()) {
        VerifyResult result = flowVerifier.verify(new ApplicationId("test", newSpec.getName()), flowSpec);
        Assert.assertTrue(result.getStatus() == VerifyResult.Status.FAILED);
    }
}
Also used : ApplicationSpecification(co.cask.cdap.api.app.ApplicationSpecification) FlowSpecification(co.cask.cdap.api.flow.FlowSpecification) ApplicationSpecificationAdapter(co.cask.cdap.internal.app.ApplicationSpecificationAdapter) ReflectionSchemaGenerator(co.cask.cdap.internal.io.ReflectionSchemaGenerator) VerifyResult(co.cask.cdap.app.verification.VerifyResult) ApplicationId(co.cask.cdap.proto.id.ApplicationId) Test(org.junit.Test)

Example 15 with ReflectionSchemaGenerator

use of co.cask.cdap.internal.io.ReflectionSchemaGenerator in project cdap by caskdata.

the class FlowVerificationTest method testFlowWithMoreOutputThanWhatInputCanHandle.

@Test
public void testFlowWithMoreOutputThanWhatInputCanHandle() throws Exception {
    ApplicationSpecification appSpec = Specifications.from(new WebCrawlApp());
    ApplicationSpecificationAdapter adapter = ApplicationSpecificationAdapter.create(new ReflectionSchemaGenerator());
    ApplicationSpecification newSpec = adapter.fromJson(adapter.toJson(appSpec));
    FlowVerification flowSpec = new FlowVerification();
    for (Map.Entry<String, FlowSpecification> entry : newSpec.getFlows().entrySet()) {
        VerifyResult result = flowSpec.verify(new ApplicationId("test", newSpec.getName()), entry.getValue());
        // that is not connected to any input to flowlet CountByField.
        if (entry.getValue().getName().equals("WordCountFlow")) {
            Assert.assertTrue(result.getStatus() == VerifyResult.Status.FAILED);
        } else {
            Assert.assertTrue(result.getStatus() == VerifyResult.Status.SUCCESS);
        }
    }
}
Also used : ApplicationSpecification(co.cask.cdap.api.app.ApplicationSpecification) WebCrawlApp(co.cask.cdap.WebCrawlApp) FlowSpecification(co.cask.cdap.api.flow.FlowSpecification) ApplicationSpecificationAdapter(co.cask.cdap.internal.app.ApplicationSpecificationAdapter) ReflectionSchemaGenerator(co.cask.cdap.internal.io.ReflectionSchemaGenerator) VerifyResult(co.cask.cdap.app.verification.VerifyResult) ApplicationId(co.cask.cdap.proto.id.ApplicationId) Map(java.util.Map) Test(org.junit.Test)

Aggregations

ReflectionSchemaGenerator (co.cask.cdap.internal.io.ReflectionSchemaGenerator)42 Test (org.junit.Test)37 Schema (co.cask.cdap.api.data.schema.Schema)23 ApplicationSpecification (co.cask.cdap.api.app.ApplicationSpecification)12 ApplicationSpecificationAdapter (co.cask.cdap.internal.app.ApplicationSpecificationAdapter)10 Id (co.cask.cdap.common.id.Id)6 BinaryDecoder (co.cask.cdap.common.io.BinaryDecoder)6 BinaryEncoder (co.cask.cdap.common.io.BinaryEncoder)6 ApplicationClass (co.cask.cdap.api.artifact.ApplicationClass)5 Location (org.apache.twill.filesystem.Location)5 Table (co.cask.cdap.api.dataset.table.Table)4 PluginClass (co.cask.cdap.api.plugin.PluginClass)4 InspectionApp (co.cask.cdap.internal.app.runtime.artifact.app.inspection.InspectionApp)4 Gson (com.google.gson.Gson)4 TransactionExecutor (org.apache.tephra.TransactionExecutor)4 WordCountApp (co.cask.cdap.WordCountApp)3 CloseableClassLoader (co.cask.cdap.api.artifact.CloseableClassLoader)3 StructuredRecord (co.cask.cdap.api.data.format.StructuredRecord)3 VerifyResult (co.cask.cdap.app.verification.VerifyResult)3 ApplicationId (co.cask.cdap.proto.id.ApplicationId)3