Search in sources :

Example 1 with SchemaTypeAdapter

use of co.cask.cdap.internal.io.SchemaTypeAdapter in project cdap by caskdata.

the class SchemaTest method testGenerateSchema.

@Test
public void testGenerateSchema() throws UnsupportedTypeException {
    Schema schema = (new ReflectionSchemaGenerator()).generate((new TypeToken<Child<Node>>() {
    }).getType());
    Gson gson = new GsonBuilder().registerTypeAdapter(Schema.class, new SchemaTypeAdapter()).create();
    Assert.assertEquals(schema, gson.fromJson(gson.toJson(schema), Schema.class));
}
Also used : SchemaTypeAdapter(co.cask.cdap.internal.io.SchemaTypeAdapter) GsonBuilder(com.google.gson.GsonBuilder) Schema(co.cask.cdap.api.data.schema.Schema) Gson(com.google.gson.Gson) ReflectionSchemaGenerator(co.cask.cdap.internal.io.ReflectionSchemaGenerator) Test(org.junit.Test)

Example 2 with SchemaTypeAdapter

use of co.cask.cdap.internal.io.SchemaTypeAdapter in project cdap by caskdata.

the class DefaultPreviewStore method get.

@Override
public Map<String, List<JsonElement>> get(ApplicationId applicationId, String tracerName) {
    // PreviewStore is a singleton and we have to create gson for each operation since gson is not thread safe.
    Gson gson = new GsonBuilder().registerTypeAdapter(Schema.class, new SchemaTypeAdapter()).create();
    byte[] startRowKey = new MDSKey.Builder().add(applicationId.getNamespace()).add(applicationId.getApplication()).add(tracerName).build().getKey();
    byte[] stopRowKey = new MDSKey(Bytes.stopKeyForPrefix(startRowKey)).getKey();
    Map<String, List<JsonElement>> result = new HashMap<>();
    try (Scanner scanner = table.scan(startRowKey, stopRowKey, null, null, null)) {
        Row indexRow;
        while ((indexRow = scanner.next()) != null) {
            Map<byte[], byte[]> columns = indexRow.getColumns();
            String propertyName = Bytes.toString(columns.get(PROPERTY));
            JsonElement value = gson.fromJson(Bytes.toString(columns.get(VALUE)), JsonElement.class);
            List<JsonElement> values = result.get(propertyName);
            if (values == null) {
                values = new ArrayList<>();
                result.put(propertyName, values);
            }
            values.add(value);
        }
    } catch (IOException e) {
        String message = String.format("Error while reading preview data for application '%s' and tracer '%s'.", applicationId, tracerName);
        throw new RuntimeException(message, e);
    }
    return result;
}
Also used : Scanner(co.cask.cdap.api.dataset.table.Scanner) GsonBuilder(com.google.gson.GsonBuilder) HashMap(java.util.HashMap) Schema(co.cask.cdap.api.data.schema.Schema) Gson(com.google.gson.Gson) MDSKey(co.cask.cdap.data2.dataset2.lib.table.MDSKey) IOException(java.io.IOException) SchemaTypeAdapter(co.cask.cdap.internal.io.SchemaTypeAdapter) JsonElement(com.google.gson.JsonElement) ArrayList(java.util.ArrayList) List(java.util.List) Row(co.cask.cdap.api.dataset.table.Row)

Example 3 with SchemaTypeAdapter

use of co.cask.cdap.internal.io.SchemaTypeAdapter in project cdap by caskdata.

the class StreamDataFileReader method verifySchema.

private void verifySchema(Map<String, String> properties) throws IOException {
    String schemaKey = StreamDataFileConstants.Property.Key.SCHEMA;
    String schemaStr = properties.get(schemaKey);
    if (schemaStr == null) {
        throw new IOException("Missing '" + schemaKey + "' property.");
    }
    try {
        Schema schema = new SchemaTypeAdapter().read(new JsonReader(new StringReader(schemaStr)));
        if (!StreamEventDataCodec.STREAM_DATA_SCHEMA.equals(schema)) {
            throw new IOException("Unsupported schema " + schemaStr);
        }
    } catch (JsonSyntaxException e) {
        throw new IOException("Invalid schema.", e);
    }
}
Also used : SchemaTypeAdapter(co.cask.cdap.internal.io.SchemaTypeAdapter) JsonSyntaxException(com.google.gson.JsonSyntaxException) Schema(co.cask.cdap.api.data.schema.Schema) StringReader(java.io.StringReader) JsonReader(com.google.gson.stream.JsonReader) IOException(java.io.IOException)

Example 4 with SchemaTypeAdapter

use of co.cask.cdap.internal.io.SchemaTypeAdapter in project cdap by caskdata.

the class DefaultPreviewStore method put.

@Override
public void put(ApplicationId applicationId, String tracerName, String propertyName, Object value) {
    // PreviewStore is a singleton and we have to create gson for each operation since gson is not thread safe.
    Gson gson = new GsonBuilder().registerTypeAdapter(Schema.class, new SchemaTypeAdapter()).create();
    MDSKey mdsKey = new MDSKey.Builder().add(applicationId.getNamespace()).add(applicationId.getApplication()).add(tracerName).add(counter.getAndIncrement()).build();
    try {
        table.put(mdsKey.getKey(), TRACER, Bytes.toBytes(tracerName), 1L);
        table.put(mdsKey.getKey(), PROPERTY, Bytes.toBytes(propertyName), 1L);
        table.put(mdsKey.getKey(), VALUE, Bytes.toBytes(gson.toJson(value)), 1L);
    } catch (IOException e) {
        String message = String.format("Error while putting property '%s' for application '%s' and tracer '%s' in" + " preview table.", propertyName, applicationId, tracerName);
        throw new RuntimeException(message, e);
    }
}
Also used : SchemaTypeAdapter(co.cask.cdap.internal.io.SchemaTypeAdapter) GsonBuilder(com.google.gson.GsonBuilder) Schema(co.cask.cdap.api.data.schema.Schema) Gson(com.google.gson.Gson) MDSKey(co.cask.cdap.data2.dataset2.lib.table.MDSKey) IOException(java.io.IOException)

Aggregations

Schema (co.cask.cdap.api.data.schema.Schema)4 SchemaTypeAdapter (co.cask.cdap.internal.io.SchemaTypeAdapter)4 Gson (com.google.gson.Gson)3 GsonBuilder (com.google.gson.GsonBuilder)3 IOException (java.io.IOException)3 MDSKey (co.cask.cdap.data2.dataset2.lib.table.MDSKey)2 Row (co.cask.cdap.api.dataset.table.Row)1 Scanner (co.cask.cdap.api.dataset.table.Scanner)1 ReflectionSchemaGenerator (co.cask.cdap.internal.io.ReflectionSchemaGenerator)1 JsonElement (com.google.gson.JsonElement)1 JsonSyntaxException (com.google.gson.JsonSyntaxException)1 JsonReader (com.google.gson.stream.JsonReader)1 StringReader (java.io.StringReader)1 ArrayList (java.util.ArrayList)1 HashMap (java.util.HashMap)1 List (java.util.List)1 Test (org.junit.Test)1