Search in sources :

Example 81 with ObjectMapper

use of org.codehaus.jackson.map.ObjectMapper in project pinot by linkedin.

the class DelegatingAvroKeyInputFormat method getSourceNameFromPath.

public static String getSourceNameFromPath(FileSplit fileSplit, Configuration configuration) throws IOException, JsonParseException, JsonMappingException {
    String content = configuration.get("schema.path.mapping");
    Map<String, String> schemaPathMapping = new ObjectMapper().readValue(content, MAP_STRING_STRING_TYPE);
    LOGGER.info("Schema Path Mapping: {}", schemaPathMapping);
    String sourceName = null;
    for (String path : schemaPathMapping.keySet()) {
        if (fileSplit.getPath().toString().indexOf(path) > -1) {
            sourceName = schemaPathMapping.get(path);
            break;
        }
    }
    return sourceName;
}
Also used : ObjectMapper(org.codehaus.jackson.map.ObjectMapper)

Example 82 with ObjectMapper

use of org.codehaus.jackson.map.ObjectMapper in project pinot by linkedin.

the class DelegatingAvroKeyInputFormat method createRecordReader.

public org.apache.hadoop.mapreduce.RecordReader<org.apache.avro.mapred.AvroKey<T>, NullWritable> createRecordReader(InputSplit split, TaskAttemptContext context) throws IOException, InterruptedException {
    LOGGER.info("DelegatingAvroKeyInputFormat.createRecordReader()  for split:{}", split);
    FileSplit fileSplit = (FileSplit) split;
    Configuration configuration = context.getConfiguration();
    String sourceName = getSourceNameFromPath(fileSplit, configuration);
    LOGGER.info("Source Name for path {} : {}", fileSplit.getPath(), sourceName);
    Map<String, String> schemaJSONMapping = new ObjectMapper().readValue(configuration.get("schema.json.mapping"), MAP_STRING_STRING_TYPE);
    LOGGER.info("Schema JSON Mapping: {}", schemaJSONMapping);
    String sourceSchemaJSON = schemaJSONMapping.get(sourceName);
    Schema schema = new Schema.Parser().parse(sourceSchemaJSON);
    return new AvroKeyRecordReader<T>(schema);
}
Also used : Configuration(org.apache.hadoop.conf.Configuration) Schema(org.apache.avro.Schema) FileSplit(org.apache.hadoop.mapreduce.lib.input.FileSplit) ObjectMapper(org.codehaus.jackson.map.ObjectMapper) AvroKeyRecordReader(org.apache.avro.mapreduce.AvroKeyRecordReader)

Example 83 with ObjectMapper

use of org.codehaus.jackson.map.ObjectMapper in project pinot by linkedin.

the class DelegatingAvroKeyInputFormat method getSourceNameFromPath.

public static String getSourceNameFromPath(FileSplit fileSplit, Configuration configuration) throws IOException, JsonParseException, JsonMappingException {
    String content = configuration.get("schema.path.mapping");
    Map<String, String> schemaPathMapping = new ObjectMapper().readValue(content, MAP_STRING_STRING_TYPE);
    LOGGER.info("Schema Path Mapping: {}", schemaPathMapping);
    String sourceName = null;
    for (String path : schemaPathMapping.keySet()) {
        if (fileSplit.getPath().toString().indexOf(path) > -1) {
            sourceName = schemaPathMapping.get(path);
            break;
        }
    }
    return sourceName;
}
Also used : ObjectMapper(org.codehaus.jackson.map.ObjectMapper)

Example 84 with ObjectMapper

use of org.codehaus.jackson.map.ObjectMapper in project hbase by apache.

the class TestMultiRowResource method testMultiCellGetWithColsJSON.

@Test
public void testMultiCellGetWithColsJSON() throws IOException, JAXBException {
    String row_5_url = "/" + TABLE + "/" + ROW_1 + "/" + COLUMN_1;
    String row_6_url = "/" + TABLE + "/" + ROW_2 + "/" + COLUMN_2;
    StringBuilder path = new StringBuilder();
    path.append("/");
    path.append(TABLE);
    path.append("/multiget");
    path.append("/" + COLUMN_1 + "," + CFB);
    path.append("?row=");
    path.append(ROW_1);
    path.append("&row=");
    path.append(ROW_2);
    client.post(row_5_url, Constants.MIMETYPE_BINARY, Bytes.toBytes(VALUE_1), extraHdr);
    client.post(row_6_url, Constants.MIMETYPE_BINARY, Bytes.toBytes(VALUE_2), extraHdr);
    Response response = client.get(path.toString(), Constants.MIMETYPE_JSON);
    assertEquals(response.getCode(), 200);
    ObjectMapper mapper = new JacksonJaxbJsonProvider().locateMapper(CellSetModel.class, MediaType.APPLICATION_JSON_TYPE);
    CellSetModel cellSet = (CellSetModel) mapper.readValue(response.getBody(), CellSetModel.class);
    assertEquals(2, cellSet.getRows().size());
    assertEquals(ROW_1, Bytes.toString(cellSet.getRows().get(0).getKey()));
    assertEquals(VALUE_1, Bytes.toString(cellSet.getRows().get(0).getCells().get(0).getValue()));
    assertEquals(ROW_2, Bytes.toString(cellSet.getRows().get(1).getKey()));
    assertEquals(VALUE_2, Bytes.toString(cellSet.getRows().get(1).getCells().get(0).getValue()));
    client.delete(row_5_url, extraHdr);
    client.delete(row_6_url, extraHdr);
}
Also used : Response(org.apache.hadoop.hbase.rest.client.Response) CellSetModel(org.apache.hadoop.hbase.rest.model.CellSetModel) JacksonJaxbJsonProvider(org.codehaus.jackson.jaxrs.JacksonJaxbJsonProvider) ObjectMapper(org.codehaus.jackson.map.ObjectMapper) Test(org.junit.Test)

Example 85 with ObjectMapper

use of org.codehaus.jackson.map.ObjectMapper in project hbase by apache.

the class TestTableScan method testScanningUnknownColumnJson.

@Test
public void testScanningUnknownColumnJson() throws IOException, JAXBException {
    // Test scanning particular columns with limit.
    StringBuilder builder = new StringBuilder();
    builder.append("/*");
    builder.append("?");
    builder.append(Constants.SCAN_COLUMN + "=a:test");
    Response response = client.get("/" + TABLE + builder.toString(), Constants.MIMETYPE_JSON);
    assertEquals(200, response.getCode());
    assertEquals(Constants.MIMETYPE_JSON, response.getHeader("content-type"));
    ObjectMapper mapper = new JacksonJaxbJsonProvider().locateMapper(CellSetModel.class, MediaType.APPLICATION_JSON_TYPE);
    CellSetModel model = mapper.readValue(response.getStream(), CellSetModel.class);
    int count = TestScannerResource.countCellSet(model);
    assertEquals(0, count);
}
Also used : Response(org.apache.hadoop.hbase.rest.client.Response) CellSetModel(org.apache.hadoop.hbase.rest.model.CellSetModel) JacksonJaxbJsonProvider(org.codehaus.jackson.jaxrs.JacksonJaxbJsonProvider) ObjectMapper(org.codehaus.jackson.map.ObjectMapper) Test(org.junit.Test)

Aggregations

ObjectMapper (org.codehaus.jackson.map.ObjectMapper)386 IOException (java.io.IOException)85 Test (org.junit.Test)61 JsonNode (org.codehaus.jackson.JsonNode)50 HashMap (java.util.HashMap)46 ArrayList (java.util.ArrayList)45 Test (org.testng.annotations.Test)37 Map (java.util.Map)34 List (java.util.List)26 File (java.io.File)22 StringWriter (java.io.StringWriter)22 JSONObject (org.json.JSONObject)18 SimpleModule (org.codehaus.jackson.map.module.SimpleModule)17 JsonMappingException (org.codehaus.jackson.map.JsonMappingException)16 ByteArrayInputStream (java.io.ByteArrayInputStream)14 Version (org.codehaus.jackson.Version)14 JsonFactory (org.codehaus.jackson.JsonFactory)13 JSONObject (org.codehaus.jettison.json.JSONObject)13 ByteArrayOutputStream (java.io.ByteArrayOutputStream)11 JsonGenerator (org.codehaus.jackson.JsonGenerator)11