Examples with StringWindow io.georocket.util.StringWindow used on opensource projects

Example 1 with StringWindow

use of io.georocket.util.StringWindow in project georocket by georocket.

the class ImporterVerticle method importJSON.

/**
 * Imports a JSON file from the given input stream into the store
 * @param f the JSON file to read
 * @param correlationId a unique identifier for this import process
 * @param filename the name of the file currently being imported
 * @param timestamp denotes when the import process has started
 * @param layer the layer where the file should be stored (may be null)
 * @param tags the list of tags to attach to the file (may be null)
 * @param properties the map of properties to attach to the file (may be null)
 * @return a single that will emit when the file has been imported
 */
protected Single<Integer> importJSON(ReadStream<Buffer> f, String correlationId, String filename, long timestamp, String layer, List<String> tags, Map<String, Object> properties) {
    UTF8BomFilter bomFilter = new UTF8BomFilter();
    StringWindow window = new StringWindow();
    GeoJsonSplitter splitter = new GeoJsonSplitter(window);
    AtomicInteger processing = new AtomicInteger(0);
    return f.toObservable().map(buf -> (io.vertx.core.buffer.Buffer) buf.getDelegate()).map(bomFilter::filter).doOnNext(window::append).lift(new JsonParserOperator()).flatMap(splitter::onEventObservable).flatMapSingle(result -> {
        IndexMeta indexMeta = new IndexMeta(correlationId, filename, timestamp, tags, properties, null);
        return addToStoreWithPause(result, layer, indexMeta, f, processing);
    }).count().toSingle();
}

Example 2 with StringWindow

use of io.georocket.util.StringWindow in project georocket by georocket.

the class GeoJsonSplitterTest method split.

private List<Tuple2<GeoJsonChunkMeta, JsonObject>> split(String file) throws IOException {
    byte[] json = IOUtils.toByteArray(GeoJsonSplitterTest.class.getResource(file));
    List<Tuple2<GeoJsonChunkMeta, JsonObject>> chunks = new ArrayList<>();
    StringWindow window = new StringWindow();
    GeoJsonSplitter splitter = new GeoJsonSplitter(window);
    Observable.just(json).map(Buffer::buffer).doOnNext(window::append).lift(new JsonParserOperator()).flatMap(splitter::onEventObservable).toBlocking().forEach(result -> {
        JsonObject o = new JsonObject(result.getChunk());
        chunks.add(Tuple.tuple((GeoJsonChunkMeta) result.getMeta(), o));
    });
    return chunks;
}