Search in sources :

Example 1 with ElasticsearchClientFactory

use of io.georocket.index.elasticsearch.ElasticsearchClientFactory in project georocket by georocket.

the class MetadataVerticle method start.

@Override
public void start(Future<Void> startFuture) {
    // load and copy all indexer factories now and not lazily to avoid
    // concurrent modifications to the service loader's internal cache
    indexerFactories = ImmutableList.copyOf(ServiceLoader.load(IndexerFactory.class));
    queryCompiler = createQueryCompiler();
    queryCompiler.setQueryCompilers(indexerFactories);
    new ElasticsearchClientFactory(vertx).createElasticsearchClient(INDEX_NAME).doOnNext(es -> {
        client = es;
    }).flatMap(v -> client.ensureIndex()).flatMap(v -> ensureMapping()).subscribe(es -> {
        registerMessageConsumers();
        startFuture.complete();
    }, startFuture::fail);
}
Also used : URL(java.net.URL) TimeoutException(java.util.concurrent.TimeoutException) HashMap(java.util.HashMap) Function(java.util.function.Function) DefaultQueryCompiler(io.georocket.query.DefaultQueryCompiler) LoggerFactory(io.vertx.core.logging.LoggerFactory) AbstractVerticle(io.vertx.rxjava.core.AbstractVerticle) Observable(rx.Observable) Single(rx.Single) ImmutableList(com.google.common.collect.ImmutableList) Map(java.util.Map) JsonObject(io.vertx.core.json.JsonObject) Logger(io.vertx.core.logging.Logger) Resources(com.google.common.io.Resources) ThrowableHelper.throwableToMessage(io.georocket.util.ThrowableHelper.throwableToMessage) IOException(java.io.IOException) ServiceLoader(java.util.ServiceLoader) Future(io.vertx.core.Future) Collectors(java.util.stream.Collectors) FileNotFoundException(java.io.FileNotFoundException) StandardCharsets(java.nio.charset.StandardCharsets) Objects(java.util.Objects) JsonArray(io.vertx.core.json.JsonArray) ElasticsearchClientFactory(io.georocket.index.elasticsearch.ElasticsearchClientFactory) List(java.util.List) ElasticsearchClient(io.georocket.index.elasticsearch.ElasticsearchClient) MapUtils(io.georocket.util.MapUtils) ThrowableHelper.throwableToCode(io.georocket.util.ThrowableHelper.throwableToCode) AddressConstants(io.georocket.constants.AddressConstants) DefaultMetaIndexerFactory(io.georocket.index.generic.DefaultMetaIndexerFactory) ConfigConstants(io.georocket.constants.ConfigConstants) ElasticsearchClientFactory(io.georocket.index.elasticsearch.ElasticsearchClientFactory)

Example 2 with ElasticsearchClientFactory

use of io.georocket.index.elasticsearch.ElasticsearchClientFactory in project georocket by georocket.

the class IndexerVerticle method start.

@Override
public void start(Future<Void> startFuture) {
    // True if the indexer and other verticles should report their activities
    // to the Vert.x event bus (mostly useful for GeoRocket plug-ins)
    reportActivities = config().getBoolean(ConfigConstants.REPORT_ACTIVITIES, false);
    maxBulkSize = config().getInteger(ConfigConstants.INDEX_MAX_BULK_SIZE, ConfigConstants.DEFAULT_INDEX_MAX_BULK_SIZE);
    maxParallelInserts = config().getInteger(ConfigConstants.INDEX_MAX_PARALLEL_INSERTS, ConfigConstants.DEFAULT_INDEX_MAX_PARALLEL_INSERTS);
    // load and copy all indexer factories now and not lazily to avoid
    // concurrent modifications to the service loader's internal cache
    indexerFactories = ImmutableList.copyOf(ServiceLoader.load(IndexerFactory.class));
    xmlIndexerFactories = ImmutableList.copyOf(Seq.seq(indexerFactories).filter(f -> f instanceof XMLIndexerFactory).cast(XMLIndexerFactory.class));
    jsonIndexerFactories = ImmutableList.copyOf(Seq.seq(indexerFactories).filter(f -> f instanceof JsonIndexerFactory).cast(JsonIndexerFactory.class));
    metaIndexerFactories = ImmutableList.copyOf(Seq.seq(indexerFactories).filter(f -> f instanceof MetaIndexerFactory).cast(MetaIndexerFactory.class));
    store = new RxStore(StoreFactory.createStore(getVertx()));
    queryCompiler = createQueryCompiler();
    queryCompiler.setQueryCompilers(indexerFactories);
    new ElasticsearchClientFactory(vertx).createElasticsearchClient(INDEX_NAME).doOnNext(es -> {
        client = es;
    }).flatMap(v -> client.ensureIndex()).flatMap(v -> ensureMapping()).subscribe(es -> {
        registerMessageConsumers();
        startFuture.complete();
    }, err -> {
        startFuture.fail(err);
    });
}
Also used : MetaIndexer(io.georocket.index.xml.MetaIndexer) GeoJsonChunkMeta(io.georocket.storage.GeoJsonChunkMeta) IndexMeta(io.georocket.storage.IndexMeta) StreamEvent(io.georocket.util.StreamEvent) XMLChunkMeta(io.georocket.storage.XMLChunkMeta) XMLParserOperator(io.georocket.util.XMLParserOperator) ChunkMeta(io.georocket.storage.ChunkMeta) RxStore(io.georocket.storage.RxStore) Tuple2(org.jooq.lambda.tuple.Tuple2) Tuple3(org.jooq.lambda.tuple.Tuple3) JsonParserOperator(io.georocket.util.JsonParserOperator) Map(java.util.Map) JsonObject(io.vertx.core.json.JsonObject) Logger(io.vertx.core.logging.Logger) MetaIndexerFactory(io.georocket.index.xml.MetaIndexerFactory) Message(io.vertx.rxjava.core.eventbus.Message) ServiceLoader(java.util.ServiceLoader) Collectors(java.util.stream.Collectors) Future(io.vertx.core.Future) List(java.util.List) ElasticsearchClientFactory(io.georocket.index.elasticsearch.ElasticsearchClientFactory) Stream(java.util.stream.Stream) Tuple(org.jooq.lambda.tuple.Tuple) MapUtils(io.georocket.util.MapUtils) Buffer(io.vertx.core.buffer.Buffer) MimeTypeUtils.belongsTo(io.georocket.util.MimeTypeUtils.belongsTo) RxHelper(io.vertx.rx.java.RxHelper) AddressConstants(io.georocket.constants.AddressConstants) ChunkReadStream(io.georocket.storage.ChunkReadStream) Operator(rx.Observable.Operator) HashMap(java.util.HashMap) Seq(org.jooq.lambda.Seq) DefaultQueryCompiler(io.georocket.query.DefaultQueryCompiler) LoggerFactory(io.vertx.core.logging.LoggerFactory) ArrayList(java.util.ArrayList) AbstractVerticle(io.vertx.rxjava.core.AbstractVerticle) Observable(rx.Observable) Func1(rx.functions.Func1) ImmutableList(com.google.common.collect.ImmutableList) XMLIndexerFactory(io.georocket.index.xml.XMLIndexerFactory) StoreFactory(io.georocket.storage.StoreFactory) JsonIndexerFactory(io.georocket.index.xml.JsonIndexerFactory) NoStackTraceThrowable(io.vertx.core.impl.NoStackTraceThrowable) ThrowableHelper.throwableToMessage(io.georocket.util.ThrowableHelper.throwableToMessage) StreamIndexer(io.georocket.index.xml.StreamIndexer) TimeUnit(java.util.concurrent.TimeUnit) JsonArray(io.vertx.core.json.JsonArray) ThrowableHelper.throwableToCode(io.georocket.util.ThrowableHelper.throwableToCode) ElasticsearchClient(io.georocket.index.elasticsearch.ElasticsearchClient) RxUtils(io.georocket.util.RxUtils) DefaultMetaIndexerFactory(io.georocket.index.generic.DefaultMetaIndexerFactory) ConfigConstants(io.georocket.constants.ConfigConstants) JsonChunkMeta(io.georocket.storage.JsonChunkMeta) ElasticsearchClientFactory(io.georocket.index.elasticsearch.ElasticsearchClientFactory) JsonIndexerFactory(io.georocket.index.xml.JsonIndexerFactory) MetaIndexerFactory(io.georocket.index.xml.MetaIndexerFactory) DefaultMetaIndexerFactory(io.georocket.index.generic.DefaultMetaIndexerFactory) RxStore(io.georocket.storage.RxStore) XMLIndexerFactory(io.georocket.index.xml.XMLIndexerFactory)

Aggregations

ImmutableList (com.google.common.collect.ImmutableList)2 AddressConstants (io.georocket.constants.AddressConstants)2 ConfigConstants (io.georocket.constants.ConfigConstants)2 ElasticsearchClient (io.georocket.index.elasticsearch.ElasticsearchClient)2 ElasticsearchClientFactory (io.georocket.index.elasticsearch.ElasticsearchClientFactory)2 DefaultMetaIndexerFactory (io.georocket.index.generic.DefaultMetaIndexerFactory)2 DefaultQueryCompiler (io.georocket.query.DefaultQueryCompiler)2 MapUtils (io.georocket.util.MapUtils)2 ThrowableHelper.throwableToCode (io.georocket.util.ThrowableHelper.throwableToCode)2 ThrowableHelper.throwableToMessage (io.georocket.util.ThrowableHelper.throwableToMessage)2 Future (io.vertx.core.Future)2 JsonArray (io.vertx.core.json.JsonArray)2 JsonObject (io.vertx.core.json.JsonObject)2 Logger (io.vertx.core.logging.Logger)2 LoggerFactory (io.vertx.core.logging.LoggerFactory)2 AbstractVerticle (io.vertx.rxjava.core.AbstractVerticle)2 HashMap (java.util.HashMap)2 Resources (com.google.common.io.Resources)1 JsonIndexerFactory (io.georocket.index.xml.JsonIndexerFactory)1 MetaIndexer (io.georocket.index.xml.MetaIndexer)1