Search in sources :

Example 1 with IndexBaseBuilder

use of io.crate.planner.fetch.IndexBaseBuilder in project crate by crate.

the class RoutingBuilder method buildReaderAllocations.

ReaderAllocations buildReaderAllocations() {
    Map<RelationName, Collection<String>> indicesByTable = new HashMap<>();
    IndexBaseBuilder indexBaseBuilder = new IndexBaseBuilder();
    Map<String, Map<Integer, String>> shardNodes = new HashMap<>();
    Map<RelationName, List<Routing>> routingListByTable = routingListByTableStack.removeLast();
    assert routingListByTable != null : "Call to `buildReaderAllocations` without prior `newAllocations` call";
    for (var tableRouting : routingListByTable.entrySet()) {
        RelationName table = tableRouting.getKey();
        List<Routing> routingList = tableRouting.getValue();
        for (Routing routing : routingList) {
            allocateRoutingNodes(shardNodes, routing.locations());
            for (Map.Entry<String, Map<String, IntIndexedContainer>> entry : routing.locations().entrySet()) {
                Map<String, IntIndexedContainer> shardsByIndex = entry.getValue();
                Collection<String> indices = indicesByTable.computeIfAbsent(table, ignored -> new ArrayList<>());
                indices.addAll(shardsByIndex.keySet());
                for (Map.Entry<String, IntIndexedContainer> shardsByIndexEntry : shardsByIndex.entrySet()) {
                    indexBaseBuilder.allocate(shardsByIndexEntry.getKey(), shardsByIndexEntry.getValue());
                }
            }
        }
    }
    return new ReaderAllocations(indexBaseBuilder.build(), shardNodes, indicesByTable);
}
Also used : HashMap(java.util.HashMap) ShardRouting(org.elasticsearch.cluster.routing.ShardRouting) Routing(io.crate.metadata.Routing) IntIndexedContainer(com.carrotsearch.hppc.IntIndexedContainer) IndexBaseBuilder(io.crate.planner.fetch.IndexBaseBuilder) RelationName(io.crate.metadata.RelationName) Collection(java.util.Collection) ArrayList(java.util.ArrayList) List(java.util.List) HashMap(java.util.HashMap) Map(java.util.Map)

Example 2 with IndexBaseBuilder

use of io.crate.planner.fetch.IndexBaseBuilder in project crate by crate.

the class IndexBaseBuilderTest method testDoubleIndex.

@Test
public void testDoubleIndex() throws Exception {
    IndexBaseBuilder builder = new IndexBaseBuilder();
    builder.allocate("i1", IntArrayList.from(1, 4));
    builder.allocate("i2", IntArrayList.from(1, 2));
    builder.allocate("i1", IntArrayList.from(1, 5));
    builder.allocate("i1", IntArrayList.from(1, 3));
    builder.allocate("i3", IntArrayList.from(1, 3));
    TreeMap<String, Integer> bases = builder.build();
    assertThat(bases.size(), is(3));
    assertThat(bases.get("i1"), is(0));
    assertThat(bases.get("i2"), is(6));
    assertThat(bases.get("i3"), is(9));
}
Also used : IndexBaseBuilder(io.crate.planner.fetch.IndexBaseBuilder) Test(org.junit.Test)

Example 3 with IndexBaseBuilder

use of io.crate.planner.fetch.IndexBaseBuilder in project crate by crate.

the class FetchTaskTest method testSearcherIsAcquiredForShard.

@Test
public void testSearcherIsAcquiredForShard() throws Exception {
    IntArrayList shards = IntArrayList.from(1, 2);
    Routing routing = new Routing(Map.of("dummy", Map.of("i1", shards)));
    IndexBaseBuilder ibb = new IndexBaseBuilder();
    ibb.allocate("i1", shards);
    Map<RelationName, Collection<String>> tableIndices = new HashMap<>();
    tableIndices.put(new RelationName(Schemas.DOC_SCHEMA_NAME, "i1"), List.of("i1"));
    Metadata metadata = Metadata.builder().put(IndexMetadata.builder("i1").settings(Settings.builder().put(SETTING_NUMBER_OF_SHARDS, 1).put(SETTING_NUMBER_OF_REPLICAS, 0).put(SETTING_VERSION_CREATED, Version.CURRENT)).build(), true).build();
    final FetchTask context = new FetchTask(UUID.randomUUID(), new FetchPhase(1, null, ibb.build(), tableIndices, List.of(createReference("i1", new ColumnIdent("x"), DataTypes.STRING))), "dummy", new SharedShardContexts(mock(IndicesService.class, RETURNS_MOCKS), UnaryOperator.identity()), metadata, relationName -> null, List.of(routing));
    context.start();
    assertThat(context.searcher(1), Matchers.notNullValue());
    assertThat(context.searcher(2), Matchers.notNullValue());
}
Also used : ColumnIdent(io.crate.metadata.ColumnIdent) SharedShardContexts(io.crate.execution.jobs.SharedShardContexts) HashMap(java.util.HashMap) FetchPhase(io.crate.execution.dsl.phases.FetchPhase) IndexMetadata(org.elasticsearch.cluster.metadata.IndexMetadata) Metadata(org.elasticsearch.cluster.metadata.Metadata) Routing(io.crate.metadata.Routing) RelationName(io.crate.metadata.RelationName) Collection(java.util.Collection) IntArrayList(com.carrotsearch.hppc.IntArrayList) IndexBaseBuilder(io.crate.planner.fetch.IndexBaseBuilder) CrateDummyClusterServiceUnitTest(io.crate.test.integration.CrateDummyClusterServiceUnitTest) Test(org.junit.Test)

Example 4 with IndexBaseBuilder

use of io.crate.planner.fetch.IndexBaseBuilder in project crate by crate.

the class IndexBaseBuilderTest method testMaxShard.

@Test
public void testMaxShard() throws Exception {
    IndexBaseBuilder builder = new IndexBaseBuilder();
    builder.allocate("i1", IntArrayList.from(1, 4));
    builder.allocate("i2", IntArrayList.from(1, 5));
    TreeMap<String, Integer> bases = builder.build();
    assertThat(bases.size(), is(2));
    assertThat(bases.get("i1"), is(0));
    assertThat(bases.get("i2"), is(5));
}
Also used : IndexBaseBuilder(io.crate.planner.fetch.IndexBaseBuilder) Test(org.junit.Test)

Aggregations

IndexBaseBuilder (io.crate.planner.fetch.IndexBaseBuilder)4 Test (org.junit.Test)3 RelationName (io.crate.metadata.RelationName)2 Routing (io.crate.metadata.Routing)2 Collection (java.util.Collection)2 HashMap (java.util.HashMap)2 IntArrayList (com.carrotsearch.hppc.IntArrayList)1 IntIndexedContainer (com.carrotsearch.hppc.IntIndexedContainer)1 FetchPhase (io.crate.execution.dsl.phases.FetchPhase)1 SharedShardContexts (io.crate.execution.jobs.SharedShardContexts)1 ColumnIdent (io.crate.metadata.ColumnIdent)1 CrateDummyClusterServiceUnitTest (io.crate.test.integration.CrateDummyClusterServiceUnitTest)1 ArrayList (java.util.ArrayList)1 List (java.util.List)1 Map (java.util.Map)1 IndexMetadata (org.elasticsearch.cluster.metadata.IndexMetadata)1 Metadata (org.elasticsearch.cluster.metadata.Metadata)1 ShardRouting (org.elasticsearch.cluster.routing.ShardRouting)1