Search in sources :

Example 1 with Int2ObjectAVLTreeMap

use of it.unimi.dsi.fastutil.ints.Int2ObjectAVLTreeMap in project conquery by bakdata.

the class BucketManager method create.

public static BucketManager create(Worker worker, WorkerStorage storage, int entityBucketSize) {
    Int2ObjectMap<Entity> entities = new Int2ObjectAVLTreeMap<>();
    Map<Connector, Int2ObjectMap<Map<Bucket, CBlock>>> connectorCBlocks = new HashMap<>();
    Map<Table, Int2ObjectMap<List<Bucket>>> tableBuckets = new HashMap<>();
    IntArraySet assignedBucketNumbers = worker.getInfo().getIncludedBuckets();
    log.trace("Trying to load these buckets that map to: {}", assignedBucketNumbers);
    for (Bucket bucket : storage.getAllBuckets()) {
        if (!assignedBucketNumbers.contains(bucket.getBucket())) {
            log.warn("Found Bucket[{}] in Storage that does not belong to this Worker according to the Worker information.", bucket.getId());
        }
        registerBucket(bucket, entities, tableBuckets);
    }
    for (CBlock cBlock : storage.getAllCBlocks()) {
        registerCBlock(cBlock, connectorCBlocks);
    }
    return new BucketManager(worker.getJobManager(), storage, worker, entities, connectorCBlocks, tableBuckets, entityBucketSize);
}
Also used : Entity(com.bakdata.conquery.models.query.entity.Entity) ConceptTreeConnector(com.bakdata.conquery.models.datasets.concepts.tree.ConceptTreeConnector) Connector(com.bakdata.conquery.models.datasets.concepts.Connector) Table(com.bakdata.conquery.models.datasets.Table) HashMap(java.util.HashMap) Int2ObjectAVLTreeMap(it.unimi.dsi.fastutil.ints.Int2ObjectAVLTreeMap) Int2ObjectMap(it.unimi.dsi.fastutil.ints.Int2ObjectMap) IntArraySet(it.unimi.dsi.fastutil.ints.IntArraySet)

Aggregations

Table (com.bakdata.conquery.models.datasets.Table)1 Connector (com.bakdata.conquery.models.datasets.concepts.Connector)1 ConceptTreeConnector (com.bakdata.conquery.models.datasets.concepts.tree.ConceptTreeConnector)1 Entity (com.bakdata.conquery.models.query.entity.Entity)1 Int2ObjectAVLTreeMap (it.unimi.dsi.fastutil.ints.Int2ObjectAVLTreeMap)1 Int2ObjectMap (it.unimi.dsi.fastutil.ints.Int2ObjectMap)1 IntArraySet (it.unimi.dsi.fastutil.ints.IntArraySet)1 HashMap (java.util.HashMap)1