Search in sources :

Example 6 with DictionaryMapping

use of com.bakdata.conquery.models.dictionary.DictionaryMapping in project conquery by bakdata.

the class ImportJob method importPrimaryDictionary.

private DictionaryMapping importPrimaryDictionary(Dictionary primaryDictionary) {
    Dictionary orig = namespace.getStorage().getPrimaryDictionaryRaw();
    Dictionary primaryDict = Dictionary.copyUncompressed(orig);
    DictionaryMapping primaryMapping = DictionaryMapping.createAndImport(primaryDictionary, primaryDict);
    log.debug("Mapped {} new ids", primaryMapping.getNumberOfNewIds());
    // if no new ids we shouldn't recompress and store
    if (primaryMapping.getNumberOfNewIds() == 0) {
        log.trace("No new ids");
        return primaryMapping;
    }
    namespace.getStorage().updatePrimaryDictionary(primaryDict);
    return primaryMapping;
}
Also used : Dictionary(com.bakdata.conquery.models.dictionary.Dictionary) DictionaryMapping(com.bakdata.conquery.models.dictionary.DictionaryMapping)

Aggregations

DictionaryMapping (com.bakdata.conquery.models.dictionary.DictionaryMapping)6 Dictionary (com.bakdata.conquery.models.dictionary.Dictionary)4 ColumnStore (com.bakdata.conquery.models.events.stores.root.ColumnStore)3 IntegerStore (com.bakdata.conquery.models.events.stores.root.IntegerStore)3 StringStore (com.bakdata.conquery.models.events.stores.root.StringStore)3 IntegerParser (com.bakdata.conquery.models.preproc.parser.specific.IntegerParser)3 ProgressReporter (com.bakdata.conquery.util.progressreporter.ProgressReporter)3 IntArrayList (it.unimi.dsi.fastutil.ints.IntArrayList)3 IntList (it.unimi.dsi.fastutil.ints.IntList)3 IntOpenHashSet (it.unimi.dsi.fastutil.ints.IntOpenHashSet)3 IntSet (it.unimi.dsi.fastutil.ints.IntSet)3 NamespaceStorage (com.bakdata.conquery.io.storage.NamespaceStorage)2 ConqueryConfig (com.bakdata.conquery.models.config.ConqueryConfig)2 com.bakdata.conquery.models.datasets (com.bakdata.conquery.models.datasets)2 Bucket (com.bakdata.conquery.models.events.Bucket)2 MajorTypeId (com.bakdata.conquery.models.events.MajorTypeId)2 JSONException (com.bakdata.conquery.models.exceptions.JSONException)2 IdMutex (com.bakdata.conquery.models.identifiable.IdMutex)2 com.bakdata.conquery.models.identifiable.ids.specific (com.bakdata.conquery.models.identifiable.ids.specific)2 com.bakdata.conquery.models.messages.namespaces.specific (com.bakdata.conquery.models.messages.namespaces.specific)2