Search in sources :

Example 11 with QueryConfig

use of org.vitrivr.cineast.core.config.QueryConfig in project cineast by vitrivr.

the class VisualTextCoEmbedding method getSimilar.

@Override
public List<ScoreElement> getSimilar(String segmentId, ReadableQueryConfig qc) {
    // Ensure the correct distance function is used
    QueryConfig queryConfig = QueryConfig.clone(qc);
    queryConfig.setDistance(DISTANCE);
    return super.getSimilar(segmentId, queryConfig);
}
Also used : QueryConfig(org.vitrivr.cineast.core.config.QueryConfig) ReadableQueryConfig(org.vitrivr.cineast.core.config.ReadableQueryConfig)

Example 12 with QueryConfig

use of org.vitrivr.cineast.core.config.QueryConfig in project cineast by vitrivr.

the class CLIPImage method getSimilar.

@Override
public List<ScoreElement> getSimilar(SegmentContainer sc, ReadableQueryConfig qc) {
    if (sc.getMostRepresentativeFrame() == VideoFrame.EMPTY_VIDEO_FRAME) {
        return Collections.emptyList();
    }
    QueryConfig queryConfig = QueryConfig.clone(qc);
    queryConfig.setDistance(DISTANCE);
    float[] embeddingArray = embedImage(sc.getMostRepresentativeFrame().getImage().getBufferedImage());
    return getSimilar(embeddingArray, queryConfig);
}
Also used : QueryConfig(org.vitrivr.cineast.core.config.QueryConfig) ReadableQueryConfig(org.vitrivr.cineast.core.config.ReadableQueryConfig)

Example 13 with QueryConfig

use of org.vitrivr.cineast.core.config.QueryConfig in project cineast by vitrivr.

the class ConceptMasks method getSimilar.

@Override
public List<ScoreElement> getSimilar(SegmentContainer sc, ReadableQueryConfig qc) {
    Optional<SemanticMap> optional = sc.getSemanticMap();
    if (!optional.isPresent()) {
        return Collections.emptyList();
    }
    DeepLabLabel[][] labels = optional.get().getLabels();
    List<DeepLabLabel> list = linearize(labels);
    ArrayList<LinkedList<DeepLabLabel>> partitions = GridPartitioner.partition(list, labels.length, labels[0].length, GRID_PARTITIONS, GRID_PARTITIONS);
    float[] vector = new float[2 * GRID_PARTITIONS * GRID_PARTITIONS];
    float[] weights = new float[2 * GRID_PARTITIONS * GRID_PARTITIONS];
    for (int i = 0; i < GRID_PARTITIONS * GRID_PARTITIONS; ++i) {
        DeepLabLabel dominantLabel = DeepLabLabel.getDominantLabel(partitions.get(i));
        // TODO expose this to the API
        float weight = dominantLabel == DeepLabLabel.NOTHING ? 0f : 1f;
        weights[2 * i] = weight;
        weights[2 * i + 1] = weight;
        vector[2 * i] = dominantLabel.getEmbeddX();
        vector[2 * i + 1] = dominantLabel.getEmbeddY();
    }
    return this.getSimilar(vector, new QueryConfig(qc).setDistanceWeights(weights));
}
Also used : DeepLabLabel(org.vitrivr.cineast.core.features.neuralnet.tf.models.deeplab.DeepLabLabel) QueryConfig(org.vitrivr.cineast.core.config.QueryConfig) ReadableQueryConfig(org.vitrivr.cineast.core.config.ReadableQueryConfig) SemanticMap(org.vitrivr.cineast.core.data.SemanticMap) LinkedList(java.util.LinkedList)

Example 14 with QueryConfig

use of org.vitrivr.cineast.core.config.QueryConfig in project cineast by vitrivr.

the class AverageColorRaster method getSimilar.

private List<ScoreElement> getSimilar(float[] raster, float[] hist, ReadableQueryConfig rqc) {
    final int limit = rqc.getResultsPerModule();
    final QueryConfig qc = new QueryConfig(rqc).setDistanceIfEmpty(Distance.chisquared);
    List<Map<String, PrimitiveTypeProvider>> rows = this.selector.getNearestNeighbourRows(limit * 5, hist, "hist", qc);
    ArrayList<ScoreElement> scores = new ArrayList<>(rows.size());
    for (Map<String, PrimitiveTypeProvider> map : rows) {
        String id = map.get(GENERIC_ID_COLUMN_QUALIFIER).getString();
        double score = register(raster, map.get("raster").getFloatArray());
        scores.add(new SegmentScoreElement(id, score));
    }
    scores.sort(ScoreElement.SCORE_COMPARATOR.reversed());
    if (scores.size() > limit) {
        return scores.subList(0, limit);
    } else {
        return scores;
    }
}
Also used : QueryConfig(org.vitrivr.cineast.core.config.QueryConfig) ReadableQueryConfig(org.vitrivr.cineast.core.config.ReadableQueryConfig) SegmentScoreElement(org.vitrivr.cineast.core.data.score.SegmentScoreElement) ScoreElement(org.vitrivr.cineast.core.data.score.ScoreElement) SegmentScoreElement(org.vitrivr.cineast.core.data.score.SegmentScoreElement) ArrayList(java.util.ArrayList) Map(java.util.Map) PrimitiveTypeProvider(org.vitrivr.cineast.core.data.providers.primitive.PrimitiveTypeProvider)

Example 15 with QueryConfig

use of org.vitrivr.cineast.core.config.QueryConfig in project cineast by vitrivr.

the class ConceptMasksAde20k method getSimilar.

@Override
public List<ScoreElement> getSimilar(SegmentContainer sc, ReadableQueryConfig qc) {
    Optional<SemanticMap> optional = sc.getSemanticMap();
    if (!optional.isPresent()) {
        return Collections.emptyList();
    }
    DeepLabLabel[][] labels = optional.get().getLabels();
    List<DeepLabLabel> list = linearize(labels);
    ArrayList<LinkedList<DeepLabLabel>> ade20kPartitions = GridPartitioner.partition(list, labels.length, labels[0].length, GRID_PARTITIONS, GRID_PARTITIONS);
    float[] vector = new float[2 * GRID_PARTITIONS * GRID_PARTITIONS];
    float[] weights = new float[2 * GRID_PARTITIONS * GRID_PARTITIONS];
    for (int i = 0; i < GRID_PARTITIONS * GRID_PARTITIONS; ++i) {
        DeepLabLabel dominantLabel = DeepLabLabel.getDominantLabel(ade20kPartitions.get(i));
        // TODO expose this to the API
        float weight = dominantLabel == DeepLabLabel.NOTHING ? 0f : 1f;
        weights[2 * i] = weight;
        weights[2 * i + 1] = weight;
        vector[2 * i] = dominantLabel.getEmbeddX();
        vector[2 * i + 1] = dominantLabel.getEmbeddY();
    }
    return this.getSimilar(vector, new QueryConfig(qc).setDistanceWeights(weights));
}
Also used : DeepLabLabel(org.vitrivr.cineast.core.features.neuralnet.tf.models.deeplab.DeepLabLabel) QueryConfig(org.vitrivr.cineast.core.config.QueryConfig) ReadableQueryConfig(org.vitrivr.cineast.core.config.ReadableQueryConfig) SemanticMap(org.vitrivr.cineast.core.data.SemanticMap) LinkedList(java.util.LinkedList)

Aggregations

QueryConfig (org.vitrivr.cineast.core.config.QueryConfig)19 ReadableQueryConfig (org.vitrivr.cineast.core.config.ReadableQueryConfig)15 ArrayList (java.util.ArrayList)9 HashSet (java.util.HashSet)3 List (java.util.List)3 Map (java.util.Map)3 LogManager (org.apache.logging.log4j.LogManager)3 Logger (org.apache.logging.log4j.Logger)3 SegmentDistanceElement (org.vitrivr.cineast.core.data.distance.SegmentDistanceElement)3 SegmentScoreElement (org.vitrivr.cineast.core.data.score.SegmentScoreElement)3 HashMap (java.util.HashMap)2 LinkedList (java.util.LinkedList)2 Set (java.util.Set)2 Collectors (java.util.stream.Collectors)2 CorrespondenceFunction (org.vitrivr.cineast.core.data.CorrespondenceFunction)2 SemanticMap (org.vitrivr.cineast.core.data.SemanticMap)2 StringDoublePair (org.vitrivr.cineast.core.data.StringDoublePair)2 MediaObjectMetadataDescriptor (org.vitrivr.cineast.core.data.entities.MediaObjectMetadataDescriptor)2 DeepLabLabel (org.vitrivr.cineast.core.features.neuralnet.tf.models.deeplab.DeepLabLabel)2 ContinuousRetrievalLogic (org.vitrivr.cineast.standalone.util.ContinuousRetrievalLogic)2