Search in sources :

Example 11 with VertexiumException

use of org.vertexium.VertexiumException in project vertexium by visallo.

the class ElasticsearchGraphQueryIterable method reduceStatisticsResults.

private static StatisticsResult reduceStatisticsResults(List<Aggregation> aggs) {
    List<StatisticsResult> results = new ArrayList<>();
    for (Aggregation agg : aggs) {
        if (agg instanceof ExtendedStats) {
            ExtendedStats extendedStats = (ExtendedStats) agg;
            long count = extendedStats.getCount();
            double sum = extendedStats.getSum();
            double min = extendedStats.getMin();
            double max = extendedStats.getMax();
            double standardDeviation = extendedStats.getStdDeviation();
            results.add(new StatisticsResult(count, sum, min, max, standardDeviation));
        } else {
            throw new VertexiumException("Aggregation is not a statistics: " + agg.getClass().getName());
        }
    }
    return StatisticsResult.combine(results);
}
Also used : MultiBucketsAggregation(org.elasticsearch.search.aggregations.bucket.MultiBucketsAggregation) Aggregation(org.elasticsearch.search.aggregations.Aggregation) InternalExtendedStats(org.elasticsearch.search.aggregations.metrics.stats.extended.InternalExtendedStats) ExtendedStats(org.elasticsearch.search.aggregations.metrics.stats.extended.ExtendedStats) VertexiumException(org.vertexium.VertexiumException)

Example 12 with VertexiumException

use of org.vertexium.VertexiumException in project vertexium by visallo.

the class FlushObjectQueue method requeueFlushObject.

private void requeueFlushObject(FlushObject flushObject, int additionalTimeToSleep) {
    try {
        Thread.sleep(Math.max(0, flushObject.getNextRetryTime() - System.currentTimeMillis()));
    } catch (InterruptedException ex) {
        throw new VertexiumException("failed to sleep", ex);
    }
    long timeToWait = Math.min(((flushObject.getRetryCount() + 1) * 10) + additionalTimeToSleep, 1 * 60 * 1000);
    long nextRetryTime = System.currentTimeMillis() + timeToWait;
    queue.add(new FlushObject(flushObject.getElementId(), flushObject.getExtendedDataRowId(), flushObject.getActionRequestBuilder(), flushObject.getActionRequestBuilder().execute(), flushObject.getRetryCount() + 1, nextRetryTime));
}
Also used : VertexiumException(org.vertexium.VertexiumException)

Example 13 with VertexiumException

use of org.vertexium.VertexiumException in project vertexium by visallo.

the class ObjectUtils method compare.

public static int compare(Object leftObj, Object rightObj) {
    if (leftObj == null && rightObj == null) {
        return 0;
    }
    if (leftObj == null) {
        return 1;
    }
    if (rightObj == null) {
        return -1;
    }
    if (leftObj instanceof Stream && rightObj instanceof Stream) {
        return compareStreams((Stream) leftObj, (Stream) rightObj);
    }
    if (leftObj instanceof Stream) {
        leftObj = ((Stream) leftObj).collect(Collectors.toList());
    }
    if (rightObj instanceof Stream) {
        rightObj = ((Stream) rightObj).collect(Collectors.toList());
    }
    if (leftObj instanceof Collection && rightObj instanceof Collection) {
        return compareCollections((Collection) leftObj, (Collection) rightObj);
    }
    if (!(leftObj instanceof Comparable)) {
        throw new ClassCastException(leftObj.getClass().getName() + " does not implement " + Comparable.class.getName());
    }
    Comparable left = (Comparable) leftObj;
    try {
        if (leftObj instanceof Number && rightObj instanceof Number) {
            Number leftNumber = (Number) leftObj;
            Number rightNumber = (Number) rightObj;
            if (leftObj instanceof Double || leftObj instanceof Float || rightObj instanceof Double || rightObj instanceof Float) {
                return Double.compare(leftNumber.doubleValue(), rightNumber.doubleValue());
            }
            return Long.compare(leftNumber.longValue(), rightNumber.longValue());
        }
        if (left instanceof Number && !(rightObj instanceof Number)) {
            return -1;
        }
        if (rightObj instanceof Number && !(left instanceof Number)) {
            return 1;
        }
        return left.compareTo(rightObj);
    } catch (Exception ex) {
        if (ex instanceof ClassCastException) {
            throw ex;
        }
        throw new VertexiumException(String.format("Could not compare \"%s\" (%s) to \"%s\" (%s)", leftObj, leftObj == null ? "null" : leftObj.getClass().getName(), rightObj, rightObj == null ? "null" : rightObj.getClass().getName()), ex);
    }
}
Also used : Collection(java.util.Collection) Stream(java.util.stream.Stream) VertexiumException(org.vertexium.VertexiumException) VertexiumCypherTypeErrorException(org.vertexium.cypher.exceptions.VertexiumCypherTypeErrorException) VertexiumException(org.vertexium.VertexiumException)

Example 14 with VertexiumException

use of org.vertexium.VertexiumException in project vertexium by visallo.

the class QuickKryoVertexiumSerializer method expand.

protected byte[] expand(byte[] bytes) {
    if (!enableCompression) {
        return bytes;
    }
    Inflater inflater = new Inflater();
    try {
        inflater.setInput(bytes);
        ByteArrayOutputStream outputStream = new ByteArrayOutputStream(bytes.length);
        byte[] buffer = new byte[1024];
        while (!inflater.finished()) {
            int count = inflater.inflate(buffer);
            outputStream.write(buffer, 0, count);
        }
        outputStream.close();
        return outputStream.toByteArray();
    } catch (Exception ex) {
        throw new VertexiumException("Could not decompress bytes", ex);
    } finally {
        inflater.end();
    }
}
Also used : Inflater(java.util.zip.Inflater) ByteArrayOutputStream(java.io.ByteArrayOutputStream) VertexiumException(org.vertexium.VertexiumException) VertexiumException(org.vertexium.VertexiumException)

Example 15 with VertexiumException

use of org.vertexium.VertexiumException in project vertexium by visallo.

the class DeleteHistoricalLegacyStreamingPropertyValueData method execute.

public void execute(Options options, Authorizations authorizations) {
    try {
        org.apache.accumulo.core.security.Authorizations accumuloAuthorizations = graph.toAccumuloAuthorizations(authorizations);
        Scanner scanner = graph.getConnector().createScanner(graph.getDataTableName(), accumuloAuthorizations);
        BatchWriter writer = graph.getConnector().createBatchWriter(graph.getDataTableName(), graph.getConfiguration().createBatchWriterConfig());
        String lastRowIdPrefix = null;
        List<Key> rowsToDelete = new ArrayList<>();
        try {
            int rowCount = 0;
            for (Map.Entry<Key, Value> row : scanner) {
                if (rowCount % 10000 == 0) {
                    writer.flush();
                    LOGGER.debug("looking at row: %s (row count: %d)", row.getKey().getRow().toString(), rowCount);
                }
                rowCount++;
                if (!EMPTY_TEXT.equals(row.getKey().getColumnFamily())) {
                    continue;
                }
                if (!EMPTY_TEXT.equals(row.getKey().getColumnQualifier())) {
                    continue;
                }
                String rowId = row.getKey().getRow().toString();
                String[] rowIdParts = rowId.split("" + DataTableRowKey.VALUE_SEPARATOR);
                if (rowIdParts.length < 3) {
                    continue;
                }
                if (lastRowIdPrefix == null || !isSameProperty(lastRowIdPrefix, rowId)) {
                    deleteRows(writer, rowsToDelete, options);
                    rowsToDelete.clear();
                    lastRowIdPrefix = rowIdParts[0] + DataTableRowKey.VALUE_SEPARATOR + rowIdParts[1] + DataTableRowKey.VALUE_SEPARATOR + rowIdParts[2];
                }
                rowsToDelete.add(row.getKey());
            }
            deleteRows(writer, rowsToDelete, options);
        } finally {
            writer.flush();
            scanner.close();
        }
    } catch (Exception ex) {
        throw new VertexiumException("Could not delete old SPV data", ex);
    }
}
Also used : Scanner(org.apache.accumulo.core.client.Scanner) ArrayList(java.util.ArrayList) VertexiumException(org.vertexium.VertexiumException) MutationsRejectedException(org.apache.accumulo.core.client.MutationsRejectedException) VertexiumException(org.vertexium.VertexiumException) Value(org.apache.accumulo.core.data.Value) BatchWriter(org.apache.accumulo.core.client.BatchWriter) Map(java.util.Map) DataTableRowKey(org.vertexium.accumulo.keys.DataTableRowKey) Key(org.apache.accumulo.core.data.Key)

Aggregations

VertexiumException (org.vertexium.VertexiumException)34 Aggregation (org.elasticsearch.search.aggregations.Aggregation)6 MultiBucketsAggregation (org.elasticsearch.search.aggregations.bucket.MultiBucketsAggregation)6 IOException (java.io.IOException)5 ArrayList (java.util.ArrayList)4 Map (java.util.Map)3 Key (org.apache.accumulo.core.data.Key)3 Value (org.apache.accumulo.core.data.Value)3 DataTableRowKey (org.vertexium.accumulo.keys.DataTableRowKey)3 StreamingPropertyValue (org.vertexium.property.StreamingPropertyValue)3 ByteArrayOutputStream (java.io.ByteArrayOutputStream)2 InputStream (java.io.InputStream)2 List (java.util.List)2 ScannerBase (org.apache.accumulo.core.client.ScannerBase)2 Span (org.apache.accumulo.core.trace.Span)2 GeoHashGrid (org.elasticsearch.search.aggregations.bucket.geogrid.GeoHashGrid)2 InternalGeoHashGrid (org.elasticsearch.search.aggregations.bucket.geogrid.InternalGeoHashGrid)2 Histogram (org.elasticsearch.search.aggregations.bucket.histogram.Histogram)2 InternalDateHistogram (org.elasticsearch.search.aggregations.bucket.histogram.InternalDateHistogram)2 InternalHistogram (org.elasticsearch.search.aggregations.bucket.histogram.InternalHistogram)2