Search in sources :

Example 11 with Transaction

use of org.apache.fluo.api.client.Transaction in project incubator-rya by apache.

the class BatchIT method countResults.

private int countResults(FluoClient fluoClient, String nodeId, Column bsColumn) {
    try (Transaction tx = fluoClient.newTransaction()) {
        int count = 0;
        Optional<NodeType> type = NodeType.fromNodeId(nodeId);
        Bytes prefixBytes = Bytes.of(type.get().getNodeTypePrefix());
        RowScanner scanner = tx.scanner().over(Span.prefix(prefixBytes)).fetch(bsColumn).byRow().build();
        Iterator<ColumnScanner> colScanners = scanner.iterator();
        while (colScanners.hasNext()) {
            ColumnScanner colScanner = colScanners.next();
            BindingSetRow bsRow = BindingSetRow.makeFromShardedRow(prefixBytes, colScanner.getRow());
            if (bsRow.getNodeId().equals(nodeId)) {
                Iterator<ColumnValue> vals = colScanner.iterator();
                while (vals.hasNext()) {
                    vals.next();
                    count++;
                }
            }
        }
        tx.commit();
        return count;
    }
}
Also used : Bytes(org.apache.fluo.api.data.Bytes) Transaction(org.apache.fluo.api.client.Transaction) NodeType(org.apache.rya.indexing.pcj.fluo.app.NodeType) BindingSetRow(org.apache.rya.indexing.pcj.fluo.app.BindingSetRow) RowScanner(org.apache.fluo.api.client.scanner.RowScanner) ColumnScanner(org.apache.fluo.api.client.scanner.ColumnScanner) ColumnValue(org.apache.fluo.api.data.ColumnValue)

Example 12 with Transaction

use of org.apache.fluo.api.client.Transaction in project incubator-rya by apache.

the class CreateFluoPcj method writeFluoQuery.

private void writeFluoQuery(FluoClient fluo, FluoQuery fluoQuery, String pcjId) {
    try (Transaction tx = fluo.newTransaction()) {
        // Write the query's structure to Fluo.
        new FluoQueryMetadataDAO().write(tx, fluoQuery);
        // Flush the changes to Fluo.
        tx.commit();
    }
}
Also used : Transaction(org.apache.fluo.api.client.Transaction) FluoQueryMetadataDAO(org.apache.rya.indexing.pcj.fluo.app.query.FluoQueryMetadataDAO)

Example 13 with Transaction

use of org.apache.fluo.api.client.Transaction in project incubator-rya by apache.

the class DeleteFluoPcj method deleteData.

/**
 * Deletes all results (BindingSets or Statements) associated with the specified nodeId.
 *
 * @param nodeId - nodeId whose {@link BindingSet}s will be deleted. (not null)
 * @param client - Used to delete the data. (not null)
 */
private void deleteData(final FluoClient client, final String nodeId) {
    requireNonNull(client);
    requireNonNull(nodeId);
    final NodeType type = NodeType.fromNodeId(nodeId).get();
    Transaction tx = client.newTransaction();
    Bytes prefixBytes = Bytes.of(type.getNodeTypePrefix());
    SpanBatchDeleteInformation batch = SpanBatchDeleteInformation.builder().setColumn(type.getResultColumn()).setSpan(Span.prefix(prefixBytes)).setBatchSize(batchSize).setNodeId(Optional.of(nodeId)).build();
    BatchInformationDAO.addBatch(tx, nodeId, batch);
    tx.commit();
}
Also used : Bytes(org.apache.fluo.api.data.Bytes) Transaction(org.apache.fluo.api.client.Transaction) NodeType(org.apache.rya.indexing.pcj.fluo.app.NodeType) SpanBatchDeleteInformation(org.apache.rya.indexing.pcj.fluo.app.batch.SpanBatchDeleteInformation)

Example 14 with Transaction

use of org.apache.fluo.api.client.Transaction in project incubator-rya by apache.

the class InsertTriples method insert.

/**
 * Insert a batch of triples into Fluo.
 *
 * @param fluo - A connection to the Fluo table that will be updated. (not null)
 * @param triples - The triples to insert. (not null)
 * @param visibility - The visibility/permissions required to view the triples once stored.
 * Note: The same visibility will be applied to each triple.(not null)
 */
public void insert(final FluoClient fluo, final Collection<RyaStatement> triples, final Optional<String> visibility) {
    checkNotNull(fluo);
    checkNotNull(triples);
    checkNotNull(visibility);
    try (Transaction tx = fluo.newTransaction()) {
        for (final RyaStatement triple : triples) {
            try {
                tx.set(spoFormat(triple), FluoQueryColumns.TRIPLES, Bytes.of(visibility.or("")));
            } catch (final TripleRowResolverException e) {
                log.error("Could not convert a Triple into the SPO format: " + triple);
            }
        }
        tx.commit();
    }
}
Also used : TripleRowResolverException(org.apache.rya.api.resolver.triple.TripleRowResolverException) Transaction(org.apache.fluo.api.client.Transaction) RyaStatement(org.apache.rya.api.domain.RyaStatement)

Example 15 with Transaction

use of org.apache.fluo.api.client.Transaction in project incubator-rya by apache.

the class FluoQueryMetadataCacheTest method testCache.

@Test
public void testCache() {
    FluoQueryMetadataDAO mockDAO = Mockito.mock(FluoQueryMetadataDAO.class);
    Transaction mockTx = Mockito.mock(Transaction.class);
    String nodeId = NodeType.generateNewFluoIdForType(NodeType.STATEMENT_PATTERN);
    StatementPatternMetadata metadata = StatementPatternMetadata.builder(nodeId).setParentNodeId("parent").setStatementPattern("pattern").setVarOrder(new VariableOrder("xyz")).build();
    when(mockDAO.readStatementPatternMetadata(mockTx, nodeId)).thenReturn(metadata);
    FluoQueryMetadataCache cache = new FluoQueryMetadataCache(mockDAO, 20, 2);
    assertEquals(metadata, cache.readStatementPatternMetadata(mockTx, nodeId));
    cache.readStatementPatternMetadata(mockTx, nodeId);
    cache.readStatementPatternMetadata(mockTx, nodeId);
    cache.readStatementPatternMetadata(mockTx, nodeId);
    cache.readStatementPatternMetadata(mockTx, nodeId);
    Mockito.verify(mockDAO, Mockito.times(1)).readStatementPatternMetadata(mockTx, nodeId);
}
Also used : Transaction(org.apache.fluo.api.client.Transaction) VariableOrder(org.apache.rya.indexing.pcj.storage.accumulo.VariableOrder) Test(org.junit.Test)

Aggregations

Transaction (org.apache.fluo.api.client.Transaction)28 Test (org.junit.Test)17 FluoClient (org.apache.fluo.api.client.FluoClient)15 Snapshot (org.apache.fluo.api.client.Snapshot)13 VariableOrder (org.apache.rya.indexing.pcj.storage.accumulo.VariableOrder)11 NodeType (org.apache.rya.indexing.pcj.fluo.app.NodeType)5 Bytes (org.apache.fluo.api.data.Bytes)4 SpanBatchDeleteInformation (org.apache.rya.indexing.pcj.fluo.app.batch.SpanBatchDeleteInformation)3 HashSet (java.util.HashSet)2 Column (org.apache.fluo.api.data.Column)2 RyaStatement (org.apache.rya.api.domain.RyaStatement)2 AggregationElement (org.apache.rya.api.function.aggregation.AggregationElement)2 TripleRowResolverException (org.apache.rya.api.resolver.triple.TripleRowResolverException)2 FluoQueryMetadataDAO (org.apache.rya.indexing.pcj.fluo.app.query.FluoQueryMetadataDAO)2 ColumnScanner (org.apache.fluo.api.client.scanner.ColumnScanner)1 RowScanner (org.apache.fluo.api.client.scanner.RowScanner)1 ColumnValue (org.apache.fluo.api.data.ColumnValue)1 RyaURI (org.apache.rya.api.domain.RyaURI)1 FluoQueryStringBuilder (org.apache.rya.indexing.pcj.fluo.api.ListFluoQueries.FluoQueryStringBuilder)1 BindingSetRow (org.apache.rya.indexing.pcj.fluo.app.BindingSetRow)1