Examples with RamAccountingContext io.crate.breaker.RamAccountingContext used on opensource projects

Example 1 with RamAccountingContext

use of io.crate.breaker.RamAccountingContext in project crate by crate.

the class JobsLogsTest method createScheduler.

@Before
public void createScheduler() {
    scheduler = Executors.newSingleThreadScheduledExecutor();
    ramAccountingContext = new RamAccountingContext("testRamAccountingContext", breakerService.getBreaker(CrateCircuitBreakerService.JOBS_LOG));
}

Example 2 with RamAccountingContext

use of io.crate.breaker.RamAccountingContext in project crate by crate.

the class RemoteCollectorFactory method createCollector.

/**
     * create a RemoteCollector
     * The RemoteCollector will collect data from another node using a wormhole as if it was collecting on this node.
     * <p>
     * This should only be used if a shard is not available on the current node due to a relocation
     */
public CrateCollector.Builder createCollector(String index, Integer shardId, RoutedCollectPhase collectPhase, final RamAccountingContext ramAccountingContext) {
    // new job because subContexts can't be merged into an existing job
    final UUID childJobId = UUID.randomUUID();
    IndexShardRoutingTable shardRoutings = clusterService.state().routingTable().shardRoutingTable(index, shardId);
    // for update operations primaryShards must be used
    // (for others that wouldn't be the case, but at this point it is not easily visible which is the case)
    ShardRouting shardRouting = shardRoutings.primaryShard();
    final String remoteNodeId = shardRouting.currentNodeId();
    assert remoteNodeId != null : "primaryShard not assigned :(";
    final String localNodeId = clusterService.localNode().getId();
    final RoutedCollectPhase newCollectPhase = createNewCollectPhase(childJobId, collectPhase, index, shardId, remoteNodeId);
    return consumer -> new RemoteCollector(childJobId, localNodeId, remoteNodeId, transportActionProvider.transportJobInitAction(), transportActionProvider.transportKillJobsNodeAction(), jobContextService, ramAccountingContext, consumer, newCollectPhase);
}

Example 3 with RamAccountingContext

use of io.crate.breaker.RamAccountingContext in project crate by crate.

the class RamAccountingQueue method offer.

@Override
public boolean offer(T o) {
    context.addBytesWithoutBreaking(sizeEstimator.estimateSize(o));
    if (context.exceededBreaker() && exceeded.compareAndSet(false, true)) {
        LOGGER.error("Memory limit for breaker [{}] was exceeded. Queue [{}] is cleared.", breaker.getName(), context.contextId());
        // clear queue, close context and create new one
        close();
        context = new RamAccountingContext(contextId(), breaker);
        // add bytes to new context
        context.addBytesWithoutBreaking(sizeEstimator.estimateSize(o));
        exceeded.set(false);
    }
    return delegate.offer(o);
}

Example 4 with RamAccountingContext

use of io.crate.breaker.RamAccountingContext in project crate by crate.

the class LuceneBatchIteratorTest method testLuceneBatchIterator.

@Test
public void testLuceneBatchIterator() throws Exception {
    BatchIteratorTester tester = new BatchIteratorTester(() -> {
        return new LuceneBatchIterator(indexSearcher, new MatchAllDocsQuery(), null, false, new CollectorContext(mock(IndexFieldDataService.class), new CollectorFieldsVisitor(0)), new RamAccountingContext("dummy", new NoopCircuitBreaker("dummy")), columnRefs, columnRefs);
    });
    tester.verifyResultAndEdgeCaseBehaviour(expectedResult);
}