Search in sources :

Example 6 with Closeable

use of java.io.Closeable in project hbase by apache.

the class MetaTableAccessor method scanMeta.

/**
   * Performs a scan of META table.
   * @param connection connection we're using
   * @param startRow Where to start the scan. Pass null if want to begin scan
   *                 at first row.
   * @param stopRow Where to stop the scan. Pass null if want to scan all rows
   *                from the start one
   * @param type scanned part of meta
   * @param maxRows maximum rows to return
   * @param visitor Visitor invoked against each row.
   * @throws IOException
   */
public static void scanMeta(Connection connection, @Nullable final byte[] startRow, @Nullable final byte[] stopRow, QueryType type, int maxRows, final Visitor visitor) throws IOException {
    int rowUpperLimit = maxRows > 0 ? maxRows : Integer.MAX_VALUE;
    Scan scan = getMetaScan(connection, rowUpperLimit);
    for (byte[] family : type.getFamilies()) {
        scan.addFamily(family);
    }
    if (startRow != null)
        scan.setStartRow(startRow);
    if (stopRow != null)
        scan.setStopRow(stopRow);
    if (LOG.isTraceEnabled()) {
        LOG.trace("Scanning META" + " starting at row=" + Bytes.toStringBinary(startRow) + " stopping at row=" + Bytes.toStringBinary(stopRow) + " for max=" + rowUpperLimit + " with caching=" + scan.getCaching());
    }
    int currentRow = 0;
    try (Table metaTable = getMetaHTable(connection)) {
        try (ResultScanner scanner = metaTable.getScanner(scan)) {
            Result data;
            while ((data = scanner.next()) != null) {
                if (data.isEmpty())
                    continue;
                // Break if visit returns false.
                if (!visitor.visit(data))
                    break;
                if (++currentRow >= rowUpperLimit)
                    break;
            }
        }
    }
    if (visitor != null && visitor instanceof Closeable) {
        try {
            ((Closeable) visitor).close();
        } catch (Throwable t) {
            ExceptionUtil.rethrowIfInterrupt(t);
            LOG.debug("Got exception in closing the meta scanner visitor", t);
        }
    }
}
Also used : Table(org.apache.hadoop.hbase.client.Table) ResultScanner(org.apache.hadoop.hbase.client.ResultScanner) Closeable(java.io.Closeable) Scan(org.apache.hadoop.hbase.client.Scan) Result(org.apache.hadoop.hbase.client.Result)

Example 7 with Closeable

use of java.io.Closeable in project kafka by apache.

the class Utils method closeAll.

/**
     * Closes all the provided closeables.
     * @throws IOException if any of the close methods throws an IOException.
     *         The first IOException is thrown with subsequent exceptions
     *         added as suppressed exceptions.
     */
public static void closeAll(Closeable... closeables) throws IOException {
    IOException exception = null;
    for (Closeable closeable : closeables) {
        try {
            closeable.close();
        } catch (IOException e) {
            if (exception != null)
                exception.addSuppressed(e);
            else
                exception = e;
        }
    }
    if (exception != null)
        throw exception;
}
Also used : Closeable(java.io.Closeable) IOException(java.io.IOException)

Example 8 with Closeable

use of java.io.Closeable in project druid by druid-io.

the class TestKafkaExtractionCluster method testSimpleRename.

@Test(timeout = 60_000L)
public void testSimpleRename() throws InterruptedException {
    final Properties kafkaProducerProperties = makeProducerProperties();
    final Producer<byte[], byte[]> producer = new Producer<>(new ProducerConfig(kafkaProducerProperties));
    closer.register(new Closeable() {

        @Override
        public void close() throws IOException {
            producer.close();
        }
    });
    checkServer();
    assertUpdated(null, "foo");
    assertReverseUpdated(ImmutableList.<String>of(), "foo");
    long events = factory.getCompletedEventCount();
    log.info("-------------------------     Sending foo bar     -------------------------------");
    producer.send(new KeyedMessage<>(topicName, StringUtils.toUtf8("foo"), StringUtils.toUtf8("bar")));
    long start = System.currentTimeMillis();
    while (events == factory.getCompletedEventCount()) {
        Thread.sleep(100);
        if (System.currentTimeMillis() > start + 60_000) {
            throw new ISE("Took too long to update event");
        }
    }
    log.info("-------------------------     Checking foo bar     -------------------------------");
    assertUpdated("bar", "foo");
    assertReverseUpdated(Collections.singletonList("foo"), "bar");
    assertUpdated(null, "baz");
    checkServer();
    events = factory.getCompletedEventCount();
    log.info("-------------------------     Sending baz bat     -------------------------------");
    producer.send(new KeyedMessage<>(topicName, StringUtils.toUtf8("baz"), StringUtils.toUtf8("bat")));
    while (events == factory.getCompletedEventCount()) {
        Thread.sleep(10);
        if (System.currentTimeMillis() > start + 60_000) {
            throw new ISE("Took too long to update event");
        }
    }
    log.info("-------------------------     Checking baz bat     -------------------------------");
    Assert.assertEquals("bat", factory.get().apply("baz"));
    Assert.assertEquals(Collections.singletonList("baz"), factory.get().unapply("bat"));
}
Also used : Producer(kafka.javaapi.producer.Producer) Closeable(java.io.Closeable) ProducerConfig(kafka.producer.ProducerConfig) ISE(io.druid.java.util.common.ISE) IOException(java.io.IOException) Properties(java.util.Properties) Test(org.junit.Test)

Example 9 with Closeable

use of java.io.Closeable in project druid by druid-io.

the class WrappingSequenceTest method testSanity.

@Test
public void testSanity() throws Exception {
    final AtomicInteger closedCounter = new AtomicInteger(0);
    Closeable closeable = new Closeable() {

        @Override
        public void close() throws IOException {
            closedCounter.incrementAndGet();
        }
    };
    final List<Integer> nums = Arrays.asList(1, 2, 3, 4, 5);
    SequenceTestHelper.testAll(Sequences.withBaggage(Sequences.simple(nums), closeable), nums);
    Assert.assertEquals(3, closedCounter.get());
    closedCounter.set(0);
    SequenceTestHelper.testClosed(closedCounter, Sequences.withBaggage(new UnsupportedSequence(), closeable));
}
Also used : AtomicInteger(java.util.concurrent.atomic.AtomicInteger) AtomicInteger(java.util.concurrent.atomic.AtomicInteger) Closeable(java.io.Closeable) Test(org.junit.Test)

Example 10 with Closeable

use of java.io.Closeable in project druid by druid-io.

the class WithEffectSequenceTest method testEffectExecutedIfWrappedSequenceThrowsExceptionFromClose.

@Test
public void testEffectExecutedIfWrappedSequenceThrowsExceptionFromClose() {
    Sequence<Integer> baseSeq = Sequences.simple(Arrays.asList(1, 2, 3));
    Sequence<Integer> throwingSeq = Sequences.withBaggage(baseSeq, new Closeable() {

        @Override
        public void close() throws IOException {
            throw new RuntimeException();
        }
    });
    final AtomicBoolean effectExecuted = new AtomicBoolean();
    Sequence<Integer> seqWithEffect = Sequences.withEffect(throwingSeq, new Runnable() {

        @Override
        public void run() {
            effectExecuted.set(true);
        }
    }, MoreExecutors.sameThreadExecutor());
    try {
        Sequences.toList(seqWithEffect, new ArrayList<Integer>());
        Assert.fail("expected RuntimeException");
    } catch (RuntimeException e) {
        // expected
        Assert.assertTrue(effectExecuted.get());
    }
}
Also used : AtomicInteger(java.util.concurrent.atomic.AtomicInteger) AtomicBoolean(java.util.concurrent.atomic.AtomicBoolean) Closeable(java.io.Closeable) IOException(java.io.IOException) Test(org.junit.Test)

Aggregations

Closeable (java.io.Closeable)216 IOException (java.io.IOException)88 Test (org.junit.Test)56 ArrayList (java.util.ArrayList)29 File (java.io.File)26 HashMap (java.util.HashMap)12 VirtualFile (org.jboss.vfs.VirtualFile)12 URL (java.net.URL)9 Path (java.nio.file.Path)9 AtomicBoolean (java.util.concurrent.atomic.AtomicBoolean)8 Map (java.util.Map)7 ISE (io.druid.java.util.common.ISE)6 InputStream (java.io.InputStream)6 MountHandle (org.jboss.as.server.deployment.module.MountHandle)6 ResourceRoot (org.jboss.as.server.deployment.module.ResourceRoot)6 ProgramController (co.cask.cdap.app.runtime.ProgramController)5 ProgramType (co.cask.cdap.proto.ProgramType)4 ProgramId (co.cask.cdap.proto.id.ProgramId)4 Pair (io.druid.java.util.common.Pair)4 FileOutputStream (java.io.FileOutputStream)4