Examples with InputStreamFileInput org.embulk.spi.util.InputStreamFileInput used on opensource projects

Example 1 with InputStreamFileInput

use of org.embulk.spi.util.InputStreamFileInput in project embulk by embulk.

the class TestInputStreamFileInput method testMultipleProvider.

@Test
public void testMultipleProvider() throws IOException {
    InputStreamFileInput subject = new InputStreamFileInput(runtime.getBufferAllocator(), provider(new ByteArrayInputStream("abcdef".getBytes("UTF-8")), new ByteArrayInputStream("ghijkl".getBytes("UTF-8")), new ByteArrayInputStream("mnopqr".getBytes("UTF-8"))));
    assertEquals(true, subject.nextFile());
    assertEquals("abcdef", bufferToString(subject.poll()));
    assertEquals(true, subject.nextFile());
    assertEquals("ghijkl", bufferToString(subject.poll()));
    assertEquals(true, subject.nextFile());
    assertEquals("mnopqr", bufferToString(subject.poll()));
    subject.close();
}

Example 2 with InputStreamFileInput

use of org.embulk.spi.util.InputStreamFileInput in project embulk by embulk.

the class TestInputStreamFileInput method testPollFirstException.

@Test
public void testPollFirstException() throws IOException {
    InputStreamFileInput subject = new InputStreamFileInput(runtime.getBufferAllocator(), provider(new ByteArrayInputStream("abcdef".getBytes("UTF-8"))));
    try {
        subject.poll();
        fail();
    } catch (IllegalStateException ile) {
    // OK
    }
    subject.close();
}

Example 3 with InputStreamFileInput

use of org.embulk.spi.util.InputStreamFileInput in project embulk by embulk.

the class TestJsonParserPlugin method fileInput.

private FileInput fileInput(String... lines) throws Exception {
    StringBuilder sb = new StringBuilder();
    for (String line : lines) {
        sb.append(line).append("\n");
    }
    ByteArrayInputStream in = new ByteArrayInputStream(sb.toString().getBytes("UTF-8"));
    return new InputStreamFileInput(runtime.getBufferAllocator(), provider(in));
}

Example 4 with InputStreamFileInput

use of org.embulk.spi.util.InputStreamFileInput in project embulk by embulk.

the class Bzip2FileDecoderPlugin method open.

@Override
public FileInput open(TaskSource taskSource, FileInput fileInput) {
    PluginTask task = taskSource.loadTask(PluginTask.class);
    final FileInputInputStream files = new FileInputInputStream(fileInput);
    return new InputStreamFileInput(task.getBufferAllocator(), new InputStreamFileInput.Provider() {

        public InputStream openNext() throws IOException {
            if (!files.nextFile()) {
                return null;
            }
            return new BZip2CompressorInputStream(files, true);
        }

        public void close() throws IOException {
            files.close();
        }
    });
}

Example 5 with InputStreamFileInput

use of org.embulk.spi.util.InputStreamFileInput in project embulk by embulk.

the class GzipFileDecoderPlugin method open.

@Override
public FileInput open(TaskSource taskSource, FileInput fileInput) {
    PluginTask task = taskSource.loadTask(PluginTask.class);
    final FileInputInputStream files = new FileInputInputStream(fileInput);
    return new InputStreamFileInput(task.getBufferAllocator(), new InputStreamFileInput.Provider() {

        public InputStream openNext() throws IOException {
            if (!files.nextFile()) {
                return null;
            }
            return new GZIPInputStream(files, 8 * 1024);
        }

        public void close() throws IOException {
            files.close();
        }
    });
}