Search in sources :

Example 41 with EOFException

use of java.io.EOFException in project flink by apache.

the class KryoClearedBufferTest method testOutputBufferedBeingClearedInCaseOfException.

/**
	 * Tests that the kryo output buffer is cleared in case of an exception. Flink uses the
	 * EOFException to signal that a buffer is full. In such a case, the record which was tried
	 * to be written will be rewritten. Therefore, eventually buffered data of this record has
	 * to be cleared.
	 */
@Test
public void testOutputBufferedBeingClearedInCaseOfException() throws Exception {
    ExecutionConfig executionConfig = new ExecutionConfig();
    executionConfig.registerTypeWithKryoSerializer(TestRecord.class, new TestRecordSerializer());
    executionConfig.registerKryoType(TestRecord.class);
    KryoSerializer<TestRecord> kryoSerializer = new KryoSerializer<TestRecord>(TestRecord.class, executionConfig);
    int size = 94;
    int bufferSize = 150;
    TestRecord testRecord = new TestRecord(size);
    TestDataOutputView target = new TestDataOutputView(bufferSize);
    kryoSerializer.serialize(testRecord, target);
    try {
        kryoSerializer.serialize(testRecord, target);
        Assert.fail("Expected an EOFException.");
    } catch (EOFException eofException) {
    // expected exception
    // now the Kryo Output should have been cleared
    }
    TestRecord actualRecord = kryoSerializer.deserialize(new DataInputViewStreamWrapper(new ByteArrayInputStream(target.getBuffer())));
    Assert.assertEquals(testRecord, actualRecord);
    target.clear();
    // if the kryo output has been cleared then we can serialize our test record into the target
    // because the target buffer 150 bytes can host one TestRecord (total serialization size 100)
    kryoSerializer.serialize(testRecord, target);
    byte[] buffer = target.getBuffer();
    int counter = 0;
    for (int i = 0; i < buffer.length; i++) {
        if (buffer[i] == 42) {
            counter++;
        }
    }
    Assert.assertEquals(size, counter);
}
Also used : ByteArrayInputStream(java.io.ByteArrayInputStream) EOFException(java.io.EOFException) ExecutionConfig(org.apache.flink.api.common.ExecutionConfig) DataInputViewStreamWrapper(org.apache.flink.core.memory.DataInputViewStreamWrapper) Test(org.junit.Test)

Example 42 with EOFException

use of java.io.EOFException in project flink by apache.

the class MemorySegmentTestBase method testDataInputOutputStreamUnderflowOverflow.

@Test
public void testDataInputOutputStreamUnderflowOverflow() {
    try {
        final int segmentSize = 1337;
        // segment with random contents
        MemorySegment seg = createSegment(segmentSize);
        byte[] bytes = new byte[segmentSize];
        random.nextBytes(bytes);
        seg.put(0, bytes);
        // a stream that we cannot fully write to
        DataOutputStream out = new DataOutputStream(new OutputStream() {

            int bytesSoFar = 0;

            @Override
            public void write(int b) throws IOException {
                bytesSoFar++;
                if (bytesSoFar > segmentSize / 2) {
                    throw new IOException("overflow");
                }
            }
        });
        // write the segment in chunks into the stream
        try {
            int pos = 0;
            while (pos < pageSize) {
                int len = random.nextInt(segmentSize / 10);
                len = Math.min(len, pageSize - pos);
                seg.get(out, pos, len);
                pos += len;
            }
            fail("Should fail with an IOException");
        } catch (IOException e) {
        // expected
        }
        DataInputStream in = new DataInputStream(new ByteArrayInputStream(new byte[segmentSize / 2]));
        try {
            int pos = 0;
            while (pos < pageSize) {
                int len = random.nextInt(segmentSize / 10);
                len = Math.min(len, pageSize - pos);
                seg.put(in, pos, len);
                pos += len;
            }
            fail("Should fail with an EOFException");
        } catch (EOFException e) {
        // expected
        }
    } catch (Exception e) {
        e.printStackTrace();
        fail(e.getMessage());
    }
}
Also used : ByteArrayInputStream(java.io.ByteArrayInputStream) DataOutputStream(java.io.DataOutputStream) OutputStream(java.io.OutputStream) ByteArrayOutputStream(java.io.ByteArrayOutputStream) DataOutputStream(java.io.DataOutputStream) EOFException(java.io.EOFException) IOException(java.io.IOException) DataInputStream(java.io.DataInputStream) BufferOverflowException(java.nio.BufferOverflowException) IOException(java.io.IOException) EOFException(java.io.EOFException) BufferUnderflowException(java.nio.BufferUnderflowException) Test(org.junit.Test)

Example 43 with EOFException

use of java.io.EOFException in project ExoPlayer by google.

the class DefaultExtractorInputTest method testSkipFullyLarge.

public void testSkipFullyLarge() throws Exception {
    // Tests skipping an amount of data that's larger than any internal scratch space.
    int largeSkipSize = 1024 * 1024;
    FakeDataSource.Builder builder = new FakeDataSource.Builder();
    builder.appendReadData(new byte[largeSkipSize]);
    FakeDataSource testDataSource = builder.build();
    testDataSource.open(new DataSpec(Uri.parse(TEST_URI)));
    DefaultExtractorInput input = new DefaultExtractorInput(testDataSource, 0, C.LENGTH_UNSET);
    input.skipFully(largeSkipSize);
    assertEquals(largeSkipSize, input.getPosition());
    // Check that we fail with EOFException we skip again.
    try {
        input.skipFully(1);
        fail();
    } catch (EOFException e) {
    // Expected.
    }
}
Also used : FakeDataSource(com.google.android.exoplayer2.testutil.FakeDataSource) EOFException(java.io.EOFException) DataSpec(com.google.android.exoplayer2.upstream.DataSpec)

Example 44 with EOFException

use of java.io.EOFException in project ExoPlayer by google.

the class DefaultOggSeekerUtilMethodsTest method testReadGranuleOfLastPageAfterLastHeader.

public void testReadGranuleOfLastPageAfterLastHeader() throws IOException, InterruptedException {
    FakeExtractorInput input = TestData.createInput(TestUtil.buildTestData(100, random), false);
    try {
        assertReadGranuleOfLastPage(input, 60000);
        fail();
    } catch (EOFException e) {
    // ignored
    }
}
Also used : FakeExtractorInput(com.google.android.exoplayer2.testutil.FakeExtractorInput) EOFException(java.io.EOFException)

Example 45 with EOFException

use of java.io.EOFException in project ExoPlayer by google.

the class DefaultOggSeekerUtilMethodsTest method testSkipToNextPageNoMatch.

public void testSkipToNextPageNoMatch() throws Exception {
    FakeExtractorInput extractorInput = TestData.createInput(new byte[] { 'g', 'g', 'S', 'O', 'g', 'g' }, false);
    try {
        skipToNextPage(extractorInput);
        fail();
    } catch (EOFException e) {
    // expected
    }
}
Also used : FakeExtractorInput(com.google.android.exoplayer2.testutil.FakeExtractorInput) EOFException(java.io.EOFException)

Aggregations

EOFException (java.io.EOFException)624 IOException (java.io.IOException)275 FileInputStream (java.io.FileInputStream)81 DataInputStream (java.io.DataInputStream)79 Test (org.junit.Test)59 ByteArrayInputStream (java.io.ByteArrayInputStream)53 InputStream (java.io.InputStream)43 RandomAccessFile (java.io.RandomAccessFile)42 ArrayList (java.util.ArrayList)42 File (java.io.File)41 FileNotFoundException (java.io.FileNotFoundException)41 ByteBuffer (java.nio.ByteBuffer)40 BufferedInputStream (java.io.BufferedInputStream)31 Path (org.apache.hadoop.fs.Path)21 InterruptedIOException (java.io.InterruptedIOException)20 ObjectInputStream (java.io.ObjectInputStream)20 ByteArrayOutputStream (java.io.ByteArrayOutputStream)19 SocketTimeoutException (java.net.SocketTimeoutException)19 FSDataInputStream (org.apache.hadoop.fs.FSDataInputStream)17 SocketException (java.net.SocketException)16