Examples with FSDataOutputStream org.apache.hadoop.fs.FSDataOutputStream used on opensource projects

Example 11 with FSDataOutputStream

use of org.apache.hadoop.fs.FSDataOutputStream in project hadoop by apache.

the class TestCopyPreserveFlag method initialize.

@Before
public void initialize() throws Exception {
    conf = new Configuration(false);
    conf.set("fs.file.impl", LocalFileSystem.class.getName());
    fs = FileSystem.getLocal(conf);
    testDir = new FileSystemTestHelper().getTestRootPath(fs);
    // don't want scheme on the path, just an absolute path
    testDir = new Path(fs.makeQualified(testDir).toUri().getPath());
    FileSystem.setDefaultUri(conf, fs.getUri());
    fs.setWorkingDirectory(testDir);
    fs.mkdirs(DIR_FROM);
    fs.mkdirs(DIR_TO1);
    fs.createNewFile(FROM);
    FSDataOutputStream output = fs.create(FROM, true);
    for (int i = 0; i < 100; ++i) {
        output.writeInt(i);
        output.writeChar('\n');
    }
    output.close();
    fs.setTimes(FROM, MODIFICATION_TIME, ACCESS_TIME);
    fs.setPermission(FROM, PERMISSIONS);
    fs.setTimes(DIR_FROM, MODIFICATION_TIME, ACCESS_TIME);
    fs.setPermission(DIR_FROM, PERMISSIONS);
}

Example 12 with FSDataOutputStream

use of org.apache.hadoop.fs.FSDataOutputStream in project hadoop by apache.

the class AbstractContractAppendTest method testAppendToExistingFile.

@Test
public void testAppendToExistingFile() throws Throwable {
    byte[] original = dataset(8192, 'A', 'Z');
    byte[] appended = dataset(8192, '0', '9');
    createFile(getFileSystem(), target, false, original);
    FSDataOutputStream outputStream = getFileSystem().append(target);
    outputStream.write(appended);
    outputStream.close();
    byte[] bytes = ContractTestUtils.readDataset(getFileSystem(), target, original.length + appended.length);
    ContractTestUtils.validateFileContent(bytes, new byte[][] { original, appended });
}

Example 13 with FSDataOutputStream

use of org.apache.hadoop.fs.FSDataOutputStream in project hadoop by apache.

the class AbstractContractAppendTest method testAppendMissingTarget.

@Test
public void testAppendMissingTarget() throws Throwable {
    try {
        FSDataOutputStream out = getFileSystem().append(target);
        //got here: trouble
        out.close();
        fail("expected a failure");
    } catch (Exception e) {
        //expected
        handleExpectedException(e);
    }
}

Example 14 with FSDataOutputStream

use of org.apache.hadoop.fs.FSDataOutputStream in project hadoop by apache.

the class AbstractContractAppendTest method testAppendToEmptyFile.

@Test
public void testAppendToEmptyFile() throws Throwable {
    touch(getFileSystem(), target);
    byte[] dataset = dataset(256, 'a', 'z');
    try (FSDataOutputStream outputStream = getFileSystem().append(target)) {
        outputStream.write(dataset);
    }
    byte[] bytes = ContractTestUtils.readDataset(getFileSystem(), target, dataset.length);
    ContractTestUtils.compareByteArrays(dataset, bytes, dataset.length);
}

Example 15 with FSDataOutputStream

use of org.apache.hadoop.fs.FSDataOutputStream in project hadoop by apache.

the class AbstractContractCreateTest method testCreatedFileIsVisibleOnFlush.

@Test
public void testCreatedFileIsVisibleOnFlush() throws Throwable {
    describe("verify that a newly created file exists once a flush has taken " + "place");
    Path path = path("testCreatedFileIsVisibleOnFlush");
    FileSystem fs = getFileSystem();
    try (FSDataOutputStream out = fs.create(path, false, 4096, (short) 1, 1024)) {
        out.write('a');
        out.flush();
        if (!fs.exists(path)) {
            if (isSupported(IS_BLOBSTORE)) {
                // object store: downgrade to a skip so that the failure is visible
                // in test results
                skip("Filesystem is an object store and newly created files are not " + "immediately visible");
            }
            assertPathExists("expected path to be visible before file closed", path);
        }
    }
}