Search in sources :

Example 26 with FileNotFoundException

use of java.io.FileNotFoundException in project flink by apache.

the class DataSinkTaskTest method testUnionDataSinkTask.

@Test
public void testUnionDataSinkTask() {
    int keyCnt = 10;
    int valCnt = 20;
    super.initEnvironment(MEMORY_MANAGER_SIZE, NETWORK_BUFFER_SIZE);
    final IteratorWrappingTestSingleInputGate<?>[] readers = new IteratorWrappingTestSingleInputGate[4];
    readers[0] = super.addInput(new UniformRecordGenerator(keyCnt, valCnt, 0, 0, false), 0, false);
    readers[1] = super.addInput(new UniformRecordGenerator(keyCnt, valCnt, keyCnt, 0, false), 0, false);
    readers[2] = super.addInput(new UniformRecordGenerator(keyCnt, valCnt, keyCnt * 2, 0, false), 0, false);
    readers[3] = super.addInput(new UniformRecordGenerator(keyCnt, valCnt, keyCnt * 3, 0, false), 0, false);
    DataSinkTask<Record> testTask = new DataSinkTask<>();
    super.registerFileOutputTask(testTask, MockOutputFormat.class, new File(tempTestPath).toURI().toString());
    try {
        // which checks forwards existing notifications on registerListener calls.
        for (IteratorWrappingTestSingleInputGate<?> reader : readers) {
            reader.notifyNonEmpty();
        }
        testTask.invoke();
    } catch (Exception e) {
        LOG.debug("Exception while invoking the test task.", e);
        Assert.fail("Invoke method caused exception.");
    }
    File tempTestFile = new File(this.tempTestPath);
    Assert.assertTrue("Temp output file does not exist", tempTestFile.exists());
    FileReader fr = null;
    BufferedReader br = null;
    try {
        fr = new FileReader(tempTestFile);
        br = new BufferedReader(fr);
        HashMap<Integer, HashSet<Integer>> keyValueCountMap = new HashMap<>(keyCnt);
        while (br.ready()) {
            String line = br.readLine();
            Integer key = Integer.parseInt(line.substring(0, line.indexOf("_")));
            Integer val = Integer.parseInt(line.substring(line.indexOf("_") + 1, line.length()));
            if (!keyValueCountMap.containsKey(key)) {
                keyValueCountMap.put(key, new HashSet<Integer>());
            }
            keyValueCountMap.get(key).add(val);
        }
        Assert.assertTrue("Invalid key count in out file. Expected: " + keyCnt + " Actual: " + keyValueCountMap.keySet().size(), keyValueCountMap.keySet().size() == keyCnt * 4);
        for (Integer key : keyValueCountMap.keySet()) {
            Assert.assertTrue("Invalid value count for key: " + key + ". Expected: " + valCnt + " Actual: " + keyValueCountMap.get(key).size(), keyValueCountMap.get(key).size() == valCnt);
        }
    } catch (FileNotFoundException e) {
        Assert.fail("Out file got lost...");
    } catch (IOException ioe) {
        Assert.fail("Caught IOE while reading out file");
    } finally {
        if (br != null) {
            try {
                br.close();
            } catch (Throwable t) {
            }
        }
        if (fr != null) {
            try {
                fr.close();
            } catch (Throwable t) {
            }
        }
    }
}
Also used : HashMap(java.util.HashMap) FileNotFoundException(java.io.FileNotFoundException) IOException(java.io.IOException) IOException(java.io.IOException) FileNotFoundException(java.io.FileNotFoundException) IteratorWrappingTestSingleInputGate(org.apache.flink.runtime.io.network.partition.consumer.IteratorWrappingTestSingleInputGate) BufferedReader(java.io.BufferedReader) Record(org.apache.flink.types.Record) FileReader(java.io.FileReader) UniformRecordGenerator(org.apache.flink.runtime.operators.testutils.UniformRecordGenerator) File(java.io.File) HashSet(java.util.HashSet) Test(org.junit.Test)

Example 27 with FileNotFoundException

use of java.io.FileNotFoundException in project hadoop by apache.

the class FileSystem method rename.

/**
   * Renames Path src to Path dst
   * <ul>
   *   <li>Fails if src is a file and dst is a directory.</li>
   *   <li>Fails if src is a directory and dst is a file.</li>
   *   <li>Fails if the parent of dst does not exist or is a file.</li>
   * </ul>
   * <p>
   * If OVERWRITE option is not passed as an argument, rename fails
   * if the dst already exists.
   * <p>
   * If OVERWRITE option is passed as an argument, rename overwrites
   * the dst if it is a file or an empty directory. Rename fails if dst is
   * a non-empty directory.
   * <p>
   * Note that atomicity of rename is dependent on the file system
   * implementation. Please refer to the file system documentation for
   * details. This default implementation is non atomic.
   * <p>
   * This method is deprecated since it is a temporary method added to
   * support the transition from FileSystem to FileContext for user
   * applications.
   *
   * @param src path to be renamed
   * @param dst new path after rename
   * @throws FileNotFoundException src path does not exist, or the parent
   * path of dst does not exist.
   * @throws FileAlreadyExistsException dest path exists and is a file
   * @throws ParentNotDirectoryException if the parent path of dest is not
   * a directory
   * @throws IOException on failure
   */
@Deprecated
protected void rename(final Path src, final Path dst, final Rename... options) throws IOException {
    // Default implementation
    final FileStatus srcStatus = getFileLinkStatus(src);
    if (srcStatus == null) {
        throw new FileNotFoundException("rename source " + src + " not found.");
    }
    boolean overwrite = false;
    if (null != options) {
        for (Rename option : options) {
            if (option == Rename.OVERWRITE) {
                overwrite = true;
            }
        }
    }
    FileStatus dstStatus;
    try {
        dstStatus = getFileLinkStatus(dst);
    } catch (IOException e) {
        dstStatus = null;
    }
    if (dstStatus != null) {
        if (srcStatus.isDirectory() != dstStatus.isDirectory()) {
            throw new IOException("Source " + src + " Destination " + dst + " both should be either file or directory");
        }
        if (!overwrite) {
            throw new FileAlreadyExistsException("rename destination " + dst + " already exists.");
        }
        // Delete the destination that is a file or an empty directory
        if (dstStatus.isDirectory()) {
            FileStatus[] list = listStatus(dst);
            if (list != null && list.length != 0) {
                throw new IOException("rename cannot overwrite non empty destination directory " + dst);
            }
        }
        delete(dst, false);
    } else {
        final Path parent = dst.getParent();
        final FileStatus parentStatus = getFileStatus(parent);
        if (parentStatus == null) {
            throw new FileNotFoundException("rename destination parent " + parent + " not found.");
        }
        if (!parentStatus.isDirectory()) {
            throw new ParentNotDirectoryException("rename destination parent " + parent + " is a file.");
        }
    }
    if (!rename(src, dst)) {
        throw new IOException("rename from " + src + " to " + dst + " failed.");
    }
}
Also used : FileNotFoundException(java.io.FileNotFoundException) IOException(java.io.IOException) MultipleIOException(org.apache.hadoop.io.MultipleIOException) Rename(org.apache.hadoop.fs.Options.Rename)

Example 28 with FileNotFoundException

use of java.io.FileNotFoundException in project hadoop by apache.

the class JavaKeyStoreProvider method flush.

@Override
public void flush() throws IOException {
    Path newPath = constructNewPath(path);
    Path oldPath = constructOldPath(path);
    Path resetPath = path;
    writeLock.lock();
    try {
        if (!changed) {
            return;
        }
        // Might exist if a backup has been restored etc.
        try {
            renameOrFail(newPath, new Path(newPath.toString() + "_ORPHANED_" + System.currentTimeMillis()));
        } catch (FileNotFoundException ignored) {
        }
        try {
            renameOrFail(oldPath, new Path(oldPath.toString() + "_ORPHANED_" + System.currentTimeMillis()));
        } catch (FileNotFoundException ignored) {
        }
        // put all of the updates into the keystore
        for (Map.Entry<String, Metadata> entry : cache.entrySet()) {
            try {
                keyStore.setKeyEntry(entry.getKey(), new KeyMetadata(entry.getValue()), password, null);
            } catch (KeyStoreException e) {
                throw new IOException("Can't set metadata key " + entry.getKey(), e);
            }
        }
        // Save old File first
        boolean fileExisted = backupToOld(oldPath);
        if (fileExisted) {
            resetPath = oldPath;
        }
        // Write to _NEW path first :
        try {
            writeToNew(newPath);
        } catch (IOException ioe) {
            // rename _OLD back to curent and throw Exception
            revertFromOld(oldPath, fileExisted);
            resetPath = path;
            throw ioe;
        }
        // Rename _NEW to CURRENT and delete _OLD
        cleanupNewAndOld(newPath, oldPath);
        changed = false;
    } catch (IOException ioe) {
        resetKeyStoreState(resetPath);
        throw ioe;
    } finally {
        writeLock.unlock();
    }
}
Also used : Path(org.apache.hadoop.fs.Path) FileNotFoundException(java.io.FileNotFoundException) KeyStoreException(java.security.KeyStoreException) IOException(java.io.IOException) HashMap(java.util.HashMap) Map(java.util.Map)

Example 29 with FileNotFoundException

use of java.io.FileNotFoundException in project hadoop by apache.

the class RawLocalFileSystem method setTimes.

/**
   * Sets the {@link Path}'s last modified time and last access time to
   * the given valid times.
   *
   * @param mtime the modification time to set (only if no less than zero).
   * @param atime the access time to set (only if no less than zero).
   * @throws IOException if setting the times fails.
   */
@Override
public void setTimes(Path p, long mtime, long atime) throws IOException {
    try {
        BasicFileAttributeView view = Files.getFileAttributeView(pathToFile(p).toPath(), BasicFileAttributeView.class);
        FileTime fmtime = (mtime >= 0) ? FileTime.fromMillis(mtime) : null;
        FileTime fatime = (atime >= 0) ? FileTime.fromMillis(atime) : null;
        view.setTimes(fmtime, fatime, null);
    } catch (NoSuchFileException e) {
        throw new FileNotFoundException("File " + p + " does not exist");
    }
}
Also used : BasicFileAttributeView(java.nio.file.attribute.BasicFileAttributeView) NoSuchFileException(java.nio.file.NoSuchFileException) FileNotFoundException(java.io.FileNotFoundException) FileTime(java.nio.file.attribute.FileTime)

Example 30 with FileNotFoundException

use of java.io.FileNotFoundException in project groovy by apache.

the class JavaStubGenerator method generateClass.

public void generateClass(ClassNode classNode) throws FileNotFoundException {
    // Only attempt to render our self if our super-class is resolved, else wait for it
    if (requireSuperResolved && !classNode.getSuperClass().isResolved()) {
        return;
    }
    // owner should take care for us
    if (classNode instanceof InnerClassNode)
        return;
    // don't generate stubs for private classes, as they are only visible in the same file
    if ((classNode.getModifiers() & Opcodes.ACC_PRIVATE) != 0)
        return;
    String fileName = classNode.getName().replace('.', '/');
    mkdirs(outputPath, fileName);
    toCompile.add(fileName);
    File file = new File(outputPath, fileName + ".java");
    FileOutputStream fos = new FileOutputStream(file);
    Charset charset = Charset.forName(encoding);
    PrintWriter out = new PrintWriter(new OutputStreamWriter(fos, charset));
    try {
        String packageName = classNode.getPackageName();
        if (packageName != null) {
            out.println("package " + packageName + ";\n");
        }
        printImports(out, classNode);
        printClassContents(out, classNode);
    } finally {
        try {
            out.close();
        } catch (Exception e) {
        // ignore
        }
        try {
            fos.close();
        } catch (IOException e) {
        // ignore
        }
    }
}
Also used : FileOutputStream(java.io.FileOutputStream) Charset(java.nio.charset.Charset) OutputStreamWriter(java.io.OutputStreamWriter) IOException(java.io.IOException) File(java.io.File) IOException(java.io.IOException) FileNotFoundException(java.io.FileNotFoundException) PrintWriter(java.io.PrintWriter)

Aggregations

FileNotFoundException (java.io.FileNotFoundException)3218 IOException (java.io.IOException)1836 File (java.io.File)1277 FileInputStream (java.io.FileInputStream)814 FileOutputStream (java.io.FileOutputStream)492 InputStream (java.io.InputStream)466 BufferedReader (java.io.BufferedReader)262 FileReader (java.io.FileReader)230 ArrayList (java.util.ArrayList)205 Path (org.apache.hadoop.fs.Path)202 XmlPullParserException (org.xmlpull.v1.XmlPullParserException)189 InputStreamReader (java.io.InputStreamReader)171 Test (org.junit.Test)171 XmlPullParser (org.xmlpull.v1.XmlPullParser)166 BufferedInputStream (java.io.BufferedInputStream)138 ParcelFileDescriptor (android.os.ParcelFileDescriptor)131 Properties (java.util.Properties)120 URL (java.net.URL)119 FileStatus (org.apache.hadoop.fs.FileStatus)119 RandomAccessFile (java.io.RandomAccessFile)101