Search in sources :

Example 1 with FileSystemCommand

use of alluxio.wire.FileSystemCommand in project alluxio by Alluxio.

the class DefaultFileSystemMaster method workerHeartbeat.

@Override
public FileSystemCommand workerHeartbeat(long workerId, List<Long> persistedFiles, WorkerHeartbeatContext context) throws IOException {
    List<String> persistedUfsFingerprints = context.getOptions().getPersistedFileFingerprintsList();
    boolean hasPersistedFingerprints = persistedUfsFingerprints.size() == persistedFiles.size();
    for (int i = 0; i < persistedFiles.size(); i++) {
        long fileId = persistedFiles.get(i);
        String ufsFingerprint = hasPersistedFingerprints ? persistedUfsFingerprints.get(i) : Constants.INVALID_UFS_FINGERPRINT;
        try {
            // Permission checking for each file is performed inside setAttribute
            setAttribute(getPath(fileId), SetAttributeContext.mergeFrom(SetAttributePOptions.newBuilder().setPersisted(true)).setUfsFingerprint(ufsFingerprint));
        } catch (FileDoesNotExistException | AccessControlException | InvalidPathException e) {
            LOG.error("Failed to set file {} as persisted, because {}", fileId, e);
        }
    }
    // TODO(zac) Clean up master and worker code since this is taken care of by job service now.
    // Worker should not persist any files. Instead, files are persisted through job service.
    List<PersistFile> filesToPersist = new ArrayList<>();
    FileSystemCommandOptions commandOptions = new FileSystemCommandOptions();
    commandOptions.setPersistOptions(new PersistCommandOptions(filesToPersist));
    return new FileSystemCommand(CommandType.PERSIST, commandOptions);
}
Also used : FileDoesNotExistException(alluxio.exception.FileDoesNotExistException) ArrayList(java.util.ArrayList) AccessControlException(alluxio.exception.AccessControlException) FileSystemCommandOptions(alluxio.wire.FileSystemCommandOptions) FileSystemCommand(alluxio.wire.FileSystemCommand) Fingerprint(alluxio.underfs.Fingerprint) InvalidPathException(alluxio.exception.InvalidPathException) PersistFile(alluxio.wire.PersistFile) PersistCommandOptions(alluxio.wire.PersistCommandOptions)

Example 2 with FileSystemCommand

use of alluxio.wire.FileSystemCommand in project alluxio by Alluxio.

the class FileSystemMasterTest method workerHeartbeat.

/**
 * Tests the {@link FileSystemMaster#workerHeartbeat} method.
 */
@Test
public void workerHeartbeat() throws Exception {
    long blockId = createFileWithSingleBlock(ROOT_FILE_URI);
    long fileId = mFileSystemMaster.getFileId(ROOT_FILE_URI);
    mFileSystemMaster.scheduleAsyncPersistence(ROOT_FILE_URI, ScheduleAsyncPersistenceContext.defaults());
    FileSystemCommand command = mFileSystemMaster.workerHeartbeat(mWorkerId1, Lists.newArrayList(fileId), WorkerHeartbeatContext.defaults());
    assertEquals(alluxio.wire.CommandType.PERSIST, command.getCommandType());
    assertEquals(0, command.getCommandOptions().getPersistOptions().getFilesToPersist().size());
}
Also used : FileSystemCommand(alluxio.wire.FileSystemCommand) Test(org.junit.Test)

Aggregations

FileSystemCommand (alluxio.wire.FileSystemCommand)2 AccessControlException (alluxio.exception.AccessControlException)1 FileDoesNotExistException (alluxio.exception.FileDoesNotExistException)1 InvalidPathException (alluxio.exception.InvalidPathException)1 Fingerprint (alluxio.underfs.Fingerprint)1 FileSystemCommandOptions (alluxio.wire.FileSystemCommandOptions)1 PersistCommandOptions (alluxio.wire.PersistCommandOptions)1 PersistFile (alluxio.wire.PersistFile)1 ArrayList (java.util.ArrayList)1 Test (org.junit.Test)1