Search in sources :

Example 1 with ListStatusPOptions

use of alluxio.grpc.ListStatusPOptions in project alluxio by Alluxio.

the class BaseFileSystem method iterateStatus.

@Override
public void iterateStatus(AlluxioURI path, final ListStatusPOptions options, Consumer<? super URIStatus> action) throws FileDoesNotExistException, IOException, AlluxioException {
    checkUri(path);
    rpc(client -> {
        // TODO(calvin): Fix the exception handling in the master
        ListStatusPOptions mergedOptions = FileSystemOptions.listStatusDefaults(mFsContext.getPathConf(path)).toBuilder().mergeFrom(options).build();
        client.iterateStatus(path, mergedOptions, action);
        return null;
    });
}
Also used : ListStatusPOptions(alluxio.grpc.ListStatusPOptions)

Example 2 with ListStatusPOptions

use of alluxio.grpc.ListStatusPOptions in project alluxio by Alluxio.

the class FileSystem method loadMetadata.

/**
 * Convenience method for {@link #loadMetadata(AlluxioURI, ListStatusPOptions)} with default
 * options.
 *
 * @param path the path for which to load metadata from UFS
 * @throws FileDoesNotExistException if the given path does not exist
 */
default void loadMetadata(AlluxioURI path) throws FileDoesNotExistException, IOException, AlluxioException {
    ListStatusPOptions options = ListStatusPOptions.newBuilder().setLoadMetadataType(LoadMetadataPType.ALWAYS).setRecursive(LoadMetadataPOptions.getDefaultInstance().getRecursive()).setLoadMetadataOnly(true).build();
    loadMetadata(path, options);
}
Also used : ListStatusPOptions(alluxio.grpc.ListStatusPOptions)

Example 3 with ListStatusPOptions

use of alluxio.grpc.ListStatusPOptions in project alluxio by Alluxio.

the class BaseFileSystemTest method listStatus.

/**
 * Tests for the {@link BaseFileSystem#listStatus(AlluxioURI, ListStatusPOptions)} method.
 */
@Test
public void listStatus() throws Exception {
    AlluxioURI file = new AlluxioURI("/file");
    List<URIStatus> infos = new ArrayList<>();
    infos.add(new URIStatus(new FileInfo()));
    ListStatusPOptions listStatusOptions = ListStatusPOptions.getDefaultInstance();
    when(mFileSystemMasterClient.listStatus(file, FileSystemOptions.listStatusDefaults(mConf).toBuilder().mergeFrom(listStatusOptions).build())).thenReturn(infos);
    assertSame(infos, mFileSystem.listStatus(file, listStatusOptions));
    verify(mFileSystemMasterClient).listStatus(file, FileSystemOptions.listStatusDefaults(mConf).toBuilder().mergeFrom(listStatusOptions).build());
    verifyFilesystemContextAcquiredAndReleased();
}
Also used : FileInfo(alluxio.wire.FileInfo) ArrayList(java.util.ArrayList) AlluxioURI(alluxio.AlluxioURI) ListStatusPOptions(alluxio.grpc.ListStatusPOptions) PrepareForTest(org.powermock.core.classloader.annotations.PrepareForTest) Test(org.junit.Test)

Example 4 with ListStatusPOptions

use of alluxio.grpc.ListStatusPOptions in project alluxio by Alluxio.

the class UfsSyncIntegrationTest method listDirSyncOnlyTouchingChildren.

// https://github.com/Alluxio/alluxio/issues/12372
@Test
public void listDirSyncOnlyTouchingChildren() throws Exception {
    String dir1 = PathUtils.concatPath(EXISTING_DIR, "dir_should_sync");
    String dir2 = PathUtils.concatPath(dir1, "dir_should_not_sync");
    new File(ufsPath(dir1)).mkdirs();
    new File(ufsPath(dir2)).mkdirs();
    ListStatusPOptions optionsAlways = ListStatusPOptions.newBuilder().setLoadMetadataType(LoadMetadataPType.NEVER).setRecursive(false).setCommonOptions(PSYNC_ALWAYS).build();
    checkListStatus(EXISTING_DIR, optionsAlways, true);
    ListStatusPOptions optionsNever = ListStatusPOptions.newBuilder().setLoadMetadataType(LoadMetadataPType.NEVER).setCommonOptions(PSYNC_NEVER).setRecursive(false).build();
    checkListStatus(dir2, optionsNever, false);
}
Also used : File(java.io.File) ListStatusPOptions(alluxio.grpc.ListStatusPOptions) BaseIntegrationTest(alluxio.testutils.BaseIntegrationTest) Test(org.junit.Test)

Example 5 with ListStatusPOptions

use of alluxio.grpc.ListStatusPOptions in project alluxio by Alluxio.

the class UfsSyncIntegrationTest method getStatusDirSyncOnlyTouchingChildren.

// https://github.com/Alluxio/alluxio/issues/12372
@Test
public void getStatusDirSyncOnlyTouchingChildren() throws Exception {
    String dir1 = PathUtils.concatPath(EXISTING_DIR, "dir_should_sync");
    String dir2 = PathUtils.concatPath(dir1, "dir_should_not_sync");
    new File(ufsPath(dir1)).mkdirs();
    new File(ufsPath(dir2)).mkdirs();
    GetStatusPOptions optionsAlways = GetStatusPOptions.newBuilder().setLoadMetadataType(LoadMetadataPType.NEVER).setCommonOptions(PSYNC_ALWAYS).build();
    checkGetStatus(EXISTING_DIR, optionsAlways, true);
    ListStatusPOptions optionsNever = ListStatusPOptions.newBuilder().setLoadMetadataType(LoadMetadataPType.NEVER).setCommonOptions(PSYNC_NEVER).setRecursive(false).build();
    checkListStatus(dir2, optionsNever, false);
}
Also used : File(java.io.File) GetStatusPOptions(alluxio.grpc.GetStatusPOptions) ListStatusPOptions(alluxio.grpc.ListStatusPOptions) BaseIntegrationTest(alluxio.testutils.BaseIntegrationTest) Test(org.junit.Test)

Aggregations

ListStatusPOptions (alluxio.grpc.ListStatusPOptions)34 Test (org.junit.Test)24 BaseIntegrationTest (alluxio.testutils.BaseIntegrationTest)21 AlluxioURI (alluxio.AlluxioURI)17 URIStatus (alluxio.client.file.URIStatus)14 File (java.io.File)9 GetStatusPOptions (alluxio.grpc.GetStatusPOptions)7 HashMap (java.util.HashMap)3 FileSystem (alluxio.client.file.FileSystem)2 AlluxioException (alluxio.exception.AlluxioException)2 FileDoesNotExistException (alluxio.exception.FileDoesNotExistException)2 ListBucketResult (alluxio.proxy.s3.ListBucketResult)2 FileWriter (java.io.FileWriter)2 IOException (java.io.IOException)2 AtomicLong (java.util.concurrent.atomic.AtomicLong)2 AuthenticatedUserRule (alluxio.AuthenticatedUserRule)1 Constants (alluxio.Constants)1 UnderFileSystemFactoryRegistryRule (alluxio.UnderFileSystemFactoryRegistryRule)1 FileOutStream (alluxio.client.file.FileOutStream)1 PropertyKey (alluxio.conf.PropertyKey)1