Search in sources :

Example 1 with FileSystemMasterClient

use of alluxio.worker.file.FileSystemMasterClient in project alluxio by Alluxio.

the class BlockWorkerDataReaderTest method before.

@Before
public void before() throws Exception {
    BlockMasterClient blockMasterClient = mock(BlockMasterClient.class);
    BlockMasterClientPool blockMasterClientPool = spy(new BlockMasterClientPool());
    when(blockMasterClientPool.createNewResource()).thenReturn(blockMasterClient);
    TieredBlockStore blockStore = new TieredBlockStore();
    FileSystemMasterClient fileSystemMasterClient = mock(FileSystemMasterClient.class);
    Sessions sessions = mock(Sessions.class);
    // Connect to the real UFS for UFS read testing
    UfsManager ufsManager = mock(UfsManager.class);
    mRootUfs = ServerConfiguration.getString(PropertyKey.MASTER_MOUNT_TABLE_ROOT_UFS);
    UfsManager.UfsClient ufsClient = new UfsManager.UfsClient(() -> UnderFileSystem.Factory.create(mRootUfs, UnderFileSystemConfiguration.defaults(ServerConfiguration.global())), new AlluxioURI(mRootUfs));
    when(ufsManager.get(anyLong())).thenReturn(ufsClient);
    mBlockWorker = new DefaultBlockWorker(blockMasterClientPool, fileSystemMasterClient, sessions, blockStore, ufsManager);
    URIStatus dummyStatus = new URIStatus(new FileInfo().setBlockIds(Collections.singletonList(BLOCK_ID)));
    InStreamOptions options = new InStreamOptions(dummyStatus, FileSystemOptions.openFileDefaults(mConf), mConf);
    mDataReaderFactory = new BlockWorkerDataReader.Factory(mBlockWorker, BLOCK_ID, CHUNK_SIZE, options);
}
Also used : BlockMasterClientPool(alluxio.worker.block.BlockMasterClientPool) UfsManager(alluxio.underfs.UfsManager) Sessions(alluxio.Sessions) URIStatus(alluxio.client.file.URIStatus) TieredBlockStore(alluxio.worker.block.TieredBlockStore) InStreamOptions(alluxio.client.file.options.InStreamOptions) BlockWorkerDataReader(alluxio.client.block.stream.BlockWorkerDataReader) FileSystemMasterClient(alluxio.worker.file.FileSystemMasterClient) FileInfo(alluxio.wire.FileInfo) BlockMasterClient(alluxio.worker.block.BlockMasterClient) DefaultBlockWorker(alluxio.worker.block.DefaultBlockWorker) AlluxioURI(alluxio.AlluxioURI) Before(org.junit.Before)

Example 2 with FileSystemMasterClient

use of alluxio.worker.file.FileSystemMasterClient in project alluxio by Alluxio.

the class BlockWorkerRegisterStreamIntegrationTest method initBlockWorker.

public void initBlockWorker() throws Exception {
    // Prepare a block worker
    mBlockMasterClientPool = spy(new BlockMasterClientPool());
    when(mBlockMasterClientPool.createNewResource()).thenReturn(mBlockMasterClient);
    when(mBlockMasterClientPool.acquire()).thenReturn(mBlockMasterClient);
    TieredBlockStore mBlockStore = spy(new TieredBlockStore());
    FileSystemMasterClient mFileSystemMasterClient = mock(FileSystemMasterClient.class);
    Sessions mSessions = mock(Sessions.class);
    UfsManager mUfsManager = mock(UfsManager.class);
    mBlockWorker = new DefaultBlockWorker(mBlockMasterClientPool, mFileSystemMasterClient, mSessions, mBlockStore, mUfsManager);
}
Also used : BlockMasterClientPool(alluxio.worker.block.BlockMasterClientPool) FileSystemMasterClient(alluxio.worker.file.FileSystemMasterClient) UfsManager(alluxio.underfs.UfsManager) Sessions(alluxio.Sessions) DefaultBlockWorker(alluxio.worker.block.DefaultBlockWorker) TieredBlockStore(alluxio.worker.block.TieredBlockStore)

Example 3 with FileSystemMasterClient

use of alluxio.worker.file.FileSystemMasterClient in project alluxio by Alluxio.

the class PinIntegrationTest method before.

@Before
public final void before() throws Exception {
    mFileSystem = mLocalAlluxioClusterResource.get().getClient();
    mFSMasterClient = new FileSystemMasterClient(new InetSocketAddress(mLocalAlluxioClusterResource.get().getHostname(), mLocalAlluxioClusterResource.get().getMasterRpcPort()));
    mSetPinned = SetAttributeOptions.defaults().setPinned(true);
    mUnsetPinned = SetAttributeOptions.defaults().setPinned(false);
}
Also used : FileSystemMasterClient(alluxio.worker.file.FileSystemMasterClient) InetSocketAddress(java.net.InetSocketAddress) Before(org.junit.Before)

Example 4 with FileSystemMasterClient

use of alluxio.worker.file.FileSystemMasterClient in project alluxio by Alluxio.

the class PinIntegrationTest method before.

@Before
public final void before() throws Exception {
    mFileSystem = mLocalAlluxioClusterResource.get().getClient();
    mFSMasterClient = new FileSystemMasterClient(MasterClientContext.newBuilder(ClientContext.create(ServerConfiguration.global())).build());
    mSetPinned = SetAttributePOptions.newBuilder().setPinned(true).build();
    mUnsetPinned = SetAttributePOptions.newBuilder().setPinned(false).build();
    mFileSystem.mount(new AlluxioURI("/mnt/"), new AlluxioURI(mLocalUfsPath));
}
Also used : FileSystemMasterClient(alluxio.worker.file.FileSystemMasterClient) AlluxioURI(alluxio.AlluxioURI) Before(org.junit.Before)

Example 5 with FileSystemMasterClient

use of alluxio.worker.file.FileSystemMasterClient in project alluxio by Alluxio.

the class CacheRequestManagerTest method before.

/**
 * Sets up all dependencies before a test runs.
 */
@Before
public void before() throws IOException {
    BlockMasterClient blockMasterClient = mock(BlockMasterClient.class);
    BlockMasterClientPool blockMasterClientPool = spy(new BlockMasterClientPool());
    when(blockMasterClientPool.createNewResource()).thenReturn(blockMasterClient);
    TieredBlockStore blockStore = new TieredBlockStore();
    FileSystemMasterClient fileSystemMasterClient = mock(FileSystemMasterClient.class);
    Sessions sessions = mock(Sessions.class);
    // Connect to the real UFS for testing
    UfsManager ufsManager = mock(UfsManager.class);
    mRootUfs = ServerConfiguration.getString(PropertyKey.MASTER_MOUNT_TABLE_ROOT_UFS);
    UfsManager.UfsClient ufsClient = new UfsManager.UfsClient(() -> UnderFileSystem.Factory.create(mRootUfs, UnderFileSystemConfiguration.defaults(ServerConfiguration.global())), new AlluxioURI(mRootUfs));
    when(ufsManager.get(anyLong())).thenReturn(ufsClient);
    mBlockWorker = spy(new DefaultBlockWorker(blockMasterClientPool, fileSystemMasterClient, sessions, blockStore, ufsManager));
    FileSystemContext context = mock(FileSystemContext.class);
    mCacheRequestManager = spy(new CacheRequestManager(GrpcExecutors.CACHE_MANAGER_EXECUTOR, mBlockWorker, context));
    // Write an actual file to UFS
    String testFilePath = File.createTempFile("temp", null, new File(mRootUfs)).getAbsolutePath();
    byte[] buffer = BufferUtils.getIncreasingByteArray(CHUNK_SIZE);
    BufferUtils.writeBufferToFile(testFilePath, buffer);
    // create options
    BlockInfo info = new BlockInfo().setBlockId(BLOCK_ID).setLength(CHUNK_SIZE);
    URIStatus dummyStatus = new URIStatus(new FileInfo().setPersisted(true).setUfsPath(testFilePath).setBlockIds(Collections.singletonList(BLOCK_ID)).setLength(CHUNK_SIZE).setBlockSizeBytes(CHUNK_SIZE).setFileBlockInfos(Collections.singletonList(new FileBlockInfo().setBlockInfo(info))));
    OpenFilePOptions readOptions = FileSystemOptions.openFileDefaults(mConf);
    InStreamOptions options = new InStreamOptions(dummyStatus, readOptions, mConf);
    mOpenUfsBlockOptions = options.getOpenUfsBlockOptions(BLOCK_ID);
}
Also used : UfsManager(alluxio.underfs.UfsManager) Sessions(alluxio.Sessions) URIStatus(alluxio.client.file.URIStatus) FileBlockInfo(alluxio.wire.FileBlockInfo) InStreamOptions(alluxio.client.file.options.InStreamOptions) FileSystemMasterClient(alluxio.worker.file.FileSystemMasterClient) FileInfo(alluxio.wire.FileInfo) BlockInfo(alluxio.wire.BlockInfo) FileBlockInfo(alluxio.wire.FileBlockInfo) FileSystemContext(alluxio.client.file.FileSystemContext) OpenFilePOptions(alluxio.grpc.OpenFilePOptions) File(java.io.File) AlluxioURI(alluxio.AlluxioURI) Before(org.junit.Before)

Aggregations

FileSystemMasterClient (alluxio.worker.file.FileSystemMasterClient)6 Before (org.junit.Before)5 AlluxioURI (alluxio.AlluxioURI)3 Sessions (alluxio.Sessions)3 UfsManager (alluxio.underfs.UfsManager)3 URIStatus (alluxio.client.file.URIStatus)2 InStreamOptions (alluxio.client.file.options.InStreamOptions)2 FileInfo (alluxio.wire.FileInfo)2 BlockMasterClientPool (alluxio.worker.block.BlockMasterClientPool)2 DefaultBlockWorker (alluxio.worker.block.DefaultBlockWorker)2 TieredBlockStore (alluxio.worker.block.TieredBlockStore)2 BlockWorkerDataReader (alluxio.client.block.stream.BlockWorkerDataReader)1 FileSystemContext (alluxio.client.file.FileSystemContext)1 OpenFilePOptions (alluxio.grpc.OpenFilePOptions)1 MasterClientContext (alluxio.master.MasterClientContext)1 BlockInfo (alluxio.wire.BlockInfo)1 FileBlockInfo (alluxio.wire.FileBlockInfo)1 BlockMasterClient (alluxio.worker.block.BlockMasterClient)1 File (java.io.File)1 InetSocketAddress (java.net.InetSocketAddress)1