Examples with AlluxioURI alluxio.AlluxioURI used on opensource projects

Example 16 with AlluxioURI

use of alluxio.AlluxioURI in project alluxio by Alluxio.

the class KeyValueSystemIntegrationTest method createStoreOfMultiplePartitions.

/**
   * Creates a store with the specified number of partitions.
   *
   * NOTE: calling this method will set {@link PropertyKey#KEY_VALUE_PARTITION_SIZE_BYTES_MAX} to
   * {@link Constants#MB}.
   *
   * @param partitionNumber the number of partitions
   * @param keyValuePairs the key-value pairs in the store, null if you don't want to know them
   * @return the URI to the created store
   */
private AlluxioURI createStoreOfMultiplePartitions(int partitionNumber, List<KeyValuePair> keyValuePairs) throws Exception {
    // These sizes are carefully selected, one partition holds only one key-value pair.
    // Each partition is at most 1 MB
    final long maxPartitionSize = Constants.MB;
    Configuration.set(PropertyKey.KEY_VALUE_PARTITION_SIZE_BYTES_MAX, String.valueOf(maxPartitionSize));
    // 4Byte key
    final int keyLength = 4;
    // 500KB value
    final int valueLength = 500 * Constants.KB;
    AlluxioURI storeUri = new AlluxioURI(PathUtils.uniqPath());
    mWriter = sKeyValueSystem.createStore(storeUri);
    for (int i = 0; i < partitionNumber; i++) {
        byte[] key = BufferUtils.getIncreasingByteArray(i, keyLength);
        byte[] value = BufferUtils.getIncreasingByteArray(i, valueLength);
        mWriter.put(key, value);
        if (keyValuePairs != null) {
            keyValuePairs.add(new KeyValuePair(key, value));
        }
    }
    mWriter.close();
    Assert.assertEquals(partitionNumber, getPartitionNumber(storeUri));
    return storeUri;
}

Example 17 with AlluxioURI

use of alluxio.AlluxioURI in project alluxio by Alluxio.

the class KeyValueSystemIntegrationTest method createStoreOfSize.

/**
   * Creates a store with the specified number of key-value pairs. The key-value pairs are in the
   * format specified in {@link #genBaseKey(int)} and {@link #genBaseValue(int)} with id starts
   * from 0.
   *
   * The created store's size is {@link Assert}ed before return.
   *
   * @param size the number of key-value pairs
   * @param pairs the key-value pairs in the store, null if you don't want to know them
   * @return the URI to the store
   */
private AlluxioURI createStoreOfSize(int size, List<KeyValuePair> pairs) throws Exception {
    AlluxioURI path = new AlluxioURI(PathUtils.uniqPath());
    KeyValueStoreWriter writer = sKeyValueSystem.createStore(path);
    for (int i = 0; i < size; i++) {
        byte[] key = genBaseKey(i).getBytes();
        byte[] value = genBaseValue(i).getBytes();
        writer.put(key, value);
        if (pairs != null) {
            pairs.add(new KeyValuePair(key, value));
        }
    }
    writer.close();
    Assert.assertEquals(size, sKeyValueSystem.openStore(path).size());
    return path;
}

Example 18 with AlluxioURI

use of alluxio.AlluxioURI in project alluxio by Alluxio.

the class HdfsFileInputStreamIntegrationTest method positionedReadCacheNoPartialCache.

@Test
public void positionedReadCacheNoPartialCache() throws IOException, AlluxioException {
    createUfsInStreamNoPartialcache(ReadType.CACHE);
    mUfsInputStream.readFully(0, new byte[FILE_LEN - 1]);
    URIStatus statusUfsOnlyFile = mFileSystem.getStatus(new AlluxioURI(UFS_ONLY_FILE));
    Assert.assertEquals(0, statusUfsOnlyFile.getInMemoryPercentage());
}

Example 19 with AlluxioURI

use of alluxio.AlluxioURI in project alluxio by Alluxio.

the class HdfsFileInputStreamIntegrationTest method after.

@After
public final void after() throws IOException, AlluxioException {
    mInMemInputStream.close();
    mFileSystem.delete(new AlluxioURI(IN_MEMORY_FILE));
    if (mUfsInputStream != null) {
        mUfsInputStream.close();
        mFileSystem.delete(new AlluxioURI(UFS_ONLY_FILE));
    }
    ClientTestUtils.resetClient();
}

Example 20 with AlluxioURI

use of alluxio.AlluxioURI in project alluxio by Alluxio.

the class HdfsFileInputStreamIntegrationTest method positionedReadCache.

@Test
public void positionedReadCache() throws IOException, AlluxioException {
    createUfsInStream(ReadType.CACHE);
    mUfsInputStream.readFully(0, new byte[FILE_LEN]);
    URIStatus statusUfsOnlyFile = mFileSystem.getStatus(new AlluxioURI(UFS_ONLY_FILE));
    Assert.assertEquals(100, statusUfsOnlyFile.getInMemoryPercentage());
}