Search in sources :

Example 1 with ImmutableEntry

use of org.apache.drill.common.collections.ImmutableEntry in project drill by apache.

the class LocalPersistentStore method getRange.

@Override
public Iterator<Map.Entry<String, V>> getRange(int skip, int take) {
    try (AutoCloseableLock lock = readLock.open()) {
        try {
            List<FileStatus> f = fs.list(false, basePath);
            if (f == null || f.isEmpty()) {
                return Collections.emptyIterator();
            }
            List<String> files = Lists.newArrayList();
            for (FileStatus stat : f) {
                String s = stat.getPath().getName();
                if (s.endsWith(DRILL_SYS_FILE_SUFFIX)) {
                    files.add(s.substring(0, s.length() - DRILL_SYS_FILE_SUFFIX.length()));
                }
            }
            Collections.sort(files);
            return Iterables.transform(Iterables.limit(Iterables.skip(files, skip), take), new Function<String, Entry<String, V>>() {

                @Nullable
                @Override
                public Entry<String, V> apply(String key) {
                    return new ImmutableEntry<>(key, get(key));
                }
            }).iterator();
        } catch (IOException e) {
            throw new RuntimeException(e);
        }
    }
}
Also used : Function(com.google.common.base.Function) FileStatus(org.apache.hadoop.fs.FileStatus) ImmutableEntry(org.apache.drill.common.collections.ImmutableEntry) AutoCloseableLock(org.apache.drill.common.concurrent.AutoCloseableLock) IOException(java.io.IOException)

Example 2 with ImmutableEntry

use of org.apache.drill.common.collections.ImmutableEntry in project drill by axbaretto.

the class LocalPersistentStore method getRange.

@Override
public Iterator<Map.Entry<String, V>> getRange(int skip, int take) {
    try {
        // list only files with sys file suffix
        PathFilter sysFileSuffixFilter = new PathFilter() {

            @Override
            public boolean accept(Path path) {
                return path.getName().endsWith(DRILL_SYS_FILE_SUFFIX);
            }
        };
        List<FileStatus> fileStatuses = DrillFileSystemUtil.listFiles(fs, basePath, false, sysFileSuffixFilter);
        if (fileStatuses.isEmpty()) {
            return Collections.emptyIterator();
        }
        List<String> files = Lists.newArrayList();
        for (FileStatus stat : fileStatuses) {
            String s = stat.getPath().getName();
            files.add(s.substring(0, s.length() - DRILL_SYS_FILE_SUFFIX.length()));
        }
        Collections.sort(files);
        return Iterables.transform(Iterables.limit(Iterables.skip(files, skip), take), new Function<String, Entry<String, V>>() {

            @Nullable
            @Override
            public Entry<String, V> apply(String key) {
                return new ImmutableEntry<>(key, get(key));
            }
        }).iterator();
    } catch (IOException e) {
        throw new RuntimeException(e);
    }
}
Also used : Path(org.apache.hadoop.fs.Path) Function(com.google.common.base.Function) PathFilter(org.apache.hadoop.fs.PathFilter) FileStatus(org.apache.hadoop.fs.FileStatus) ImmutableEntry(org.apache.drill.common.collections.ImmutableEntry) IOException(java.io.IOException)

Example 3 with ImmutableEntry

use of org.apache.drill.common.collections.ImmutableEntry in project drill by axbaretto.

the class TestZookeeperClient method testEntriesReturnsRelativePaths.

@Test
public void testEntriesReturnsRelativePaths() throws Exception {
    final ChildData child = Mockito.mock(ChildData.class);
    Mockito.when(child.getPath()).thenReturn(abspath);
    Mockito.when(child.getData()).thenReturn(data);
    final List<ChildData> children = Lists.newArrayList(child);
    Mockito.when(client.getCache().getCurrentData()).thenReturn(children);
    final Iterator<Map.Entry<String, byte[]>> entries = client.entries();
    // returned entry must contain the given relative path
    final Map.Entry<String, byte[]> expected = new ImmutableEntry<>(path, data);
    assertEquals("entries do not match", expected, entries.next());
}
Also used : ImmutableEntry(org.apache.drill.common.collections.ImmutableEntry) ImmutableEntry(org.apache.drill.common.collections.ImmutableEntry) ChildData(org.apache.curator.framework.recipes.cache.ChildData) Map(java.util.Map) Test(org.junit.Test)

Example 4 with ImmutableEntry

use of org.apache.drill.common.collections.ImmutableEntry in project drill by apache.

the class TestZookeeperClient method testEntriesReturnsRelativePaths.

@Test
public void testEntriesReturnsRelativePaths() {
    final ChildData child = Mockito.mock(ChildData.class);
    Mockito.when(child.getPath()).thenReturn(abspath);
    Mockito.when(child.getData()).thenReturn(data);
    final List<ChildData> children = Lists.newArrayList(child);
    Mockito.when(client.getCache().getCurrentData()).thenReturn(children);
    final Iterator<Map.Entry<String, byte[]>> entries = client.entries();
    // returned entry must contain the given relative path
    final Map.Entry<String, byte[]> expected = new ImmutableEntry<>(path, data);
    assertEquals("entries do not match", expected, entries.next());
}
Also used : ImmutableEntry(org.apache.drill.common.collections.ImmutableEntry) ImmutableEntry(org.apache.drill.common.collections.ImmutableEntry) ChildData(org.apache.curator.framework.recipes.cache.ChildData) Map(java.util.Map) BaseTest(org.apache.drill.test.BaseTest) Test(org.junit.Test)

Aggregations

ImmutableEntry (org.apache.drill.common.collections.ImmutableEntry)4 Function (com.google.common.base.Function)2 IOException (java.io.IOException)2 Map (java.util.Map)2 ChildData (org.apache.curator.framework.recipes.cache.ChildData)2 FileStatus (org.apache.hadoop.fs.FileStatus)2 Test (org.junit.Test)2 AutoCloseableLock (org.apache.drill.common.concurrent.AutoCloseableLock)1 BaseTest (org.apache.drill.test.BaseTest)1 Path (org.apache.hadoop.fs.Path)1 PathFilter (org.apache.hadoop.fs.PathFilter)1