Search in sources :

Example 11 with FileBasedLock

use of io.prestosql.spi.filesystem.FileBasedLock in project hetu-core by openlookeng.

the class FileBasedSeedStore method add.

@Override
public Set<Seed> add(Collection<Seed> seeds) throws IOException {
    Lock lock = new FileBasedLock(fs, seedDir.resolve(name));
    try {
        lock.lock();
        Set<String> writtens = new HashSet<>(0);
        Set<Seed> outputs = getSeeds();
        // overwrite all seeds
        outputs.removeAll(seeds);
        outputs.addAll(seeds);
        for (Seed seed : outputs) {
            writtens.add(seed.serialize());
        }
        writeToFile(seedFilePath, String.join(COMMA, writtens), true);
        return outputs;
    } catch (UncheckedIOException e) {
        throw new IOException(e);
    } finally {
        lock.unlock();
    }
}
Also used : FileBasedLock(io.prestosql.spi.filesystem.FileBasedLock) Seed(io.prestosql.spi.seedstore.Seed) UncheckedIOException(java.io.UncheckedIOException) IOException(java.io.IOException) UncheckedIOException(java.io.UncheckedIOException) FileBasedLock(io.prestosql.spi.filesystem.FileBasedLock) Lock(java.util.concurrent.locks.Lock) HashSet(java.util.HashSet)

Example 12 with FileBasedLock

use of io.prestosql.spi.filesystem.FileBasedLock in project hetu-core by openlookeng.

the class FileBasedSeedStore method remove.

@Override
public Set<Seed> remove(Collection<Seed> seeds) throws IOException {
    Lock lock = new FileBasedLock(fs, seedDir.resolve(name));
    try {
        lock.lock();
        Set<String> writtens = new HashSet<>(0);
        Set<Seed> outputs = getSeeds();
        outputs.removeAll(seeds);
        for (Seed seed : outputs) {
            writtens.add(seed.serialize());
        }
        writeToFile(seedFilePath, String.join(COMMA, writtens), true);
        return outputs;
    } catch (UncheckedIOException e) {
        throw new IOException(e);
    } finally {
        lock.unlock();
    }
}
Also used : FileBasedLock(io.prestosql.spi.filesystem.FileBasedLock) Seed(io.prestosql.spi.seedstore.Seed) UncheckedIOException(java.io.UncheckedIOException) IOException(java.io.IOException) UncheckedIOException(java.io.UncheckedIOException) FileBasedLock(io.prestosql.spi.filesystem.FileBasedLock) Lock(java.util.concurrent.locks.Lock) HashSet(java.util.HashSet)

Example 13 with FileBasedLock

use of io.prestosql.spi.filesystem.FileBasedLock in project hetu-core by openlookeng.

the class TestFileBasedLockOnHdfs method testAcquiredLock.

@Test
public void testAcquiredLock() throws IOException, InterruptedException {
    Path testDir = Paths.get(testLockRoot + "/testAcquiredLock");
    FileBasedLock lock = new FileBasedLock(fs, testDir, 1000L, FileBasedLock.DEFAULT_RETRY_INTERVAL, FileBasedLock.DEFAULT_REFRESH_RATE);
    OutputStream os = fs.newOutputStream(testDir.resolve(".lockInfo"));
    os.write("test".getBytes(StandardCharsets.UTF_8));
    os.close();
    assertFalse(lock.acquiredLock());
    Thread.sleep(1200L);
    // Expired, can be acquired
    assertTrue(lock.acquiredLock());
}
Also used : Path(java.nio.file.Path) FileBasedLock(io.prestosql.spi.filesystem.FileBasedLock) OutputStream(java.io.OutputStream) Test(org.testng.annotations.Test) AfterTest(org.testng.annotations.AfterTest)

Example 14 with FileBasedLock

use of io.prestosql.spi.filesystem.FileBasedLock in project hetu-core by openlookeng.

the class TestFileBasedLockOnLocal method testReuse.

@Test(expectedExceptions = IllegalStateException.class)
public void testReuse() throws IOException {
    Path testDir = Paths.get(testLockRoot + "/testReuse");
    FileBasedLock lock = new FileBasedLock(fs, testDir);
    lock.lock();
    lock.unlock();
    // Should throw an exception
    lock.lock();
}
Also used : Path(java.nio.file.Path) FileBasedLock(io.prestosql.spi.filesystem.FileBasedLock) BeforeTest(org.testng.annotations.BeforeTest) Test(org.testng.annotations.Test) AfterTest(org.testng.annotations.AfterTest)

Example 15 with FileBasedLock

use of io.prestosql.spi.filesystem.FileBasedLock in project hetu-core by openlookeng.

the class TestFileBasedLockOnLocal method testTryLock.

@Test
public void testTryLock() throws InterruptedException, IOException {
    Path testDir = Paths.get(testLockRoot + "/testTryLock");
    FileBasedLock lock = new FileBasedLock(fs, testDir, 1000, FileBasedLock.DEFAULT_RETRY_INTERVAL, FileBasedLock.DEFAULT_REFRESH_RATE);
    if (lock.tryLock()) {
        assertTrue(lock.isLocked());
        lock.unlock();
    }
    assertFalse(lock.isLocked());
    lock = new FileBasedLock(fs, testDir, 1000, FileBasedLock.DEFAULT_RETRY_INTERVAL, FileBasedLock.DEFAULT_REFRESH_RATE);
    if (lock.tryLock(1500, TimeUnit.MILLISECONDS)) {
        assertTrue(lock.isLocked());
        lock.unlock();
    }
    assertFalse(lock.isLocked());
}
Also used : Path(java.nio.file.Path) FileBasedLock(io.prestosql.spi.filesystem.FileBasedLock) BeforeTest(org.testng.annotations.BeforeTest) Test(org.testng.annotations.Test) AfterTest(org.testng.annotations.AfterTest)

Aggregations

FileBasedLock (io.prestosql.spi.filesystem.FileBasedLock)17 Path (java.nio.file.Path)13 AfterTest (org.testng.annotations.AfterTest)10 Test (org.testng.annotations.Test)10 Lock (java.util.concurrent.locks.Lock)7 IOException (java.io.IOException)6 OutputStream (java.io.OutputStream)6 UncheckedIOException (java.io.UncheckedIOException)5 HashSet (java.util.HashSet)5 BeforeTest (org.testng.annotations.BeforeTest)5 Seed (io.prestosql.spi.seedstore.Seed)4 Preconditions.checkArgument (com.google.common.base.Preconditions.checkArgument)3 ImmutableList (com.google.common.collect.ImmutableList)3 Logger (io.airlift.log.Logger)3 SecurePathWhiteList (io.hetu.core.common.util.SecurePathWhiteList)3 HetuFileSystemClient (io.prestosql.spi.filesystem.HetuFileSystemClient)3 Paths (java.nio.file.Paths)3 List (java.util.List)3 Map (java.util.Map)3 Collectors (java.util.stream.Collectors)3