Search in sources :

Example 26 with TemporaryFolder

use of org.junit.rules.TemporaryFolder in project flink by apache.

the class LolCacheFactory method create.

@Override
public DownloadCache create() throws IOException {
    final TemporaryFolder folder = new TemporaryFolder();
    folder.create();
    return new LolCache(folder);
}
Also used : TemporaryFolder(org.junit.rules.TemporaryFolder)

Example 27 with TemporaryFolder

use of org.junit.rules.TemporaryFolder in project flink by apache.

the class StreamOperatorSnapshotRestoreTest method beforeClass.

@BeforeClass
public static void beforeClass() throws IOException {
    temporaryFolder = new TemporaryFolder();
    temporaryFolder.create();
}
Also used : TemporaryFolder(org.junit.rules.TemporaryFolder) BeforeClass(org.junit.BeforeClass)

Example 28 with TemporaryFolder

use of org.junit.rules.TemporaryFolder in project flink by apache.

the class TaskStateManagerImplTest method testForwardingSubtaskLocalStateBaseDirFromLocalStateStore.

/**
 * This tests if the {@link TaskStateManager} properly returns the subtask local state dir from
 * the corresponding {@link TaskLocalStateStoreImpl}.
 */
@Test
public void testForwardingSubtaskLocalStateBaseDirFromLocalStateStore() throws IOException {
    JobID jobID = new JobID(42L, 43L);
    AllocationID allocationID = new AllocationID(4711L, 23L);
    JobVertexID jobVertexID = new JobVertexID(12L, 34L);
    ExecutionAttemptID executionAttemptID = new ExecutionAttemptID();
    TestCheckpointResponder checkpointResponderMock = new TestCheckpointResponder();
    Executor directExecutor = Executors.directExecutor();
    TemporaryFolder tmpFolder = new TemporaryFolder();
    try {
        tmpFolder.create();
        File[] allocBaseDirs = new File[] { tmpFolder.newFolder(), tmpFolder.newFolder(), tmpFolder.newFolder() };
        LocalRecoveryDirectoryProviderImpl directoryProvider = new LocalRecoveryDirectoryProviderImpl(allocBaseDirs, jobID, jobVertexID, 0);
        LocalRecoveryConfig localRecoveryConfig = new LocalRecoveryConfig(directoryProvider);
        TaskLocalStateStore taskLocalStateStore = new TaskLocalStateStoreImpl(jobID, allocationID, jobVertexID, 13, localRecoveryConfig, directExecutor);
        InMemoryStateChangelogStorage changelogStorage = new InMemoryStateChangelogStorage();
        TaskStateManager taskStateManager = taskStateManager(jobID, executionAttemptID, checkpointResponderMock, null, taskLocalStateStore, changelogStorage);
        LocalRecoveryConfig localRecoveryConfFromTaskLocalStateStore = taskLocalStateStore.getLocalRecoveryConfig();
        LocalRecoveryConfig localRecoveryConfFromTaskStateManager = taskStateManager.createLocalRecoveryConfig();
        for (int i = 0; i < 10; ++i) {
            Assert.assertEquals(allocBaseDirs[i % allocBaseDirs.length], localRecoveryConfFromTaskLocalStateStore.getLocalStateDirectoryProvider().get().allocationBaseDirectory(i));
            Assert.assertEquals(allocBaseDirs[i % allocBaseDirs.length], localRecoveryConfFromTaskStateManager.getLocalStateDirectoryProvider().get().allocationBaseDirectory(i));
        }
        Assert.assertEquals(localRecoveryConfFromTaskLocalStateStore.isLocalRecoveryEnabled(), localRecoveryConfFromTaskStateManager.isLocalRecoveryEnabled());
    } finally {
        tmpFolder.delete();
    }
}
Also used : ExecutionAttemptID(org.apache.flink.runtime.executiongraph.ExecutionAttemptID) AllocationID(org.apache.flink.runtime.clusterframework.types.AllocationID) JobVertexID(org.apache.flink.runtime.jobgraph.JobVertexID) Executor(java.util.concurrent.Executor) InMemoryStateChangelogStorage(org.apache.flink.runtime.state.changelog.inmemory.InMemoryStateChangelogStorage) TestCheckpointResponder(org.apache.flink.runtime.taskmanager.TestCheckpointResponder) TemporaryFolder(org.junit.rules.TemporaryFolder) File(java.io.File) JobID(org.apache.flink.api.common.JobID) Test(org.junit.Test)

Example 29 with TemporaryFolder

use of org.junit.rules.TemporaryFolder in project flink by apache.

the class CheckpointStreamWithResultProviderTest method beforeClass.

@BeforeClass
public static void beforeClass() throws IOException {
    temporaryFolder = new TemporaryFolder();
    temporaryFolder.create();
}
Also used : TemporaryFolder(org.junit.rules.TemporaryFolder) BeforeClass(org.junit.BeforeClass)

Example 30 with TemporaryFolder

use of org.junit.rules.TemporaryFolder in project flink by apache.

the class RocksDBResource method before.

@Override
protected void before() throws Throwable {
    this.temporaryFolder = new TemporaryFolder();
    this.temporaryFolder.create();
    final File rocksFolder = temporaryFolder.newFolder();
    this.dbOptions = optionsFactory.createDBOptions(new DBOptions().setUseFsync(false).setInfoLogLevel(InfoLogLevel.HEADER_LEVEL).setStatsDumpPeriodSec(0), handlesToClose).setCreateIfMissing(true);
    this.columnFamilyOptions = optionsFactory.createColumnOptions(new ColumnFamilyOptions(), handlesToClose);
    this.writeOptions = new WriteOptions();
    this.writeOptions.disableWAL();
    this.readOptions = new ReadOptions();
    this.columnFamilyHandles = new ArrayList<>(1);
    this.rocksDB = RocksDB.open(dbOptions, rocksFolder.getAbsolutePath(), Collections.singletonList(new ColumnFamilyDescriptor("default".getBytes(), columnFamilyOptions)), columnFamilyHandles);
    this.batchWrapper = new RocksDBWriteBatchWrapper(rocksDB, writeOptions);
}
Also used : ColumnFamilyOptions(org.rocksdb.ColumnFamilyOptions) WriteOptions(org.rocksdb.WriteOptions) ReadOptions(org.rocksdb.ReadOptions) TemporaryFolder(org.junit.rules.TemporaryFolder) DBOptions(org.rocksdb.DBOptions) ColumnFamilyDescriptor(org.rocksdb.ColumnFamilyDescriptor) File(java.io.File)

Aggregations

TemporaryFolder (org.junit.rules.TemporaryFolder)88 File (java.io.File)33 Test (org.junit.Test)25 Before (org.junit.Before)24 BeforeClass (org.junit.BeforeClass)13 Path (java.nio.file.Path)7 URL (java.net.URL)6 IOException (java.io.IOException)3 ArrayList (java.util.ArrayList)3 UserState (alluxio.security.user.UserState)2 ProjectFilesystem (com.facebook.buck.io.ProjectFilesystem)2 RocksDBPersistenceProvider (com.iota.iri.storage.rocksDB.RocksDBPersistenceProvider)2 DocumentMetaData (de.tudarmstadt.ukp.dkpro.core.api.metadata.type.DocumentMetaData)2 ByteArrayInputStream (java.io.ByteArrayInputStream)2 URISyntaxException (java.net.URISyntaxException)2 BdpDataPlanePlugin (org.batfish.bdp.BdpDataPlanePlugin)2 Batfish (org.batfish.main.Batfish)2 DefaultSafeModeManager (alluxio.master.DefaultSafeModeManager)1 InMemoryReferenceSequenceFile (au.edu.wehi.idsv.picard.InMemoryReferenceSequenceFile)1 SynchronousReferenceLookupAdapter (au.edu.wehi.idsv.picard.SynchronousReferenceLookupAdapter)1