Examples with BlobServer org.apache.flink.runtime.blob.BlobServer used on opensource projects

Example 11 with BlobServer

use of org.apache.flink.runtime.blob.BlobServer in project flink by apache.

the class DefaultExecutionGraphDeploymentWithBlobServerTest method setupBlobServer.

@Before
public void setupBlobServer() throws IOException {
    Configuration config = new Configuration();
    // always offload the serialized job and task information
    config.setInteger(BlobServerOptions.OFFLOAD_MINSIZE, 0);
    blobServer = Mockito.spy(new BlobServer(config, TEMPORARY_FOLDER.newFolder(), new VoidBlobStore()));
    blobWriter = blobServer;
    blobCache = blobServer;
    seenHashes.clear();
    // verify that we do not upload the same content more than once
    doAnswer(invocation -> {
        PermanentBlobKey key = (PermanentBlobKey) invocation.callRealMethod();
        assertTrue(seenHashes.add(key.getHash()));
        return key;
    }).when(blobServer).putPermanent(any(JobID.class), Matchers.<byte[]>any());
    blobServer.start();
}

Example 12 with BlobServer

use of org.apache.flink.runtime.blob.BlobServer in project flink by apache.

the class BlobServerResource method before.

protected void before() throws Throwable {
    temporaryFolder.create();
    Configuration config = new Configuration();
    blobServer = new BlobServer(config, temporaryFolder.newFolder(), new VoidBlobStore());
    blobServer.start();
}

Example 13 with BlobServer

use of org.apache.flink.runtime.blob.BlobServer in project flink by apache.

the class DispatcherTest method testOnlyRecoveredJobsAreRetainedInTheBlobServer.

@Test
public void testOnlyRecoveredJobsAreRetainedInTheBlobServer() throws Exception {
    final JobID jobId1 = new JobID();
    final JobID jobId2 = new JobID();
    final byte[] fileContent = { 1, 2, 3, 4 };
    final BlobServer blobServer = getBlobServer();
    final PermanentBlobKey blobKey1 = blobServer.putPermanent(jobId1, fileContent);
    final PermanentBlobKey blobKey2 = blobServer.putPermanent(jobId2, fileContent);
    dispatcher = createTestingDispatcherBuilder().setRecoveredJobs(Collections.singleton(new JobGraph(jobId1, "foobar"))).build();
    Assertions.assertThat(blobServer.getFile(jobId1, blobKey1)).hasBinaryContent(fileContent);
    Assertions.assertThatThrownBy(() -> blobServer.getFile(jobId2, blobKey2)).isInstanceOf(NoSuchFileException.class);
}

Example 14 with BlobServer

use of org.apache.flink.runtime.blob.BlobServer in project flink by apache.

the class ClientUtilsTest method setup.

@BeforeClass
public static void setup() throws IOException {
    Configuration config = new Configuration();
    blobServer = new BlobServer(config, temporaryFolder.newFolder(), new VoidBlobStore());
    blobServer.start();
}

Example 15 with BlobServer

use of org.apache.flink.runtime.blob.BlobServer in project flink by apache.

the class JobSubmitHandlerTest method setup.

@Before
public void setup() throws IOException {
    Configuration config = new Configuration(configuration);
    blobServer = new BlobServer(config, TEMPORARY_FOLDER.newFolder(), new VoidBlobStore());
    blobServer.start();
}