Search in sources :

Example 41 with VoidBlobStore

use of org.apache.flink.runtime.blob.VoidBlobStore in project flink-mirror by flink-ci.

the class ZooKeeperLeaderRetrievalTest method before.

@Before
public void before() throws Exception {
    testingServer = new TestingServer();
    config = new Configuration();
    config.setString(HighAvailabilityOptions.HA_MODE, "zookeeper");
    config.setString(HighAvailabilityOptions.HA_ZOOKEEPER_QUORUM, testingServer.getConnectString());
    highAvailabilityServices = new ZooKeeperHaServices(ZooKeeperUtils.startCuratorFramework(config, testingFatalErrorHandlerResource.getFatalErrorHandler()), TestingUtils.defaultExecutor(), config, new VoidBlobStore());
}
Also used : TestingServer(org.apache.curator.test.TestingServer) ZooKeeperHaServices(org.apache.flink.runtime.highavailability.zookeeper.ZooKeeperHaServices) VoidBlobStore(org.apache.flink.runtime.blob.VoidBlobStore) Configuration(org.apache.flink.configuration.Configuration) Before(org.junit.Before)

Example 42 with VoidBlobStore

use of org.apache.flink.runtime.blob.VoidBlobStore in project flink-mirror by flink-ci.

the class JobSubmitHandlerTest method setup.

@Before
public void setup() throws IOException {
    Configuration config = new Configuration(configuration);
    blobServer = new BlobServer(config, TEMPORARY_FOLDER.newFolder(), new VoidBlobStore());
    blobServer.start();
}
Also used : VoidBlobStore(org.apache.flink.runtime.blob.VoidBlobStore) Configuration(org.apache.flink.configuration.Configuration) BlobServer(org.apache.flink.runtime.blob.BlobServer) Before(org.junit.Before)

Example 43 with VoidBlobStore

use of org.apache.flink.runtime.blob.VoidBlobStore in project flink-mirror by flink-ci.

the class AbstractTaskManagerFileHandlerTest method setup.

@BeforeClass
public static void setup() throws IOException, HandlerRequestException {
    final Configuration configuration = new Configuration();
    blobServer = new BlobServer(configuration, temporaryFolder.newFolder(), new VoidBlobStore());
    handlerRequest = HandlerRequest.resolveParametersAndCreate(EmptyRequestBody.getInstance(), new TaskManagerFileMessageParameters(), Collections.singletonMap(TaskManagerIdPathParameter.KEY, EXPECTED_TASK_MANAGER_ID.getResourceIdString()), Collections.emptyMap(), Collections.emptyList());
}
Also used : VoidBlobStore(org.apache.flink.runtime.blob.VoidBlobStore) Configuration(org.apache.flink.configuration.Configuration) BlobServer(org.apache.flink.runtime.blob.BlobServer) TaskManagerFileMessageParameters(org.apache.flink.runtime.rest.messages.taskmanager.TaskManagerFileMessageParameters) BeforeClass(org.junit.BeforeClass)

Example 44 with VoidBlobStore

use of org.apache.flink.runtime.blob.VoidBlobStore in project flink by splunk.

the class KubernetesHaServicesTest method testInternalCleanupShouldCleanupConfigMaps.

@Test
public void testInternalCleanupShouldCleanupConfigMaps() throws Exception {
    new Context() {

        {
            runTest(() -> {
                final KubernetesHaServices kubernetesHaServices = new KubernetesHaServices(flinkKubeClient, executorService, configuration, new VoidBlobStore());
                kubernetesHaServices.internalCleanup();
                final Map<String, String> labels = deleteConfigMapByLabelsFuture.get(TIMEOUT, TimeUnit.MILLISECONDS);
                assertThat(labels.size(), is(3));
                assertThat(labels.get(LABEL_CONFIGMAP_TYPE_KEY), is(LABEL_CONFIGMAP_TYPE_HIGH_AVAILABILITY));
            });
        }
    };
}
Also used : VoidBlobStore(org.apache.flink.runtime.blob.VoidBlobStore) Test(org.junit.Test)

Example 45 with VoidBlobStore

use of org.apache.flink.runtime.blob.VoidBlobStore in project flink by splunk.

the class DefaultExecutionGraphDeploymentWithBlobServerTest method setupBlobServer.

@Before
public void setupBlobServer() throws IOException {
    Configuration config = new Configuration();
    // always offload the serialized job and task information
    config.setInteger(BlobServerOptions.OFFLOAD_MINSIZE, 0);
    blobServer = Mockito.spy(new BlobServer(config, TEMPORARY_FOLDER.newFolder(), new VoidBlobStore()));
    blobWriter = blobServer;
    blobCache = blobServer;
    seenHashes.clear();
    // verify that we do not upload the same content more than once
    doAnswer(invocation -> {
        PermanentBlobKey key = (PermanentBlobKey) invocation.callRealMethod();
        assertTrue(seenHashes.add(key.getHash()));
        return key;
    }).when(blobServer).putPermanent(any(JobID.class), Matchers.<byte[]>any());
    blobServer.start();
}
Also used : VoidBlobStore(org.apache.flink.runtime.blob.VoidBlobStore) Configuration(org.apache.flink.configuration.Configuration) PermanentBlobKey(org.apache.flink.runtime.blob.PermanentBlobKey) BlobServer(org.apache.flink.runtime.blob.BlobServer) JobID(org.apache.flink.api.common.JobID) Before(org.junit.Before)

Aggregations

VoidBlobStore (org.apache.flink.runtime.blob.VoidBlobStore)51 Configuration (org.apache.flink.configuration.Configuration)42 BlobServer (org.apache.flink.runtime.blob.BlobServer)39 Before (org.junit.Before)18 Test (org.junit.Test)18 InetSocketAddress (java.net.InetSocketAddress)15 JobID (org.apache.flink.api.common.JobID)15 PermanentBlobCache (org.apache.flink.runtime.blob.PermanentBlobCache)15 PermanentBlobKey (org.apache.flink.runtime.blob.PermanentBlobKey)12 UserCodeClassLoader (org.apache.flink.util.UserCodeClassLoader)9 BeforeClass (org.junit.BeforeClass)9 URL (java.net.URL)6 ArrayList (java.util.ArrayList)6 File (java.io.File)3 IOException (java.io.IOException)3 Collection (java.util.Collection)3 TestingServer (org.apache.curator.test.TestingServer)3 KubernetesConfigMap (org.apache.flink.kubernetes.kubeclient.resources.KubernetesConfigMap)3 BlobCacheSizeTracker (org.apache.flink.runtime.blob.BlobCacheSizeTracker)3 StandaloneCheckpointRecoveryFactory (org.apache.flink.runtime.checkpoint.StandaloneCheckpointRecoveryFactory)3