Search in sources :

Example 31 with MemorySize

use of org.apache.flink.configuration.MemorySize in project flink by apache.

the class ProcessMemoryUtilsTestBase method testConfigLegacyHeapMemory.

private void testConfigLegacyHeapMemory(Configuration configuration, MemorySize expected) {
    MemorySize newOptionValue = getConfigurationWithLegacyHeapSizeMappedToNewConfigOption(configuration).get(getNewOptionForLegacyHeapOption());
    assertThat(newOptionValue, is(expected));
}
Also used : MemorySize(org.apache.flink.configuration.MemorySize)

Example 32 with MemorySize

use of org.apache.flink.configuration.MemorySize in project flink by apache.

the class ProcessMemoryUtilsTestBase method getConfigurationWithJvmMetaspaceAndTotalFlinkMemory.

private Configuration getConfigurationWithJvmMetaspaceAndTotalFlinkMemory(long jvmMetaspaceSizeMb, long totalProcessMemorySizeMb) {
    MemorySize jvmMetaspaceSize = MemorySize.ofMebiBytes(jvmMetaspaceSizeMb);
    MemorySize totalProcessMemorySize = MemorySize.ofMebiBytes(totalProcessMemorySizeMb);
    Configuration conf = new Configuration();
    conf.set(options.getJvmOptions().getJvmMetaspaceOption(), jvmMetaspaceSize);
    conf.set(options.getTotalProcessMemoryOption(), totalProcessMemorySize);
    return conf;
}
Also used : MemorySize(org.apache.flink.configuration.MemorySize) Configuration(org.apache.flink.configuration.Configuration)

Example 33 with MemorySize

use of org.apache.flink.configuration.MemorySize in project flink by apache.

the class ProcessMemoryUtilsTestBase method testConfigLegacyHeapMB.

@Test
public void testConfigLegacyHeapMB() {
    MemorySize jvmHeapSize = MemorySize.parse("1g");
    Configuration conf = new Configuration();
    conf.set(legacyMemoryOptions.getHeapMb(), jvmHeapSize.getMebiBytes());
    testConfigLegacyHeapMemory(conf, jvmHeapSize);
}
Also used : MemorySize(org.apache.flink.configuration.MemorySize) Configuration(org.apache.flink.configuration.Configuration) Test(org.junit.Test)

Example 34 with MemorySize

use of org.apache.flink.configuration.MemorySize in project flink by apache.

the class ProcessMemoryUtilsTestBase method testConfigTotalProcessMemoryAddUpFailure.

@Test
public void testConfigTotalProcessMemoryAddUpFailure() {
    MemorySize totalProcessMemory = MemorySize.parse("699m");
    MemorySize totalFlinkMemory = MemorySize.parse("500m");
    MemorySize jvmMetaspace = MemorySize.parse("100m");
    MemorySize jvmOverhead = MemorySize.parse("100m");
    Configuration conf = new Configuration();
    conf.set(options.getTotalProcessMemoryOption(), totalProcessMemory);
    conf.set(options.getTotalFlinkMemoryOption(), totalFlinkMemory);
    conf.set(options.getJvmOptions().getJvmMetaspaceOption(), jvmMetaspace);
    conf.set(options.getJvmOptions().getJvmOverheadMin(), jvmOverhead);
    conf.set(options.getJvmOptions().getJvmOverheadMax(), jvmOverhead);
    validateFail(conf);
}
Also used : MemorySize(org.apache.flink.configuration.MemorySize) Configuration(org.apache.flink.configuration.Configuration) Test(org.junit.Test)

Example 35 with MemorySize

use of org.apache.flink.configuration.MemorySize in project flink by apache.

the class ProcessMemoryUtilsTestBase method testConfigTotalProcessMemorySize.

@Test
public void testConfigTotalProcessMemorySize() {
    MemorySize totalProcessMemorySize = MemorySize.parse("2g");
    Configuration conf = new Configuration();
    conf.set(options.getTotalProcessMemoryOption(), totalProcessMemorySize);
    T processSpec = processSpecFromConfig(conf);
    assertThat(processSpec.getTotalProcessMemorySize(), is(totalProcessMemorySize));
}
Also used : MemorySize(org.apache.flink.configuration.MemorySize) Configuration(org.apache.flink.configuration.Configuration) Test(org.junit.Test)

Aggregations

MemorySize (org.apache.flink.configuration.MemorySize)81 Test (org.junit.Test)56 Configuration (org.apache.flink.configuration.Configuration)51 IllegalConfigurationException (org.apache.flink.configuration.IllegalConfigurationException)6 Path (org.apache.flink.core.fs.Path)5 File (java.io.File)4 ArrayList (java.util.ArrayList)3 CPUResource (org.apache.flink.api.common.resources.CPUResource)3 FileSourceSplit (org.apache.flink.connector.file.src.FileSourceSplit)2 BulkFormat (org.apache.flink.connector.file.src.reader.BulkFormat)2 ExecutionJobVertex (org.apache.flink.runtime.executiongraph.ExecutionJobVertex)2 IntermediateResultPartitionTest (org.apache.flink.runtime.executiongraph.IntermediateResultPartitionTest)2 SlotSharingGroup (org.apache.flink.runtime.jobmanager.scheduler.SlotSharingGroup)2 TaskExecutorFlinkMemory (org.apache.flink.runtime.util.config.memory.taskmanager.TaskExecutorFlinkMemory)2 CoreMatchers.containsString (org.hamcrest.CoreMatchers.containsString)2 HashMap (java.util.HashMap)1 List (java.util.List)1 Map (java.util.Map)1 UUID (java.util.UUID)1 ResourceSpec (org.apache.flink.api.common.operators.ResourceSpec)1