Examples with VisibleForTesting org.apache.flink.annotation.VisibleForTesting used on opensource projects

Example 6 with VisibleForTesting

use of org.apache.flink.annotation.VisibleForTesting in project flink by apache.

the class JobManagerProcessUtils method createDefaultJobManagerProcessSpec.

@VisibleForTesting
public static JobManagerProcessSpec createDefaultJobManagerProcessSpec(int totalProcessMemoryMb) {
    Configuration configuration = new Configuration();
    configuration.set(JobManagerOptions.TOTAL_PROCESS_MEMORY, MemorySize.ofMebiBytes(totalProcessMemoryMb));
    return processSpecFromConfig(configuration);
}

Example 7 with VisibleForTesting

use of org.apache.flink.annotation.VisibleForTesting in project flink by apache.

the class ConfigOptionsDocGenerator method generateTablesForClass.

@VisibleForTesting
static List<Tuple2<ConfigGroup, String>> generateTablesForClass(Class<?> optionsClass) {
    ConfigGroups configGroups = optionsClass.getAnnotation(ConfigGroups.class);
    List<OptionWithMetaInfo> allOptions = extractConfigOptions(optionsClass);
    if (allOptions.isEmpty()) {
        return Collections.emptyList();
    }
    List<Tuple2<ConfigGroup, String>> tables;
    if (configGroups != null) {
        tables = new ArrayList<>(configGroups.groups().length + 1);
        Tree tree = new Tree(configGroups.groups(), allOptions);
        for (ConfigGroup group : configGroups.groups()) {
            List<OptionWithMetaInfo> configOptions = tree.findConfigOptions(group);
            if (!configOptions.isEmpty()) {
                sortOptions(configOptions);
                tables.add(Tuple2.of(group, toHtmlTable(configOptions)));
            }
        }
        List<OptionWithMetaInfo> configOptions = tree.getDefaultOptions();
        if (!configOptions.isEmpty()) {
            sortOptions(configOptions);
            tables.add(Tuple2.of(null, toHtmlTable(configOptions)));
        }
    } else {
        sortOptions(allOptions);
        tables = Collections.singletonList(Tuple2.of(null, toHtmlTable(allOptions)));
    }
    return tables;
}

Example 8 with VisibleForTesting

use of org.apache.flink.annotation.VisibleForTesting in project flink by apache.

the class OpenApiSpecGenerator method createDocumentationFile.

@VisibleForTesting
static void createDocumentationFile(DocumentingRestEndpoint restEndpoint, RestAPIVersion apiVersion, Path outputFile) throws IOException {
    final OpenAPI openApi = new OpenAPI();
    // eagerly initialize some data-structures to simplify operations later on
    openApi.setPaths(new io.swagger.v3.oas.models.Paths());
    openApi.setComponents(new Components());
    setInfo(openApi, apiVersion);
    List<MessageHeaders> specs = restEndpoint.getSpecs().stream().filter(spec -> spec.getSupportedAPIVersions().contains(apiVersion)).filter(OpenApiSpecGenerator::shouldBeDocumented).collect(Collectors.toList());
    specs.forEach(spec -> add(spec, openApi));
    final List<Schema> asyncOperationSchemas = collectAsyncOperationResultVariants(specs);
    // this adds the schema for every JSON object
    openApi.components(new Components().schemas(new HashMap<>(modelConverterContext.getDefinedModels())));
    injectAsyncOperationResultSchema(openApi, asyncOperationSchemas);
    overrideIdSchemas(openApi);
    overrideSerializeThrowableSchema(openApi);
    Files.deleteIfExists(outputFile);
    Files.write(outputFile, Yaml.pretty(openApi).getBytes(StandardCharsets.UTF_8));
}

Example 9 with VisibleForTesting

use of org.apache.flink.annotation.VisibleForTesting in project flink by apache.

the class ConfigOptionsDocGenerator method extractConfigOptions.

@VisibleForTesting
static List<OptionWithMetaInfo> extractConfigOptions(Class<?> clazz) {
    try {
        List<OptionWithMetaInfo> configOptions = new ArrayList<>(8);
        Field[] fields = clazz.getFields();
        for (Field field : fields) {
            if (isConfigOption(field) && shouldBeDocumented(field)) {
                configOptions.add(new OptionWithMetaInfo((ConfigOption<?>) field.get(null), field));
            }
        }
        return configOptions;
    } catch (Exception e) {
        throw new RuntimeException("Failed to extract config options from class " + clazz + '.', e);
    }
}

Example 10 with VisibleForTesting

use of org.apache.flink.annotation.VisibleForTesting in project flink by apache.

the class BootstrapTransformation method getConfig.

@VisibleForTesting
StreamConfig getConfig(OperatorID operatorID, @Nullable StateBackend stateBackend, Configuration additionalConfig, StreamOperator<TaggedOperatorSubtaskState> operator) {
    // Eagerly perform a deep copy of the configuration, otherwise it will result in undefined
    // behavior
    // when deploying with multiple bootstrap transformations.
    Configuration deepCopy = new Configuration(dataSet.getExecutionEnvironment().getConfiguration());
    deepCopy.addAll(additionalConfig);
    final StreamConfig config = new StreamConfig(deepCopy);
    config.setChainStart();
    config.setCheckpointingEnabled(true);
    config.setCheckpointMode(CheckpointingMode.EXACTLY_ONCE);
    if (keyType != null) {
        TypeSerializer<?> keySerializer = keyType.createSerializer(dataSet.getExecutionEnvironment().getConfig());
        config.setStateKeySerializer(keySerializer);
        config.setStatePartitioner(0, originalKeySelector);
    }
    config.setStreamOperator(operator);
    config.setOperatorName(operatorID.toHexString());
    config.setOperatorID(operatorID);
    config.setStateBackend(stateBackend);
    // This means leaving this stateBackend unwrapped.
    config.setChangelogStateBackendEnabled(TernaryBoolean.FALSE);
    config.setManagedMemoryFractionOperatorOfUseCase(ManagedMemoryUseCase.STATE_BACKEND, 1.0);
    return config;
}