Search in sources :

Example 11 with Nullable

use of com.microsoft.azuretools.azurecommons.helpers.Nullable in project azure-tools-for-java by Microsoft.

the class HDInsightHelperImpl method openItem.

private void openItem(@NotNull final Project project, @NotNull final IClusterDetail myClusterDetail, @NotNull final String uuid, @Nullable final VirtualFile closeableFile) {
    final LightVirtualFile virtualFile = new LightVirtualFile(myClusterDetail.getName() + ": Job View");
    virtualFile.putUserData(JobViewEditorProvider.JOB_VIEW_KEY, myClusterDetail);
    virtualFile.setFileType(new FileType() {

        @NotNull
        @Override
        public String getName() {
            return this.getClass().getName();
        }

        @NotNull
        @Override
        public String getDescription() {
            return "job view dummy file";
        }

        @NotNull
        @Override
        public String getDefaultExtension() {
            return "";
        }

        @Nullable
        @Override
        public Icon getIcon() {
            return PluginUtil.getIcon(CommonConst.SPARK_JOBVIEW_ICONPATH);
        }

        @Override
        public boolean isBinary() {
            return true;
        }

        @Override
        public boolean isReadOnly() {
            return true;
        }

        @Nullable
        @Override
        public String getCharset(@NotNull final VirtualFile virtualFile, @NotNull final byte[] bytes) {
            return "UTF8";
        }
    });
    virtualFile.putUserData(JobViewEditorProvider.JOB_VIEW_UUID, uuid);
    openItem(project, virtualFile, closeableFile);
}
Also used : VirtualFile(com.intellij.openapi.vfs.VirtualFile) LightVirtualFile(com.intellij.testFramework.LightVirtualFile) FileType(com.intellij.openapi.fileTypes.FileType) LightVirtualFile(com.intellij.testFramework.LightVirtualFile) Icon(javax.swing.Icon) NotNull(com.microsoft.azuretools.azurecommons.helpers.NotNull) Nullable(com.microsoft.azuretools.azurecommons.helpers.Nullable)

Example 12 with Nullable

use of com.microsoft.azuretools.azurecommons.helpers.Nullable in project azure-tools-for-java by Microsoft.

the class SparkBatchJob method parseAmHostHttpAddressHost.

/**
 * Parse host from host:port combination string
 *
 * @param driverHttpAddress the host:port combination string to parse
 * @return the host got, otherwise null
 */
@Nullable
String parseAmHostHttpAddressHost(@Nullable String driverHttpAddress) {
    if (driverHttpAddress == null) {
        return null;
    }
    final Pattern driverRegex = Pattern.compile("(?<host>[^:]+):(?<port>\\d+)");
    final Matcher driverMatcher = driverRegex.matcher(driverHttpAddress);
    return driverMatcher.matches() ? driverMatcher.group("host") : null;
}
Also used : Pattern(java.util.regex.Pattern) Matcher(java.util.regex.Matcher) Nullable(com.microsoft.azuretools.azurecommons.helpers.Nullable)

Example 13 with Nullable

use of com.microsoft.azuretools.azurecommons.helpers.Nullable in project azure-tools-for-java by Microsoft.

the class RequestDetail method getRequestDetail.

@Nullable
public static RequestDetail getRequestDetail(@NotNull URI myUrl) {
    String[] queries = myUrl.getQuery() == null ? null : myUrl.getQuery().split("&");
    String path = myUrl.getPath();
    Matcher matcher = clusterPattern.matcher(path);
    if (matcher.find()) {
        return new RequestDetail(matcher.group(1), matcher.group(2), queries);
    }
    return null;
}
Also used : Matcher(java.util.regex.Matcher) Nullable(com.microsoft.azuretools.azurecommons.helpers.Nullable)

Example 14 with Nullable

use of com.microsoft.azuretools.azurecommons.helpers.Nullable in project azure-tools-for-java by Microsoft.

the class WebHDFSDeploy method getArtifactUploadedPath.

@Nullable
public String getArtifactUploadedPath(String rootPath) throws URISyntaxException {
    final List<NameValuePair> params = new WebHdfsParamsBuilder("OPEN").build();
    final URIBuilder uriBuilder = new URIBuilder(rootPath);
    uriBuilder.addParameters(params);
    return uriBuilder.build().toString();
}
Also used : NameValuePair(org.apache.http.NameValuePair) WebHdfsParamsBuilder(com.microsoft.azure.hdinsight.sdk.storage.webhdfs.WebHdfsParamsBuilder) URIBuilder(org.apache.http.client.utils.URIBuilder) Nullable(com.microsoft.azuretools.azurecommons.helpers.Nullable)

Example 15 with Nullable

use of com.microsoft.azuretools.azurecommons.helpers.Nullable in project azure-tools-for-java by Microsoft.

the class LivySparkBatchJobRunConfiguration method getState.

@Nullable
@Override
public RunProfileState getState(@NotNull final Executor executor, @NotNull final ExecutionEnvironment executionEnvironment) throws ExecutionException {
    Operation operation = executionEnvironment.getUserData(TelemetryKeys.OPERATION);
    final String debugTarget = executionEnvironment.getUserData(SparkBatchJobDebuggerRunner.DEBUG_TARGET_KEY);
    final boolean isExecutor = StringUtils.equals(debugTarget, SparkBatchJobDebuggerRunner.DEBUG_EXECUTOR);
    RunProfileStateWithAppInsightsEvent state = null;
    final Artifact selectedArtifact = ArtifactUtil.getArtifactWithOutputPaths(getProject()).stream().filter(artifact -> artifact.getName().equals(getSubmitModel().getArtifactName())).findFirst().orElse(null);
    if (executor instanceof SparkBatchJobDebugExecutor) {
        final ISparkBatchJob remoteDebugBatch = sparkRemoteBatch;
        if (!(remoteDebugBatch instanceof SparkBatchRemoteDebugJob)) {
            throw new ExecutionException("Spark Batch Job is not prepared for " + executor.getId());
        }
        if (isExecutor) {
            setRunMode(RunMode.REMOTE_DEBUG_EXECUTOR);
            state = new SparkBatchRemoteDebugExecutorState(getModel().getSubmitModel(), operation, remoteDebugBatch);
        } else {
            if (selectedArtifact != null) {
                BuildArtifactsBeforeRunTaskProvider.setBuildArtifactBeforeRun(getProject(), this, selectedArtifact);
            }
            setRunMode(RunMode.REMOTE);
            state = new SparkBatchRemoteDebugState(getModel().getSubmitModel(), operation, sparkRemoteBatch);
        }
    } else if (executor instanceof SparkBatchJobRunExecutor) {
        final ISparkBatchJob remoteBatch = sparkRemoteBatch;
        if (remoteBatch == null) {
            throw new ExecutionException("Spark Batch Job is not prepared for " + executor.getId());
        }
        if (selectedArtifact != null) {
            BuildArtifactsBeforeRunTaskProvider.setBuildArtifactBeforeRun(getProject(), this, selectedArtifact);
        }
        setRunMode(RunMode.REMOTE);
        state = new SparkBatchRemoteRunState(getModel().getSubmitModel(), operation, remoteBatch);
    } else if (executor instanceof DefaultDebugExecutor) {
        setRunMode(RunMode.LOCAL);
        if (operation == null) {
            operation = TelemetryManager.createOperation(TelemetryConstants.HDINSIGHT, TelemetryConstants.DEBUG_LOCAL_SPARK_JOB);
            operation.start();
        }
        state = new SparkBatchLocalDebugState(getProject(), getModel().getLocalRunConfigurableModel(), operation);
    } else if (executor instanceof DefaultRunExecutor) {
        setRunMode(RunMode.LOCAL);
        if (operation == null) {
            operation = TelemetryManager.createOperation(TelemetryConstants.HDINSIGHT, TelemetryConstants.RUN_LOCAL_SPARK_JOB);
            operation.start();
        }
        state = new SparkBatchLocalRunState(getProject(), getModel().getLocalRunConfigurableModel(), operation);
    }
    if (state != null) {
        final Map<String, String> props = getActionProperties().entrySet().stream().collect(Collectors.toMap((Map.Entry<Object, Object> entry) -> entry.getKey() == null ? null : entry.getKey().toString(), (Map.Entry<Object, Object> entry) -> entry.getValue() == null ? "" : entry.getValue().toString()));
        final String configurationId = Optional.ofNullable(executionEnvironment.getRunnerAndConfigurationSettings()).map(settings -> settings.getType().getId()).orElse("");
        props.put("configurationId", configurationId);
        state.createAppInsightEvent(executor, props);
        EventUtil.logEvent(EventType.info, operation, props);
        // Clear the action properties
        getActionProperties().clear();
    }
    return state;
}
Also used : ISecureStore(com.microsoft.azure.toolkit.ide.common.store.ISecureStore) java.util(java.util) BLOB(com.microsoft.azure.hdinsight.spark.common.SparkSubmitStorageType.BLOB) TelemetryKeys(com.microsoft.intellij.telemetry.TelemetryKeys) NotNull(com.microsoft.azuretools.azurecommons.helpers.NotNull) ExecutionException(com.intellij.execution.ExecutionException) Artifact(com.intellij.packaging.artifacts.Artifact) CompileStepBeforeRun(com.intellij.compiler.options.CompileStepBeforeRun) InvalidDataException(com.intellij.openapi.util.InvalidDataException) StringUtils(org.apache.commons.lang3.StringUtils) com.intellij.execution.configurations(com.intellij.execution.configurations) SparkSubmitJobUploadStorageModelKt.getSecureStoreServiceOf(com.microsoft.azure.hdinsight.spark.common.SparkSubmitJobUploadStorageModelKt.getSecureStoreServiceOf) ILogger(com.microsoft.azure.hdinsight.common.logger.ILogger) Observable(rx.Observable) ExecutionEnvironment(com.intellij.execution.runners.ExecutionEnvironment) BuildArtifactsBeforeRunTaskProvider(com.intellij.packaging.impl.run.BuildArtifactsBeforeRunTaskProvider) SparkSubmissionJobUploadStorageWithUploadPathPanel(com.microsoft.azure.hdinsight.spark.ui.SparkSubmissionJobUploadStorageWithUploadPathPanel) SparkApplicationType(com.microsoft.azure.hdinsight.spark.run.action.SparkApplicationType) Disposer(com.intellij.openapi.util.Disposer) SparkBatchJobConfigurable(com.microsoft.azure.hdinsight.spark.ui.SparkBatchJobConfigurable) Project(com.intellij.openapi.project.Project) BuildArtifactsBeforeRunTask(com.intellij.packaging.impl.run.BuildArtifactsBeforeRunTask) Module(com.intellij.openapi.module.Module) BeforeRunTask(com.intellij.execution.BeforeRunTask) SettingsEditor(com.intellij.openapi.options.SettingsEditor) DefaultRunExecutor(com.intellij.execution.executors.DefaultRunExecutor) DefaultDebugExecutor(com.intellij.execution.executors.DefaultDebugExecutor) ProgramRunner(com.intellij.execution.runners.ProgramRunner) Nullable(com.microsoft.azuretools.azurecommons.helpers.Nullable) Operation(com.microsoft.azuretools.telemetrywrapper.Operation) ADLS_GEN2(com.microsoft.azure.hdinsight.spark.common.SparkSubmitStorageType.ADLS_GEN2) AzureStoreManager(com.microsoft.azure.toolkit.ide.common.store.AzureStoreManager) Executor(com.intellij.execution.Executor) com.microsoft.azure.hdinsight.spark.run(com.microsoft.azure.hdinsight.spark.run) Collectors(java.util.stream.Collectors) File(java.io.File) JavaExecutionUtil(com.intellij.execution.JavaExecutionUtil) EventType(com.microsoft.azuretools.telemetrywrapper.EventType) Stream(java.util.stream.Stream) ArtifactUtil(com.intellij.packaging.impl.artifacts.ArtifactUtil) TelemetryConstants(com.microsoft.azuretools.telemetry.TelemetryConstants) TelemetryManager(com.microsoft.azuretools.telemetrywrapper.TelemetryManager) UsePassword(com.microsoft.azure.hdinsight.spark.common.SparkBatchRemoteDebugJobSshAuth.SSHAuthType.UsePassword) EventUtil(com.microsoft.azuretools.telemetrywrapper.EventUtil) WriteExternalException(com.intellij.openapi.util.WriteExternalException) Element(org.jdom.Element) com.microsoft.azure.hdinsight.spark.common(com.microsoft.azure.hdinsight.spark.common) Operation(com.microsoft.azuretools.telemetrywrapper.Operation) Artifact(com.intellij.packaging.artifacts.Artifact) DefaultDebugExecutor(com.intellij.execution.executors.DefaultDebugExecutor) ExecutionException(com.intellij.execution.ExecutionException) DefaultRunExecutor(com.intellij.execution.executors.DefaultRunExecutor) Nullable(com.microsoft.azuretools.azurecommons.helpers.Nullable)

Aggregations

Nullable (com.microsoft.azuretools.azurecommons.helpers.Nullable)22 Project (com.intellij.openapi.project.Project)4 VirtualFile (com.intellij.openapi.vfs.VirtualFile)4 ILogger (com.microsoft.azure.hdinsight.common.logger.ILogger)3 NotNull (com.microsoft.azuretools.azurecommons.helpers.NotNull)3 ConsoleView (com.intellij.execution.ui.ConsoleView)2 Module (com.intellij.openapi.module.Module)2 ProgressIndicator (com.intellij.openapi.progress.ProgressIndicator)2 ProgressManager (com.intellij.openapi.progress.ProgressManager)2 LightVirtualFile (com.intellij.testFramework.LightVirtualFile)2 HDIException (com.microsoft.azure.hdinsight.sdk.common.HDIException)2 Azure (com.microsoft.azure.management.Azure)2 Action (com.microsoft.azure.toolkit.lib.common.action.Action)2 ActionView (com.microsoft.azure.toolkit.lib.common.action.ActionView)2 AzureString (com.microsoft.azure.toolkit.lib.common.bundle.AzureString)2 AzureMessager (com.microsoft.azure.toolkit.lib.common.messager.AzureMessager)2 AzureOperation (com.microsoft.azure.toolkit.lib.common.operation.AzureOperation)2 AzureOperationBundle.title (com.microsoft.azure.toolkit.lib.common.operation.AzureOperationBundle.title)2 AzureTask (com.microsoft.azure.toolkit.lib.common.task.AzureTask)2 AzureTaskManager (com.microsoft.azure.toolkit.lib.common.task.AzureTaskManager)2