Search in sources :

Example 6 with Nullable

use of com.microsoft.azuretools.azurecommons.helpers.Nullable in project azure-tools-for-java by Microsoft.

the class ClusterDetail method getDefaultStorageAccount.

@Nullable
private IHDIStorageAccount getDefaultStorageAccount(Map<String, String> coresiteMap, ClusterIdentity clusterIdentity) throws HDIException {
    String defaultStorageRootPath = getDefaultStorageRootPath();
    if (defaultStorageRootPath == null) {
        throw new HDIException("Failed to get default storage root path");
    }
    StorageAccountType storageType = StorageAccountType.parseUri(URI.create(defaultStorageRootPath));
    switch(storageType) {
        case ADLS:
            return new ADLSStorageAccount(this, true, clusterIdentity, URI.create(defaultStorageRootPath));
        case BLOB:
            WasbUri wasbUri = WasbUri.parse(defaultStorageRootPath);
            String storageAccountName = wasbUri.getStorageAccount() + ".blob." + wasbUri.getEndpointSuffix();
            String defaultContainerName = wasbUri.getContainer();
            String defaultStorageAccountKey = StorageAccountKeyPrefix + storageAccountName;
            String storageAccountKey = null;
            if (coresiteMap.containsKey(defaultStorageAccountKey)) {
                storageAccountKey = coresiteMap.get(defaultStorageAccountKey);
            }
            if (storageAccountKey == null) {
                return null;
            }
            return new HDStorageAccount(this, storageAccountName, storageAccountKey, true, defaultContainerName);
        case ADLSGen2:
            AbfsUri abfsUri = AbfsUri.parse(defaultStorageRootPath);
            String accountName = abfsUri.getAccountName();
            String fileSystem = abfsUri.getFileSystem();
            return new ADLSGen2StorageAccount(this, accountName, null, true, fileSystem, abfsUri.getUri().getScheme());
        default:
            return null;
    }
}
Also used : WasbUri(com.microsoft.azure.hdinsight.common.WasbUri) HDIException(com.microsoft.azure.hdinsight.sdk.common.HDIException) AbfsUri(com.microsoft.azure.hdinsight.common.AbfsUri) Nullable(com.microsoft.azuretools.azurecommons.helpers.Nullable)

Example 7 with Nullable

use of com.microsoft.azuretools.azurecommons.helpers.Nullable in project azure-tools-for-java by Microsoft.

the class HttpObservable method request.

/*
     * Core request
     */
public Observable<CloseableHttpResponse> request(@NotNull final HttpRequestBase httpRequest, @Nullable final HttpEntity entity, final List<NameValuePair> parameters, final List<Header> addOrReplaceHeaders) {
    return Observable.fromCallable(() -> {
        URIBuilder builder = new URIBuilder(httpRequest.getURI());
        // Add parameters
        builder.setParameters(getDefaultParameters());
        Optional.ofNullable(parameters).filter(pairs -> !pairs.isEmpty()).ifPresent(builder::addParameters);
        httpRequest.setURI(builder.build());
        // Set the default headers and update Headers
        httpRequest.setHeaders(getDefaultHeaders());
        Optional.ofNullable(addOrReplaceHeaders).ifPresent(headers -> headers.forEach(httpRequest::setHeader));
        // Set entity for non-entity
        if (httpRequest instanceof HttpEntityEnclosingRequestBase && entity != null) {
            ((HttpEntityEnclosingRequestBase) httpRequest).setEntity(entity);
            // Update the content type by entity
            httpRequest.setHeader(entity.getContentType());
        }
        return getHttpClient().execute(httpRequest, getHttpContext());
    });
}
Also used : TrustStrategy(org.apache.http.ssl.TrustStrategy) SSLContext(javax.net.ssl.SSLContext) java.util(java.util) HeaderGroup(org.apache.http.message.HeaderGroup) com.microsoft.azure.hdinsight.sdk.common.errorresponse(com.microsoft.azure.hdinsight.sdk.common.errorresponse) NotNull(com.microsoft.azuretools.azurecommons.helpers.NotNull) ServiceManager(com.microsoft.azuretools.service.ServiceManager) HttpClientContext(org.apache.http.client.protocol.HttpClientContext) Exceptions(rx.exceptions.Exceptions) ObjectConvertUtils(com.microsoft.azure.hdinsight.sdk.rest.ObjectConvertUtils) CookieSpecs(org.apache.http.client.config.CookieSpecs) RequestConfig(org.apache.http.client.config.RequestConfig) KeyStoreException(java.security.KeyStoreException) StringUtils(org.apache.commons.lang3.StringUtils) Base64(org.apache.commons.codec.binary.Base64) AuthSchemes(org.apache.http.client.config.AuthSchemes) ILogger(com.microsoft.azure.hdinsight.common.logger.ILogger) EntityUtils(org.apache.http.util.EntityUtils) Observable(rx.Observable) CookieStore(org.apache.http.client.CookieStore) DefaultLoader(com.microsoft.tooling.msservices.components.DefaultLoader) org.apache.http.client.methods(org.apache.http.client.methods) NoopHostnameVerifier(org.apache.http.conn.ssl.NoopHostnameVerifier) SSLConnectionSocketFactory(org.apache.http.conn.ssl.SSLConnectionSocketFactory) CloseableHttpClient(org.apache.http.impl.client.CloseableHttpClient) Nullable(com.microsoft.azuretools.azurecommons.helpers.Nullable) URIBuilder(org.apache.http.client.utils.URIBuilder) Exceptions.propagate(rx.exceptions.Exceptions.propagate) CommonConst(com.microsoft.azure.hdinsight.common.CommonConst) IOException(java.io.IOException) KeyManagementException(java.security.KeyManagementException) SSLContextBuilder(org.apache.http.ssl.SSLContextBuilder) org.apache.http(org.apache.http) StandardCharsets(java.nio.charset.StandardCharsets) BasicCookieStore(org.apache.http.impl.client.BasicCookieStore) BasicHttpContext(org.apache.http.protocol.BasicHttpContext) HttpContext(org.apache.http.protocol.HttpContext) StreamUtil(com.microsoft.azure.hdinsight.common.StreamUtil) BasicHeader(org.apache.http.message.BasicHeader) NoSuchAlgorithmException(java.security.NoSuchAlgorithmException) DefaultHostnameVerifier(org.apache.http.conn.ssl.DefaultHostnameVerifier) UnknownServiceException(java.net.UnknownServiceException) HttpClients(org.apache.http.impl.client.HttpClients) ExceptionUtils(org.apache.commons.lang3.exception.ExceptionUtils) URIBuilder(org.apache.http.client.utils.URIBuilder)

Example 8 with Nullable

use of com.microsoft.azuretools.azurecommons.helpers.Nullable in project azure-tools-for-java by Microsoft.

the class SparkBatchJobRunner method doExecute.

@Nullable
@Override
protected RunContentDescriptor doExecute(@NotNull RunProfileState state, @NotNull ExecutionEnvironment environment) throws ExecutionException {
    final SparkBatchRemoteRunProfileState submissionState = (SparkBatchRemoteRunProfileState) state;
    final SparkSubmitModel submitModel = submissionState.getSubmitModel();
    final Project project = submitModel.getProject();
    // Prepare the run table console view UI
    final SparkJobLogConsoleView jobOutputView = new SparkJobLogConsoleView(project);
    final String artifactPath = submitModel.getArtifactPath().orElse(null);
    assert artifactPath != null : "artifactPath should be checked in LivySparkBatchJobRunConfiguration::checkSubmissionConfigurationBeforeRun";
    // To address issue https://github.com/microsoft/azure-tools-for-java/issues/4021.
    // In this issue, when user click rerun button, we are still using the legacy ctrlSubject which has already sent
    // "onComplete" message when the job is done in the previous time. To avoid this issue,  We clone a new Spark
    // batch job instance to re-initialize everything in the object.
    final ISparkBatchJob sparkBatch = submissionState.getSparkBatch().clone();
    final PublishSubject<SparkLogLine> ctrlSubject = (PublishSubject<SparkLogLine>) sparkBatch.getCtrlSubject();
    final SparkBatchJobRemoteProcess remoteProcess = new SparkBatchJobRemoteProcess(new IdeaSchedulers(project), sparkBatch, artifactPath, submitModel.getSubmissionParameter().getMainClassName(), ctrlSubject);
    final SparkBatchJobRunProcessHandler processHandler = new SparkBatchJobRunProcessHandler(remoteProcess, "Package and deploy the job to Spark cluster", null);
    // After attaching, the console view can read the process inputStreams and display them
    jobOutputView.attachToProcess(processHandler);
    remoteProcess.start();
    final Operation operation = environment.getUserData(TelemetryKeys.OPERATION);
    // After we define a new AnAction class, IntelliJ will construct a new AnAction instance for us.
    // Use one action instance can keep behaviours like isEnabled() consistent
    final SparkBatchJobDisconnectAction disconnectAction = (SparkBatchJobDisconnectAction) ActionManager.getInstance().getAction("Actions.SparkJobDisconnect");
    disconnectAction.init(remoteProcess, operation);
    sendTelemetryForParameters(submitModel, operation);
    final ExecutionResult result = new DefaultExecutionResult(jobOutputView, processHandler, Separator.getInstance(), disconnectAction);
    submissionState.setExecutionResult(result);
    final ConsoleView consoleView = jobOutputView.getSecondaryConsoleView();
    submissionState.setConsoleView(consoleView);
    addConsoleViewFilter(remoteProcess.getSparkJob(), consoleView);
    submissionState.setRemoteProcessCtrlLogHandler(processHandler);
    ctrlSubject.subscribe(messageWithType -> {
    }, err -> disconnectAction.setEnabled(false), () -> disconnectAction.setEnabled(false));
    return super.doExecute(state, environment);
}
Also used : DefaultExecutionResult(com.intellij.execution.DefaultExecutionResult) SparkJobLogConsoleView(com.microsoft.azure.hdinsight.spark.ui.SparkJobLogConsoleView) ConsoleView(com.intellij.execution.ui.ConsoleView) IdeaSchedulers(com.microsoft.intellij.rxjava.IdeaSchedulers) ExecutionResult(com.intellij.execution.ExecutionResult) DefaultExecutionResult(com.intellij.execution.DefaultExecutionResult) Operation(com.microsoft.azuretools.telemetrywrapper.Operation) Project(com.intellij.openapi.project.Project) SparkBatchJobDisconnectAction(com.microsoft.azure.hdinsight.spark.run.action.SparkBatchJobDisconnectAction) PublishSubject(rx.subjects.PublishSubject) SparkJobLogConsoleView(com.microsoft.azure.hdinsight.spark.ui.SparkJobLogConsoleView) SparkLogLine(com.microsoft.azure.hdinsight.spark.common.log.SparkLogLine) Nullable(com.microsoft.azuretools.azurecommons.helpers.Nullable)

Example 9 with Nullable

use of com.microsoft.azuretools.azurecommons.helpers.Nullable in project azure-tools-for-java by Microsoft.

the class UIHelperImpl method getOpenedFile.

@Nullable
@Override
public <T extends StorageServiceTreeItem> Object getOpenedFile(@NotNull Object projectObject, @NotNull String accountName, @NotNull T item) {
    FileEditorManager fileEditorManager = FileEditorManager.getInstance((Project) projectObject);
    for (VirtualFile editedFile : fileEditorManager.getOpenFiles()) {
        T editedItem = editedFile.getUserData((Key<T>) name2Key.get(item.getClass()));
        StorageAccount editedStorageAccount = editedFile.getUserData(STORAGE_KEY);
        ClientStorageAccount editedClientStorageAccount = editedFile.getUserData(CLIENT_STORAGE_KEY);
        if (((editedStorageAccount != null && editedStorageAccount.name().equals(accountName)) || (editedClientStorageAccount != null && editedClientStorageAccount.getName().equals(accountName))) && editedItem != null && editedItem.getName().equals(item.getName())) {
            return editedFile;
        }
    }
    return null;
}
Also used : VirtualFile(com.intellij.openapi.vfs.VirtualFile) LightVirtualFile(com.intellij.testFramework.LightVirtualFile) FileEditorManager(com.intellij.openapi.fileEditor.FileEditorManager) ClientStorageAccount(com.microsoft.tooling.msservices.model.storage.ClientStorageAccount) StorageAccount(com.microsoft.azure.management.storage.StorageAccount) ClientStorageAccount(com.microsoft.tooling.msservices.model.storage.ClientStorageAccount) Nullable(com.microsoft.azuretools.azurecommons.helpers.Nullable)

Example 10 with Nullable

use of com.microsoft.azuretools.azurecommons.helpers.Nullable in project azure-tools-for-java by Microsoft.

the class CreateFunctionAppAction method openDialog.

@AzureOperation(name = "function.open_creation_dialog", type = AzureOperation.Type.ACTION)
private void openDialog(final Project project, @Nullable final FunctionAppConfig data) {
    final FunctionAppCreationDialog dialog = new FunctionAppCreationDialog(project);
    if (Objects.nonNull(data)) {
        dialog.setData(data);
    }
    dialog.setOkActionListener((config) -> {
        dialog.close();
        this.createFunctionApp(config).subscribe(functionApp -> {
        }, (error) -> {
            final String title = String.format("Reopen dialog \"%s\"", dialog.getTitle());
            final Consumer<Object> act = t -> AzureTaskManager.getInstance().runLater("open dialog", () -> this.openDialog(project, config));
            final Action<?> action = new Action<>(act, new ActionView.Builder(title));
            AzureMessager.getMessager().error(error, null, action);
        });
    });
    dialog.show();
}
Also used : FunctionAppCreationDialog(com.microsoft.azure.toolkit.intellij.function.FunctionAppCreationDialog) AzureOperation(com.microsoft.azure.toolkit.lib.common.operation.AzureOperation) AzureBundle.message(com.microsoft.intellij.ui.messages.AzureBundle.message) AzureUIRefreshCore(com.microsoft.azuretools.utils.AzureUIRefreshCore) IntellijAzureMessager(com.microsoft.azure.toolkit.intellij.common.messager.IntellijAzureMessager) AzureTask(com.microsoft.azure.toolkit.lib.common.task.AzureTask) Single(rx.Single) AzureOperationBundle.title(com.microsoft.azure.toolkit.lib.common.operation.AzureOperationBundle.title) Project(com.intellij.openapi.project.Project) AzureString(com.microsoft.azure.toolkit.lib.common.bundle.AzureString) FunctionAppService(com.microsoft.azure.toolkit.lib.function.FunctionAppService) ProgressManager(com.intellij.openapi.progress.ProgressManager) AzureActionEnum(com.microsoft.tooling.msservices.serviceexplorer.AzureActionEnum) NodeActionListener(com.microsoft.tooling.msservices.serviceexplorer.NodeActionListener) AzureMessager(com.microsoft.azure.toolkit.lib.common.messager.AzureMessager) Nullable(com.microsoft.azuretools.azurecommons.helpers.Nullable) IAzureMessage(com.microsoft.azure.toolkit.lib.common.messager.IAzureMessage) AzureUIRefreshEvent(com.microsoft.azuretools.utils.AzureUIRefreshEvent) Operation(com.microsoft.azuretools.telemetrywrapper.Operation) AzureSignInAction(com.microsoft.intellij.actions.AzureSignInAction) Name(com.microsoft.tooling.msservices.helpers.Name) FunctionAppConfig(com.microsoft.azure.toolkit.lib.function.FunctionAppConfig) AzureTelemetry(com.microsoft.azure.toolkit.lib.common.telemetry.AzureTelemetry) Objects(java.util.Objects) Consumer(java.util.function.Consumer) ProgressIndicator(com.intellij.openapi.progress.ProgressIndicator) FunctionModule(com.microsoft.tooling.msservices.serviceexplorer.azure.function.FunctionModule) ActionView(com.microsoft.azure.toolkit.lib.common.action.ActionView) TelemetryConstants(com.microsoft.azuretools.telemetry.TelemetryConstants) IFunctionApp(com.microsoft.azure.toolkit.lib.appservice.service.IFunctionApp) Action(com.microsoft.azure.toolkit.lib.common.action.Action) TelemetryManager(com.microsoft.azuretools.telemetrywrapper.TelemetryManager) NodeActionEvent(com.microsoft.tooling.msservices.serviceexplorer.NodeActionEvent) AzureTaskManager(com.microsoft.azure.toolkit.lib.common.task.AzureTaskManager) AzureSignInAction(com.microsoft.intellij.actions.AzureSignInAction) Action(com.microsoft.azure.toolkit.lib.common.action.Action) ActionView(com.microsoft.azure.toolkit.lib.common.action.ActionView) AzureString(com.microsoft.azure.toolkit.lib.common.bundle.AzureString) FunctionAppCreationDialog(com.microsoft.azure.toolkit.intellij.function.FunctionAppCreationDialog) AzureOperation(com.microsoft.azure.toolkit.lib.common.operation.AzureOperation)

Aggregations

Nullable (com.microsoft.azuretools.azurecommons.helpers.Nullable)22 Project (com.intellij.openapi.project.Project)4 VirtualFile (com.intellij.openapi.vfs.VirtualFile)4 ILogger (com.microsoft.azure.hdinsight.common.logger.ILogger)3 NotNull (com.microsoft.azuretools.azurecommons.helpers.NotNull)3 ConsoleView (com.intellij.execution.ui.ConsoleView)2 Module (com.intellij.openapi.module.Module)2 ProgressIndicator (com.intellij.openapi.progress.ProgressIndicator)2 ProgressManager (com.intellij.openapi.progress.ProgressManager)2 LightVirtualFile (com.intellij.testFramework.LightVirtualFile)2 HDIException (com.microsoft.azure.hdinsight.sdk.common.HDIException)2 Azure (com.microsoft.azure.management.Azure)2 Action (com.microsoft.azure.toolkit.lib.common.action.Action)2 ActionView (com.microsoft.azure.toolkit.lib.common.action.ActionView)2 AzureString (com.microsoft.azure.toolkit.lib.common.bundle.AzureString)2 AzureMessager (com.microsoft.azure.toolkit.lib.common.messager.AzureMessager)2 AzureOperation (com.microsoft.azure.toolkit.lib.common.operation.AzureOperation)2 AzureOperationBundle.title (com.microsoft.azure.toolkit.lib.common.operation.AzureOperationBundle.title)2 AzureTask (com.microsoft.azure.toolkit.lib.common.task.AzureTask)2 AzureTaskManager (com.microsoft.azure.toolkit.lib.common.task.AzureTaskManager)2