Search in sources :

Example 6 with Application

use of io.cdap.cdap.api.app.Application in project cdap by cdapio.

the class DefaultArtifactInspector method inspectApplications.

private ArtifactClasses.Builder inspectApplications(Id.Artifact artifactId, ArtifactClasses.Builder builder, Location artifactLocation, ClassLoader artifactClassLoader) throws IOException, InvalidArtifactException {
    // right now we force users to include the application main class as an attribute in their manifest,
    // which forces them to have a single application class.
    // in the future, we may want to let users do this or maybe specify a list of classes or
    // a package that will be searched for applications, to allow multiple applications in a single artifact.
    String mainClassName;
    try {
        Manifest manifest = BundleJarUtil.getManifest(artifactLocation);
        if (manifest == null) {
            return builder;
        }
        Attributes manifestAttributes = manifest.getMainAttributes();
        if (manifestAttributes == null) {
            return builder;
        }
        mainClassName = manifestAttributes.getValue(ManifestFields.MAIN_CLASS);
    } catch (ZipException e) {
        throw new InvalidArtifactException(String.format("Couldn't unzip artifact %s, please check it is a valid jar file.", artifactId), e);
    }
    if (mainClassName == null) {
        return builder;
    }
    try {
        Class<?> mainClass = artifactClassLoader.loadClass(mainClassName);
        if (!(Application.class.isAssignableFrom(mainClass))) {
            // possible for 3rd party plugin artifacts to have the main class set
            return builder;
        }
        Application app = (Application) mainClass.newInstance();
        java.lang.reflect.Type configType;
        // we can deserialize the config into that object. Otherwise it'll just be a Config
        try {
            configType = Artifacts.getConfigType(app.getClass());
        } catch (Exception e) {
            throw new InvalidArtifactException(String.format("Could not resolve config type for Application class %s in artifact %s. " + "The type must extend Config and cannot be parameterized.", mainClassName, artifactId));
        }
        Schema configSchema = configType == Config.class ? null : schemaGenerator.generate(configType);
        builder.addApp(new ApplicationClass(mainClassName, "", configSchema, getArtifactRequirements(app.getClass())));
    } catch (ClassNotFoundException e) {
        throw new InvalidArtifactException(String.format("Could not find Application main class %s in artifact %s.", mainClassName, artifactId));
    } catch (UnsupportedTypeException e) {
        throw new InvalidArtifactException(String.format("Config for Application %s in artifact %s has an unsupported schema. " + "The type must extend Config and cannot be parameterized.", mainClassName, artifactId));
    } catch (InstantiationException | IllegalAccessException e) {
        throw new InvalidArtifactException(String.format("Could not instantiate Application class %s in artifact %s.", mainClassName, artifactId), e);
    }
    return builder;
}
Also used : PluginConfig(io.cdap.cdap.api.plugin.PluginConfig) Config(io.cdap.cdap.api.Config) Schema(io.cdap.cdap.api.data.schema.Schema) Attributes(java.util.jar.Attributes) ApplicationClass(io.cdap.cdap.api.artifact.ApplicationClass) ZipException(java.util.zip.ZipException) Manifest(java.util.jar.Manifest) InvalidMetadataException(io.cdap.cdap.common.InvalidMetadataException) ZipException(java.util.zip.ZipException) UnsupportedTypeException(io.cdap.cdap.api.data.schema.UnsupportedTypeException) EOFException(java.io.EOFException) InvalidArtifactException(io.cdap.cdap.common.InvalidArtifactException) IOException(java.io.IOException) UnsupportedTypeException(io.cdap.cdap.api.data.schema.UnsupportedTypeException) Application(io.cdap.cdap.api.app.Application) InvalidArtifactException(io.cdap.cdap.common.InvalidArtifactException)

Example 7 with Application

use of io.cdap.cdap.api.app.Application in project cdap by cdapio.

the class InMemoryConfigurator method config.

/**
 * Executes the <code>Application.configure</code> within the same JVM.
 * <p>
 * This method could be dangerous and should be used only in standalone mode.
 * </p>
 *
 * @return A instance of {@link ListenableFuture}.
 */
@Override
public ListenableFuture<ConfigResponse> config() {
    // Create the classloader
    EntityImpersonator classLoaderImpersonator = new EntityImpersonator(artifactId.toEntityId(), impersonator);
    try (CloseableClassLoader classLoader = artifactRepository.createArtifactClassLoader(new ArtifactDescriptor(artifactId.getNamespace().getId(), artifactId.toArtifactId(), artifactLocation), classLoaderImpersonator)) {
        SettableFuture<ConfigResponse> result = SettableFuture.create();
        Object appMain = classLoader.loadClass(appClassName).newInstance();
        if (!(appMain instanceof Application)) {
            throw new IllegalStateException(String.format("Application main class is of invalid type: %s", appMain.getClass().getName()));
        }
        Application<?> app = (Application<?>) appMain;
        ConfigResponse response = createResponse(app, classLoader);
        result.set(response);
        return result;
    } catch (Throwable t) {
        return Futures.immediateFailedFuture(t);
    }
}
Also used : ArtifactDescriptor(io.cdap.cdap.internal.app.runtime.artifact.ArtifactDescriptor) EntityImpersonator(io.cdap.cdap.security.impersonation.EntityImpersonator) ConfigResponse(io.cdap.cdap.app.deploy.ConfigResponse) CloseableClassLoader(io.cdap.cdap.api.artifact.CloseableClassLoader) Application(io.cdap.cdap.api.app.Application)

Example 8 with Application

use of io.cdap.cdap.api.app.Application in project cdap by caskdata.

the class InMemoryConfigurator method config.

/**
 * Executes the <code>Application.configure</code> within the same JVM.
 * <p>
 * This method could be dangerous and should be used only in standalone mode.
 * </p>
 *
 * @return A instance of {@link ListenableFuture}.
 */
@Override
public ListenableFuture<ConfigResponse> config() {
    // Create the classloader
    EntityImpersonator classLoaderImpersonator = new EntityImpersonator(artifactId.toEntityId(), impersonator);
    try (CloseableClassLoader classLoader = artifactRepository.createArtifactClassLoader(new ArtifactDescriptor(artifactId.getNamespace().getId(), artifactId.toArtifactId(), artifactLocation), classLoaderImpersonator)) {
        SettableFuture<ConfigResponse> result = SettableFuture.create();
        Object appMain = classLoader.loadClass(appClassName).newInstance();
        if (!(appMain instanceof Application)) {
            throw new IllegalStateException(String.format("Application main class is of invalid type: %s", appMain.getClass().getName()));
        }
        Application<?> app = (Application<?>) appMain;
        ConfigResponse response = createResponse(app, classLoader);
        result.set(response);
        return result;
    } catch (Throwable t) {
        return Futures.immediateFailedFuture(t);
    }
}
Also used : ArtifactDescriptor(io.cdap.cdap.internal.app.runtime.artifact.ArtifactDescriptor) EntityImpersonator(io.cdap.cdap.security.impersonation.EntityImpersonator) ConfigResponse(io.cdap.cdap.app.deploy.ConfigResponse) CloseableClassLoader(io.cdap.cdap.api.artifact.CloseableClassLoader) Application(io.cdap.cdap.api.app.Application)

Example 9 with Application

use of io.cdap.cdap.api.app.Application in project cdap by caskdata.

the class ApplicationLifecycleService method getLatestAppArtifactForUpgrade.

/**
 * Finds latest application artifact for given application and current artifact for upgrading application.
 * If no artifact found then returns current artifact as the candidate.
 *
 * @param appId application Id to find latest app artifact for.
 * @param currentArtifactId current artifact used by application.
 * @param allowedArtifactScopes artifact scopes to search in for finding candidate artifacts.
 * @param allowSnapshot whether to consider snapshot version of artifacts or not for upgrade.
 * @return {@link ArtifactSummary} for the artifact to be used for upgrade purpose.
 * @throws NotFoundException if there is no artifact available for given artifact.
 * @throws Exception if there was an exception during finding candidate artifact.
 */
private ArtifactSummary getLatestAppArtifactForUpgrade(ApplicationId appId, ArtifactId currentArtifactId, Set<ArtifactScope> allowedArtifactScopes, boolean allowSnapshot) throws Exception {
    List<ArtifactSummary> availableArtifacts = new ArrayList<>();
    // At the least, current artifact should be in the set of available artifacts.
    availableArtifacts.add(ArtifactSummary.from(currentArtifactId));
    // Find candidate artifacts from all scopes we need to consider.
    for (ArtifactScope scope : allowedArtifactScopes) {
        NamespaceId artifactNamespaceToConsider = ArtifactScope.SYSTEM.equals(scope) ? NamespaceId.SYSTEM : appId.getParent();
        List<ArtifactSummary> artifacts;
        try {
            artifacts = artifactRepository.getArtifactSummaries(artifactNamespaceToConsider, currentArtifactId.getName(), Integer.MAX_VALUE, ArtifactSortOrder.ASC);
        } catch (ArtifactNotFoundException e) {
            // This can happen if we are looking for candidate artifact in multiple namespace.
            continue;
        }
        for (ArtifactSummary artifactSummary : artifacts) {
            ArtifactVersion artifactVersion = new ArtifactVersion(artifactSummary.getVersion());
            // Consider if it is a non-snapshot version artifact or it is a snapshot version than allowSnapshot is true.
            if ((artifactVersion.isSnapshot() && allowSnapshot) || !artifactVersion.isSnapshot()) {
                availableArtifacts.add(artifactSummary);
            }
        }
    }
    // Find the artifact with latest version.
    Optional<ArtifactSummary> newArtifactCandidate = availableArtifacts.stream().max(Comparator.comparing(artifactSummary -> new ArtifactVersion(artifactSummary.getVersion())));
    io.cdap.cdap.proto.id.ArtifactId currentArtifact = new io.cdap.cdap.proto.id.ArtifactId(appId.getNamespace(), currentArtifactId.getName(), currentArtifactId.getVersion().getVersion());
    return newArtifactCandidate.orElseThrow(() -> new ArtifactNotFoundException(currentArtifact));
}
Also used : RunRecordDetail(io.cdap.cdap.internal.app.store.RunRecordDetail) ApplicationDetail(io.cdap.cdap.proto.ApplicationDetail) GsonBuilder(com.google.gson.GsonBuilder) ScanApplicationsRequest(io.cdap.cdap.app.store.ScanApplicationsRequest) ArtifactSummary(io.cdap.cdap.api.artifact.ArtifactSummary) ArtifactSortOrder(io.cdap.cdap.proto.artifact.ArtifactSortOrder) Map(java.util.Map) MetricDeleteQuery(io.cdap.cdap.api.metrics.MetricDeleteQuery) ApplicationNotFoundException(io.cdap.cdap.common.ApplicationNotFoundException) PreferencesService(io.cdap.cdap.config.PreferencesService) ProgramTerminator(io.cdap.cdap.internal.app.deploy.ProgramTerminator) Set(java.util.Set) ApplicationClass(io.cdap.cdap.api.artifact.ApplicationClass) AppDeploymentInfo(io.cdap.cdap.internal.app.deploy.pipeline.AppDeploymentInfo) StandardCharsets(java.nio.charset.StandardCharsets) Id(io.cdap.cdap.common.id.Id) InstanceId(io.cdap.cdap.proto.id.InstanceId) MetricsSystemClient(io.cdap.cdap.api.metrics.MetricsSystemClient) ApplicationId(io.cdap.cdap.proto.id.ApplicationId) Joiner(com.google.common.base.Joiner) ZipOutputStream(java.util.zip.ZipOutputStream) Iterables(com.google.common.collect.Iterables) WorkflowSpecification(io.cdap.cdap.api.workflow.WorkflowSpecification) PluginInstanceDetail(io.cdap.cdap.proto.PluginInstanceDetail) AccessEnforcer(io.cdap.cdap.security.spi.authorization.AccessEnforcer) ArrayList(java.util.ArrayList) LinkedHashMap(java.util.LinkedHashMap) ProgramRunId(io.cdap.cdap.proto.id.ProgramRunId) ArtifactVersionRange(io.cdap.cdap.api.artifact.ArtifactVersionRange) AdminEventPublisher(io.cdap.cdap.internal.profile.AdminEventPublisher) ManagerFactory(io.cdap.cdap.app.deploy.ManagerFactory) CapabilityNotAvailableException(io.cdap.cdap.internal.capability.CapabilityNotAvailableException) JsonWriter(com.google.gson.stream.JsonWriter) ArtifactDetail(io.cdap.cdap.internal.app.runtime.artifact.ArtifactDetail) Nullable(javax.annotation.Nullable) ApplicationUpdateResult(io.cdap.cdap.api.app.ApplicationUpdateResult) Throwables(com.google.common.base.Throwables) Impersonator(io.cdap.cdap.security.impersonation.Impersonator) IOException(java.io.IOException) File(java.io.File) CannotBeDeletedException(io.cdap.cdap.common.CannotBeDeletedException) ExecutionException(java.util.concurrent.ExecutionException) ProgramSpecification(io.cdap.cdap.api.ProgramSpecification) ArtifactRepository(io.cdap.cdap.internal.app.runtime.artifact.ArtifactRepository) ArtifactId(io.cdap.cdap.api.artifact.ArtifactId) NamespaceId(io.cdap.cdap.proto.id.NamespaceId) AccessException(io.cdap.cdap.api.security.AccessException) Inject(com.google.inject.Inject) LoggerFactory(org.slf4j.LoggerFactory) Principal(io.cdap.cdap.proto.security.Principal) AppDeploymentRuntimeInfo(io.cdap.cdap.internal.app.deploy.pipeline.AppDeploymentRuntimeInfo) Gson(com.google.gson.Gson) AuthenticationContext(io.cdap.cdap.security.spi.authentication.AuthenticationContext) CloseableClassLoader(io.cdap.cdap.api.artifact.CloseableClassLoader) EntityType(io.cdap.cdap.proto.element.EntityType) MetadataMutation(io.cdap.cdap.spi.metadata.MetadataMutation) ZipEntry(java.util.zip.ZipEntry) Application(io.cdap.cdap.api.app.Application) BatchingConsumer(io.cdap.cdap.common.utils.BatchingConsumer) Artifacts(io.cdap.cdap.internal.app.runtime.artifact.Artifacts) AccessPermission(io.cdap.cdap.proto.security.AccessPermission) Collection(java.util.Collection) ApplicationSpecification(io.cdap.cdap.api.app.ApplicationSpecification) MessagingService(io.cdap.cdap.messaging.MessagingService) MetadataServiceClient(io.cdap.cdap.data2.metadata.writer.MetadataServiceClient) Collectors(java.util.stream.Collectors) Base64(java.util.Base64) List(java.util.List) MultiThreadMessagingContext(io.cdap.cdap.messaging.context.MultiThreadMessagingContext) Type(java.lang.reflect.Type) ApplicationConfigUpdateAction(io.cdap.cdap.api.app.ApplicationConfigUpdateAction) CaseInsensitiveEnumTypeAdapterFactory(io.cdap.cdap.common.io.CaseInsensitiveEnumTypeAdapterFactory) JsonIOException(com.google.gson.JsonIOException) AppFabric(io.cdap.cdap.common.conf.Constants.AppFabric) Entry(java.util.Map.Entry) Optional(java.util.Optional) Constants(io.cdap.cdap.common.conf.Constants) Manager(io.cdap.cdap.app.deploy.Manager) InvalidArtifactException(io.cdap.cdap.common.InvalidArtifactException) ArtifactScope(io.cdap.cdap.api.artifact.ArtifactScope) ArtifactAlreadyExistsException(io.cdap.cdap.common.ArtifactAlreadyExistsException) NotFoundException(io.cdap.cdap.common.NotFoundException) StandardPermission(io.cdap.cdap.proto.security.StandardPermission) ArtifactRange(io.cdap.cdap.api.artifact.ArtifactRange) HashMap(java.util.HashMap) ApplicationWithPrograms(io.cdap.cdap.internal.app.deploy.pipeline.ApplicationWithPrograms) EntityId(io.cdap.cdap.proto.id.EntityId) ProgramType(io.cdap.cdap.proto.ProgramType) HashSet(java.util.HashSet) ImmutableList(com.google.common.collect.ImmutableList) AbstractIdleService(com.google.common.util.concurrent.AbstractIdleService) OutputStreamWriter(java.io.OutputStreamWriter) DefaultApplicationUpdateContext(io.cdap.cdap.internal.app.DefaultApplicationUpdateContext) SimpleEntry(java.util.AbstractMap.SimpleEntry) UsageRegistry(io.cdap.cdap.data2.registry.UsageRegistry) Logger(org.slf4j.Logger) Scheduler(io.cdap.cdap.scheduler.Scheduler) ApplicationFilter(io.cdap.cdap.app.store.ApplicationFilter) ProgramId(io.cdap.cdap.proto.id.ProgramId) Store(io.cdap.cdap.app.store.Store) EntityImpersonator(io.cdap.cdap.security.impersonation.EntityImpersonator) Consumer(java.util.function.Consumer) KerberosPrincipalId(io.cdap.cdap.proto.id.KerberosPrincipalId) ArtifactNotFoundException(io.cdap.cdap.common.ArtifactNotFoundException) CConfiguration(io.cdap.cdap.common.conf.CConfiguration) ArtifactVersion(io.cdap.cdap.api.artifact.ArtifactVersion) CapabilityReader(io.cdap.cdap.internal.capability.CapabilityReader) OwnerAdmin(io.cdap.cdap.security.impersonation.OwnerAdmin) AppRequest(io.cdap.cdap.proto.artifact.AppRequest) Comparator(java.util.Comparator) Plugin(io.cdap.cdap.api.plugin.Plugin) Collections(java.util.Collections) SecurityUtil(io.cdap.cdap.security.impersonation.SecurityUtil) ArtifactId(io.cdap.cdap.api.artifact.ArtifactId) ArrayList(java.util.ArrayList) ArtifactScope(io.cdap.cdap.api.artifact.ArtifactScope) ArtifactSummary(io.cdap.cdap.api.artifact.ArtifactSummary) ArtifactVersion(io.cdap.cdap.api.artifact.ArtifactVersion) NamespaceId(io.cdap.cdap.proto.id.NamespaceId) ArtifactNotFoundException(io.cdap.cdap.common.ArtifactNotFoundException)

Example 10 with Application

use of io.cdap.cdap.api.app.Application in project cdap by caskdata.

the class DistributedProgramRunnerTxTimeoutTest method setup.

@BeforeClass
public static void setup() {
    Application app = new AppWithAllProgramTypes();
    DefaultAppConfigurer configurer = new DefaultAppConfigurer(Id.Namespace.DEFAULT, new Id.Artifact(Id.Namespace.DEFAULT, "artifact", new ArtifactVersion("0.1")), app);
    app.configure(configurer, () -> null);
    appSpec = configurer.createSpecification("app", "1.0");
    Injector injector = Guice.createInjector(new AppFabricTestModule(cConf));
    cConf.setInt(TxConstants.Manager.CFG_TX_MAX_TIMEOUT, 60);
    serviceRunner = new DistributedServiceProgramRunner(cConf, yConf, null, ClusterMode.ON_PREMISE, null, injector);
    workerRunner = new DistributedWorkerProgramRunner(cConf, yConf, null, ClusterMode.ON_PREMISE, null, injector);
    mapreduceRunner = new DistributedMapReduceProgramRunner(cConf, yConf, null, ClusterMode.ON_PREMISE, null, injector);
    sparkRunner = new DistributedSparkProgramRunner(SparkCompat.SPARK2_2_11, cConf, yConf, null, null, ClusterMode.ON_PREMISE, null, injector);
    workflowRunner = new DistributedWorkflowProgramRunner(cConf, yConf, null, ClusterMode.ON_PREMISE, null, null, injector);
}
Also used : DefaultAppConfigurer(io.cdap.cdap.app.DefaultAppConfigurer) ArtifactVersion(io.cdap.cdap.api.artifact.ArtifactVersion) Injector(com.google.inject.Injector) AppFabricTestModule(io.cdap.cdap.internal.guice.AppFabricTestModule) ProgramId(io.cdap.cdap.proto.id.ProgramId) Id(io.cdap.cdap.common.id.Id) AbstractApplication(io.cdap.cdap.api.app.AbstractApplication) Application(io.cdap.cdap.api.app.Application) DistributedSparkProgramRunner(io.cdap.cdap.app.runtime.spark.distributed.DistributedSparkProgramRunner) BeforeClass(org.junit.BeforeClass)

Aggregations

Application (io.cdap.cdap.api.app.Application)14 IOException (java.io.IOException)10 Type (java.lang.reflect.Type)8 ApplicationClass (io.cdap.cdap.api.artifact.ApplicationClass)6 ArtifactSummary (io.cdap.cdap.api.artifact.ArtifactSummary)6 CloseableClassLoader (io.cdap.cdap.api.artifact.CloseableClassLoader)6 AccessException (io.cdap.cdap.api.security.AccessException)6 InvalidArtifactException (io.cdap.cdap.common.InvalidArtifactException)6 AppRequest (io.cdap.cdap.proto.artifact.AppRequest)6 ApplicationId (io.cdap.cdap.proto.id.ApplicationId)6 EntityImpersonator (io.cdap.cdap.security.impersonation.EntityImpersonator)6 JsonIOException (com.google.gson.JsonIOException)4 ArtifactVersion (io.cdap.cdap.api.artifact.ArtifactVersion)4 ApplicationNotFoundException (io.cdap.cdap.common.ApplicationNotFoundException)4 ArtifactAlreadyExistsException (io.cdap.cdap.common.ArtifactAlreadyExistsException)4 ArtifactNotFoundException (io.cdap.cdap.common.ArtifactNotFoundException)4 CannotBeDeletedException (io.cdap.cdap.common.CannotBeDeletedException)4 NotFoundException (io.cdap.cdap.common.NotFoundException)4 Id (io.cdap.cdap.common.id.Id)4 DefaultApplicationUpdateContext (io.cdap.cdap.internal.app.DefaultApplicationUpdateContext)4