Search in sources :

Example 1 with SystemMetadataWriter

use of io.cdap.cdap.data2.metadata.system.SystemMetadataWriter in project cdap by caskdata.

the class ExistingEntitySystemMetadataWriter method writeSystemMetadataForApps.

private void writeSystemMetadataForApps(NamespaceId namespace) {
    for (ApplicationSpecification appSpec : store.getAllApplications(namespace)) {
        ApplicationId app = namespace.app(appSpec.getName());
        SystemMetadataWriter writer = new AppSystemMetadataWriter(metadataStore, app, appSpec);
        writer.write();
        writeSystemMetadataForPrograms(app, appSpec);
    }
}
Also used : ApplicationSpecification(co.cask.cdap.api.app.ApplicationSpecification) DatasetSystemMetadataWriter(co.cask.cdap.data2.metadata.system.DatasetSystemMetadataWriter) ProgramSystemMetadataWriter(co.cask.cdap.data2.metadata.system.ProgramSystemMetadataWriter) ViewSystemMetadataWriter(co.cask.cdap.data2.metadata.system.ViewSystemMetadataWriter) SystemMetadataWriter(co.cask.cdap.data2.metadata.system.SystemMetadataWriter) AppSystemMetadataWriter(co.cask.cdap.data2.metadata.system.AppSystemMetadataWriter) ArtifactSystemMetadataWriter(co.cask.cdap.data2.metadata.system.ArtifactSystemMetadataWriter) StreamSystemMetadataWriter(co.cask.cdap.data2.metadata.system.StreamSystemMetadataWriter) ApplicationId(co.cask.cdap.proto.id.ApplicationId) AppSystemMetadataWriter(co.cask.cdap.data2.metadata.system.AppSystemMetadataWriter)

Example 2 with SystemMetadataWriter

use of io.cdap.cdap.data2.metadata.system.SystemMetadataWriter in project cdap by caskdata.

the class ExistingEntitySystemMetadataWriter method writeSystemMetadataForDatasets.

private void writeSystemMetadataForDatasets(NamespaceId namespace, DatasetFramework dsFramework) throws DatasetManagementException, IOException, NamespaceNotFoundException {
    SystemDatasetInstantiatorFactory systemDatasetInstantiatorFactory = new SystemDatasetInstantiatorFactory(locationFactory, dsFramework, cConf);
    try (SystemDatasetInstantiator systemDatasetInstantiator = systemDatasetInstantiatorFactory.create()) {
        for (DatasetSpecificationSummary summary : dsFramework.getInstances(namespace)) {
            final DatasetId dsInstance = namespace.dataset(summary.getName());
            DatasetProperties dsProperties = DatasetProperties.of(summary.getProperties());
            String dsType = summary.getType();
            Dataset dataset = null;
            try {
                try {
                    dataset = impersonator.doAs(dsInstance, new Callable<Dataset>() {

                        @Override
                        public Dataset call() throws Exception {
                            return systemDatasetInstantiator.getDataset(dsInstance);
                        }
                    });
                } catch (Exception e) {
                    LOG.warn("Exception while instantiating dataset {}", dsInstance, e);
                }
                SystemMetadataWriter writer = new DatasetSystemMetadataWriter(metadataStore, dsInstance, dsProperties, dataset, dsType, summary.getDescription());
                writer.write();
            } finally {
                if (dataset != null) {
                    dataset.close();
                }
            }
        }
    }
}
Also used : SystemDatasetInstantiatorFactory(co.cask.cdap.data.dataset.SystemDatasetInstantiatorFactory) DatasetSystemMetadataWriter(co.cask.cdap.data2.metadata.system.DatasetSystemMetadataWriter) SystemDatasetInstantiator(co.cask.cdap.data.dataset.SystemDatasetInstantiator) Dataset(co.cask.cdap.api.dataset.Dataset) DatasetProperties(co.cask.cdap.api.dataset.DatasetProperties) DatasetSystemMetadataWriter(co.cask.cdap.data2.metadata.system.DatasetSystemMetadataWriter) ProgramSystemMetadataWriter(co.cask.cdap.data2.metadata.system.ProgramSystemMetadataWriter) ViewSystemMetadataWriter(co.cask.cdap.data2.metadata.system.ViewSystemMetadataWriter) SystemMetadataWriter(co.cask.cdap.data2.metadata.system.SystemMetadataWriter) AppSystemMetadataWriter(co.cask.cdap.data2.metadata.system.AppSystemMetadataWriter) ArtifactSystemMetadataWriter(co.cask.cdap.data2.metadata.system.ArtifactSystemMetadataWriter) StreamSystemMetadataWriter(co.cask.cdap.data2.metadata.system.StreamSystemMetadataWriter) DatasetSpecificationSummary(co.cask.cdap.proto.DatasetSpecificationSummary) Callable(java.util.concurrent.Callable) DatasetManagementException(co.cask.cdap.api.dataset.DatasetManagementException) NamespaceNotFoundException(co.cask.cdap.common.NamespaceNotFoundException) IOException(java.io.IOException) DatasetId(co.cask.cdap.proto.id.DatasetId)

Example 3 with SystemMetadataWriter

use of io.cdap.cdap.data2.metadata.system.SystemMetadataWriter in project cdap by caskdata.

the class ExistingEntitySystemMetadataWriter method writeSystemMetadataForPrograms.

private void writeSystemMetadataForPrograms(ApplicationId app, ProgramType programType, Collection<? extends ProgramSpecification> programSpecs) {
    for (ProgramSpecification programSpec : programSpecs) {
        ProgramId programId = app.program(programType, programSpec.getName());
        SystemMetadataWriter writer = new ProgramSystemMetadataWriter(metadataStore, programId, programSpec, true);
        writer.write();
    }
}
Also used : ProgramSpecification(co.cask.cdap.api.ProgramSpecification) DatasetSystemMetadataWriter(co.cask.cdap.data2.metadata.system.DatasetSystemMetadataWriter) ProgramSystemMetadataWriter(co.cask.cdap.data2.metadata.system.ProgramSystemMetadataWriter) ViewSystemMetadataWriter(co.cask.cdap.data2.metadata.system.ViewSystemMetadataWriter) SystemMetadataWriter(co.cask.cdap.data2.metadata.system.SystemMetadataWriter) AppSystemMetadataWriter(co.cask.cdap.data2.metadata.system.AppSystemMetadataWriter) ArtifactSystemMetadataWriter(co.cask.cdap.data2.metadata.system.ArtifactSystemMetadataWriter) StreamSystemMetadataWriter(co.cask.cdap.data2.metadata.system.StreamSystemMetadataWriter) ProgramId(co.cask.cdap.proto.id.ProgramId) ProgramSystemMetadataWriter(co.cask.cdap.data2.metadata.system.ProgramSystemMetadataWriter)

Example 4 with SystemMetadataWriter

use of io.cdap.cdap.data2.metadata.system.SystemMetadataWriter in project cdap by caskdata.

the class ExistingEntitySystemMetadataWriter method writeSystemMetadataForArtifacts.

private void writeSystemMetadataForArtifacts(NamespaceId namespace) throws IOException {
    for (ArtifactDetail artifactDetail : artifactStore.getArtifacts(namespace)) {
        co.cask.cdap.api.artifact.ArtifactId artifact = artifactDetail.getDescriptor().getArtifactId();
        ArtifactInfo artifactInfo = new ArtifactInfo(artifact, artifactDetail.getMeta().getClasses(), artifactDetail.getMeta().getProperties());
        ArtifactId artifactId = namespace.artifact(artifact.getName(), artifact.getVersion().getVersion());
        SystemMetadataWriter writer = new ArtifactSystemMetadataWriter(metadataStore, artifactId, artifactInfo);
        writer.write();
    }
}
Also used : ArtifactInfo(co.cask.cdap.api.artifact.ArtifactInfo) ArtifactId(co.cask.cdap.proto.id.ArtifactId) DatasetSystemMetadataWriter(co.cask.cdap.data2.metadata.system.DatasetSystemMetadataWriter) ProgramSystemMetadataWriter(co.cask.cdap.data2.metadata.system.ProgramSystemMetadataWriter) ViewSystemMetadataWriter(co.cask.cdap.data2.metadata.system.ViewSystemMetadataWriter) SystemMetadataWriter(co.cask.cdap.data2.metadata.system.SystemMetadataWriter) AppSystemMetadataWriter(co.cask.cdap.data2.metadata.system.AppSystemMetadataWriter) ArtifactSystemMetadataWriter(co.cask.cdap.data2.metadata.system.ArtifactSystemMetadataWriter) StreamSystemMetadataWriter(co.cask.cdap.data2.metadata.system.StreamSystemMetadataWriter) ArtifactSystemMetadataWriter(co.cask.cdap.data2.metadata.system.ArtifactSystemMetadataWriter) ArtifactDetail(co.cask.cdap.internal.app.runtime.artifact.ArtifactDetail)

Example 5 with SystemMetadataWriter

use of io.cdap.cdap.data2.metadata.system.SystemMetadataWriter in project cdap by caskdata.

the class ExistingEntitySystemMetadataWriter method writeSystemMetadataForStreams.

private void writeSystemMetadataForStreams(NamespaceId namespace) throws Exception {
    for (StreamSpecification streamSpec : store.getAllStreams(namespace)) {
        StreamId streamId = namespace.stream(streamSpec.getName());
        SystemMetadataWriter writer = new StreamSystemMetadataWriter(metadataStore, streamId, streamAdmin.getConfig(streamId), streamSpec.getDescription());
        writer.write();
        for (StreamViewId view : streamAdmin.listViews(streamId)) {
            writer = new ViewSystemMetadataWriter(metadataStore, view, viewAdmin.get(view), true);
            writer.write();
        }
    }
}
Also used : StreamSystemMetadataWriter(co.cask.cdap.data2.metadata.system.StreamSystemMetadataWriter) StreamId(co.cask.cdap.proto.id.StreamId) ViewSystemMetadataWriter(co.cask.cdap.data2.metadata.system.ViewSystemMetadataWriter) StreamSpecification(co.cask.cdap.api.data.stream.StreamSpecification) DatasetSystemMetadataWriter(co.cask.cdap.data2.metadata.system.DatasetSystemMetadataWriter) ProgramSystemMetadataWriter(co.cask.cdap.data2.metadata.system.ProgramSystemMetadataWriter) ViewSystemMetadataWriter(co.cask.cdap.data2.metadata.system.ViewSystemMetadataWriter) SystemMetadataWriter(co.cask.cdap.data2.metadata.system.SystemMetadataWriter) AppSystemMetadataWriter(co.cask.cdap.data2.metadata.system.AppSystemMetadataWriter) ArtifactSystemMetadataWriter(co.cask.cdap.data2.metadata.system.ArtifactSystemMetadataWriter) StreamSystemMetadataWriter(co.cask.cdap.data2.metadata.system.StreamSystemMetadataWriter) StreamViewId(co.cask.cdap.proto.id.StreamViewId)

Aggregations

SystemMetadataWriter (co.cask.cdap.data2.metadata.system.SystemMetadataWriter)8 AppSystemMetadataWriter (co.cask.cdap.data2.metadata.system.AppSystemMetadataWriter)6 DatasetSystemMetadataWriter (co.cask.cdap.data2.metadata.system.DatasetSystemMetadataWriter)6 ProgramSystemMetadataWriter (co.cask.cdap.data2.metadata.system.ProgramSystemMetadataWriter)6 StreamSystemMetadataWriter (co.cask.cdap.data2.metadata.system.StreamSystemMetadataWriter)6 ArtifactSystemMetadataWriter (co.cask.cdap.data2.metadata.system.ArtifactSystemMetadataWriter)5 ViewSystemMetadataWriter (co.cask.cdap.data2.metadata.system.ViewSystemMetadataWriter)5 IOException (java.io.IOException)3 ApplicationSpecification (co.cask.cdap.api.app.ApplicationSpecification)2 Dataset (co.cask.cdap.api.dataset.Dataset)2 NotFoundException (co.cask.cdap.common.NotFoundException)2 ApplicationId (co.cask.cdap.proto.id.ApplicationId)2 ProgramSpecification (co.cask.cdap.api.ProgramSpecification)1 ArtifactInfo (co.cask.cdap.api.artifact.ArtifactInfo)1 StreamSpecification (co.cask.cdap.api.data.stream.StreamSpecification)1 DatasetManagementException (co.cask.cdap.api.dataset.DatasetManagementException)1 DatasetProperties (co.cask.cdap.api.dataset.DatasetProperties)1 IncompatibleUpdateException (co.cask.cdap.api.dataset.IncompatibleUpdateException)1 BadRequestException (co.cask.cdap.common.BadRequestException)1 NamespaceNotFoundException (co.cask.cdap.common.NamespaceNotFoundException)1