Examples with StreamSpecification co.cask.cdap.api.data.stream.StreamSpecification used on opensource projects

Example 1 with StreamSpecification

use of co.cask.cdap.api.data.stream.StreamSpecification in project cdap by caskdata.

the class DefaultDatasetConfigurer method addStream.

@Override
public void addStream(Stream stream) {
    checkArgument(stream != null, "Stream cannot be null.");
    StreamSpecification spec = stream.configure();
    StreamSpecification existingSpec = streams.get(spec.getName());
    if (existingSpec != null && !existingSpec.equals(spec)) {
        throw new IllegalArgumentException(String.format("Stream '%s' was added multiple times with different specs. " + "Please resolve the conflict so that there is only one spec for the stream.", spec.getName()));
    }
    streams.put(spec.getName(), spec);
}

Example 2 with StreamSpecification

use of co.cask.cdap.api.data.stream.StreamSpecification in project cdap by caskdata.

the class FileStreamAdmin method getProperties.

@Override
public StreamProperties getProperties(StreamId streamId) throws Exception {
    // User should have any access on the stream to read its properties
    ensureAccess(streamId);
    // get the principal which will be used for impersonation to display as owner
    String ownerPrincipal = ownerAdmin.getOwnerPrincipal(streamId);
    StreamConfig config = getConfig(streamId);
    StreamSpecification spec = streamMetaStore.getStream(streamId);
    return new StreamProperties(config.getTTL(), config.getFormat(), config.getNotificationThresholdMB(), spec.getDescription(), ownerPrincipal);
}

Example 3 with StreamSpecification

use of co.cask.cdap.api.data.stream.StreamSpecification in project cdap by caskdata.

the class InMemoryStreamAdmin method dropAllInNamespace.

@Override
public void dropAllInNamespace(NamespaceId namespace) throws Exception {
    queueService.resetStreamsWithPrefix(QueueName.prefixForNamedspacedStream(namespace.getNamespace()));
    for (StreamSpecification spec : streamMetaStore.listStreams(namespace)) {
        // Remove metadata for the stream
        StreamId stream = namespace.stream(spec.getName());
        metadataStore.removeMetadata(stream);
        streamMetaStore.removeStream(stream);
    }
}

Example 4 with StreamSpecification

use of co.cask.cdap.api.data.stream.StreamSpecification in project cdap by caskdata.

the class ExistingEntitySystemMetadataWriter method writeSystemMetadataForStreams.

private void writeSystemMetadataForStreams(NamespaceId namespace) throws Exception {
    for (StreamSpecification streamSpec : store.getAllStreams(namespace)) {
        StreamId streamId = namespace.stream(streamSpec.getName());
        SystemMetadataWriter writer = new StreamSystemMetadataWriter(metadataStore, streamId, streamAdmin.getConfig(streamId), streamSpec.getDescription());
        writer.write();
        for (StreamViewId view : streamAdmin.listViews(streamId)) {
            writer = new ViewSystemMetadataWriter(metadataStore, view, viewAdmin.get(view), true);
            writer.write();
        }
    }
}

Example 5 with StreamSpecification

use of co.cask.cdap.api.data.stream.StreamSpecification in project cdap by caskdata.

the class LocalStreamService method initialize.

@Override
protected void initialize() throws Exception {
    for (Map.Entry<NamespaceId, StreamSpecification> streamSpecEntry : streamMetaStore.listStreams().entries()) {
        StreamId streamId = streamSpecEntry.getKey().stream(streamSpecEntry.getValue().getName());
        StreamConfig config;
        try {
            config = streamAdmin.getConfig(streamId);
        } catch (FileNotFoundException e) {
            // TODO: this kind of inconsistency should not happen. [CDAP-5722]
            LOG.warn("Inconsistent stream state: Stream '{}' exists in meta store " + "but its configuration file does not exist", streamId);
            continue;
        } catch (Exception e) {
            LOG.warn("Inconsistent stream state: Stream '{}' exists in meta store " + "but its configuration cannot be read:", streamId, e);
            continue;
        }
        long eventsSizes = getStreamEventsSize(streamId);
        createSizeAggregator(streamId, eventsSizes, config.getNotificationThresholdMB());
    }
}