Search in sources :

Example 1 with BroadcastRef

use of org.atlasapi.content.BroadcastRef in project atlas-deer by atlasapi.

the class AbstractScheduleStore method sendUpdateMessage.

private void sendUpdateMessage(Publisher source, List<ScheduleHierarchy> content, ScheduleBlocksUpdate update, Channel channel, Interval interval) throws WriteException {
    try {
        String messageId = UUID.randomUUID().toString();
        Timestamp messageTimestamp = Timestamp.of(DateTime.now(DateTimeZones.UTC));
        ScheduleRef updateRef = scheduleRef(content, channel, interval);
        ImmutableSet<BroadcastRef> staleRefs = broadcastRefs(update.getStaleEntries());
        ScheduleUpdateMessage message = new ScheduleUpdateMessage(messageId, messageTimestamp, new ScheduleUpdate(source, updateRef, staleRefs));
        Id channelId = message.getScheduleUpdate().getSchedule().getChannel();
        messageSender.sendMessage(message, Longs.toByteArray(channelId.longValue()));
    } catch (MessagingException e) {
        throw new WriteException(e);
    }
}
Also used : WriteException(org.atlasapi.entity.util.WriteException) MessagingException(com.metabroadcast.common.queue.MessagingException) BroadcastRef(org.atlasapi.content.BroadcastRef) Id(org.atlasapi.entity.Id) Timestamp(com.metabroadcast.common.time.Timestamp)

Example 2 with BroadcastRef

use of org.atlasapi.content.BroadcastRef in project atlas-deer by atlasapi.

the class CqlContentStore method removeItemRefsFromContainers.

private Collection<? extends Statement> removeItemRefsFromContainers(Item item, ContainerRef... containerRefs) {
    ItemRef ref = item.toRef();
    ImmutableMap<ItemRef, Iterable<BroadcastRef>> upcoming = ImmutableMap.of(item.toRef(), item.getUpcomingBroadcastRefs());
    Set<Ref> upcomingSerialised = upcoming.keySet().stream().map(refTranslator::serialize).collect(MoreCollectors.toImmutableSet());
    ImmutableMap<ItemRef, Iterable<LocationSummary>> availableLocations = ImmutableMap.of(item.toRef(), item.getAvailableLocations());
    Set<Ref> availableLocationsSerialised = availableLocations.keySet().stream().map(refTranslator::serialize).collect(MoreCollectors.toImmutableSet());
    Ref itemRefKey = refTranslator.serialize(ref);
    return Arrays.stream(containerRefs).filter(Objects::nonNull).flatMap(containerRef -> Lists.newArrayList(accessor.removeItemRefsFromContainer(containerRef.getId().longValue(), ImmutableSet.of(itemRefKey), upcomingSerialised, availableLocationsSerialised), accessor.removeItemSummariesFromContainer(containerRef.getId().longValue(), ImmutableSet.of(itemRefKey))).stream()).collect(Collectors.toList());
}
Also used : org.atlasapi.content(org.atlasapi.content) Broadcast(org.atlasapi.content.Broadcast) MissingResourceException(org.atlasapi.entity.util.MissingResourceException) LoggerFactory(org.slf4j.LoggerFactory) Resolved(org.atlasapi.entity.util.Resolved) MappingManager(com.datastax.driver.mapping.MappingManager) ConsistencyLevel(com.datastax.driver.core.ConsistencyLevel) Optional(com.google.common.base.Optional) Session(com.datastax.driver.core.Session) ContainerRef(org.atlasapi.content.ContainerRef) com.google.common.collect(com.google.common.collect) IdGenerator(com.metabroadcast.common.ids.IdGenerator) BatchStatement(com.datastax.driver.core.BatchStatement) Longs(com.google.common.primitives.Longs) Function(com.google.common.base.Function) OptionalMap(com.metabroadcast.common.collect.OptionalMap) Collectors(java.util.stream.Collectors) Promise(com.metabroadcast.promise.Promise) MoreCollectors(com.metabroadcast.common.stream.MoreCollectors) Statement(com.datastax.driver.core.Statement) ContainerSummary(org.atlasapi.content.ContainerSummary) java.util(java.util) ListenableFuture(com.google.common.util.concurrent.ListenableFuture) Clock(com.metabroadcast.common.time.Clock) Description(org.atlasapi.content.v2.model.udt.Description) ImmutableOptionalMap(com.metabroadcast.common.collect.ImmutableOptionalMap) Timestamp(com.metabroadcast.common.time.Timestamp) LocationSummary(org.atlasapi.content.LocationSummary) EquivalenceGraph(org.atlasapi.equivalence.EquivalenceGraph) MapStream(com.codepoetics.protonpack.maps.MapStream) StreamSupport(java.util.stream.StreamSupport) BroadcastRef(org.atlasapi.content.BroadcastRef) SeriesRef(org.atlasapi.content.SeriesRef) org.atlasapi.content.v2.model.udt(org.atlasapi.content.v2.model.udt) WriteResult(org.atlasapi.entity.util.WriteResult) org.atlasapi.content.v2.serialization(org.atlasapi.content.v2.serialization) Nullable(javax.annotation.Nullable) MetricRegistry(com.codahale.metrics.MetricRegistry) Logger(org.slf4j.Logger) DateTime(org.joda.time.DateTime) Preconditions.checkNotNull(com.google.common.base.Preconditions.checkNotNull) Throwables(com.google.common.base.Throwables) Id(org.atlasapi.entity.Id) ResourceUpdatedMessage(org.atlasapi.messaging.ResourceUpdatedMessage) ItemSummary(org.atlasapi.content.ItemSummary) ExecutionException(java.util.concurrent.ExecutionException) Futures(com.google.common.util.concurrent.Futures) EquivalenceGraphStore(org.atlasapi.equivalence.EquivalenceGraphStore) Mapper(com.datastax.driver.mapping.Mapper) MessageSender(com.metabroadcast.common.queue.MessageSender) Instant(org.joda.time.Instant) WriteException(org.atlasapi.entity.util.WriteException) ContainerRef(org.atlasapi.content.ContainerRef) BroadcastRef(org.atlasapi.content.BroadcastRef) SeriesRef(org.atlasapi.content.SeriesRef)

Example 3 with BroadcastRef

use of org.atlasapi.content.BroadcastRef in project atlas-deer by atlasapi.

the class CassandraEquivalentScheduleStore method writeSchedule.

@Override
protected void writeSchedule(ScheduleUpdate update, Map<ScheduleRef.Entry, EquivalentScheduleEntry> content) throws WriteException {
    DateTime now = clock.now();
    ImmutableMultimap<LocalDate, BroadcastRow> currentBroadcastRows = resolveBroadcasts(update.getSource(), update.getSchedule().getChannel(), update.getSchedule().getInterval());
    ImmutableList<EquivalentScheduleEntry> updateEntries = getEquivalentScheduleEntries(update.getSchedule(), content);
    ImmutableSet<BroadcastRef> updateBroadcastRefs = updateEntries.stream().map(EquivalentScheduleEntry::getBroadcast).map(Broadcast::toRef).collect(MoreCollectors.toImmutableSet());
    Set<BroadcastRow> staleBroadcasts = getStaleBroadcasts(updateBroadcastRefs, currentBroadcastRows);
    List<Statement> deletes = deleteStatements(update.getSource(), update.getSchedule().getChannel(), staleBroadcasts);
    log.info("Processing equivalent schedule update for {} {} {}: content: {}, " + "currentEntries:{}, update:{}, stale broadcasts from update:{}, " + "stale broadcasts:{}", update.getSource(), update.getSchedule().getChannel().longValue(), update.getSchedule().getInterval(), content, currentBroadcastRows.values().stream().map(BroadcastRow::toString).collect(Collectors.joining(",")), updateLog(updateBroadcastRefs), updateLog(update.getStaleBroadcasts()), staleBroadcasts.stream().map(BroadcastRow::toString).collect(Collectors.joining(",")));
    ImmutableList<Statement> updates = updateEntries.stream().flatMap(entry -> statementsForEntry(update.getSource(), entry, now).stream()).collect(MoreCollectors.toImmutableList());
    if (updates.isEmpty() && deletes.isEmpty()) {
        return;
    }
    BatchStatement updateBatch = new BatchStatement();
    updateBatch.addAll(Iterables.concat(updates, deletes));
    try {
        session.execute(updateBatch.setConsistencyLevel(write));
        log.info("Processed equivalent schedule update for {} {} {}, updates: {}, " + "deletes: {}", update.getSource(), update.getSchedule().getChannel().longValue(), update.getSchedule().getInterval(), updates.size(), deletes.size());
    } catch (NoHostAvailableException | QueryExecutionException e) {
        throw new WriteException(e);
    }
}
Also used : QueryBuilder.set(com.datastax.driver.core.querybuilder.QueryBuilder.set) QueryBuilder(com.datastax.driver.core.querybuilder.QueryBuilder) DateTimeZone(org.joda.time.DateTimeZone) Channel(org.atlasapi.channel.Channel) Date(java.util.Date) Column.dateColumn(org.atlasapi.util.Column.dateColumn) ResultSetFuture(com.datastax.driver.core.ResultSetFuture) Broadcast(org.atlasapi.content.Broadcast) LoggerFactory(org.slf4j.LoggerFactory) ImmutableCollection(com.google.common.collect.ImmutableCollection) ByteBuffer(java.nio.ByteBuffer) ConsistencyLevel(com.datastax.driver.core.ConsistencyLevel) EquivalenceGraphSerializer(org.atlasapi.equivalence.EquivalenceGraphSerializer) Preconditions.checkArgument(com.google.common.base.Preconditions.checkArgument) Map(java.util.Map) Session(com.datastax.driver.core.Session) BatchStatement(com.datastax.driver.core.BatchStatement) ImmutableSetMultimap(com.google.common.collect.ImmutableSetMultimap) ContentSerializer(org.atlasapi.content.ContentSerializer) InvalidProtocolBufferException(com.google.protobuf.InvalidProtocolBufferException) ContentSerializationVisitor(org.atlasapi.content.ContentSerializationVisitor) Function(com.google.common.base.Function) ImmutableSet(com.google.common.collect.ImmutableSet) QueryBuilder.update(com.datastax.driver.core.querybuilder.QueryBuilder.update) Column.bytesColumn(org.atlasapi.util.Column.bytesColumn) Item(org.atlasapi.content.Item) Set(java.util.Set) NoHostAvailableException(com.datastax.driver.core.exceptions.NoHostAvailableException) Collectors(java.util.stream.Collectors) Content(org.atlasapi.content.Content) Objects(java.util.Objects) ByteString(com.google.protobuf.ByteString) ContentResolver(org.atlasapi.content.ContentResolver) List(java.util.List) Stream(java.util.stream.Stream) Annotation(org.atlasapi.annotation.Annotation) EquivalenceRef(org.atlasapi.equivalence.EquivalenceRef) MoreCollectors(com.metabroadcast.common.stream.MoreCollectors) Optional(java.util.Optional) Statement(com.datastax.driver.core.Statement) Equivalent(org.atlasapi.equivalence.Equivalent) ContentProtos(org.atlasapi.serialization.protobuf.ContentProtos) Iterables(com.google.common.collect.Iterables) ByteArrayOutputStream(java.io.ByteArrayOutputStream) EquivalenceGraphFilter(org.atlasapi.equivalence.EquivalenceGraphFilter) ListenableFuture(com.google.common.util.concurrent.ListenableFuture) Row(com.datastax.driver.core.Row) Duration(org.joda.time.Duration) BroadcastSerializer(org.atlasapi.content.BroadcastSerializer) QueryBuilder.bindMarker(com.datastax.driver.core.querybuilder.QueryBuilder.bindMarker) Clock(com.metabroadcast.common.time.Clock) EquivalenceGraph(org.atlasapi.equivalence.EquivalenceGraph) PreparedStatement(com.datastax.driver.core.PreparedStatement) QueryExecutionException(com.datastax.driver.core.exceptions.QueryExecutionException) Interval(org.joda.time.Interval) ResultSet(com.datastax.driver.core.ResultSet) ImmutableList(com.google.common.collect.ImmutableList) Publisher(org.atlasapi.media.entity.Publisher) StreamSupport(java.util.stream.StreamSupport) Column(org.atlasapi.util.Column) BroadcastRef(org.atlasapi.content.BroadcastRef) ImmutableMultimap(com.google.common.collect.ImmutableMultimap) MetricRegistry(com.codahale.metrics.MetricRegistry) Logger(org.slf4j.Logger) QueryBuilder.eq(com.datastax.driver.core.querybuilder.QueryBuilder.eq) DateTime(org.joda.time.DateTime) Preconditions.checkNotNull(com.google.common.base.Preconditions.checkNotNull) Throwables(com.google.common.base.Throwables) IOException(java.io.IOException) Column.textColumn(org.atlasapi.util.Column.textColumn) Id(org.atlasapi.entity.Id) SetMultimap(com.google.common.collect.SetMultimap) LocalDate(org.joda.time.LocalDate) Futures(com.google.common.util.concurrent.Futures) EquivalenceGraphStore(org.atlasapi.equivalence.EquivalenceGraphStore) QueryBuilder.select(com.datastax.driver.core.querybuilder.QueryBuilder.select) Column.bigIntColumn(org.atlasapi.util.Column.bigIntColumn) WriteException(org.atlasapi.entity.util.WriteException) InputStream(java.io.InputStream) WriteException(org.atlasapi.entity.util.WriteException) BatchStatement(com.datastax.driver.core.BatchStatement) Statement(com.datastax.driver.core.Statement) PreparedStatement(com.datastax.driver.core.PreparedStatement) BroadcastRef(org.atlasapi.content.BroadcastRef) LocalDate(org.joda.time.LocalDate) DateTime(org.joda.time.DateTime) QueryExecutionException(com.datastax.driver.core.exceptions.QueryExecutionException) NoHostAvailableException(com.datastax.driver.core.exceptions.NoHostAvailableException) BatchStatement(com.datastax.driver.core.BatchStatement)

Example 4 with BroadcastRef

use of org.atlasapi.content.BroadcastRef in project atlas-deer by atlasapi.

the class CassandraEquivalentScheduleStore method getStaleBroadcasts.

private Set<BroadcastRow> getStaleBroadcasts(Set<BroadcastRef> updateBroadcastRefs, ImmutableMultimap<LocalDate, BroadcastRow> currentBroadcastRows) {
    ImmutableMultimap<LocalDate, BroadcastRef> updateBroadcastsByDay = getBroadcastsByDay(updateBroadcastRefs);
    ImmutableSet.Builder<BroadcastRow> staleBroadcasts = ImmutableSet.builder();
    for (LocalDate day : currentBroadcastRows.keySet()) {
        ImmutableCollection<BroadcastRow> currentBroadcastsInDay = currentBroadcastRows.get(day);
        ImmutableSet<String> updateBroadcastIdsInDay = updateBroadcastsByDay.get(day).stream().map(BroadcastRef::getSourceId).collect(MoreCollectors.toImmutableSet());
        ImmutableSet<BroadcastRow> staleBroadcastsInDay = currentBroadcastsInDay.stream().filter(broadcast -> !updateBroadcastIdsInDay.contains(broadcast.getBroadcastSourceId())).collect(MoreCollectors.toImmutableSet());
        staleBroadcasts.addAll(staleBroadcastsInDay);
    }
    return staleBroadcasts.build();
}
Also used : QueryBuilder.set(com.datastax.driver.core.querybuilder.QueryBuilder.set) QueryBuilder(com.datastax.driver.core.querybuilder.QueryBuilder) DateTimeZone(org.joda.time.DateTimeZone) Channel(org.atlasapi.channel.Channel) Date(java.util.Date) Column.dateColumn(org.atlasapi.util.Column.dateColumn) ResultSetFuture(com.datastax.driver.core.ResultSetFuture) Broadcast(org.atlasapi.content.Broadcast) LoggerFactory(org.slf4j.LoggerFactory) ImmutableCollection(com.google.common.collect.ImmutableCollection) ByteBuffer(java.nio.ByteBuffer) ConsistencyLevel(com.datastax.driver.core.ConsistencyLevel) EquivalenceGraphSerializer(org.atlasapi.equivalence.EquivalenceGraphSerializer) Preconditions.checkArgument(com.google.common.base.Preconditions.checkArgument) Map(java.util.Map) Session(com.datastax.driver.core.Session) BatchStatement(com.datastax.driver.core.BatchStatement) ImmutableSetMultimap(com.google.common.collect.ImmutableSetMultimap) ContentSerializer(org.atlasapi.content.ContentSerializer) InvalidProtocolBufferException(com.google.protobuf.InvalidProtocolBufferException) ContentSerializationVisitor(org.atlasapi.content.ContentSerializationVisitor) Function(com.google.common.base.Function) ImmutableSet(com.google.common.collect.ImmutableSet) QueryBuilder.update(com.datastax.driver.core.querybuilder.QueryBuilder.update) Column.bytesColumn(org.atlasapi.util.Column.bytesColumn) Item(org.atlasapi.content.Item) Set(java.util.Set) NoHostAvailableException(com.datastax.driver.core.exceptions.NoHostAvailableException) Collectors(java.util.stream.Collectors) Content(org.atlasapi.content.Content) Objects(java.util.Objects) ByteString(com.google.protobuf.ByteString) ContentResolver(org.atlasapi.content.ContentResolver) List(java.util.List) Stream(java.util.stream.Stream) Annotation(org.atlasapi.annotation.Annotation) EquivalenceRef(org.atlasapi.equivalence.EquivalenceRef) MoreCollectors(com.metabroadcast.common.stream.MoreCollectors) Optional(java.util.Optional) Statement(com.datastax.driver.core.Statement) Equivalent(org.atlasapi.equivalence.Equivalent) ContentProtos(org.atlasapi.serialization.protobuf.ContentProtos) Iterables(com.google.common.collect.Iterables) ByteArrayOutputStream(java.io.ByteArrayOutputStream) EquivalenceGraphFilter(org.atlasapi.equivalence.EquivalenceGraphFilter) ListenableFuture(com.google.common.util.concurrent.ListenableFuture) Row(com.datastax.driver.core.Row) Duration(org.joda.time.Duration) BroadcastSerializer(org.atlasapi.content.BroadcastSerializer) QueryBuilder.bindMarker(com.datastax.driver.core.querybuilder.QueryBuilder.bindMarker) Clock(com.metabroadcast.common.time.Clock) EquivalenceGraph(org.atlasapi.equivalence.EquivalenceGraph) PreparedStatement(com.datastax.driver.core.PreparedStatement) QueryExecutionException(com.datastax.driver.core.exceptions.QueryExecutionException) Interval(org.joda.time.Interval) ResultSet(com.datastax.driver.core.ResultSet) ImmutableList(com.google.common.collect.ImmutableList) Publisher(org.atlasapi.media.entity.Publisher) StreamSupport(java.util.stream.StreamSupport) Column(org.atlasapi.util.Column) BroadcastRef(org.atlasapi.content.BroadcastRef) ImmutableMultimap(com.google.common.collect.ImmutableMultimap) MetricRegistry(com.codahale.metrics.MetricRegistry) Logger(org.slf4j.Logger) QueryBuilder.eq(com.datastax.driver.core.querybuilder.QueryBuilder.eq) DateTime(org.joda.time.DateTime) Preconditions.checkNotNull(com.google.common.base.Preconditions.checkNotNull) Throwables(com.google.common.base.Throwables) IOException(java.io.IOException) Column.textColumn(org.atlasapi.util.Column.textColumn) Id(org.atlasapi.entity.Id) SetMultimap(com.google.common.collect.SetMultimap) LocalDate(org.joda.time.LocalDate) Futures(com.google.common.util.concurrent.Futures) EquivalenceGraphStore(org.atlasapi.equivalence.EquivalenceGraphStore) QueryBuilder.select(com.datastax.driver.core.querybuilder.QueryBuilder.select) Column.bigIntColumn(org.atlasapi.util.Column.bigIntColumn) WriteException(org.atlasapi.entity.util.WriteException) InputStream(java.io.InputStream) ImmutableSet(com.google.common.collect.ImmutableSet) BroadcastRef(org.atlasapi.content.BroadcastRef) ByteString(com.google.protobuf.ByteString) LocalDate(org.joda.time.LocalDate)

Example 5 with BroadcastRef

use of org.atlasapi.content.BroadcastRef in project atlas-deer by atlasapi.

the class CqlContentGenerator method setContainerFields.

private static void setContainerFields(Container container) {
    setContentFields(container);
    container.setItemRefs(ImmutableList.of(new ItemRef(Id.valueOf(8775), Publisher.BT, "sort key", DateTime.now(DateTimeZone.UTC))));
    container.setUpcomingContent(ImmutableMap.of(new ItemRef(Id.valueOf(221), Publisher.BT, "sort key", DateTime.now(DateTimeZone.UTC)), ImmutableList.of(new BroadcastRef("source id", Id.valueOf(32199), new Interval(DateTime.now(DateTimeZone.UTC).minusHours(2), DateTime.now(DateTimeZone.UTC))))));
    container.setAvailableContent(ImmutableMap.of(new ItemRef(Id.valueOf(23087), Publisher.BT, "sort key", DateTime.now(DateTimeZone.UTC)), ImmutableList.of(new LocationSummary(Boolean.TRUE, "uri", DateTime.now(DateTimeZone.UTC).minusHours(1), DateTime.now(DateTimeZone.UTC)))));
    container.setItemSummaries(ImmutableList.of(new ItemSummary(new ItemRef(Id.valueOf(32), Publisher.BT, "sort key", DateTime.now(DateTimeZone.UTC)), "title", "description", "image", 2015, ImmutableList.of(new Certificate("class", Countries.GB)))));
}
Also used : LocationSummary(org.atlasapi.content.LocationSummary) ItemRef(org.atlasapi.content.ItemRef) BroadcastRef(org.atlasapi.content.BroadcastRef) ItemSummary(org.atlasapi.content.ItemSummary) Interval(org.joda.time.Interval) Certificate(org.atlasapi.content.Certificate)

Aggregations

BroadcastRef (org.atlasapi.content.BroadcastRef)8 Id (org.atlasapi.entity.Id)5 WriteException (org.atlasapi.entity.util.WriteException)5 Interval (org.joda.time.Interval)5 MetricRegistry (com.codahale.metrics.MetricRegistry)4 BatchStatement (com.datastax.driver.core.BatchStatement)4 ConsistencyLevel (com.datastax.driver.core.ConsistencyLevel)4 Session (com.datastax.driver.core.Session)4 Statement (com.datastax.driver.core.Statement)4 Function (com.google.common.base.Function)4 Preconditions.checkNotNull (com.google.common.base.Preconditions.checkNotNull)4 Throwables (com.google.common.base.Throwables)4 Futures (com.google.common.util.concurrent.Futures)4 ListenableFuture (com.google.common.util.concurrent.ListenableFuture)4 MoreCollectors (com.metabroadcast.common.stream.MoreCollectors)4 Clock (com.metabroadcast.common.time.Clock)4 Collectors (java.util.stream.Collectors)4 StreamSupport (java.util.stream.StreamSupport)4 Broadcast (org.atlasapi.content.Broadcast)4 ItemSummary (org.atlasapi.content.ItemSummary)4