Search in sources :

Example 1 with SplitEntities

use of google.registry.beam.datastore.BulkDeleteDatastorePipeline.SplitEntities in project nomulus by google.

the class BulkDeleteDatastorePipelineTest method splitEntitiesByKind.

@Test
void splitEntitiesByKind() {
    TupleTagList tags = getDeletionTags(2);
    PCollection<String> kinds = testPipeline.apply("InjectKinds", Create.of("A", "B"));
    PCollectionView<Map<String, TupleTag<Entity>>> kindToTagMapping = BulkDeleteDatastorePipeline.mapKindsToDeletionTags(kinds, tags).apply(View.asMap());
    Entity entityA = createTestEntity("A", 1);
    Entity entityB = createTestEntity("B", 2);
    PCollection<Entity> entities = testPipeline.apply("InjectEntities", Create.of(entityA, entityB));
    PCollectionTuple allCollections = entities.apply("SplitByKind", ParDo.of(new SplitEntities(kindToTagMapping)).withSideInputs(kindToTagMapping).withOutputTags(getOneDeletionTag("placeholder"), tags));
    PAssert.that(allCollections.get((TupleTag<Entity>) tags.get(0))).containsInAnyOrder(entityA);
    PAssert.that(allCollections.get((TupleTag<Entity>) tags.get(1))).containsInAnyOrder(entityB);
    testPipeline.run();
}
Also used : Entity(com.google.datastore.v1.Entity) SplitEntities(google.registry.beam.datastore.BulkDeleteDatastorePipeline.SplitEntities) TupleTagList(org.apache.beam.sdk.values.TupleTagList) PCollectionTuple(org.apache.beam.sdk.values.PCollectionTuple) Map(java.util.Map) ImmutableMap(com.google.common.collect.ImmutableMap) Test(org.junit.jupiter.api.Test)

Aggregations

ImmutableMap (com.google.common.collect.ImmutableMap)1 Entity (com.google.datastore.v1.Entity)1 SplitEntities (google.registry.beam.datastore.BulkDeleteDatastorePipeline.SplitEntities)1 Map (java.util.Map)1 PCollectionTuple (org.apache.beam.sdk.values.PCollectionTuple)1 TupleTagList (org.apache.beam.sdk.values.TupleTagList)1 Test (org.junit.jupiter.api.Test)1