Search in sources :

Example 1 with DatabeanFielder

use of io.datarouter.model.serialize.fielder.DatabeanFielder in project datarouter by hotpads.

the class DirectoryGroupQueueNode method putMulti.

@Override
public void putMulti(Collection<D> databeans, Config config) {
    DatabeanFielder<PK, D> fielder = getFieldInfo().getSampleFielder();
    StringDatabeanCodec codec = getFieldInfo().getSampleFielder().getStringDatabeanCodec();
    List<byte[]> databeansAsBytes = Scanner.of(databeans).map(databean -> codec.toBytes(databean, fielder)).list();
    List<List<byte[]>> groups = codec.makeGroups(databeansAsBytes, MAX_MESSAGE_BYTES);
    Scanner.of(groups).map(codec::concatGroup).forEach(directoryQueue::putMessage);
}
Also used : DirectoryQueue(io.datarouter.filesystem.raw.queue.DirectoryQueue) Scanner(io.datarouter.scanner.Scanner) StringDatabeanCodec(io.datarouter.model.serialize.StringDatabeanCodec) Collection(java.util.Collection) Supplier(java.util.function.Supplier) PrimaryKey(io.datarouter.model.key.primary.PrimaryKey) BaseQueueMessage(io.datarouter.storage.queue.BaseQueueMessage) Objects(java.util.Objects) List(java.util.List) Databean(io.datarouter.model.databean.Databean) PhysicalGroupQueueStorageNode(io.datarouter.storage.node.op.raw.GroupQueueStorage.PhysicalGroupQueueStorageNode) Config(io.datarouter.storage.config.Config) Optional(java.util.Optional) DatabeanFielder(io.datarouter.model.serialize.fielder.DatabeanFielder) NodeParams(io.datarouter.storage.node.NodeParams) GroupQueueMessage(io.datarouter.storage.queue.GroupQueueMessage) List(java.util.List) StringDatabeanCodec(io.datarouter.model.serialize.StringDatabeanCodec)

Example 2 with DatabeanFielder

use of io.datarouter.model.serialize.fielder.DatabeanFielder in project datarouter by hotpads.

the class DirectoryGroupQueueNode method peekMulti.

@Override
public List<GroupQueueMessage<PK, D>> peekMulti(Config config) {
    int limit = config.findLimit().orElse(1);
    DatabeanFielder<PK, D> fielder = getFieldInfo().getSampleFielder();
    StringDatabeanCodec codec = getFieldInfo().getSampleFielder().getStringDatabeanCodec();
    Supplier<D> databeanSupplier = getFieldInfo().getDatabeanSupplier();
    return Scanner.generate(directoryQueue::peek).limit(limit).advanceWhile(Optional::isPresent).map(Optional::get).map(directoryQueueMessage -> {
        byte[] id = directoryQueueMessage.getBytesId();
        List<D> databeans = codec.fromStringMulti(directoryQueueMessage.content, fielder, databeanSupplier);
        return new GroupQueueMessage<>(id, databeans);
    }).list();
}
Also used : DirectoryQueue(io.datarouter.filesystem.raw.queue.DirectoryQueue) Scanner(io.datarouter.scanner.Scanner) StringDatabeanCodec(io.datarouter.model.serialize.StringDatabeanCodec) Collection(java.util.Collection) Supplier(java.util.function.Supplier) PrimaryKey(io.datarouter.model.key.primary.PrimaryKey) BaseQueueMessage(io.datarouter.storage.queue.BaseQueueMessage) Objects(java.util.Objects) List(java.util.List) Databean(io.datarouter.model.databean.Databean) PhysicalGroupQueueStorageNode(io.datarouter.storage.node.op.raw.GroupQueueStorage.PhysicalGroupQueueStorageNode) Config(io.datarouter.storage.config.Config) Optional(java.util.Optional) DatabeanFielder(io.datarouter.model.serialize.fielder.DatabeanFielder) NodeParams(io.datarouter.storage.node.NodeParams) GroupQueueMessage(io.datarouter.storage.queue.GroupQueueMessage) Optional(java.util.Optional) List(java.util.List) StringDatabeanCodec(io.datarouter.model.serialize.StringDatabeanCodec)

Aggregations

DirectoryQueue (io.datarouter.filesystem.raw.queue.DirectoryQueue)2 Databean (io.datarouter.model.databean.Databean)2 PrimaryKey (io.datarouter.model.key.primary.PrimaryKey)2 StringDatabeanCodec (io.datarouter.model.serialize.StringDatabeanCodec)2 DatabeanFielder (io.datarouter.model.serialize.fielder.DatabeanFielder)2 Scanner (io.datarouter.scanner.Scanner)2 Config (io.datarouter.storage.config.Config)2 NodeParams (io.datarouter.storage.node.NodeParams)2 PhysicalGroupQueueStorageNode (io.datarouter.storage.node.op.raw.GroupQueueStorage.PhysicalGroupQueueStorageNode)2 BaseQueueMessage (io.datarouter.storage.queue.BaseQueueMessage)2 GroupQueueMessage (io.datarouter.storage.queue.GroupQueueMessage)2 Collection (java.util.Collection)2 List (java.util.List)2 Objects (java.util.Objects)2 Optional (java.util.Optional)2 Supplier (java.util.function.Supplier)2