Search in sources :

Example 1 with UcxCallback

use of org.openucx.jucx.UcxCallback in project twister2 by DSC-SPIDAL.

the class TWSUCXChannel method sendMessage.

@Override
public boolean sendMessage(int id, ChannelMessage message, ChannelListener callback) {
    AtomicInteger buffersLeft = new AtomicInteger(message.getBuffers().size());
    for (DataBuffer buffer : message.getBuffers()) {
        buffer.getByteBuffer().limit(buffer.getSize());
        buffer.getByteBuffer().position(0);
        int tag = this.workerId * tagWIdOffset + message.getHeader().getEdge();
        LOG.log(Level.FINE, () -> String.format("SENDING to %d[%d] : %s, TAG[%d]", id, message.getHeader().getEdge(), buffer.getByteBuffer(), tag));
        this.endpoints.get(id).sendTaggedNonBlocking(buffer.getByteBuffer(), tag, new UcxCallback() {

            @Override
            public void onSuccess(UcpRequest request) {
                pendingSendRequests.decrementAndGet();
                if (buffersLeft.decrementAndGet() == 0) {
                    callback.onSendComplete(id, message.getHeader().getEdge(), message);
                }
            }

            @Override
            public void onError(int ucsStatus, String errorMsg) {
                // This is a catastrophic failure
                LOG.severe("UCX send request failed to worker " + id + " with status " + ucsStatus + ". Error : " + errorMsg);
                throw new Twister2RuntimeException("Send request to worker : " + id + " failed. " + errorMsg);
            }
        });
        this.pendingSendRequests.incrementAndGet();
    }
    return true;
}
Also used : Twister2RuntimeException(edu.iu.dsc.tws.api.exceptions.Twister2RuntimeException) AtomicInteger(java.util.concurrent.atomic.AtomicInteger) UcpRequest(org.openucx.jucx.ucp.UcpRequest) UcpEndpoint(org.openucx.jucx.ucp.UcpEndpoint) DataBuffer(edu.iu.dsc.tws.api.comms.packing.DataBuffer) UcxCallback(org.openucx.jucx.UcxCallback)

Aggregations

DataBuffer (edu.iu.dsc.tws.api.comms.packing.DataBuffer)1 Twister2RuntimeException (edu.iu.dsc.tws.api.exceptions.Twister2RuntimeException)1 AtomicInteger (java.util.concurrent.atomic.AtomicInteger)1 UcxCallback (org.openucx.jucx.UcxCallback)1 UcpEndpoint (org.openucx.jucx.ucp.UcpEndpoint)1 UcpRequest (org.openucx.jucx.ucp.UcpRequest)1