Search in sources :

Example 1 with InterProcessCommunication

use of com.cinchapi.concourse.server.plugin.io.InterProcessCommunication in project concourse by cinchapi.

the class BackgroundThreadTest method testBackgroundExecutorHasCorrectInformation.

@Test
public void testBackgroundExecutorHasCorrectInformation() throws InterruptedException {
    InterProcessCommunication outgoing = new MessageQueue();
    ConcurrentMap<AccessToken, RemoteMethodResponse> responses = Maps.newConcurrentMap();
    BackgroundExecutor executor = PluginExecutors.newCachedBackgroundExecutor(outgoing, responses);
    CountDownLatch latch = new CountDownLatch(1);
    final AtomicBoolean passed = new AtomicBoolean(true);
    executor.execute(Random.getSimpleString(), () -> {
        InterProcessCommunication myOutgoing = ((BackgroundThread) Thread.currentThread()).outgoing();
        ConcurrentMap<AccessToken, RemoteMethodResponse> myResponses = ((BackgroundThread) Thread.currentThread()).responses();
        try {
            Assert.assertSame(outgoing, myOutgoing);
            Assert.assertSame(responses, myResponses);
        } catch (AssertionError e) {
            passed.set(false);
            e.printStackTrace();
        }
        latch.countDown();
    });
    latch.await();
    Assert.assertTrue(passed.get());
}
Also used : AtomicBoolean(java.util.concurrent.atomic.AtomicBoolean) MessageQueue(com.cinchapi.concourse.server.plugin.io.MessageQueue) AccessToken(com.cinchapi.concourse.thrift.AccessToken) InterProcessCommunication(com.cinchapi.concourse.server.plugin.io.InterProcessCommunication) CountDownLatch(java.util.concurrent.CountDownLatch) Test(org.junit.Test)

Example 2 with InterProcessCommunication

use of com.cinchapi.concourse.server.plugin.io.InterProcessCommunication in project concourse by cinchapi.

the class RealTimePlugin method run.

@Override
public final void run() {
    Reflection.call(this, "setReadyState");
    // For a RealTimePlugin, the first fromServer message contains the
    // address for the stream channel
    ByteBuffer data = fromServer.read();
    RemoteMessage message = serializer.deserialize(data);
    if (message.type() == RemoteMessage.Type.ATTRIBUTE) {
        RemoteAttributeExchange attribute = (RemoteAttributeExchange) message;
        if (attribute.key().equalsIgnoreCase(STREAM_ATTRIBUTE)) {
            log.debug("Listening for streamed packets at {}", attribute.value());
            @SuppressWarnings("resource") final InterProcessCommunication stream = new MessageQueue(attribute.value());
            // Create a separate event loop to process Packets of writes
            // that come from the server.
            Thread loop = new Thread(() -> {
                ByteBuffer bytes = null;
                while ((bytes = stream.read()) != null) {
                    final Packet packet = serializer.deserialize(bytes);
                    // Each packet should be processed in a separate
                    // worker thread
                    workers.execute(() -> {
                        log.debug("Received packet from Concourse Server: {}", packet);
                        handlePacket(packet);
                    });
                }
            });
            loop.setDaemon(true);
            loop.start();
            // Start normal plugin operations
            super.run();
        } else {
            throw new IllegalStateException("Unsupported attribute " + attribute);
        }
    } else {
        throw new IllegalStateException();
    }
}
Also used : MessageQueue(com.cinchapi.concourse.server.plugin.io.MessageQueue) InterProcessCommunication(com.cinchapi.concourse.server.plugin.io.InterProcessCommunication) ByteBuffer(java.nio.ByteBuffer)

Example 3 with InterProcessCommunication

use of com.cinchapi.concourse.server.plugin.io.InterProcessCommunication in project concourse by cinchapi.

the class PluginManager method startEventLoop.

/**
 * Start a {@link Thread} that serves as an event loop; processing both
 * requests and responses {@code #fromPlugin}.
 * <p>
 * Requests are forked to a {@link RemoteInvocationThread} for processing.
 * </p>
 * <p>
 * Responses are placed on the appropriate
 * {@link RegistryData#FROM_PLUGIN_RESPONSES queue} and listeners are
 * notified.
 * </p>
 *
 * @param id the plugin id
 * @return the event loop thread
 */
private Thread startEventLoop(String id) {
    final InterProcessCommunication incoming = (InterProcessCommunication) registry.get(id, RegistryData.FROM_PLUGIN);
    final InterProcessCommunication outgoing = (InterProcessCommunication) registry.get(id, RegistryData.FROM_SERVER);
    final ConcurrentMap<AccessToken, RemoteMethodResponse> fromPluginResponses = (ConcurrentMap<AccessToken, RemoteMethodResponse>) registry.get(id, RegistryData.FROM_PLUGIN_RESPONSES);
    Thread loop = new Thread(new Runnable() {

        @Override
        public void run() {
            ByteBuffer data;
            while ((data = incoming.read()) != null) {
                RemoteMessage message = serializer.deserialize(data);
                if (message.type() == RemoteMessage.Type.REQUEST) {
                    RemoteMethodRequest request = (RemoteMethodRequest) message;
                    Logger.debug("Received REQUEST from Plugin {}: {}", id, request);
                    Thread worker = new RemoteInvocationThread(request, outgoing, server, true, fromPluginResponses);
                    worker.setUncaughtExceptionHandler((thread, throwable) -> {
                        Logger.error("While processing request '{}' from '{}', the following " + "non-recoverable error occurred:", request, id, throwable);
                    });
                    worker.start();
                } else if (message.type() == RemoteMessage.Type.RESPONSE) {
                    RemoteMethodResponse response = (RemoteMethodResponse) message;
                    Logger.debug("Received RESPONSE from Plugin {}: {}", id, response);
                    ConcurrentMaps.putAndSignal(fromPluginResponses, response.creds, response);
                } else if (message.type() == RemoteMessage.Type.STOP) {
                    break;
                } else {
                    // Ignore the message...
                    continue;
                }
            }
        }
    }, "plugin-event-loop-" + id);
    loop.setDaemon(true);
    loop.start();
    return loop;
}
Also used : Version(com.github.zafarkhaja.semver.Version) AnyStrings(com.cinchapi.common.base.AnyStrings) JsonObject(com.google.gson.JsonObject) StringUtils(org.apache.commons.lang.StringUtils) URL(java.net.URL) Logger(com.cinchapi.concourse.util.Logger) HashBasedTable(com.google.common.collect.HashBasedTable) Reflections(org.reflections.Reflections) ByteBuffer(java.nio.ByteBuffer) DirectoryStream(java.nio.file.DirectoryStream) MessageQueue(com.cinchapi.concourse.server.plugin.io.MessageQueue) URLClassLoader(java.net.URLClassLoader) Future(java.util.concurrent.Future) Resources(com.cinchapi.concourse.util.Resources) CharStreams(com.google.common.io.CharStreams) Map(java.util.Map) ConcurrentMaps(com.cinchapi.concourse.util.ConcurrentMaps) AccessToken(com.cinchapi.concourse.thrift.AccessToken) Path(java.nio.file.Path) ZipException(java.util.zip.ZipException) FileSystem(com.cinchapi.concourse.server.io.FileSystem) ComplexTObject(com.cinchapi.concourse.thrift.ComplexTObject) Set(java.util.Set) Versions(com.cinchapi.concourse.server.plugin.util.Versions) Executors(java.util.concurrent.Executors) ClasspathHelper(org.reflections.util.ClasspathHelper) Sets(com.google.common.collect.Sets) ConcourseServer(com.cinchapi.concourse.server.ConcourseServer) List(java.util.List) Queues(com.cinchapi.concourse.util.Queues) WriteEvent(com.cinchapi.concourse.server.plugin.data.WriteEvent) Modifier(java.lang.reflect.Modifier) Entry(java.util.Map.Entry) CheckedExceptions(com.cinchapi.common.base.CheckedExceptions) TransactionToken(com.cinchapi.concourse.thrift.TransactionToken) JsonParser(com.google.gson.JsonParser) Constructor(java.lang.reflect.Constructor) ArrayList(java.util.ArrayList) ConcurrentMap(java.util.concurrent.ConcurrentMap) Reflection(com.cinchapi.common.reflect.Reflection) Lists(com.google.common.collect.Lists) BINARY_QUEUE(com.cinchapi.concourse.server.GlobalState.BINARY_QUEUE) ExecutorService(java.util.concurrent.ExecutorService) ConfigurationBuilder(org.reflections.util.ConfigurationBuilder) Time(com.cinchapi.concourse.time.Time) Iterator(java.util.Iterator) AfterInstallHook(com.cinchapi.concourse.server.plugin.hook.AfterInstallHook) Files(java.nio.file.Files) BaseEncoding(com.google.common.io.BaseEncoding) MorePaths(com.cinchapi.concourse.util.MorePaths) ZipFiles(com.cinchapi.concourse.util.ZipFiles) Throwables(com.google.common.base.Throwables) IOException(java.io.IOException) InputStreamReader(java.io.InputStreamReader) Maps(com.google.common.collect.Maps) File(java.io.File) JavaApp(com.cinchapi.concourse.server.io.process.JavaApp) ExecutionException(java.util.concurrent.ExecutionException) InterProcessCommunication(com.cinchapi.concourse.server.plugin.io.InterProcessCommunication) PluginSerializer(com.cinchapi.concourse.server.plugin.io.PluginSerializer) Paths(java.nio.file.Paths) Collections(java.util.Collections) Table(com.google.common.collect.Table) ConcurrentMap(java.util.concurrent.ConcurrentMap) ByteBuffer(java.nio.ByteBuffer) AccessToken(com.cinchapi.concourse.thrift.AccessToken) InterProcessCommunication(com.cinchapi.concourse.server.plugin.io.InterProcessCommunication)

Example 4 with InterProcessCommunication

use of com.cinchapi.concourse.server.plugin.io.InterProcessCommunication in project concourse by cinchapi.

the class PluginManager method invoke.

/**
 * Invoke {@code method} that is defined in the plugin endpoint inside of
 * {@clazz}. The provided {@code creds}, {@code transaction} token and
 * {@code environment} are used to ensure proper alignment with the
 * corresponding client session on the server.
 *
 * @param plugin class or alias name of the {@link Plugin}
 * @param method the name of the method to invoke
 * @param args a list of arguments to pass to the method
 * @param creds the {@link AccessToken} submitted to ConcourseServer via the
 *            invokePlugin method
 * @param transaction the {@link TransactionToken} submitted to
 *            ConcourseServer via
 *            the invokePlugin method
 * @param environment the environment submitted to ConcourseServer via the
 *            invokePlugin method
 * @return the response from the plugin
 */
public ComplexTObject invoke(String plugin, String method, List<ComplexTObject> args, final AccessToken creds, TransactionToken transaction, String environment) {
    String clazz = getIdByAlias(plugin);
    InterProcessCommunication fromServer = (InterProcessCommunication) registry.get(clazz, RegistryData.FROM_SERVER);
    if (fromServer == null) {
        String message = ambiguous.contains(plugin) ? "Multiple plugins are " + "configured to use the alias '{}' so it is not permitted. " + "Please invoke the plugin using its full qualified name" : "No plugin with id or alias {} exists";
        throw new PluginException(AnyStrings.format(message, clazz));
    }
    RemoteMethodRequest request = new RemoteMethodRequest(method, creds, transaction, environment, args);
    ByteBuffer buffer = serializer.serialize(request);
    fromServer.write(buffer);
    ConcurrentMap<AccessToken, RemoteMethodResponse> fromPluginResponses = (ConcurrentMap<AccessToken, RemoteMethodResponse>) registry.get(clazz, RegistryData.FROM_PLUGIN_RESPONSES);
    RemoteMethodResponse response = ConcurrentMaps.waitAndRemove(fromPluginResponses, creds);
    if (!response.isError()) {
        return response.response;
    } else {
        Logger.error("Plugin Exception:", response.error);
        throw new PluginException(AnyStrings.format("An error occurred when invoking '{}' in '{}': ", method, clazz, response.error));
    }
}
Also used : AccessToken(com.cinchapi.concourse.thrift.AccessToken) ConcurrentMap(java.util.concurrent.ConcurrentMap) InterProcessCommunication(com.cinchapi.concourse.server.plugin.io.InterProcessCommunication) ByteBuffer(java.nio.ByteBuffer)

Example 5 with InterProcessCommunication

use of com.cinchapi.concourse.server.plugin.io.InterProcessCommunication in project concourse by cinchapi.

the class PluginManager method startStreamToPlugin.

/**
 * Create a {@link InterProcessCommunication} segment over which the
 * PluginManager will stream real-time {@link Packet packets} that contain
 * writes.
 *
 * @param id the plugin id
 */
private void startStreamToPlugin(String id) {
    String tempDir = getPluginTempDirectory(id);
    String streamFile = FileSystem.tempFile(tempDir, "RT-", ".shm");
    Logger.debug("Creating real-time stream for {} at {}", id, streamFile);
    InterProcessCommunication stream = new MessageQueue(streamFile);
    Logger.debug("Shared memory for real-time stream of '{} is located at " + "'{}", id, streamFile);
    RemoteAttributeExchange attribute = new RemoteAttributeExchange("stream", streamFile);
    InterProcessCommunication fromServer = (InterProcessCommunication) registry.get(id, RegistryData.FROM_SERVER);
    ByteBuffer buffer = serializer.serialize(attribute);
    fromServer.write(buffer);
    streams.add(stream);
}
Also used : MessageQueue(com.cinchapi.concourse.server.plugin.io.MessageQueue) InterProcessCommunication(com.cinchapi.concourse.server.plugin.io.InterProcessCommunication) ByteBuffer(java.nio.ByteBuffer)

Aggregations

InterProcessCommunication (com.cinchapi.concourse.server.plugin.io.InterProcessCommunication)6 MessageQueue (com.cinchapi.concourse.server.plugin.io.MessageQueue)5 AccessToken (com.cinchapi.concourse.thrift.AccessToken)4 ByteBuffer (java.nio.ByteBuffer)4 ConcurrentMap (java.util.concurrent.ConcurrentMap)2 AnyStrings (com.cinchapi.common.base.AnyStrings)1 CheckedExceptions (com.cinchapi.common.base.CheckedExceptions)1 Reflection (com.cinchapi.common.reflect.Reflection)1 ConcourseServer (com.cinchapi.concourse.server.ConcourseServer)1 BINARY_QUEUE (com.cinchapi.concourse.server.GlobalState.BINARY_QUEUE)1 FileSystem (com.cinchapi.concourse.server.io.FileSystem)1 JavaApp (com.cinchapi.concourse.server.io.process.JavaApp)1 WriteEvent (com.cinchapi.concourse.server.plugin.data.WriteEvent)1 AfterInstallHook (com.cinchapi.concourse.server.plugin.hook.AfterInstallHook)1 PluginSerializer (com.cinchapi.concourse.server.plugin.io.PluginSerializer)1 Versions (com.cinchapi.concourse.server.plugin.util.Versions)1 ComplexTObject (com.cinchapi.concourse.thrift.ComplexTObject)1 TransactionToken (com.cinchapi.concourse.thrift.TransactionToken)1 Time (com.cinchapi.concourse.time.Time)1 ConcurrentMaps (com.cinchapi.concourse.util.ConcurrentMaps)1