Search in sources :

Example 86 with ConcurrentHashMap

use of java.util.concurrent.ConcurrentHashMap in project chassis by Kixeye.

the class FlumeLoggerLoader method initialize.

@PostConstruct
public void initialize() {
    multicaster.addApplicationListener(loggingListener);
    ConcurrentHashMap<String, String> flumeConfig = new ConcurrentHashMap<>();
    List<String> sinks = new ArrayList<>();
    for (String server : servers) {
        String sinkName = server.replace(":", "-").replace(".", "_");
        String[] servers = server.split(":", 2);
        if (servers.length == 2) {
            flumeConfig.put(sinkName + ".type", "avro");
            flumeConfig.put(sinkName + ".channels", "channel-" + name);
            flumeConfig.put(sinkName + ".hostname", servers[0]);
            flumeConfig.put(sinkName + ".port", servers[1]);
        } else {
            logger.error("Invalid server format [{}], should be [hostname:port]", server);
        }
        sinks.add(sinkName);
    }
    // force some properties
    flumeConfig.put("channel.type", "file");
    flumeConfig.put("sinks", StringUtils.collectionToDelimitedString(sinks, " "));
    flumeConfig.putIfAbsent("processor.type", serversUsage);
    flumeConfig.put("channel.checkpointDir", SystemPropertyUtils.resolvePlaceholders("${user.dir}/flume-data/checkpoint"));
    flumeConfig.put("channel.dataDirs", SystemPropertyUtils.resolvePlaceholders("${user.dir}/flume-data/data"));
    agent = new EmbeddedAgent(name);
    agent.configure(flumeConfig);
    agent.start();
    appender = new FlumeLogAppender(agent, serviceName);
    installFlumeAppender();
}
Also used : EmbeddedAgent(org.apache.flume.agent.embedded.EmbeddedAgent) ArrayList(java.util.ArrayList) ConcurrentHashMap(java.util.concurrent.ConcurrentHashMap) PostConstruct(javax.annotation.PostConstruct)

Example 87 with ConcurrentHashMap

use of java.util.concurrent.ConcurrentHashMap in project azure-iot-sdk-java by Azure.

the class AmqpsTransportTest method messageSentRemovesSuccessfullyDeliveredMessageFromInProgressMap.

// Tests_SRS_AMQPSTRANSPORT_15_030: [If the message was successfully delivered,
// its callback is added to the list of callbacks to be executed.]
@Test
public void messageSentRemovesSuccessfullyDeliveredMessageFromInProgressMap() throws IOException {
    new NonStrictExpectations() {

        {
            new AmqpsIotHubConnection(mockConfig, false);
            result = mockConnection;
        }
    };
    AmqpsTransport transport = new AmqpsTransport(mockConfig, false);
    transport.open();
    Map<Integer, IotHubOutboundPacket> inProgressMessages = new ConcurrentHashMap<>();
    inProgressMessages.put(1, new IotHubOutboundPacket(new Message(), mockIotHubEventCallback, new Object()));
    inProgressMessages.put(2, new IotHubOutboundPacket(new Message(), mockIotHubEventCallback, new Object()));
    Deencapsulation.setField(transport, "inProgressMessages", inProgressMessages);
    transport.messageSent(1, true);
    new Verifications() {

        {
            new IotHubCallbackPacket(IotHubStatusCode.OK_EMPTY, (IotHubEventCallback) any, any);
            times = 1;
        }
    };
    Queue<IotHubOutboundPacket> waitingMessages = Deencapsulation.getField(transport, "waitingMessages");
    Queue<IotHubCallbackPacket> callbackList = Deencapsulation.getField(transport, "callbackList");
    Assert.assertTrue(inProgressMessages.size() == 1);
    Assert.assertTrue(waitingMessages.size() == 0);
    Assert.assertTrue(callbackList.size() == 1);
}
Also used : AmqpsTransport(com.microsoft.azure.sdk.iot.device.transport.amqps.AmqpsTransport) AmqpsMessage(com.microsoft.azure.sdk.iot.device.transport.amqps.AmqpsMessage) IotHubCallbackPacket(com.microsoft.azure.sdk.iot.device.transport.IotHubCallbackPacket) AmqpsIotHubConnection(com.microsoft.azure.sdk.iot.device.transport.amqps.AmqpsIotHubConnection) IotHubOutboundPacket(com.microsoft.azure.sdk.iot.device.transport.IotHubOutboundPacket) ConcurrentHashMap(java.util.concurrent.ConcurrentHashMap) Test(org.junit.Test)

Example 88 with ConcurrentHashMap

use of java.util.concurrent.ConcurrentHashMap in project azure-iot-sdk-java by Azure.

the class AmqpsTransportTest method messageSentReturnsIfThereAreNoMessagesInProgress.

// Tests_SRS_AMQPSTRANSPORT_15_029: [If the hash cannot be found in the list of keys for the messages in progress, the method returns.]
@Test
public void messageSentReturnsIfThereAreNoMessagesInProgress() throws IOException {
    new NonStrictExpectations() {

        {
            new AmqpsIotHubConnection(mockConfig, false);
            result = mockConnection;
        }
    };
    AmqpsTransport transport = new AmqpsTransport(mockConfig, false);
    transport.open();
    Map<Integer, IotHubOutboundPacket> inProgressMessages = new ConcurrentHashMap<>();
    Deencapsulation.setField(transport, "inProgressMessages", inProgressMessages);
    transport.messageSent(1, true);
    new Verifications() {

        {
            new IotHubCallbackPacket(IotHubStatusCode.OK_EMPTY, (IotHubEventCallback) any, any);
            times = 0;
        }
    };
}
Also used : AmqpsIotHubConnection(com.microsoft.azure.sdk.iot.device.transport.amqps.AmqpsIotHubConnection) AmqpsTransport(com.microsoft.azure.sdk.iot.device.transport.amqps.AmqpsTransport) IotHubCallbackPacket(com.microsoft.azure.sdk.iot.device.transport.IotHubCallbackPacket) IotHubOutboundPacket(com.microsoft.azure.sdk.iot.device.transport.IotHubOutboundPacket) ConcurrentHashMap(java.util.concurrent.ConcurrentHashMap) Test(org.junit.Test)

Example 89 with ConcurrentHashMap

use of java.util.concurrent.ConcurrentHashMap in project EnrichmentMapApp by BaderLab.

the class CreateDiseaseSignatureTaskParallel method startBuildDiseaseSignatureParallel.

/**
	 * Returns immediately, need to wait on the executor to join all threads.
	 */
private Map<SimilarityKey, GenesetSimilarity> startBuildDiseaseSignatureParallel(TaskMonitor tm, ExecutorService executor, Set<String> enrichmentGeneSetNames, Map<String, GeneSet> signatureGeneSets) {
    DiscreteTaskMonitor taskMonitor = discreteTaskMonitor(tm, signatureGeneSets.size());
    // Gene universe is all enrichment genes in the map
    Set<Integer> geneUniverse = map.getAllEnrichmentGenes();
    Map<SimilarityKey, GenesetSimilarity> geneSetSimilarities = new ConcurrentHashMap<>();
    for (String hubName : signatureGeneSets.keySet()) {
        GeneSet sigGeneSet = signatureGeneSets.get(hubName);
        Set<Integer> sigGenesInUniverse = Sets.intersection(sigGeneSet.getGenes(), geneUniverse);
        // Compute similarities in batches
        executor.execute(() -> {
            loop: for (String geneSetName : enrichmentGeneSetNames) {
                for (EMDataSet dataSet : dataSets) {
                    if (Thread.interrupted())
                        break loop;
                    GeneSet enrGeneSet = dataSet.getSetOfGeneSets().getGeneSetByName(geneSetName);
                    if (enrGeneSet != null) {
                        // restrict to a common gene universe
                        Set<Integer> enrGenes = Sets.intersection(enrGeneSet.getGenes(), geneUniverse);
                        Set<Integer> union = Sets.union(sigGeneSet.getGenes(), enrGenes);
                        Set<Integer> intersection = Sets.intersection(sigGenesInUniverse, enrGenes);
                        if (!intersection.isEmpty()) {
                            double coeffecient = ComputeSimilarityTaskParallel.computeSimilarityCoeffecient(map.getParams(), intersection, union, sigGeneSet.getGenes(), enrGenes);
                            GenesetSimilarity comparison = new GenesetSimilarity(hubName, geneSetName, coeffecient, INTERACTION, intersection);
                            PostAnalysisFilterType filterType = params.getRankTestParameters().getType();
                            switch(filterType) {
                                case HYPERGEOM:
                                    int hyperUniverseSize1 = getHypergeometricUniverseSize(dataSet);
                                    hypergeometric(hyperUniverseSize1, sigGenesInUniverse, enrGenes, intersection, comparison);
                                    break;
                                case MANN_WHIT_TWO_SIDED:
                                case MANN_WHIT_GREATER:
                                case MANN_WHIT_LESS:
                                    mannWhitney(intersection, comparison, dataSet);
                                default:
                                    // want mann-whit to fall through
                                    // #70 calculate hypergeometric also
                                    int hyperUniverseSize2 = map.getNumberOfGenes();
                                    hypergeometric(hyperUniverseSize2, sigGenesInUniverse, enrGenes, intersection, comparison);
                                    break;
                            }
                            SimilarityKey key = new SimilarityKey(hubName, geneSetName, INTERACTION, dataSet.getName());
                            geneSetSimilarities.put(key, comparison);
                        }
                    }
                }
            }
            taskMonitor.inc();
        });
    }
    return geneSetSimilarities;
}
Also used : DiscreteTaskMonitor(org.baderlab.csplugins.enrichmentmap.util.DiscreteTaskMonitor) GeneSet(org.baderlab.csplugins.enrichmentmap.model.GeneSet) Set(java.util.Set) EMDataSet(org.baderlab.csplugins.enrichmentmap.model.EMDataSet) PostAnalysisFilterType(org.baderlab.csplugins.enrichmentmap.model.PostAnalysisFilterType) SimilarityKey(org.baderlab.csplugins.enrichmentmap.model.SimilarityKey) EMDataSet(org.baderlab.csplugins.enrichmentmap.model.EMDataSet) GenesetSimilarity(org.baderlab.csplugins.enrichmentmap.model.GenesetSimilarity) ConcurrentHashMap(java.util.concurrent.ConcurrentHashMap) GeneSet(org.baderlab.csplugins.enrichmentmap.model.GeneSet)

Example 90 with ConcurrentHashMap

use of java.util.concurrent.ConcurrentHashMap in project azure-iot-sdk-java by Azure.

the class AmqpsTransportTest method messageSentBuffersPreviouslySentMessageIfNotSuccessfullyDelivered.

// Tests_SRS_AMQPSTRANSPORT_15_031: [If the message was not delivered successfully, it is buffered to be sent again.]
@Test
public void messageSentBuffersPreviouslySentMessageIfNotSuccessfullyDelivered() throws IOException {
    new NonStrictExpectations() {

        {
            new AmqpsIotHubConnection(mockConfig, false);
            result = mockConnection;
        }
    };
    AmqpsTransport transport = new AmqpsTransport(mockConfig, false);
    transport.open();
    Map<Integer, IotHubOutboundPacket> inProgressMessages = new ConcurrentHashMap<>();
    inProgressMessages.put(1, new IotHubOutboundPacket(new Message(), mockIotHubEventCallback, new Object()));
    inProgressMessages.put(2, new IotHubOutboundPacket(new Message(), mockIotHubEventCallback, new Object()));
    Deencapsulation.setField(transport, "inProgressMessages", inProgressMessages);
    transport.messageSent(1, false);
    new Verifications() {

        {
            new IotHubCallbackPacket(IotHubStatusCode.OK_EMPTY, (IotHubEventCallback) any, any);
            times = 0;
        }
    };
    Queue<IotHubOutboundPacket> waitingMessages = Deencapsulation.getField(transport, "waitingMessages");
    Queue<IotHubCallbackPacket> callbackList = Deencapsulation.getField(transport, "callbackList");
    Assert.assertTrue(inProgressMessages.size() == 1);
    Assert.assertTrue(waitingMessages.size() == 1);
    Assert.assertTrue(callbackList.size() == 0);
}
Also used : AmqpsTransport(com.microsoft.azure.sdk.iot.device.transport.amqps.AmqpsTransport) AmqpsMessage(com.microsoft.azure.sdk.iot.device.transport.amqps.AmqpsMessage) IotHubCallbackPacket(com.microsoft.azure.sdk.iot.device.transport.IotHubCallbackPacket) AmqpsIotHubConnection(com.microsoft.azure.sdk.iot.device.transport.amqps.AmqpsIotHubConnection) IotHubOutboundPacket(com.microsoft.azure.sdk.iot.device.transport.IotHubOutboundPacket) ConcurrentHashMap(java.util.concurrent.ConcurrentHashMap) Test(org.junit.Test)

Aggregations

ConcurrentHashMap (java.util.concurrent.ConcurrentHashMap)420 Map (java.util.Map)106 Test (org.junit.Test)102 HashMap (java.util.HashMap)75 ArrayList (java.util.ArrayList)73 CountDownLatch (java.util.concurrent.CountDownLatch)53 AtomicInteger (java.util.concurrent.atomic.AtomicInteger)49 IOException (java.io.IOException)47 List (java.util.List)38 Set (java.util.Set)36 AtomicBoolean (java.util.concurrent.atomic.AtomicBoolean)33 HashSet (java.util.HashSet)31 AtomicLong (java.util.concurrent.atomic.AtomicLong)29 ConcurrentMap (java.util.concurrent.ConcurrentMap)28 Random (java.util.Random)25 ExecutorService (java.util.concurrent.ExecutorService)23 Collection (java.util.Collection)20 UUID (java.util.UUID)20 Iterator (java.util.Iterator)19 Configuration (org.apache.hadoop.conf.Configuration)17