Search in sources :

Example 26 with StopWatch

use of org.apache.commons.lang.time.StopWatch in project eol-globi-data by jhpoelen.

the class TaxonInteractionIndexer method collectTaxonInteractions.

public void collectTaxonInteractions(Map<Fun.Tuple3<Long, String, Long>, Long> taxonInteractions) {
    StopWatch watchForEntireRun = new StopWatch();
    watchForEntireRun.start();
    StopWatch watchForBatch = new StopWatch();
    watchForBatch.start();
    long count = 0L;
    int batchSize = 1000;
    Index<Node> taxonIndex = graphService.index().forNodes("taxons");
    IndexHits<Node> taxa = taxonIndex.query("name", "*");
    for (Node sourceTaxon : taxa) {
        final Iterable<Relationship> classifiedAs = sourceTaxon.getRelationships(Direction.INCOMING, NodeUtil.asNeo4j(RelTypes.CLASSIFIED_AS));
        for (Relationship classifiedA : classifiedAs) {
            Node specimenNode = classifiedA.getStartNode();
            final Iterable<Relationship> interactions = specimenNode.getRelationships(Direction.OUTGOING, NodeUtil.asNeo4j(InteractType.values()));
            for (Relationship interaction : interactions) {
                final Iterable<Relationship> targetClassifications = interaction.getEndNode().getRelationships(Direction.OUTGOING, NodeUtil.asNeo4j(RelTypes.CLASSIFIED_AS));
                for (Relationship targetClassification : targetClassifications) {
                    final Node targetTaxonNode = targetClassification.getEndNode();
                    final Fun.Tuple3<Long, String, Long> interactionKey = new Fun.Tuple3<Long, String, Long>(sourceTaxon.getId(), interaction.getType().name(), targetTaxonNode.getId());
                    final Long distinctInteractions = taxonInteractions.get(interactionKey);
                    taxonInteractions.put(interactionKey, distinctInteractions == null ? 1L : (distinctInteractions + 1L));
                    count++;
                }
            }
        }
        if (count % batchSize == 0) {
            watchForBatch.stop();
            final long duration = watchForBatch.getTime();
            if (duration > 0) {
                LOG.info("walked [" + batchSize + "] interactions in " + getProgressMsg(batchSize, duration));
            }
            watchForBatch.reset();
            watchForBatch.start();
        }
    }
    taxa.close();
    watchForEntireRun.stop();
    LOG.info("walked [" + count + "] interactions in " + getProgressMsg(count, watchForEntireRun.getTime()));
}
Also used : Node(org.neo4j.graphdb.Node) SpecimenNode(org.eol.globi.domain.SpecimenNode) StopWatch(org.apache.commons.lang.time.StopWatch) Relationship(org.neo4j.graphdb.Relationship) Fun(org.mapdb.Fun)

Example 27 with StopWatch

use of org.apache.commons.lang.time.StopWatch in project Gemma by PavlidisLab.

the class ExpressionExperimentDaoImpl method populateAnalysisInformation.

/**
 * Filling 'hasDifferentialExpressionAnalysis' and 'hasCoexpressionAnalysis'
 */
private void populateAnalysisInformation(Collection<ExpressionExperimentDetailsValueObject> vos) {
    Map<Long, ExpressionExperimentDetailsValueObject> voIdMap = this.getExpressionExperimentValueObjectMap(vos);
    if (voIdMap.isEmpty()) {
        return;
    }
    StopWatch timer = new StopWatch();
    timer.start();
    // noinspection unchecked
    List<Long> withCoexpression = this.getSessionFactory().getCurrentSession().createQuery("select experimentAnalyzed.id from CoexpressionAnalysis where experimentAnalyzed.id in (:ids)").setParameterList("ids", voIdMap.keySet()).list();
    for (Long id : withCoexpression) {
        voIdMap.get(id).setHasCoexpressionAnalysis(true);
    }
    // noinspection unchecked
    List<Long> withDiffEx = this.getSessionFactory().getCurrentSession().createQuery("select experimentAnalyzed.id from DifferentialExpressionAnalysis where experimentAnalyzed.id in (:ids)").setParameterList("ids", voIdMap.keySet()).list();
    for (Long id : withDiffEx) {
        voIdMap.get(id).setHasDifferentialExpressionAnalysis(true);
    }
    if (timer.getTime() > 200) {
        AbstractDao.log.info("Populate analysis info for " + voIdMap.size() + " eevos: " + timer.getTime() + "ms");
    }
}
Also used : StopWatch(org.apache.commons.lang.time.StopWatch)

Example 28 with StopWatch

use of org.apache.commons.lang.time.StopWatch in project fastjson by alibaba.

the class TestFastJson method testSerializePerformance.

@Test
public void testSerializePerformance() throws IOException {
    Object obj = createTest();
    for (int x = 0; x < 20; ++x) {
        StopWatch stopWatch = new StopWatch();
        stopWatch.start();
        for (int i = 0; i < TIMES; ++i) {
            jsonSerialize(obj);
        }
        stopWatch.stop();
        System.out.println("JSON serialize:" + stopWatch.getTime());
        stopWatch.reset();
        stopWatch.start();
        for (int i = 0; i < TIMES; ++i) {
            javaSerialize(obj);
        }
        stopWatch.stop();
        System.out.println("JAVA serialize:" + stopWatch.getTime());
        System.out.println();
    }
}
Also used : StopWatch(org.apache.commons.lang.time.StopWatch) Test(org.junit.Test)

Example 29 with StopWatch

use of org.apache.commons.lang.time.StopWatch in project whirr by apache.

the class HadoopServiceTeraSortBenchmark method runTeraGen.

private void runTeraGen() throws IOException {
    int numTaskTrackers = controller.getCluster().getInstances().size() - 1;
    long bytesPerNode = Long.parseLong(System.getProperty("terasortBytesPerNode", "1000000000"));
    long rows = numTaskTrackers * (bytesPerNode / 100);
    StopWatch stopWatch = new StopWatch();
    TeraGen teraGen = new TeraGen();
    teraGen.setConf(controller.getJobConf());
    LOG.info("Starting TeraGen with {} tasktrackers, {} bytes per node, {} rows", new Object[] { numTaskTrackers, bytesPerNode, rows });
    stopWatch.start();
    teraGen.run(new String[] { "" + rows, "input" });
    stopWatch.stop();
    LOG.info("TeraGen took {} ms", stopWatch.getTime());
}
Also used : TeraGen(org.apache.hadoop.examples.terasort.TeraGen) StopWatch(org.apache.commons.lang.time.StopWatch)

Example 30 with StopWatch

use of org.apache.commons.lang.time.StopWatch in project whirr by apache.

the class HadoopServiceTeraSortBenchmark method runTeraValidate.

private void runTeraValidate() throws Exception {
    StopWatch stopWatch = new StopWatch();
    TeraValidate teraValidate = new TeraValidate();
    teraValidate.setConf(controller.getJobConf());
    LOG.info("Starting TeraValidate");
    stopWatch.start();
    teraValidate.run(new String[] { "output", "report" });
    stopWatch.stop();
    LOG.info("TeraValidate took {} ms", stopWatch.getTime());
}
Also used : TeraValidate(org.apache.hadoop.examples.terasort.TeraValidate) StopWatch(org.apache.commons.lang.time.StopWatch)

Aggregations

StopWatch (org.apache.commons.lang.time.StopWatch)48 IOException (java.io.IOException)13 ArrayList (java.util.ArrayList)5 Test (org.junit.Test)5 File (java.io.File)4 Path (java.nio.file.Path)4 HashMap (java.util.HashMap)4 FileNotFoundException (java.io.FileNotFoundException)3 ObjectOutputStream (java.io.ObjectOutputStream)3 NoSuchAlgorithmException (java.security.NoSuchAlgorithmException)3 Map (java.util.Map)3 SchemaHolder (org.motechproject.mds.dto.SchemaHolder)3 YarnApplicationReport (com.continuuity.weave.internal.yarn.YarnApplicationReport)2 BufferedInputStream (java.io.BufferedInputStream)2 BufferedOutputStream (java.io.BufferedOutputStream)2 FileInputStream (java.io.FileInputStream)2 FileOutputStream (java.io.FileOutputStream)2 ObjectInputStream (java.io.ObjectInputStream)2 NoSuchFileException (java.nio.file.NoSuchFileException)2 HashSet (java.util.HashSet)2