Search in sources :

Example 1 with BaseEventStats

use of org.deeplearning4j.spark.stats.BaseEventStats in project deeplearning4j by deeplearning4j.

the class StatsCalculationHelper method logNextDataSetAfter.

public void logNextDataSetAfter(int numExamples) {
    long now = timeSource.currentTimeMillis();
    long duration = now - lastDataSetBefore;
    dataSetGetTimes.add(new BaseEventStats(lastDataSetBefore, duration));
    totalExampleCount += numExamples;
}
Also used : BaseEventStats(org.deeplearning4j.spark.stats.BaseEventStats)

Example 2 with BaseEventStats

use of org.deeplearning4j.spark.stats.BaseEventStats in project deeplearning4j by deeplearning4j.

the class StatsCalculationHelper method logProcessMinibatchAfter.

public void logProcessMinibatchAfter() {
    long now = timeSource.currentTimeMillis();
    long duration = now - lastProcessBefore;
    processMiniBatchTimes.add(new BaseEventStats(lastProcessBefore, duration));
}
Also used : BaseEventStats(org.deeplearning4j.spark.stats.BaseEventStats)

Example 3 with BaseEventStats

use of org.deeplearning4j.spark.stats.BaseEventStats in project deeplearning4j by deeplearning4j.

the class StatsCalculationHelper method build.

public CommonSparkTrainingStats build(SparkTrainingStats masterSpecificStats) {
    List<EventStats> totalTime = new ArrayList<>();
    totalTime.add(new ExampleCountEventStats(methodStartTime, returnTime - methodStartTime, totalExampleCount));
    List<EventStats> initTime = new ArrayList<>();
    initTime.add(new BaseEventStats(initalModelBefore, initialModelAfter - initalModelBefore));
    return new CommonSparkTrainingStats.Builder().trainingMasterSpecificStats(masterSpecificStats).workerFlatMapTotalTimeMs(totalTime).workerFlatMapGetInitialModelTimeMs(initTime).workerFlatMapDataSetGetTimesMs(dataSetGetTimes).workerFlatMapProcessMiniBatchTimesMs(processMiniBatchTimes).build();
}
Also used : BaseEventStats(org.deeplearning4j.spark.stats.BaseEventStats) EventStats(org.deeplearning4j.spark.stats.EventStats) ExampleCountEventStats(org.deeplearning4j.spark.stats.ExampleCountEventStats) ExampleCountEventStats(org.deeplearning4j.spark.stats.ExampleCountEventStats) BaseEventStats(org.deeplearning4j.spark.stats.BaseEventStats) ArrayList(java.util.ArrayList)

Aggregations

BaseEventStats (org.deeplearning4j.spark.stats.BaseEventStats)3 ArrayList (java.util.ArrayList)1 EventStats (org.deeplearning4j.spark.stats.EventStats)1 ExampleCountEventStats (org.deeplearning4j.spark.stats.ExampleCountEventStats)1