use of org.deeplearning4j.spark.stats.BaseEventStats in project deeplearning4j by deeplearning4j.
the class StatsCalculationHelper method logNextDataSetAfter.
public void logNextDataSetAfter(int numExamples) {
long now = timeSource.currentTimeMillis();
long duration = now - lastDataSetBefore;
dataSetGetTimes.add(new BaseEventStats(lastDataSetBefore, duration));
totalExampleCount += numExamples;
}
use of org.deeplearning4j.spark.stats.BaseEventStats in project deeplearning4j by deeplearning4j.
the class StatsCalculationHelper method logProcessMinibatchAfter.
public void logProcessMinibatchAfter() {
long now = timeSource.currentTimeMillis();
long duration = now - lastProcessBefore;
processMiniBatchTimes.add(new BaseEventStats(lastProcessBefore, duration));
}
use of org.deeplearning4j.spark.stats.BaseEventStats in project deeplearning4j by deeplearning4j.
the class StatsCalculationHelper method build.
public CommonSparkTrainingStats build(SparkTrainingStats masterSpecificStats) {
List<EventStats> totalTime = new ArrayList<>();
totalTime.add(new ExampleCountEventStats(methodStartTime, returnTime - methodStartTime, totalExampleCount));
List<EventStats> initTime = new ArrayList<>();
initTime.add(new BaseEventStats(initalModelBefore, initialModelAfter - initalModelBefore));
return new CommonSparkTrainingStats.Builder().trainingMasterSpecificStats(masterSpecificStats).workerFlatMapTotalTimeMs(totalTime).workerFlatMapGetInitialModelTimeMs(initTime).workerFlatMapDataSetGetTimesMs(dataSetGetTimes).workerFlatMapProcessMiniBatchTimesMs(processMiniBatchTimes).build();
}
Aggregations