Search in sources :

Example 6 with EventStats

use of org.deeplearning4j.spark.stats.EventStats in project deeplearning4j by deeplearning4j.

the class TestTrainingStatsCollection method assertExpectedNumberMachineIdsJvmIdsThreadIds.

private static void assertExpectedNumberMachineIdsJvmIdsThreadIds(List<EventStats> events, int expNMachineIDs, int expNumJvmIds, int expNumThreadIds) {
    Set<String> machineIDs = new HashSet<>();
    Set<String> jvmIDs = new HashSet<>();
    Set<Long> threadIDs = new HashSet<>();
    for (EventStats e : events) {
        machineIDs.add(e.getMachineID());
        jvmIDs.add(e.getJvmID());
        threadIDs.add(e.getThreadID());
    }
    assertTrue(machineIDs.size() == expNMachineIDs);
    assertTrue(jvmIDs.size() == expNumJvmIds);
    assertTrue(threadIDs.size() == expNumThreadIds);
}
Also used : EventStats(org.deeplearning4j.spark.stats.EventStats)

Aggregations

EventStats (org.deeplearning4j.spark.stats.EventStats)6 MultiLayerConfiguration (org.deeplearning4j.nn.conf.MultiLayerConfiguration)3 SparkTrainingStats (org.deeplearning4j.spark.api.stats.SparkTrainingStats)3 SparkDl4jMultiLayer (org.deeplearning4j.spark.impl.multilayer.SparkDl4jMultiLayer)3 ExampleCountEventStats (org.deeplearning4j.spark.stats.ExampleCountEventStats)3 Test (org.junit.Test)3 DataSet (org.nd4j.linalg.dataset.DataSet)3 LabeledPoint (org.apache.spark.mllib.regression.LabeledPoint)2 IrisDataSetIterator (org.deeplearning4j.datasets.iterator.impl.IrisDataSetIterator)2 MnistDataSetIterator (org.deeplearning4j.datasets.iterator.impl.MnistDataSetIterator)2 NeuralNetConfiguration (org.deeplearning4j.nn.conf.NeuralNetConfiguration)2 DenseLayer (org.deeplearning4j.nn.conf.layers.DenseLayer)2 BaseSparkTest (org.deeplearning4j.spark.BaseSparkTest)2 INDArray (org.nd4j.linalg.api.ndarray.INDArray)2 MultiDataSet (org.nd4j.linalg.dataset.MultiDataSet)2 DataSetIterator (org.nd4j.linalg.dataset.api.iterator.DataSetIterator)2 ByteArrayOutputStream (java.io.ByteArrayOutputStream)1 File (java.io.File)1 Field (java.lang.reflect.Field)1 Path (java.nio.file.Path)1