Search in sources :

Example 1 with VarSelMasterResult

use of ml.shifu.shifu.core.dvarsel.VarSelMasterResult in project shifu by ShifuML.

the class WrapperWorkerConductorTest method testWrapperConductor.

@Test
public void testWrapperConductor() throws IOException {
    ModelConfig modelConfig = CommonUtils.loadModelConfig("src/test/resources/example/cancer-judgement/ModelStore/ModelSet1/ModelConfig.json", RawSourceData.SourceType.LOCAL);
    List<ColumnConfig> columnConfigList = CommonUtils.loadColumnConfigList("src/test/resources/example/cancer-judgement/ModelStore/ModelSet1/ColumnConfig.json", RawSourceData.SourceType.LOCAL);
    WrapperWorkerConductor wrapper = new WrapperWorkerConductor(modelConfig, columnConfigList);
    TrainingDataSet trainingDataSet = genTrainingDataSet(modelConfig, columnConfigList);
    wrapper.retainData(trainingDataSet);
    List<Integer> columnIdList = new ArrayList<Integer>();
    for (int i = 2; i < 30; i++) {
        columnIdList.add(i);
    }
    List<CandidateSeed> seedList = new ArrayList<CandidateSeed>();
    for (int i = 0; i < 10; i++) {
        seedList.add(new CandidateSeed(0, columnIdList.subList(i + 1, i + 7)));
    }
    wrapper.consumeMasterResult(new VarSelMasterResult(seedList));
    VarSelWorkerResult workerResult = wrapper.generateVarSelResult();
    Assert.assertNotNull(workerResult);
    Assert.assertTrue(workerResult.getSeedPerfList().size() > 0);
}
Also used : ColumnConfig(ml.shifu.shifu.container.obj.ColumnConfig) ArrayList(java.util.ArrayList) VarSelWorkerResult(ml.shifu.shifu.core.dvarsel.VarSelWorkerResult) CandidateSeed(ml.shifu.shifu.core.dvarsel.CandidateSeed) ModelConfig(ml.shifu.shifu.container.obj.ModelConfig) VarSelMasterResult(ml.shifu.shifu.core.dvarsel.VarSelMasterResult) TrainingDataSet(ml.shifu.shifu.core.dvarsel.dataset.TrainingDataSet) Test(org.testng.annotations.Test)

Example 2 with VarSelMasterResult

use of ml.shifu.shifu.core.dvarsel.VarSelMasterResult in project shifu by ShifuML.

the class VotedVarSelTest method testLrApp.

// @Test
public void testLrApp() throws IOException {
    Properties props = new Properties();
    props.setProperty(GuaguaConstants.MASTER_COMPUTABLE_CLASS, VarSelWorker.class.getName());
    props.setProperty(GuaguaConstants.WORKER_COMPUTABLE_CLASS, VarSelMaster.class.getName());
    props.setProperty(GuaguaConstants.GUAGUA_ITERATION_COUNT, "20");
    props.setProperty(GuaguaConstants.GUAGUA_MASTER_RESULT_CLASS, VarSelMasterResult.class.getName());
    props.setProperty(GuaguaConstants.GUAGUA_WORKER_RESULT_CLASS, VarSelWorkerResult.class.getName());
    props.setProperty(GuaguaConstants.GUAGUA_INPUT_DIR, getClass().getResource("/example/cancer-judgement/DataStore/DataSet1/part-00").toString());
    GuaguaUnitDriver<VarSelMasterResult, VarSelWorkerResult> driver = new GuaguaMRUnitDriver<VarSelMasterResult, VarSelWorkerResult>(props);
    driver.run();
}
Also used : GuaguaMRUnitDriver(ml.shifu.guagua.hadoop.GuaguaMRUnitDriver) VarSelMaster(ml.shifu.shifu.core.dvarsel.VarSelMaster) VarSelMasterResult(ml.shifu.shifu.core.dvarsel.VarSelMasterResult) VarSelWorker(ml.shifu.shifu.core.dvarsel.VarSelWorker) VarSelWorkerResult(ml.shifu.shifu.core.dvarsel.VarSelWorkerResult) Properties(java.util.Properties)

Aggregations

VarSelMasterResult (ml.shifu.shifu.core.dvarsel.VarSelMasterResult)2 VarSelWorkerResult (ml.shifu.shifu.core.dvarsel.VarSelWorkerResult)2 ArrayList (java.util.ArrayList)1 Properties (java.util.Properties)1 GuaguaMRUnitDriver (ml.shifu.guagua.hadoop.GuaguaMRUnitDriver)1 ColumnConfig (ml.shifu.shifu.container.obj.ColumnConfig)1 ModelConfig (ml.shifu.shifu.container.obj.ModelConfig)1 CandidateSeed (ml.shifu.shifu.core.dvarsel.CandidateSeed)1 VarSelMaster (ml.shifu.shifu.core.dvarsel.VarSelMaster)1 VarSelWorker (ml.shifu.shifu.core.dvarsel.VarSelWorker)1 TrainingDataSet (ml.shifu.shifu.core.dvarsel.dataset.TrainingDataSet)1 Test (org.testng.annotations.Test)1