Search in sources :

Example 16 with Pipeline

use of com.alibaba.alink.pipeline.Pipeline in project Alink by alibaba.

the class LogisticRegressionMixVecTest method batchMixVecTest13.

@Test
public void batchMixVecTest13() {
    BatchOperator<?> trainData = (BatchOperator<?>) getData();
    Pipeline pipeline = new Pipeline().add(new VectorAssembler().setSelectedCols(new String[] { "svec", "vec", "f0", "f1", "f2", "f3" }).setOutputCol("allvec")).add(new LogisticRegression().setVectorCol("allvec").setWithIntercept(false).setStandardization(false).setLabelCol("labels").setReservedCols(new String[] { "labels" }).setPredictionCol("pred"));
    PipelineModel model = pipeline.fit(trainData);
    model.transform(trainData).collect();
}
Also used : VectorAssembler(com.alibaba.alink.pipeline.dataproc.vector.VectorAssembler) LogisticRegression(com.alibaba.alink.pipeline.classification.LogisticRegression) BatchOperator(com.alibaba.alink.operator.batch.BatchOperator) Pipeline(com.alibaba.alink.pipeline.Pipeline) PipelineModel(com.alibaba.alink.pipeline.PipelineModel) Test(org.junit.Test)

Example 17 with Pipeline

use of com.alibaba.alink.pipeline.Pipeline in project Alink by alibaba.

the class LogisticRegressionMixVecTest method batchMixVecTest4.

@Test
public void batchMixVecTest4() {
    BatchOperator<?> trainData = (BatchOperator<?>) getData();
    Pipeline pipeline = new Pipeline().add(new LogisticRegression().setVectorCol("vec").setWithIntercept(true).setStandardization(false).setLabelCol("labels").setPredictionCol("pred"));
    PipelineModel model = pipeline.fit(trainData);
    model.transform(trainData).collect();
}
Also used : LogisticRegression(com.alibaba.alink.pipeline.classification.LogisticRegression) BatchOperator(com.alibaba.alink.operator.batch.BatchOperator) Pipeline(com.alibaba.alink.pipeline.Pipeline) PipelineModel(com.alibaba.alink.pipeline.PipelineModel) Test(org.junit.Test)

Example 18 with Pipeline

use of com.alibaba.alink.pipeline.Pipeline in project Alink by alibaba.

the class LogisticRegressionMixVecTest method batchMixVecTest11.

@Test
public void batchMixVecTest11() {
    BatchOperator<?> trainData = (BatchOperator<?>) getData();
    Pipeline pipeline = new Pipeline().add(new VectorAssembler().setSelectedCols(new String[] { "svec", "vec", "f0", "f1", "f2", "f3" }).setOutputCol("allvec")).add(new LogisticRegression().setVectorCol("allvec").setWithIntercept(true).setReservedCols(new String[] { "labels" }).setStandardization(false).setLabelCol("labels").setPredictionCol("pred"));
    PipelineModel model = pipeline.fit(trainData);
    model.transform(trainData).collect();
}
Also used : VectorAssembler(com.alibaba.alink.pipeline.dataproc.vector.VectorAssembler) LogisticRegression(com.alibaba.alink.pipeline.classification.LogisticRegression) BatchOperator(com.alibaba.alink.operator.batch.BatchOperator) Pipeline(com.alibaba.alink.pipeline.Pipeline) PipelineModel(com.alibaba.alink.pipeline.PipelineModel) Test(org.junit.Test)

Example 19 with Pipeline

use of com.alibaba.alink.pipeline.Pipeline in project Alink by alibaba.

the class LogisticRegressionMixVecTest method batchMixVecTest7.

@Test
public void batchMixVecTest7() {
    BatchOperator<?> trainData = (BatchOperator<?>) getData();
    Pipeline pipeline = new Pipeline().add(new LogisticRegression().setVectorCol("svec").setWithIntercept(false).setStandardization(true).setLabelCol("labels").setPredictionCol("pred"));
    PipelineModel model = pipeline.fit(trainData);
    model.transform(trainData).collect();
}
Also used : LogisticRegression(com.alibaba.alink.pipeline.classification.LogisticRegression) BatchOperator(com.alibaba.alink.operator.batch.BatchOperator) Pipeline(com.alibaba.alink.pipeline.Pipeline) PipelineModel(com.alibaba.alink.pipeline.PipelineModel) Test(org.junit.Test)

Example 20 with Pipeline

use of com.alibaba.alink.pipeline.Pipeline in project Alink by alibaba.

the class IsotonicRegressionTest method testIsotonicReg.

@Test
public void testIsotonicReg() throws Exception {
    Table data = MLEnvironmentFactory.getDefault().createBatchTable(rows, new String[] { "id", "feature", "label" });
    Table dataStream = MLEnvironmentFactory.getDefault().createStreamTable(rows, new String[] { "id", "feature", "label" });
    IsotonicRegression op = new IsotonicRegression().setFeatureCol("feature").setLabelCol("label").setPredictionCol("result");
    PipelineModel model = new Pipeline().add(op).fit(data);
    BatchOperator<?> res = model.transform(new TableSourceBatchOp(data));
    List<Row> list = res.select(new String[] { "id", "result" }).collect();
    double[] actual = new double[] { 0.66, 0.75, 0.75, 0.75, 0.5, 0.5, 0.75, 0.66, 0.66, 0.75, 0.5, 0.5, 0.5, 0.5, 0.75 };
    for (int i = 0; i < actual.length; i++) {
        Assert.assertEquals((Double) list.get(i).getField(1), actual[(int) list.get(i).getField(0)], 0.01);
    }
// StreamOperator<?> resStream = model.transform(new TableSourceStreamOp(dataStream));
// resStream.print();
// MLEnvironmentFactory.getDefault().getStreamExecutionEnvironment().execute();
}
Also used : Table(org.apache.flink.table.api.Table) Row(org.apache.flink.types.Row) TableSourceBatchOp(com.alibaba.alink.operator.batch.source.TableSourceBatchOp) PipelineModel(com.alibaba.alink.pipeline.PipelineModel) Pipeline(com.alibaba.alink.pipeline.Pipeline) Test(org.junit.Test)

Aggregations

Pipeline (com.alibaba.alink.pipeline.Pipeline)63 Test (org.junit.Test)38 PipelineModel (com.alibaba.alink.pipeline.PipelineModel)34 LogisticRegression (com.alibaba.alink.pipeline.classification.LogisticRegression)20 Row (org.apache.flink.types.Row)18 BatchOperator (com.alibaba.alink.operator.batch.BatchOperator)16 MemSourceBatchOp (com.alibaba.alink.operator.batch.source.MemSourceBatchOp)16 VectorAssembler (com.alibaba.alink.pipeline.dataproc.vector.VectorAssembler)11 AkSourceBatchOp (com.alibaba.alink.operator.batch.source.AkSourceBatchOp)10 CollectSinkStreamOp (com.alibaba.alink.operator.stream.sink.CollectSinkStreamOp)9 EvalBinaryClassBatchOp (com.alibaba.alink.operator.batch.evaluation.EvalBinaryClassBatchOp)8 MemSourceStreamOp (com.alibaba.alink.operator.stream.source.MemSourceStreamOp)7 File (java.io.File)5 ArrayList (java.util.ArrayList)5 EvalMultiClassBatchOp (com.alibaba.alink.operator.batch.evaluation.EvalMultiClassBatchOp)4 StandardScaler (com.alibaba.alink.pipeline.dataproc.StandardScaler)4 Stopwatch (com.alibaba.alink.common.utils.Stopwatch)3 CsvSourceBatchOp (com.alibaba.alink.operator.batch.source.CsvSourceBatchOp)3 KMeans (com.alibaba.alink.pipeline.clustering.KMeans)3 VectorToTensor (com.alibaba.alink.pipeline.dataproc.VectorToTensor)3