Search in sources :

Example 16 with DoubleTensor

use of com.alibaba.alink.common.linalg.tensor.DoubleTensor in project Alink by alibaba.

the class ToTensorMapperTest method test.

@Test
public void test() throws Exception {
    final Mapper mapper = new ToTensorMapper(new TableSchema(new String[] { "vec" }, new TypeInformation<?>[] { VectorTypes.DENSE_VECTOR }), new Params().set(ToTensorParams.SELECTED_COL, "vec"));
    final DoubleTensor tensor = DoubleTensor.of(TensorUtil.getTensor("FLOAT#6#0.0 0.1 1.0 1.1 2.0 2.1 "));
    final Tensor<?> result = (Tensor<?>) mapper.map(Row.of(tensor.toVector())).getField(0);
    Assert.assertEquals(tensor, result);
}
Also used : Mapper(com.alibaba.alink.common.mapper.Mapper) DoubleTensor(com.alibaba.alink.common.linalg.tensor.DoubleTensor) DoubleTensor(com.alibaba.alink.common.linalg.tensor.DoubleTensor) Tensor(com.alibaba.alink.common.linalg.tensor.Tensor) StringTensor(com.alibaba.alink.common.linalg.tensor.StringTensor) FloatTensor(com.alibaba.alink.common.linalg.tensor.FloatTensor) TableSchema(org.apache.flink.table.api.TableSchema) ToTensorParams(com.alibaba.alink.params.dataproc.ToTensorParams) Params(org.apache.flink.ml.api.misc.param.Params) TypeInformation(org.apache.flink.api.common.typeinfo.TypeInformation) Test(org.junit.Test)

Example 17 with DoubleTensor

use of com.alibaba.alink.common.linalg.tensor.DoubleTensor in project Alink by alibaba.

the class ToTensorMapperTest method testReshape.

@Test
public void testReshape() throws Exception {
    final Mapper mapper = new ToTensorMapper(new TableSchema(new String[] { "vec" }, new TypeInformation<?>[] { VectorTypes.DENSE_VECTOR }), new Params().set(ToTensorParams.SELECTED_COL, "vec").set(ToTensorParams.TENSOR_SHAPE, new Long[] { 2L, 3L }));
    final DoubleTensor tensor = DoubleTensor.of(TensorUtil.getTensor("FLOAT#6#0.0 0.1 1.0 1.1 2.0 2.1 "));
    final DoubleTensor expect = tensor.reshape(new Shape(2L, 3L));
    final Tensor<?> result = (Tensor<?>) mapper.map(Row.of(tensor.toVector())).getField(0);
    Assert.assertEquals(expect, result);
}
Also used : Mapper(com.alibaba.alink.common.mapper.Mapper) DoubleTensor(com.alibaba.alink.common.linalg.tensor.DoubleTensor) Shape(com.alibaba.alink.common.linalg.tensor.Shape) DoubleTensor(com.alibaba.alink.common.linalg.tensor.DoubleTensor) Tensor(com.alibaba.alink.common.linalg.tensor.Tensor) StringTensor(com.alibaba.alink.common.linalg.tensor.StringTensor) FloatTensor(com.alibaba.alink.common.linalg.tensor.FloatTensor) TableSchema(org.apache.flink.table.api.TableSchema) ToTensorParams(com.alibaba.alink.params.dataproc.ToTensorParams) Params(org.apache.flink.ml.api.misc.param.Params) TypeInformation(org.apache.flink.api.common.typeinfo.TypeInformation) Test(org.junit.Test)

Example 18 with DoubleTensor

use of com.alibaba.alink.common.linalg.tensor.DoubleTensor in project Alink by alibaba.

the class ToTensorMapperTest method testFloatType.

@Test
public void testFloatType() throws Exception {
    final Mapper mapper = new ToTensorMapper(new TableSchema(new String[] { "vec" }, new TypeInformation<?>[] { VectorTypes.DENSE_VECTOR }), new Params().set(ToTensorParams.SELECTED_COL, "vec").set(ToTensorParams.TENSOR_SHAPE, new Long[] { 2L, 3L }).set(ToTensorParams.TENSOR_DATA_TYPE, DataType.FLOAT));
    Assert.assertEquals(TensorTypes.FLOAT_TENSOR, mapper.getOutputSchema().getFieldTypes()[0]);
    final DoubleTensor tensor = DoubleTensor.of(TensorUtil.getTensor("FLOAT#6#0.0 0.1 1.0 1.1 2.0 2.1 "));
    final FloatTensor expect = FloatTensor.of(tensor.reshape(new Shape(2L, 3L)));
    final Tensor<?> result = (Tensor<?>) mapper.map(Row.of(tensor.toVector())).getField(0);
    Assert.assertEquals(expect, result);
}
Also used : Mapper(com.alibaba.alink.common.mapper.Mapper) DoubleTensor(com.alibaba.alink.common.linalg.tensor.DoubleTensor) Shape(com.alibaba.alink.common.linalg.tensor.Shape) DoubleTensor(com.alibaba.alink.common.linalg.tensor.DoubleTensor) Tensor(com.alibaba.alink.common.linalg.tensor.Tensor) StringTensor(com.alibaba.alink.common.linalg.tensor.StringTensor) FloatTensor(com.alibaba.alink.common.linalg.tensor.FloatTensor) TableSchema(org.apache.flink.table.api.TableSchema) ToTensorParams(com.alibaba.alink.params.dataproc.ToTensorParams) Params(org.apache.flink.ml.api.misc.param.Params) FloatTensor(com.alibaba.alink.common.linalg.tensor.FloatTensor) TypeInformation(org.apache.flink.api.common.typeinfo.TypeInformation) Test(org.junit.Test)

Example 19 with DoubleTensor

use of com.alibaba.alink.common.linalg.tensor.DoubleTensor in project Alink by alibaba.

the class VectorToTensorMapperTest method test.

@Test
public void test() throws Exception {
    final Mapper mapper = new VectorToTensorMapper(new TableSchema(new String[] { "vec" }, new TypeInformation<?>[] { VectorTypes.DENSE_VECTOR }), new Params().set(VectorToTensorParams.SELECTED_COL, "vec"));
    final DoubleTensor tensor = DoubleTensor.of(TensorUtil.getTensor("FLOAT#6#0.0 0.1 1.0 1.1 2.0 2.1 "));
    final Tensor<?> result = (Tensor<?>) mapper.map(Row.of(tensor.toVector())).getField(0);
    Assert.assertEquals(tensor, result);
}
Also used : Mapper(com.alibaba.alink.common.mapper.Mapper) DoubleTensor(com.alibaba.alink.common.linalg.tensor.DoubleTensor) DoubleTensor(com.alibaba.alink.common.linalg.tensor.DoubleTensor) Tensor(com.alibaba.alink.common.linalg.tensor.Tensor) FloatTensor(com.alibaba.alink.common.linalg.tensor.FloatTensor) TableSchema(org.apache.flink.table.api.TableSchema) VectorToTensorParams(com.alibaba.alink.params.dataproc.VectorToTensorParams) Params(org.apache.flink.ml.api.misc.param.Params) TypeInformation(org.apache.flink.api.common.typeinfo.TypeInformation) Test(org.junit.Test)

Example 20 with DoubleTensor

use of com.alibaba.alink.common.linalg.tensor.DoubleTensor in project Alink by alibaba.

the class TensorTFTensorConversionTest method testDoubleTensor.

@Test
public void testDoubleTensor() {
    Random random = new Random(2021);
    double[][] arr = new double[3][4];
    for (int i = 0; i < 3; i += 1) {
        for (int j = 0; j < 4; j += 1) {
            arr[i][j] = random.nextDouble();
        }
    }
    DoubleTensor data = new DoubleTensor(arr);
    Tensor<TFloat64> std = TFloat64.tensorOf(StdArrays.ndCopyOf(arr));
    TensorInfo tensorInfo = createTensorInfo(DataType.DT_DOUBLE, new long[] { 3, 4 });
    // noinspection unchecked
    Tensor<TFloat64> tensor = (Tensor<TFloat64>) TensorTFTensorConversionImpl.getInstance().parseTensor(data, tensorInfo);
    Assert.assertArrayEquals(std.shape().asArray(), tensor.shape().asArray());
    Assert.assertArrayEquals(StdArrays.array2dCopyOf(std.data()), StdArrays.array2dCopyOf(tensor.data()));
    DoubleTensor encoded = (DoubleTensor) TensorTFTensorConversionImpl.getInstance().encodeTensor(tensor);
    for (int i = 0; i < 3; i += 1) {
        for (int j = 0; j < 4; j += 1) {
            Assert.assertEquals(encoded.getDouble(i, j), data.getDouble(i, j), 1e-6);
        }
    }
}
Also used : DoubleTensor(com.alibaba.alink.common.linalg.tensor.DoubleTensor) DoubleTensor(com.alibaba.alink.common.linalg.tensor.DoubleTensor) BoolTensor(com.alibaba.alink.common.linalg.tensor.BoolTensor) LongTensor(com.alibaba.alink.common.linalg.tensor.LongTensor) StringTensor(com.alibaba.alink.common.linalg.tensor.StringTensor) FloatTensor(com.alibaba.alink.common.linalg.tensor.FloatTensor) Tensor(org.tensorflow.Tensor) IntTensor(com.alibaba.alink.common.linalg.tensor.IntTensor) Random(java.util.Random) TFloat64(org.tensorflow.types.TFloat64) TensorInfo(org.tensorflow.proto.framework.TensorInfo) Test(org.junit.Test)

Aggregations

DoubleTensor (com.alibaba.alink.common.linalg.tensor.DoubleTensor)21 Test (org.junit.Test)18 FloatTensor (com.alibaba.alink.common.linalg.tensor.FloatTensor)15 StringTensor (com.alibaba.alink.common.linalg.tensor.StringTensor)10 Mapper (com.alibaba.alink.common.mapper.Mapper)10 TypeInformation (org.apache.flink.api.common.typeinfo.TypeInformation)10 Params (org.apache.flink.ml.api.misc.param.Params)10 TableSchema (org.apache.flink.table.api.TableSchema)10 Tensor (com.alibaba.alink.common.linalg.tensor.Tensor)9 Shape (com.alibaba.alink.common.linalg.tensor.Shape)8 ToTensorParams (com.alibaba.alink.params.dataproc.ToTensorParams)7 Row (org.apache.flink.types.Row)5 BoolTensor (com.alibaba.alink.common.linalg.tensor.BoolTensor)4 IntTensor (com.alibaba.alink.common.linalg.tensor.IntTensor)4 LongTensor (com.alibaba.alink.common.linalg.tensor.LongTensor)4 MemSourceBatchOp (com.alibaba.alink.operator.batch.source.MemSourceBatchOp)4 Tensor (org.tensorflow.Tensor)4 TFloat64 (org.tensorflow.types.TFloat64)4 VectorToTensorParams (com.alibaba.alink.params.dataproc.VectorToTensorParams)3 List (java.util.List)3