Java 类名:com.alibaba.alink.operator.batch.dataproc.vector.VectorImputerTrainBatchOp
Python 类名:VectorImputerTrainBatchOp
功能介绍
训练Vecotor 缺失值填充模型的组件,输出模型。
填充策略包含最大值,最小值,均值和指定数值4种。
参数说明
名称 | 中文名称 | 描述 | 类型 | 是否必须? | 取值范围 | 默认值 |
---|---|---|---|---|---|---|
selectedCol | 选中的列名 | 计算列对应的列名 | String | ✓ | 所选列类型为 [DENSE_VECTOR, SPARSE_VECTOR, STRING, VECTOR] | |
fillValue | 填充缺失值 | 自定义的填充值。当strategy为value时,读取fillValue的值 | Double | null | ||
strategy | 缺失值填充规则 | 缺失值填充的规则,支持mean,max,min或者value。选择value时,需要读取fillValue的值 | String | “MEAN”, “MIN”, “MAX”, “VALUE” | “MEAN” |
代码示例
Python 代码
from pyalink.alink import *
import pandas as pd
useLocalEnv(1)
df = pd.DataFrame([
["1:3,2:4,4:7", 1],
["1:3,2:NaN", 3],
["2:4,4:5", 4]
])
data = BatchOperator.fromDataframe(df, schemaStr="vec string, id bigint")
vecFill = VectorImputerTrainBatchOp().setSelectedCol("vec")
model = data.link(vecFill)
VectorImputerPredictBatchOp().setOutputCol("vec1").linkFrom(model, data).print()
Java 代码
import org.apache.flink.types.Row;
import com.alibaba.alink.operator.batch.BatchOperator;
import com.alibaba.alink.operator.batch.dataproc.vector.VectorImputerPredictBatchOp;
import com.alibaba.alink.operator.batch.dataproc.vector.VectorImputerTrainBatchOp;
import com.alibaba.alink.operator.batch.source.MemSourceBatchOp;
import org.junit.Test;
import java.util.Arrays;
import java.util.List;
public class VectorImputerTrainBatchOpTest {
@Test
public void testVectorImputerTrainBatchOp() throws Exception {
List <Row> df = Arrays.asList(
Row.of("1:3,2:4,4:7", 1),
Row.of("1:3,2:NaN", 3),
Row.of("2:4,4:5", 4)
);
BatchOperator <?> data = new MemSourceBatchOp(df, "vec string, id int");
BatchOperator <?> vecFill = new VectorImputerTrainBatchOp().setSelectedCol("vec");
BatchOperator <?> model = data.link(vecFill);
new VectorImputerPredictBatchOp().setOutputCol("vec1").linkFrom(model, data).print();
}
}
运行结果
| vec | id | vec1 | | —- | —- | —- |
| 1:3,2:4,4:7 | 1 | 1:3.0 2:4.0 4:7.0 |
| 1:3,2:NaN | 3 | 1:3.0 2:4.0 |
| 2:4,4:5 | 4 | 2:4.0 4:5.0 |