Java 类名：com.alibaba.alink.operator.batch.dataproc.StringIndexerTrainBatchOp
Python 类名：StringIndexerTrainBatchOp

功能介绍

StringIndexer训练组件的作用是训练一个模型用于将单列字符串映射为整数。
如将一列映射为整数，需指定 setSelectedCol 设定。
同时，该组件支持输入多列，生成一个映射词典，通过 setSelectedCols 设定其他需要补充的列名。
特征的排列顺序支持 random，frequency_asc，frequency_desc，alphabet_asc，alphabet_desc 五种排序方法。
注意：输入多列时，所有列必须为相同格式。

参数说明

名称 中文名称 描述 类型 是否必须？ 取值范围 默认值
selectedCol 选中的列名 计算列对应的列名 String ✓ 所选列类型为 [INTEGER, LONG, STRING]
modelName 模型名字 模型名字 String
selectedCols 选中的列名数组 计算列对应的列名列表 String[] 所选列类型为 [INTEGER, LONG, STRING] null
stringOrderType Token排序方法 Token排序方法 String “RANDOM”, “FREQUENCY_ASC”, “FREQUENCY_DESC”, “ALPHABET_ASC”, “ALPHABET_DESC” “RANDOM”

代码示例

Python 代码

  1. from pyalink.alink import *
  3. import pandas as pd
  5. useLocalEnv(1)
  7. df = pd.DataFrame([
  8.     ["football", "apple"],
  9.     ["football", "apple"],
  10.     ["football", "apple"],
  11.     ["basketball", "apple"],
  12.     ["basketball", "apple"],
  13.     ["tennis", "pair"],
  14.     ["tennis", "pair"],
  15.     ["pingpang", "banana"],
  16.     ["pingpang", "banana"],
  17.     ["baseball", "banana"]
  18. ])
  21. data = BatchOperator.fromDataframe(df, schemaStr='f0 string,f1 string')
  23. stringindexer = StringIndexerTrainBatchOp() \
  24.     .setSelectedCol("f0") \
  25.     .setSelectedCols(["f1"]) \
  26.     .setStringOrderType("alphabet_asc")
  28. model = stringindexer.linkFrom(data)
  29. model.print()

Java 代码

  1. import org.apache.flink.types.Row;
  3. import com.alibaba.alink.operator.batch.BatchOperator;
  4. import com.alibaba.alink.operator.batch.dataproc.StringIndexerTrainBatchOp;
  5. import com.alibaba.alink.operator.batch.source.MemSourceBatchOp;
  6. import org.junit.Test;
  8. import java.util.Arrays;
  9. import java.util.List;
  11. public class StringIndexerTrainBatchOpTest {
  12.     @Test
  13.     public void testAlphabetAsc() throws Exception {
  14.         List <Row> df = Arrays.asList(
  15.             Row.of("football", "apple"),
  16.             Row.of("football", "apple"),
  17.             Row.of("football", "apple"),
  18.             Row.of("basketball", "apple"),
  19.             Row.of("basketball", "apple"),
  20.             Row.of("tennis", "pair"),
  21.             Row.of("tennis", "pair"),
  22.             Row.of("pingpang", "banana"),
  23.             Row.of("pingpang", "banana"),
  24.             Row.of("baseball", "banana")
  25.         );
  26.         BatchOperator <?> data = new MemSourceBatchOp(df, "f0 string,f1 string");
  27.         BatchOperator <?> stringindexer = new StringIndexerTrainBatchOp()
  28.             .setSelectedCol("f0")
  29.             .setSelectedCols("f1")
  30.             .setStringOrderType("alphabet_asc");
  31.         BatchOperator model = stringindexer.linkFrom(data);
  32.         model.print();
  33.     }
  34. }

运行结果

模型表：

| token | token_index | | —- | —- |

| pingpang | 6 |

| banana | 1 |

| baseball | 2 |

| basketball | 3 |

| pair | 5 |

| apple | 0 |

| football | 4 |

| tennis | 7 |