Java 类名:com.alibaba.alink.operator.batch.dataproc.FlattenMTableBatchOp
Python 类名:FlattenMTableBatchOp

功能介绍

该组件将 MTable 展开成 Table。

参数说明

名称 中文名称 描述 类型 是否必须? 取值范围 默认值
schemaStr Schema Schema。格式为”colname coltype[, colname2, coltype2[, …]]”,例如”f0 string, f1 bigint, f2 double” String
selectedCol 选中的列名 计算列对应的列名 String 所选列类型为 [M_TABLE]
handleInvalidMethod 处理无效值的方法 处理无效值的方法,可取 error, skip String “ERROR”, “SKIP” “ERROR”
reservedCols 算法保留列名 算法保留列 String[] null

代码示例

Python 代码

  1. import numpy as np
  2. import pandas as pd
  3. from pyalink.alink import *
  4. df_data = pd.DataFrame([
  5. ["a1", "11L", 2.2],
  6. ["a1", "12L", 2.0],
  7. ["a2", "11L", 2.0],
  8. ["a2", "12L", 2.0],
  9. ["a3", "12L", 2.0],
  10. ["a3", "13L", 2.0],
  11. ["a4", "13L", 2.0],
  12. ["a4", "14L", 2.0],
  13. ["a5", "14L", 2.0],
  14. ["a5", "15L", 2.0],
  15. ["a6", "15L", 2.0],
  16. ["a6", "16L", 2.0]
  17. ])
  18. input = BatchOperator.fromDataframe(df_data, schemaStr='id string, f0 string, f1 double')
  19. zip = GroupByBatchOp()\
  20. .setGroupByPredicate("id")\
  21. .setSelectClause("id, mtable_agg(f0, f1) as m_table_col")
  22. flatten = FlattenMTableBatchOp()\
  23. .setReservedCols(["id"])\
  24. .setSelectedCol("m_table_col")\
  25. .setSchemaStr('f0 string, f1 int')
  26. zip.linkFrom(input).link(flatten).print()

Java 代码

  1. package com.alibaba.alink.operator.batch.dataproc;
  2. import org.apache.flink.types.Row;
  3. import com.alibaba.alink.operator.batch.BatchOperator;
  4. import com.alibaba.alink.operator.batch.sql.GroupByBatchOp;
  5. import com.alibaba.alink.operator.batch.source.MemSourceBatchOp;
  6. import com.alibaba.alink.testutil.AlinkTestBase;
  7. import org.junit.Test;
  8. import java.util.ArrayList;
  9. import java.util.List;
  10. /**
  11. * Test cases for gbdt.
  12. */
  13. public class FlattenMTableTest extends AlinkTestBase {
  14. @Test
  15. public void test() throws Exception {
  16. List <Row> rows = new ArrayList <>();
  17. rows.add(Row.of("a1", "11L", 2.2));
  18. rows.add(Row.of("a1", "12L", 2.0));
  19. rows.add(Row.of("a2", "11L", 2.0));
  20. rows.add(Row.of("a2", "12L", 2.0));
  21. rows.add(Row.of("a3", "12L", 2.0));
  22. rows.add(Row.of("a3", "13L", 2.0));
  23. rows.add(Row.of("a4", "13L", 2.0));
  24. rows.add(Row.of("a4", "14L", 2.0));
  25. rows.add(Row.of("a5", "14L", 2.0));
  26. rows.add(Row.of("a5", "15L", 2.0));
  27. rows.add(Row.of("a6", "15L", 2.0));
  28. rows.add(Row.of("a6", "16L", 2.0));
  29. BatchOperator input = new MemSourceBatchOp(rows, "id string, f0 string, f1 double");
  30. GroupByBatchOp zip = new GroupByBatchOp()
  31. .setGroupByPredicate("id")
  32. .setSelectClause("id, mtable_agg(f0, f1) as m_table_col");
  33. FlattenMTableBatchOp flatten = new FlattenMTableBatchOp()
  34. .setReservedCols("id")
  35. .setSelectedCol("m_table_col")
  36. .setSchemaStr("f0 string, f1 int");
  37. zip.linkFrom(input).link(flatten).print();
  38. }
  39. }

运行结果

| id | f0 | f1 | | —- | —- | —- |

| a2 | 11L | 2 |

| a2 | 12L | 2 |

| a4 | 13L | 2 |

| a4 | 14L | 2 |

| a5 | 14L | 2 |

| a5 | 15L | 2 |

| a1 | 11L | 2 |

| a1 | 12L | 2 |

| a3 | 12L | 2 |

| a3 | 13L | 2 |

| a6 | 15L | 2 |

| a6 | 16L | 2 |