Java 类名:com.alibaba.alink.operator.batch.sql.IntersectBatchOp
Python 类名:IntersectBatchOp

功能介绍

对批式数据进行sql的INTERSECT操作。(一个数据集交另一个数据集,去重)

参数说明

| 名称 | 中文名称 | 描述 | 类型 | 是否必须? | 取值范围 | 默认值 | | —- | —- | —- | —- | —- | —- | —- |

代码示例

Python 代码

  1. from pyalink.alink import *
  2. import pandas as pd
  3. useLocalEnv(1)
  4. df1 = pd.DataFrame([
  5. ['Ohio', 2000, 1.5],
  6. ['Ohio', 2001, 1.7],
  7. ['Ohio', 2002, 3.6],
  8. ['Nevada', 2001, 2.4],
  9. ['Nevada', 2001, 2.4],
  10. ['Nevada', 2003, 3.2]
  11. ])
  12. df2 = pd.DataFrame([
  13. ['Nevada', 2001, 2.4],
  14. ['Nevada', 2001, 2.4],
  15. ['Nevada', 2003, 3.2]
  16. ])
  17. batch_data1 = BatchOperator.fromDataframe(df1, schemaStr='f1 string, f2 bigint, f3 double')
  18. batch_data2 = BatchOperator.fromDataframe(df2, schemaStr='f1 string, f2 bigint, f3 double')
  19. op = IntersectBatchOp()
  20. op.linkFrom(batch_data1, batch_data2).print()

Java 代码

  1. import com.alibaba.alink.operator.batch.BatchOperator;
  2. import com.alibaba.alink.operator.batch.source.MemSourceBatchOp;
  3. import com.alibaba.alink.operator.batch.sql.IntersectBatchOp;
  4. import org.junit.Test;
  5. public class IntersectBatchOpTest {
  6. @Test
  7. public void testIntersectBatchOp() throws Exception {
  8. List <Row> df1 = Arrays.asList(
  9. Row.of("Ohio", 2000, 1.5),
  10. Row.of("Ohio", 2001, 1.7),
  11. Row.of("Ohio", 2002, 3.6),
  12. Row.of("Nevada", 2001, 2.4),
  13. Row.of("Nevada", 2001, 2.4),
  14. Row.of("Nevada", 2003, 3.2)
  15. );
  16. List <Row> df2 = Arrays.asList(
  17. Row.of("Nevada", 2001, 2.4),
  18. Row.of("Nevada", 2001, 2.4),
  19. Row.of("Nevada", 2003, 3.2)
  20. );
  21. BatchOperator <?> data1 = new MemSourceBatchOp(df1, "f1 string, f2 int, f3 double");
  22. BatchOperator <?> data2 = new MemSourceBatchOp(df2, "f1 string, f2 int, f3 double");
  23. BatchOperator <?> intersectAllOp = new IntersectBatchOp();
  24. intersectAllOp.linkFrom(data1, data2).print();
  25. }
  26. }

运行结果

| f1 | f2 | f3 | | —- | —- | —- |

| Nevada | 2001 | 2.4000 |

| Nevada | 2003 | 3.2000 |