分区表join时,过滤分区,以下两种写法都可以,其执行过程相同

    • 过滤条件写在on中 ```sql explain select a.city_name,b.name from crisps_ods.ods_crisps_mch_address a left join crisps_ods.ods_crisps_mch_detail b on a.mch_detail_id = b.id and b.p_date = ‘2022-03-31’;

    STAGE DEPENDENCIES: Stage-4 is a root stage , consists of Stage-5, Stage-1 Stage-5 has a backup stage: Stage-1 Stage-3 depends on stages: Stage-5 Stage-1 Stage-0 depends on stages: Stage-3, Stage-1

    STAGE PLANS: Stage: Stage-4 Conditional Operator

    Stage: Stage-5 Map Reduce Local Work Alias -> Map Local Tables: $hdt$_1:b Fetch Operator limit: -1 Alias -> Map Local Operator Tree: $hdt$_1:b TableScan alias: b Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE Filter Operator predicate: (p_date = ‘2022-03-31’) (type: boolean) Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE Select Operator expressions: id (type: bigint), name (type: varchar(128)) outputColumnNames: _col0, _col1 Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE HashTable Sink Operator keys: 0 _col0 (type: bigint) 1 _col0 (type: bigint)

    Stage: Stage-3 Map Reduce Map Operator Tree: TableScan alias: a Statistics: Num rows: 22040 Data size: 1278351 Basic stats: COMPLETE Column stats: NONE Select Operator expressions: mch_detail_id (type: bigint), city_name (type: varchar(50)) outputColumnNames: _col0, _col1 Statistics: Num rows: 22040 Data size: 1278351 Basic stats: COMPLETE Column stats: NONE Map Join Operator condition map: Left Outer Join0 to 1 keys: 0 _col0 (type: bigint) 1 _col0 (type: bigint) outputColumnNames: _col1, _col3 Statistics: Num rows: 24244 Data size: 1406186 Basic stats: COMPLETE Column stats: NONE Select Operator expressions: _col1 (type: varchar(50)), _col3 (type: varchar(128)) outputColumnNames: _col0, _col1 Statistics: Num rows: 24244 Data size: 1406186 Basic stats: COMPLETE Column stats: NONE File Output Operator compressed: false Statistics: Num rows: 24244 Data size: 1406186 Basic stats: COMPLETE Column stats: NONE table: input format: org.apache.hadoop.mapred.SequenceFileInputFormat output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe Local Work: Map Reduce Local Work

    Stage: Stage-1 Map Reduce Map Operator Tree: TableScan alias: a Statistics: Num rows: 22040 Data size: 1278351 Basic stats: COMPLETE Column stats: NONE Select Operator expressions: mch_detail_id (type: bigint), city_name (type: varchar(50)) outputColumnNames: _col0, _col1 Statistics: Num rows: 22040 Data size: 1278351 Basic stats: COMPLETE Column stats: NONE Reduce Output Operator key expressions: _col0 (type: bigint) sort order: + Map-reduce partition columns: _col0 (type: bigint) Statistics: Num rows: 22040 Data size: 1278351 Basic stats: COMPLETE Column stats: NONE value expressions: _col1 (type: varchar(50)) TableScan alias: b Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE Filter Operator predicate: (p_date = ‘2022-03-31’) (type: boolean) Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE Select Operator expressions: id (type: bigint), name (type: varchar(128)) outputColumnNames: _col0, _col1 Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE Reduce Output Operator key expressions: _col0 (type: bigint) sort order: + Map-reduce partition columns: _col0 (type: bigint) Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE value expressions: _col1 (type: varchar(128)) Reduce Operator Tree: Join Operator condition map: Left Outer Join0 to 1 keys: 0 _col0 (type: bigint) 1 _col0 (type: bigint) outputColumnNames: _col1, _col3 Statistics: Num rows: 24244 Data size: 1406186 Basic stats: COMPLETE Column stats: NONE Select Operator expressions: _col1 (type: varchar(50)), _col3 (type: varchar(128)) outputColumnNames: _col0, _col1 Statistics: Num rows: 24244 Data size: 1406186 Basic stats: COMPLETE Column stats: NONE File Output Operator compressed: false Statistics: Num rows: 24244 Data size: 1406186 Basic stats: COMPLETE Column stats: NONE table: input format: org.apache.hadoop.mapred.SequenceFileInputFormat output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe

    Stage: Stage-0 Fetch Operator limit: -1 Processor Tree: ListSink

    1. - 过滤条件写在where
    2. ```sql
    3. explain select
    4. a.city_name,b.name from crisps_ods.ods_crisps_mch_address a
    5. left join ( select * from crisps_ods.ods_crisps_mch_detail where p_date = '2022-03-31') b
    6. on a.mch_detail_id = b.id ;
    7. STAGE DEPENDENCIES:
    8. Stage-4 is a root stage , consists of Stage-5, Stage-1
    9. Stage-5 has a backup stage: Stage-1
    10. Stage-3 depends on stages: Stage-5
    11. Stage-1
    12. Stage-0 depends on stages: Stage-3, Stage-1
    13. STAGE PLANS:
    14. Stage: Stage-4
    15. Conditional Operator
    16. Stage: Stage-5
    17. Map Reduce Local Work
    18. Alias -> Map Local Tables:
    19. $hdt$_1:ods_crisps_mch_detail
    20. Fetch Operator
    21. limit: -1
    22. Alias -> Map Local Operator Tree:
    23. $hdt$_1:ods_crisps_mch_detail
    24. TableScan
    25. alias: ods_crisps_mch_detail
    26. Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
    27. Filter Operator
    28. predicate: (p_date = '2022-03-31') (type: boolean)
    29. Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
    30. Select Operator
    31. expressions: id (type: bigint), name (type: varchar(128))
    32. outputColumnNames: _col0, _col1
    33. Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
    34. HashTable Sink Operator
    35. keys:
    36. 0 _col0 (type: bigint)
    37. 1 _col0 (type: bigint)
    38. Stage: Stage-3
    39. Map Reduce
    40. Map Operator Tree:
    41. TableScan
    42. alias: a
    43. Statistics: Num rows: 22040 Data size: 1278351 Basic stats: COMPLETE Column stats: NONE
    44. Select Operator
    45. expressions: mch_detail_id (type: bigint), city_name (type: varchar(50))
    46. outputColumnNames: _col0, _col1
    47. Statistics: Num rows: 22040 Data size: 1278351 Basic stats: COMPLETE Column stats: NONE
    48. Map Join Operator
    49. condition map:
    50. Left Outer Join0 to 1
    51. keys:
    52. 0 _col0 (type: bigint)
    53. 1 _col0 (type: bigint)
    54. outputColumnNames: _col1, _col3
    55. Statistics: Num rows: 24244 Data size: 1406186 Basic stats: COMPLETE Column stats: NONE
    56. Select Operator
    57. expressions: _col1 (type: varchar(50)), _col3 (type: varchar(128))
    58. outputColumnNames: _col0, _col1
    59. Statistics: Num rows: 24244 Data size: 1406186 Basic stats: COMPLETE Column stats: NONE
    60. File Output Operator
    61. compressed: false
    62. Statistics: Num rows: 24244 Data size: 1406186 Basic stats: COMPLETE Column stats: NONE
    63. table:
    64. input format: org.apache.hadoop.mapred.SequenceFileInputFormat
    65. output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
    66. serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
    67. Local Work:
    68. Map Reduce Local Work
    69. Stage: Stage-1
    70. Map Reduce
    71. Map Operator Tree:
    72. TableScan
    73. alias: a
    74. Statistics: Num rows: 22040 Data size: 1278351 Basic stats: COMPLETE Column stats: NONE
    75. Select Operator
    76. expressions: mch_detail_id (type: bigint), city_name (type: varchar(50))
    77. outputColumnNames: _col0, _col1
    78. Statistics: Num rows: 22040 Data size: 1278351 Basic stats: COMPLETE Column stats: NONE
    79. Reduce Output Operator
    80. key expressions: _col0 (type: bigint)
    81. sort order: +
    82. Map-reduce partition columns: _col0 (type: bigint)
    83. Statistics: Num rows: 22040 Data size: 1278351 Basic stats: COMPLETE Column stats: NONE
    84. value expressions: _col1 (type: varchar(50))
    85. TableScan
    86. alias: ods_crisps_mch_detail
    87. Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
    88. Filter Operator
    89. predicate: (p_date = '2022-03-31') (type: boolean)
    90. Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
    91. Select Operator
    92. expressions: id (type: bigint), name (type: varchar(128))
    93. outputColumnNames: _col0, _col1
    94. Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
    95. Reduce Output Operator
    96. key expressions: _col0 (type: bigint)
    97. sort order: +
    98. Map-reduce partition columns: _col0 (type: bigint)
    99. Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
    100. value expressions: _col1 (type: varchar(128))
    101. Reduce Operator Tree:
    102. Join Operator
    103. condition map:
    104. Left Outer Join0 to 1
    105. keys:
    106. 0 _col0 (type: bigint)
    107. 1 _col0 (type: bigint)
    108. outputColumnNames: _col1, _col3
    109. Statistics: Num rows: 24244 Data size: 1406186 Basic stats: COMPLETE Column stats: NONE
    110. Select Operator
    111. expressions: _col1 (type: varchar(50)), _col3 (type: varchar(128))
    112. outputColumnNames: _col0, _col1
    113. Statistics: Num rows: 24244 Data size: 1406186 Basic stats: COMPLETE Column stats: NONE
    114. File Output Operator
    115. compressed: false
    116. Statistics: Num rows: 24244 Data size: 1406186 Basic stats: COMPLETE Column stats: NONE
    117. table:
    118. input format: org.apache.hadoop.mapred.SequenceFileInputFormat
    119. output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
    120. serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
    121. Stage: Stage-0
    122. Fetch Operator
    123. limit: -1
    124. Processor Tree:
    125. ListSink