DataStream

Function接口

计算逻辑
Map 等算子都是

org.apache.flink.streaming.api.operators包

StreamOperator接口

AbstractStreamOperator#initializeState类。

双流JOIN StreamingJoinOperator类中。

状态

程序入口

Client

执行bin/flink run xxx会调用CliFrontend类的run方法。

FlinkYarnSessionCli

YarnSessionClusterEntrypoint
image.png
YarnJobClusterEntrypoint
image.png
1.12 以上

  1. env.execute("Streaming WordCount SideOutput");

下面代码会进行分流

  1. CompletableFuture<JobClient> jobClientFuture =
  2. executorFactory
  3. .getExecutor(configuration)
  4. .execute(streamGraph, configuration, userClassloader);

到底是使用yarn或k8s的什么模式。一个模式实现一个PipelineExecutor接口。
image.png

  1. public interface PipelineExecutor {
  2. /**
  3. * Executes a {@link Pipeline} based on the provided configuration and returns a {@link
  4. * JobClient} which allows to interact with the job being executed, e.g. cancel it or take a
  5. * savepoint.
  6. *
  7. * <p><b>ATTENTION:</b> The caller is responsible for managing the lifecycle of the returned
  8. * {@link JobClient}. This means that e.g. {@code close()} should be called explicitly at the
  9. * call-site.
  10. *
  11. * @param pipeline the {@link Pipeline} to execute
  12. * @param configuration the {@link Configuration} with the required execution parameters
  13. * @param userCodeClassloader the {@link ClassLoader} to deserialize usercode
  14. * @return a {@link CompletableFuture} with the {@link JobClient} corresponding to the pipeline.
  15. */
  16. CompletableFuture<JobClient> execute(
  17. final Pipeline pipeline,
  18. final Configuration configuration,
  19. final ClassLoader userCodeClassloader)
  20. throws Exception;
  21. }

SQL

http://aitozi.com/BinaryRow-implement.html

源码解释

https://blog.csdn.net/hxcaifly/article/details/84558346