1.使用adapter同步mysql到es
    数据量大避免子表同步，使用主表驱动防止同步丢失数据 canal-admin 配置实例
    sql 查询无法建立在视图上基于mysql binlog
    子表判断条件 必须出现在select后
    canal 重启 检查同步yml 查看错误日志排除错误
    etlCondition 使用于全量同步 有入参的情况

    1. #################################################
    2. ## mysql serverId , v1.0.26+ will autoGen
    3. # canal.instance.mysql.slaveId=0
    5. # enable gtid use true/false
    6. canal.instance.gtidon=false
    8. # position info
    9. canal.instance.master.address=xxx.xxx.xxx.xxx:3306
    10. canal.instance.master.journal.name=
    11. canal.instance.master.position=
    12. canal.instance.master.timestamp=
    13. canal.instance.master.gtid=
    15. # rds oss binlog
    16. canal.instance.rds.accesskey=
    17. canal.instance.rds.secretkey=
    18. canal.instance.rds.instanceId=
    20. # table meta tsdb info
    21. canal.instance.tsdb.enable=false
    22. #canal.instance.tsdb.url=jdbc:mysql://127.0.0.1:3306/canal_tsdb
    23. #canal.instance.tsdb.dbUsername=canal
    24. #canal.instance.tsdb.dbPassword=canal
    26. #canal.instance.standby.address =
    27. #canal.instance.standby.journal.name =
    28. #canal.instance.standby.position =
    29. #canal.instance.standby.timestamp =
    30. #canal.instance.standby.gtid=
    32. # username/password
    33. canal.instance.dbUsername=xxx
    34. canal.instance.dbPassword=xxx
    35. canal.instance.defaultDatabaseName=xxx
    36. canal.instance.connectionCharset = UTF-8
    37. # enable druid Decrypt database password
    38. canal.instance.enableDruid=false
    39. #canal.instance.pwdPublicKey=MFwwDQYJKoZIhvcNAQEBBQADSwAwSAJBALK4BUxdDltRRE5/zXpVEVPUgunvscYFtEip3pmLlhrWpacX7y7GCMo2/JM6LeHmiiNdH1FWgGCpUfircSwlWKUCAwEAAQ==
    41. # table regex
    42. canal.instance.filter.regex=xxx
    43. # table black regex
    44. canal.instance.filter.black.regex="xxx.undo_log"
    45. # table field filter(format: schema1.tableName1:field1/field2,schema2.tableName2:field1/field2)
    46. #canal.instance.filter.field=test1.t_product:id/subject/keywords,test2.t_company:id/name/contact/ch
    47. # table field black filter(format: schema1.tableName1:field1/field2,schema2.tableName2:field1/field2)
    48. #canal.instance.filter.black.field=test1.t_product:subject/product_image,test2.t_company:id/name/contact/ch
    50. # mq config
    51. canal.mq.topic=xxx
    52. # dynamic topic route by schema or table regex
    53. #canal.mq.dynamicTopic=mytest1.user,mytest2\\..*,.*\\..*
    54. canal.mq.partition=0
    55. # hash partition config
    56. #canal.mq.partitionsNum=3
    57. #canal.mq.partitionHash=test.table:id^name,.*\\..*
    58. #################################################

    2.问题 数据过大产生 json 截断
    参考：
    https://www.cnblogs.com/javabg/p/14265558.html
    3.yml 列子

    1. LEFT JOIN (select request_id,GROUP_CONCAT(JSON_OBJECT( 
    2.                 'requestId',request_id,          
    3.                 'id',id)) as json
    4.                 from xxx group by request_id) as x on 主表.request_id = x.request_id

    优化

    # 优化了canal 源码
CONCAT('[',(select GROUP_CONCAT(JSON_OBJECT(
               'creator',creator,
               'creatorId',creator_id,
               'createTime',date_format(create_time,'%Y-%m-%dT%H:%i:%S+08:00'),
               'updater',updater,
               'updaterId',updater_id,
               'updateTime',date_format(update_time,'%Y-%m-%dT%H:%i:%S+08:00'),
               'id',id)) as json
               from 子表 where enabled_flag = 1 and 主表.id = 子表.id),']') as xxxList

    问题：
    数据的抓取多条 之同步一条
    两个yml id 相同 同步到同一个 idex
    解决办法
    不同服务 建立不同的同步服务 不共用一个 每个服务消耗700-800M的内存