🔗 原文链接:https://airflow.apache.org/docs/apache-airflow/stable/configurations-ref.html
1. core
Parameter |
Common Setting |
dags_folder |
~/bigdata/airflow/dags |
hostname_callable |
socket.getfqdn |
default_timezone |
Asia/Shanghai |
executor |
LocalExecutor |
sql_alchemy_conn |
mysql://airflow:airflow@localhost:3306/airflow?unix_socket=/var/run/mysqld/mysqld.sock&charset=utf8 |
sql_engine_encoding |
utf-8 |
sql_engine_collation_for_ids |
utf8mb4 |
sql_alchemy_pool_enabled |
True |
sql_alchemy_pool_size |
0 |
sql_alchemy_max_overflow |
-1 |
sql_alchemy_pool_recycle |
1800 |
sql_alchemy_pool_pre_ping |
True |
sql_alchemy_schema |
‘’ |
sql_alchemy_connect_args |
{"convert_unicode": True} |
parallelism |
32 |
dag_concurrency |
32 |
dags_are_paused_at_creation |
True |
max_active_runs_per_dag |
2 |
load_examples |
False |
load_default_connection |
False |
plugins_folder |
~/bigdata/airflow/plugins |
execute_tasks_new_python_interpreter |
False |
fernet_key |
|
donot_pickle |
True |
dagbag_import_timeout |
30.0 |
dagbag_import_error_tracebacks |
True |
dagbag_import_error_traceback_depth |
2 |
dag_file_processor_timeout |
50 |
task_runner |
StandardTaskRunner |
default_impersonation |
|
security |
|
unit_test_mode |
False |
enable_xcom_pickling |
False |
enable_xcom_pickling |
False |
killed_task_cleanup_time |
60 |
dag_run_conf_overrides_params |
True |
dag_discovery_safe_mode |
True |
default_task_retries |
5 |
min_serialized_dag_update_interval |
30 |
min_serialized_dag_fetch_interval |
10 |
store_dag_code |
True |
max_num_rendered_ti_fields_per_task |
30 |
check_slas |
True |
xcom_backend |
airflow.models.xcom.BaseXCom |
lazy_load_plugins |
True |
lazy_discover_providers |
True |
max_db_retries |
3 |
hide_sensitive_var_conn_fields |
True |
sensitive_var_conn_names |
1.1 dags_folder
The folder where your airflow pipelines live, most likely a subfolder in a code repository. This path must be absolute.
Type |
Deafult |
Environment Variable |
string |
{AIRFLOW_HOME}/dags |
AIRFLOW__CORE__DAGS_FOLDER |
2. logging
Parameter |
Common Setting |
base_log_folder |
~/bigdata/airflow/plugins |
remote_logging |
False |
remote_log_conn_id |
|
google_key_path |
|
remote_base_log_folder |
|
encrypt_s3_logs |
|
logging_level |
INFO |
fab_logging_level |
WARN |
logging_config_class |
|
colored_console_log |
True |
colored_log_format |
|
colored_formatter_class |
airflow.utils.log.colored_log.CustomTTYColoredFormatter |
log_format |
[%%(asctime)s] {{%%(filename)s:%%(lineno)d}} %%(levelname)s - %%(message)s |
simple_log_format |
%%(asctime)s %%(levelname)s - %%(message)s |
task_log_prefix_template |
{{ti.dag_id}}-{{ti.task_id}}-{{execution_date}}-{{try_number}} |
log_filename_template |
{{{{ ti.dag_id }}}}/{{{{ ti.task_id }}}}/{{{{ ts }}}}/{{{{ try_number }}}}.log |
log_processor_filename_template |
{{{{ filename }}}}.log |
dag_processor_manager_log_location |
~/bigdata/airflow/logs/dag_processor_manager/dag_processor_manager.log |
task_log_reader |
task |
extra_loggers |
3. metrics
4. secrets
5. cli
6. debug
7. api
8. lineage
9. atlas
10. operators
11. hive
12. webserver
Parameter |
Common Setting |
base_url |
|
default_ui_timezone |
Asia/Shanghai |
web_server_host |
0.0.0.0 |
web_server_port |
9000 |
web_server_ssl_cert |
|
web_server_ssl_key |
|
web_server_master_timeout |
120 |
|
13. smtp
14. sentry
15. celery_kubernetes_executor
16. celery
17. celert_broker_transport_options
18. dask
19. scheduler
20. keberos
21. github_enterprise
22. elasticsearch
23. kubernetes
24. smart_senor