原文文档：http://airflow.apache.org/docs/apache-airflow/stable/logging-monitoring/logging-tasks.html

1. Writing Logs Locally

Users can specify the directory to place log files in airflow.cfg using base_log_folder. By default, logs are placed in the AIRFLOW_HOME directory.

Note： For more information on setting the configuration, see Setting Configuration Options

The following convention is followed while naming logs: {dag_id}/{task_id}/{execution_date}/{try_number}.log

In addition, users can supply a remote location to store current logs and backups.

In the Airflow Web UI, remote logs take precedence over local logs when remote logging is enabled. If remote logs can not be found or accessed, local logs will be displayed. Note that logs are only sent to remote storage once a task is complete (including failure); In other words, remote logs for running tasks are unavailable (but local logs are available).

2. Troubleshooting

If you want to check which task handler is currently set, you can use airflow info command as in the example below.

  1. $ airflow info
  2. airflow on PATH: [True]
  4. Executor: [SequentialExecutor]
  5. Task Logging Handlers: [StackdriverTaskHandler]
  6. SQL Alchemy Conn: [sqlite://///root/airflow/airflow.db]
  7. DAGS Folder: [/root/airflow/dags]
  8. Plugins Folder: [/root/airflow/plugins]
  9. Base Log Folder: [/root/airflow/logs]

You can also use airflow config list to check that the logging configuration options have valid values.

3. Advanced configuration

Not all configuration options are available from the airflow.cfg file. Some configuration options require that the logging config class be overwritten. This can be done by logging_config_class option in airflow.cfg file. This option should specify the import path indicating to a configuration compatible with logging.config.dictConfig(). If your file is a standard import location, then you should set a PYTHONPATH environment.

Follow the steps below to enable custom logging config class:

  1. Start by setting environment variable to known directory e.g. ~/airflow/ 

    1. export PYTHONPATH=~/airflow/

  2. Create a directory to store the config file e.g. ~/airflow/config

  3. Create file called ~/airflow/config/log_config.py with following content: ```python from copy import deepcopy from airflow.config_templates.airflow_local_settings import DEFAULT_LOGGING_CONFIG

LOGGING_CONFIG = deepcopy(DEFAULT_LOGGING_CONFIG)

  2. 4. At the end of the file, add code to modify the default dictionary configuration.
  3. 4. Update _$AIRFLOW_HOME/airflow.cfg_ to contain:
  4. ```python
  5. [logging]
  6. remote_logging = True
  7. logging_config_class = log_config.LOGGING_CONFIG
  1. Restart the application.

See Modules Management for details on how Python and Airflow manage modules.

4. External Links

When using remote logging, users can configure Airflow to show a link to an external UI within the Airflow Web UI. Clicking the link redirects a user to the external UI.

Some external systems require specific configuration in Airflow for redirection to work but others do not.