📃原文链接:https://rasa.com/docs/rasa/command-line-interface

The command line interface (CLI) gives you easy-to-remember commands for common tasks. This page describes the behavior of the commands and the parameters you can pass to them.

1. Cheat Sheet

Command Effect
rasa init Creates a new project with example training data, actions, and config files.
rasa train Trains a model using your NLU data and stories, saves trained model in ./models.
rasa interactive Starts an interactive learning session to create new training data by chatting to your assistant.
rasa shell Loads your trained model and lets you talk to your assistant on the command line.
rasa run Starts a server with your trained model.
rasa run actions Starts an action server using the Rasa SDK.
rasa visualize Generates a visual representation of your stories
rasa test Tests a trained Rasa model on any files starting with test_.
rasa data split nlu Performs a 80/20 split of your NLU training data.
rasa data convert Converts training data between different formats.
rasa data validate Checks the domain, NLU and conversation data for inconsistencies.
rasa export Exports conversations from a tracker store to an event broker.
rasa -x Launches Rasa X in local mode.
rasa -h Shows all available commands.

2. rasa init

This command sets up a complete assistant for you with some example training data:

  1. $ rasa init

It creates the following files:

  1. .
  2. ├── actions
  3. ├── __init__.py
  4. └── actions.py
  5. ├── config.yml
  6. ├── credentials.yml
  7. ├── data
  8. ├── nlu.yml
  9. └── stories.yml
  10. ├── domain.yml
  11. ├── endpoints.yml
  12. ├── models
  13. └── <timestamp>.tar.gz
  14. └── tests
  15. └── test_stories.yml

It will ask you if you want to train an initial model using this data. If you answer no, the models directory will be empty.

Any of the default CLI commands will expect this project setup, so this is the best way to get started. You can run rasa train, rasa shell and rasa test without any additional configuration.

3. rasa train

The following command trains a Rasa Open Source model:

  1. $ rasa train

If you have existing models in your directory (under models/ by default), only the parts of your model that have changed will be re-trained. For example, if you edit your NLU training data and nothing else, only the NLU part will be trained.

If you want to train an NLU or dialogue model individually, you can run rasa train nlu or rasa train core. If you provide training data only for one one of these, rasa train will fall back to one of these commands by default.

rasa train will store the trained model in the directory defined by --out, models/ by default. The name of the model by default is <timestamp>.tar.gz. If you want to name your model differently, you can specify the name using the --fixed-model-name flag.

The following arguments can be used to configure the training process:

  1. usage: rasa train [-h] [-v] [-vv] [--quiet] [--data DATA [DATA ...]]
  2. [-c CONFIG] [-d DOMAIN] [--out OUT] [--dry-run]
  3. [--augmentation AUGMENTATION] [--debug-plots]
  4. [--num-threads NUM_THREADS]
  5. [--fixed-model-name FIXED_MODEL_NAME] [--persist-nlu-data]
  6. [--force] [--finetune [FINETUNE]]
  7. [--epoch-fraction EPOCH_FRACTION]
  8. {core,nlu} ...
  9. positional arguments:
  10. {core,nlu}
  11. core Trains a Rasa Core model using your stories.
  12. nlu Trains a Rasa NLU model using your NLU data.
  13. optional arguments:
  14. -h, --help show this help message and exit
  15. --data DATA [DATA ...]
  16. Paths to the Core and NLU data files. (default:
  17. ['data'])
  18. -c CONFIG, --config CONFIG
  19. The policy and NLU pipeline configuration of your bot.
  20. (default: config.yml)
  21. -d DOMAIN, --domain DOMAIN
  22. Domain specification. This can be a single YAML file,
  23. or a directory that contains several files with domain
  24. specifications in it. The content of these files will
  25. be read and merged together. (default: domain.yml)
  26. --out OUT Directory where your models should be stored.
  27. (default: models)
  28. --dry-run If enabled, no actual training will be performed.
  29. Instead, it will be determined whether a model should
  30. be re-trained and this information will be printed as
  31. the output. The return code is a 4-bit bitmask that
  32. can also be used to determine what exactly needs to be
  33. retrained: - 1 means Core needs to be retrained - 2
  34. means NLU needs to be retrained - 4 means responses in
  35. the domain should be updated - 8 means the training
  36. was forced (--force argument is specified) (default:
  37. False)
  38. --augmentation AUGMENTATION
  39. How much data augmentation to use during training.
  40. (default: 50)
  41. --debug-plots If enabled, will create plots showing checkpoints and
  42. their connections between story blocks in a file
  43. called `story_blocks_connections.html`. (default:
  44. False)
  45. --num-threads NUM_THREADS
  46. Maximum amount of threads to use when training.
  47. (default: 1)
  48. --fixed-model-name FIXED_MODEL_NAME
  49. If set, the name of the model file/directory will be
  50. set to the given name. (default: None)
  51. --persist-nlu-data Persist the NLU training data in the saved model.
  52. (default: False)
  53. --force Force a model training even if the data has not
  54. changed. (default: False)
  55. --finetune [FINETUNE]
  56. Fine-tune a previously trained model. If no model path
  57. is provided, Rasa Open Source will try to finetune the
  58. latest trained model from the model directory
  59. specified via '--out'. (default: None)
  60. --epoch-fraction EPOCH_FRACTION
  61. Fraction of epochs which are currently specified in
  62. the model configuration which should be used when
  63. finetuning a model. (default: 1.0)
  64. Python Logging Options:
  65. -v, --verbose Be verbose. Sets logging level to INFO. (default:
  66. None)
  67. -vv, --debug Print lots of debugging statements. Sets logging level
  68. to DEBUG. (default: None)
  69. --quiet Be quiet! Sets logging level to WARNING. (default:
  70. None)

3.1 Incremental training

:::warning

⚠️ CAUTION

This feature is experimental. We introduce experimental features to get feedback from our community, so we encourage you to try it out! However, the functionality might be changed or removed in the future. If you have feedback (positive or negative) please share it with us on the Rasa Forum. :::

In order to improve the performance of an assistant, it’s helpful to practice CDD and add new training examples based on how your users have talked to your assistant. You can use rasa train --finetune to initialize the pipeline with an already trained model and further finetune it on the new training dataset that includes the additional training examples. This will help reduce the training time of the new model.

By default, the command picks up the latest model in the models/ directory. If you have a specific model which you want to improve, you may specify the path to this by running rasa train --finetune <path to model to finetune>. Finetuning(微调) a model usually requires fewer epochs to train machine learning components like DIETClassifier, ResponseSelector and TEDPolicy compared to training from scratch. Either use a model configuration for finetuning which defines fewer epochs than before or use the flag --epoch-fraction.

--epoch-fraction will use a fraction of the epochs specified for each machine learning component in the model configuration file. For example, if DIETClassifier is configured to use 100 epochs, specifying --epoch-fraction 0.5 will only use 50 epochs for finetuning.

You can also finetune an NLU-only or dialogue management-only model by using rasa train nlu --finetune and rasa train core --finetune respectively.

To be able to fine tune a model, the following conditions must be met:

  1. The configuration supplied should be exactly the same as the configuration used to train the model which is being finetuned. The only parameter that you can change is epochs for the individual machine learning components and policies.
  2. The set of labels(intents, actions, entities and slots) for which the base model is trained should be exactly the same as the ones present in the training data used for finetuning. This means that you cannot add new intent, action, entity or slot labels to your training data during incremental training. You can still add new training examples for each of the existing labels. If you have added/removed labels in the training data, the pipeline needs to be trained from scratch.
  3. The model to be finetuned is trained with MINIMUM_COMPATIBLE_VERSION of the currently installed rasa version.

    4. rasa interactive

    You can use Rasa X in local mode to do interactive learning in a UI, check out the docs for more details.

If you’d rather use the command line, you can start an interactive learning session by running:

  1. $ rasa interactive

This will first train a model and then start an interactive shell session. You can then correct your assistants predictions as you talk to it. If UnexpecTEDIntentPolicy is included in the pipeline, action_unlikely_intent can be triggered at any conversation turn. Subsequently, the following message will be displayed:

  1. The bot wants to run 'action_unlikely_intent' to indicate that the last user message was unexpected
  2. at this point in the conversation. Check out UnexpecTEDIntentPolicy docs to learn more.

As the message states, this is an indication that you have explored a conversation path which is unexpected according to the current set of training stories and hence adding this path to training stories is recommended. Like other bot actions, you can choose to confirm or deny running this action.

If you provide a trained model using the --model argument, training is skipped and that model will be loaded instead.

During interactive learning, Rasa will plot the current conversation and a few similar conversations from the training data to help you keep track of where you are. You can view the visualization at http://localhost:5005/visualization.html as soon as the session has started. This diagram can take some time to generate. To skip the visualization, run rasa interactive --skip-visualization.

The following arguments can be used to configure the interactive learning session:

  1. usage: rasa interactive [-h] [-v] [-vv] [--quiet] [--e2e] [-p PORT] [-m MODEL]
  2. [--data DATA [DATA ...]] [--skip-visualization]
  3. [--conversation-id CONVERSATION_ID]
  4. [--endpoints ENDPOINTS] [-c CONFIG] [-d DOMAIN]
  5. [--out OUT] [--augmentation AUGMENTATION]
  6. [--debug-plots] [--finetune [FINETUNE]]
  7. [--epoch-fraction EPOCH_FRACTION] [--force]
  8. [--persist-nlu-data]
  9. {core} ... [model-as-positional-argument]
  10. positional arguments:
  11. {core}
  12. core Starts an interactive learning session model to create
  13. new training data for a Rasa Core model by chatting.
  14. Uses the 'RegexInterpreter', i.e. `/<intent>` input
  15. format.
  16. model-as-positional-argument
  17. Path to a trained Rasa model. If a directory is
  18. specified, it will use the latest model in this
  19. directory. (default: None)
  20. optional arguments:
  21. -h, --help show this help message and exit
  22. --e2e Save story files in e2e format. In this format user
  23. messages will be included in the stories. (default:
  24. False)
  25. -p PORT, --port PORT Port to run the server at. (default: 5005)
  26. -m MODEL, --model MODEL
  27. Path to a trained Rasa model. If a directory is
  28. specified, it will use the latest model in this
  29. directory. (default: None)
  30. --data DATA [DATA ...]
  31. Paths to the Core and NLU data files. (default:
  32. ['data'])
  33. --skip-visualization Disable plotting the visualization during interactive
  34. learning. (default: False)
  35. --conversation-id CONVERSATION_ID
  36. Specify the id of the conversation the messages are
  37. in. Defaults to a UUID that will be randomly
  38. generated. (default: 9f5889894c774daeba2f4b2c72e58787)
  39. --endpoints ENDPOINTS
  40. Configuration file for the model server and the
  41. connectors as a yml file. (default: endpoints.yml)
  42. Python Logging Options:
  43. -v, --verbose Be verbose. Sets logging level to INFO. (default:
  44. None)
  45. -vv, --debug Print lots of debugging statements. Sets logging level
  46. to DEBUG. (default: None)
  47. --quiet Be quiet! Sets logging level to WARNING. (default:
  48. None)
  49. Train Arguments:
  50. -c CONFIG, --config CONFIG
  51. The policy and NLU pipeline configuration of your bot.
  52. (default: config.yml)
  53. -d DOMAIN, --domain DOMAIN
  54. Domain specification. This can be a single YAML file,
  55. or a directory that contains several files with domain
  56. specifications in it. The content of these files will
  57. be read and merged together. (default: domain.yml)
  58. --out OUT Directory where your models should be stored.
  59. (default: models)
  60. --augmentation AUGMENTATION
  61. How much data augmentation to use during training.
  62. (default: 50)
  63. --debug-plots If enabled, will create plots showing checkpoints and
  64. their connections between story blocks in a file
  65. called `story_blocks_connections.html`. (default:
  66. False)
  67. --finetune [FINETUNE]
  68. Fine-tune a previously trained model. If no model path
  69. is provided, Rasa Open Source will try to finetune the
  70. latest trained model from the model directory
  71. specified via '--out'. (default: None)
  72. --epoch-fraction EPOCH_FRACTION
  73. Fraction of epochs which are currently specified in
  74. the model configuration which should be used when
  75. finetuning a model. (default: 1.0)
  76. --force Force a model training even if the data has not
  77. changed. (default: False)
  78. --persist-nlu-data Persist the NLU training data in the saved model.
  79. (default: False)

5. rasa shell

You can use Rasa X in local mode to talk to your assistant in a UI. Check out the Rasa X docs for more details.

If you’d rather use the command line, you can start a chat session by running:

  1. $ rasa shell

By default this will load up the latest trained model. You can specify a different model to be loaded by using the --model flag.

If you start the shell with an NLU-only model, rasa shell will output the intents and entities predicted for any message you enter.

If you have trained a combined Rasa model but only want to see what your model extracts as intents and entities from text, you can use the command rasa shell nlu.

To increase the logging level for debugging, run:

  1. $ rasa shell --debug

:::info

💡 NOTE

In order to see the typical greetings and/or session start behavior you might see in an external channel, you will need to explicitly send /session_start as the first message. Otherwise, the session start behavior will begin as described in Session configuration. :::

The following arguments can be used to configure the command:

  1. usage: rasa shell [-h] [-v] [-vv] [--quiet]
  2. [--conversation-id CONVERSATION_ID] [-m MODEL]
  3. [--log-file LOG_FILE] [--endpoints ENDPOINTS] [-p PORT]
  4. [-t AUTH_TOKEN] [--cors [CORS [CORS ...]]] [--enable-api]
  5. [--response-timeout RESPONSE_TIMEOUT]
  6. [--remote-storage REMOTE_STORAGE]
  7. [--ssl-certificate SSL_CERTIFICATE]
  8. [--ssl-keyfile SSL_KEYFILE] [--ssl-ca-file SSL_CA_FILE]
  9. [--ssl-password SSL_PASSWORD] [--credentials CREDENTIALS]
  10. [--connector CONNECTOR] [--jwt-secret JWT_SECRET]
  11. [--jwt-method JWT_METHOD]
  12. {nlu} ... [model-as-positional-argument]
  13. positional arguments:
  14. {nlu}
  15. nlu Interprets messages on the command line using your NLU
  16. model.
  17. model-as-positional-argument
  18. Path to a trained Rasa model. If a directory is
  19. specified, it will use the latest model in this
  20. directory. (default: None)
  21. optional arguments:
  22. -h, --help show this help message and exit
  23. --conversation-id CONVERSATION_ID
  24. Set the conversation ID. (default:
  25. bb923ad32d5c4149aa3529d5e9c0485d)
  26. -m MODEL, --model MODEL
  27. Path to a trained Rasa model. If a directory is
  28. specified, it will use the latest model in this
  29. directory. (default: models)
  30. --log-file LOG_FILE Store logs in specified file. (default: None)
  31. --endpoints ENDPOINTS
  32. Configuration file for the model server and the
  33. connectors as a yml file. (default: endpoints.yml)
  34. Python Logging Options:
  35. -v, --verbose Be verbose. Sets logging level to INFO. (default:
  36. None)
  37. -vv, --debug Print lots of debugging statements. Sets logging level
  38. to DEBUG. (default: None)
  39. --quiet Be quiet! Sets logging level to WARNING. (default:
  40. None)
  41. Server Settings:
  42. -p PORT, --port PORT Port to run the server at. (default: 5005)
  43. -t AUTH_TOKEN, --auth-token AUTH_TOKEN
  44. Enable token based authentication. Requests need to
  45. provide the token to be accepted. (default: None)
  46. --cors [CORS [CORS ...]]
  47. Enable CORS for the passed origin. Use * to whitelist
  48. all origins. (default: None)
  49. --enable-api Start the web server API in addition to the input
  50. channel. (default: False)
  51. --response-timeout RESPONSE_TIMEOUT
  52. Maximum time a response can take to process (sec).
  53. (default: 3600)
  54. --remote-storage REMOTE_STORAGE
  55. Set the remote location where your Rasa model is
  56. stored, e.g. on AWS. (default: None)
  57. --ssl-certificate SSL_CERTIFICATE
  58. Set the SSL Certificate to create a TLS secured
  59. server. (default: None)
  60. --ssl-keyfile SSL_KEYFILE
  61. Set the SSL Keyfile to create a TLS secured server.
  62. (default: None)
  63. --ssl-ca-file SSL_CA_FILE
  64. If your SSL certificate needs to be verified, you can
  65. specify the CA file using this parameter. (default:
  66. None)
  67. --ssl-password SSL_PASSWORD
  68. If your ssl-keyfile is protected by a password, you
  69. can specify it using this paramer. (default: None)
  70. Channels:
  71. --credentials CREDENTIALS
  72. Authentication credentials for the connector as a yml
  73. file. (default: None)
  74. --connector CONNECTOR
  75. Service to connect to. (default: None)
  76. JWT Authentication:
  77. --jwt-secret JWT_SECRET
  78. Public key for asymmetric JWT methods or shared
  79. secretfor symmetric methods. Please also make sure to
  80. use --jwt-method to select the method of the
  81. signature, otherwise this argument will be
  82. ignored.Note that this key is meant for securing the
  83. HTTP API. (default: None)
  84. --jwt-method JWT_METHOD
  85. Method used for the signature of the JWT
  86. authentication payload. (default: HS256)

6. rasa run

To start a server running your trained model, run:

  1. $ rasa run

By default the Rasa server uses HTTP for its communication. To secure the communication with SSL and run the server on HTTPS, you need to provide a valid certificate and the corresponding private key file. You can specify these files as part of the rasa run command. If you encrypted your keyfile with a password during creation, you need to add the --ssl-password as well.

  1. $ rasa run --ssl-certificate myssl.crt --ssl-keyfile myssl.key --ssl-password mypassword

The following arguments can be used to configure your Rasa server:

  1. usage: rasa run [-h] [-v] [-vv] [--quiet] [-m MODEL] [--log-file LOG_FILE]
  2. [--endpoints ENDPOINTS] [-p PORT] [-t AUTH_TOKEN]
  3. [--cors [CORS [CORS ...]]] [--enable-api]
  4. [--response-timeout RESPONSE_TIMEOUT]
  5. [--remote-storage REMOTE_STORAGE]
  6. [--ssl-certificate SSL_CERTIFICATE]
  7. [--ssl-keyfile SSL_KEYFILE] [--ssl-ca-file SSL_CA_FILE]
  8. [--ssl-password SSL_PASSWORD] [--credentials CREDENTIALS]
  9. [--connector CONNECTOR] [--jwt-secret JWT_SECRET]
  10. [--jwt-method JWT_METHOD]
  11. {actions} ... [model-as-positional-argument]
  12. positional arguments:
  13. {actions}
  14. actions Runs the action server.
  15. model-as-positional-argument
  16. Path to a trained Rasa model. If a directory is
  17. specified, it will use the latest model in this
  18. directory. (default: None)
  19. optional arguments:
  20. -h, --help show this help message and exit
  21. -m MODEL, --model MODEL
  22. Path to a trained Rasa model. If a directory is
  23. specified, it will use the latest model in this
  24. directory. (default: models)
  25. --log-file LOG_FILE Store logs in specified file. (default: None)
  26. --endpoints ENDPOINTS
  27. Configuration file for the model server and the
  28. connectors as a yml file. (default: endpoints.yml)
  29. Python Logging Options:
  30. -v, --verbose Be verbose. Sets logging level to INFO. (default:
  31. None)
  32. -vv, --debug Print lots of debugging statements. Sets logging level
  33. to DEBUG. (default: None)
  34. --quiet Be quiet! Sets logging level to WARNING. (default:
  35. None)
  36. Server Settings:
  37. -p PORT, --port PORT Port to run the server at. (default: 5005)
  38. -t AUTH_TOKEN, --auth-token AUTH_TOKEN
  39. Enable token based authentication. Requests need to
  40. provide the token to be accepted. (default: None)
  41. --cors [CORS [CORS ...]]
  42. Enable CORS for the passed origin. Use * to whitelist
  43. all origins. (default: None)
  44. --enable-api Start the web server API in addition to the input
  45. channel. (default: False)
  46. --response-timeout RESPONSE_TIMEOUT
  47. Maximum time a response can take to process (sec).
  48. (default: 3600)
  49. --remote-storage REMOTE_STORAGE
  50. Set the remote location where your Rasa model is
  51. stored, e.g. on AWS. (default: None)
  52. --ssl-certificate SSL_CERTIFICATE
  53. Set the SSL Certificate to create a TLS secured
  54. server. (default: None)
  55. --ssl-keyfile SSL_KEYFILE
  56. Set the SSL Keyfile to create a TLS secured server.
  57. (default: None)
  58. --ssl-ca-file SSL_CA_FILE
  59. If your SSL certificate needs to be verified, you can
  60. specify the CA file using this parameter. (default:
  61. None)
  62. --ssl-password SSL_PASSWORD
  63. If your ssl-keyfile is protected by a password, you
  64. can specify it using this paramer. (default: None)
  65. Channels:
  66. --credentials CREDENTIALS
  67. Authentication credentials for the connector as a yml
  68. file. (default: None)
  69. --connector CONNECTOR
  70. Service to connect to. (default: None)
  71. JWT Authentication:
  72. --jwt-secret JWT_SECRET
  73. Public key for asymmetric JWT methods or shared
  74. secretfor symmetric methods. Please also make sure to
  75. use --jwt-method to select the method of the
  76. signature, otherwise this argument will be
  77. ignored.Note that this key is meant for securing the
  78. HTTP API. (default: None)
  79. --jwt-method JWT_METHOD
  80. Method used for the signature of the JWT
  81. authentication payload. (default: HS256)

For more information on the additional parameters, see Model Storage. See the Rasa HTTP API page for detailed documentation of all the endpoints.

7. rasa run actions

To start an action server with the Rasa SDK, run:

  1. $ rasa run actions

The following arguments can be used to adapt the server settings:

  1. usage: rasa run actions [-h] [-v] [-vv] [--quiet] [-p PORT]
  2. [--cors [CORS [CORS ...]]] [--actions ACTIONS]
  3. [--ssl-keyfile SSL_KEYFILE]
  4. [--ssl-certificate SSL_CERTIFICATE]
  5. [--ssl-password SSL_PASSWORD] [--auto-reload]
  6. optional arguments:
  7. -h, --help show this help message and exit
  8. -p PORT, --port PORT port to run the server at (default: 5055)
  9. --cors [CORS [CORS ...]]
  10. enable CORS for the passed origin. Use * to whitelist
  11. all origins (default: None)
  12. --actions ACTIONS name of action package to be loaded (default: None)
  13. --ssl-keyfile SSL_KEYFILE
  14. Set the SSL certificate to create a TLS secured
  15. server. (default: None)
  16. --ssl-certificate SSL_CERTIFICATE
  17. Set the SSL certificate to create a TLS secured
  18. server. (default: None)
  19. --ssl-password SSL_PASSWORD
  20. If your ssl-keyfile is protected by a password, you
  21. can specify it using this paramer. (default: None)
  22. --auto-reload Enable auto-reloading of modules containing Action
  23. subclasses. (default: False)
  24. Python Logging Options:
  25. -v, --verbose Be verbose. Sets logging level to INFO. (default:
  26. None)
  27. -vv, --debug Print lots of debugging statements. Sets logging level
  28. to DEBUG. (default: None)
  29. --quiet Be quiet! Sets logging level to WARNING. (default:
  30. None)

8. rasa visualize

To generate a graph of your stories in the browser, run:

  1. $ rasa visualize

If your stories are located somewhere other than the default location data/, you can specify their location with the --stories flag.

The following arguments can be used to configure this command:

  1. usage: rasa visualize [-h] [-v] [-vv] [--quiet] [-d DOMAIN] [-s STORIES]
  2. [-c CONFIG] [--out OUT] [--max-history MAX_HISTORY]
  3. [-u NLU]
  4. optional arguments:
  5. -h, --help show this help message and exit
  6. -d DOMAIN, --domain DOMAIN
  7. Domain specification. This can be a single YAML file,
  8. or a directory that contains several files with domain
  9. specifications in it. The content of these files will
  10. be read and merged together. (default: domain.yml)
  11. -s STORIES, --stories STORIES
  12. File or folder containing your training stories.
  13. (default: data)
  14. -c CONFIG, --config CONFIG
  15. The policy and NLU pipeline configuration of your bot.
  16. (default: config.yml)
  17. --out OUT Filename of the output path, e.g. 'graph.html'.
  18. (default: graph.html)
  19. --max-history MAX_HISTORY
  20. Max history to consider when merging paths in the
  21. output graph. (default: 2)
  22. -u NLU, --nlu NLU File or folder containing your NLU data, used to
  23. insert example messages into the graph. (default:
  24. None)
  25. Python Logging Options:
  26. -v, --verbose Be verbose. Sets logging level to INFO. (default:
  27. None)
  28. -vv, --debug Print lots of debugging statements. Sets logging level
  29. to DEBUG. (default: None)
  30. --quiet Be quiet! Sets logging level to WARNING. (default:
  31. None)

9. rasa test

To evaluate a model on your test data, run:

  1. $ rasa test

This will test your latest trained model on any end-to-end stories you have defined in files with the test_ prefix. If you want to use a different model, you can specify it using the --model flag.

If you want to evaluate the dialogue and NLU models separately, you can use the commands below:

  1. $ rasa test core && rasa test nlu

You can find more details in Evaluating an NLU Model and Evaluating a Core Model.

The following arguments are available for rasa test:

  1. usage: rasa test [-h] [-v] [-vv] [--quiet] [-m MODEL] [-s STORIES]
  2. [--max-stories MAX_STORIES] [--endpoints ENDPOINTS]
  3. [--fail-on-prediction-errors] [--url URL]
  4. [--evaluate-model-directory] [-u NLU]
  5. [-c CONFIG [CONFIG ...]] [--cross-validation] [-f FOLDS]
  6. [-r RUNS] [-p PERCENTAGES [PERCENTAGES ...]] [--no-plot]
  7. [--successes] [--no-errors] [--no-warnings] [--out OUT]
  8. {core,nlu} ...
  9. positional arguments:
  10. {core,nlu}
  11. core Tests Rasa Core models using your test stories.
  12. nlu Tests Rasa NLU models using your test NLU data.
  13. optional arguments:
  14. -h, --help show this help message and exit
  15. -m MODEL, --model MODEL
  16. Path to a trained Rasa model. If a directory is
  17. specified, it will use the latest model in this
  18. directory. (default: models)
  19. --no-plot Don't render evaluation plots. (default: False)
  20. --successes If set successful predictions will be written to a
  21. file. (default: False)
  22. --no-errors If set incorrect predictions will NOT be written to a
  23. file. (default: False)
  24. --no-warnings If set prediction warnings will NOT be written to a
  25. file. (default: False)
  26. --out OUT Output path for any files created during the
  27. evaluation. (default: results)
  28. Python Logging Options:
  29. -v, --verbose Be verbose. Sets logging level to INFO. (default:
  30. None)
  31. -vv, --debug Print lots of debugging statements. Sets logging level
  32. to DEBUG. (default: None)
  33. --quiet Be quiet! Sets logging level to WARNING. (default:
  34. None)
  35. Core Test Arguments:
  36. -s STORIES, --stories STORIES
  37. File or folder containing your test stories. (default:
  38. .)
  39. --max-stories MAX_STORIES
  40. Maximum number of stories to test on. (default: None)
  41. --endpoints ENDPOINTS
  42. Configuration file for the connectors as a yml file.
  43. (default: endpoints.yml)
  44. --fail-on-prediction-errors
  45. If a prediction error is encountered, an exception is
  46. thrown. This can be used to validate stories during
  47. tests, e.g. on travis. (default: False)
  48. --url URL If supplied, downloads a story file from a URL and
  49. trains on it. Fetches the data by sending a GET
  50. request to the supplied URL. (default: None)
  51. --evaluate-model-directory
  52. Should be set to evaluate models trained via 'rasa
  53. train core --config <config-1> <config-2>'. All models
  54. in the provided directory are evaluated and compared
  55. against each other. (default: False)
  56. NLU Test Arguments:
  57. -u NLU, --nlu NLU File or folder containing your NLU data. (default:
  58. data)
  59. -c CONFIG [CONFIG ...], --config CONFIG [CONFIG ...]
  60. Model configuration file. If a single file is passed
  61. and cross validation mode is chosen, cross-validation
  62. is performed, if multiple configs or a folder of
  63. configs are passed, models will be trained and
  64. compared directly. (default: None)

10. rasa data split

To create a train-test split of your NLU training data, run:

  1. $ rasa data split nlu

This will create a 80/20 split of train/test data by default. You can specify the training data, the fraction, and the output directory using the following arguments:

  1. usage: rasa data split nlu [-h] [-v] [-vv] [--quiet] [-u NLU]
  2. [--training-fraction TRAINING_FRACTION]
  3. [--random-seed RANDOM_SEED] [--out OUT]
  4. optional arguments:
  5. -h, --help show this help message and exit
  6. -u NLU, --nlu NLU File or folder containing your NLU data. (default:
  7. data)
  8. --training-fraction TRAINING_FRACTION
  9. Percentage of the data which should be in the training
  10. data. (default: 0.8)
  11. --random-seed RANDOM_SEED
  12. Seed to generate the same train/test split. (default:
  13. None)
  14. --out OUT Directory where the split files should be stored.
  15. (default: train_test_split)
  16. Python Logging Options:
  17. -v, --verbose Be verbose. Sets logging level to INFO. (default:
  18. None)
  19. -vv, --debug Print lots of debugging statements. Sets logging level
  20. to DEBUG. (default: None)
  21. --quiet Be quiet! Sets logging level to WARNING. (default:
  22. None)

If you have NLG data for retrieval actions, this will be saved to seperate files:

  1. $ ls train_test_split
  2. nlg_test_data.yml test_data.yml
  3. nlg_training_data.yml training_data.yml

11. rasa data convert nlu

You can convert NLU data from

  • LUIS data format,
  • WIT data format,
  • Dialogflow data format,
  • JSON
  • Markdown

to

  • YAML or
  • JSON or
  • Markdown.

You can start the converter by running:

  1. $ rasa data convert nlu

You can specify the input file or directory, output file or directory, and the output format with the following arguments:

  1. usage: rasa data convert nlu [-h] [-v] [-vv] [--quiet] [-f {json,md,yaml}]
  2. --data DATA [DATA ...] [--out OUT] [-l LANGUAGE]
  3. optional arguments:
  4. -h, --help show this help message and exit
  5. -f {json,md,yaml}, --format {json,md,yaml}
  6. Output format the training data should be converted
  7. into. Note: currently training data can be converted
  8. to 'yaml' format only from 'md' format (default: yaml)
  9. --data DATA [DATA ...]
  10. Paths to the files or directories containing Rasa NLU
  11. data. (default: data)
  12. --out OUT File (for `json` and `md`) or existing path (for
  13. `yaml`) where to save training data in Rasa format.
  14. (default: converted_data)
  15. -l LANGUAGE, --language LANGUAGE
  16. Language of data. (default: en)
  17. Python Logging Options:
  18. -v, --verbose Be verbose. Sets logging level to INFO. (default:
  19. None)
  20. -vv, --debug Print lots of debugging statements. Sets logging level
  21. to DEBUG. (default: None)
  22. --quiet Be quiet! Sets logging level to WARNING. (default:
  23. None)

12. rasa data convert core

You can convert Core data from Markdown to YAML.

You can specify the input file or directory, output directory with the following arguments:

  1. $ rasa data convert core --help

13. rasa data convert nlg

You can convert NLG data from Markdown to YAML.

You can specify the input file or directory, output directory with the following arguments:

  1. $ rasa data convert nlg --help

14. rasa data validate

You can check your domain, NLU data, or story data for mistakes and inconsistencies. To validate your data, run this command:

  1. $ rasa data validate

The validator searches for errors in the data, e.g. two intents that have some identical training examples. The validator also checks if you have any stories where different assistant actions follow from the same dialogue history. Conflicts between stories will prevent a model from learning the correct pattern for a dialogue.

If you pass a max_history value to one or more policies in your config.yml file, provide the smallest of those values in the validator command using the --max-history <max_history> flag.

You can also validate only the story structure by running this command:

  1. $ rasa data validate stories

:::info

💡 NOTE

Running rasa data validate does not test if your rules are consistent with your stories. However, during training, the RulePolicy checks for conflicts between rules and stories. Any such conflict will abort training.

Also, if you use end-to-end stories, then this might not capture all conflicts. Specifically, if two user inputs result in different tokens yet exactly the same featurization, then conflicting actions after these inputs may exist but will not be reported by the tool. :::

To interrupt validation even for minor issues such as unused intents or responses, use the --fail-on-warnings flag.

:::info

💡 CHECK YOUR STORY NAMES

The rasa data validate stories command assumes that all your story names are unique!

💡 EXPERIMENTAL FEATURE

The rasa data validate stories command is an experimental feature. We introduce experimental features to get feedback from our community, so we encourage you to try it out! However, the functionality might be changed or removed in the future. If you have feedback (positive or negative) please share it with us on the Rasa Forum. :::

You can use rasa data validate with additional arguments, e.g. to specify the location of your data and domain files:

  1. usage: rasa data validate [-h] [-v] [-vv] [--quiet]
  2. [--max-history MAX_HISTORY] [-c CONFIG]
  3. [--fail-on-warnings] [-d DOMAIN]
  4. [--data DATA [DATA ...]]
  5. {stories} ...
  6. positional arguments:
  7. {stories}
  8. stories Checks for inconsistencies in the story files.
  9. optional arguments:
  10. -h, --help show this help message and exit
  11. --max-history MAX_HISTORY
  12. Number of turns taken into account for story structure
  13. validation. (default: None)
  14. -c CONFIG, --config CONFIG
  15. The policy and NLU pipeline configuration of your bot.
  16. (default: config.yml)
  17. --fail-on-warnings Fail validation on warnings and errors. If omitted
  18. only errors will result in a non zero exit code.
  19. (default: False)
  20. -d DOMAIN, --domain DOMAIN
  21. Domain specification. This can be a single YAML file,
  22. or a directory that contains several files with domain
  23. specifications in it. The content of these files will
  24. be read and merged together. (default: domain.yml)
  25. --data DATA [DATA ...]
  26. Paths to the files or directories containing Rasa
  27. data. (default: data)
  28. Python Logging Options:
  29. -v, --verbose Be verbose. Sets logging level to INFO. (default:
  30. None)
  31. -vv, --debug Print lots of debugging statements. Sets logging level
  32. to DEBUG. (default: None)
  33. --quiet Be quiet! Sets logging level to WARNING. (default:
  34. None)

15. rasa export

To export events from a tracker store using an event broker, run:

  1. $ rasa export

You can specify the location of the environments file, the minimum and maximum timestamps of events that should be published, as well as the conversation IDs that should be published:

  1. usage: rasa export [-h] [-v] [-vv] [--quiet] [--endpoints ENDPOINTS]
  2. [--minimum-timestamp MINIMUM_TIMESTAMP]
  3. [--maximum-timestamp MAXIMUM_TIMESTAMP]
  4. [--conversation-ids CONVERSATION_IDS]
  5. optional arguments:
  6. -h, --help show this help message and exit
  7. --endpoints ENDPOINTS
  8. Endpoint configuration file specifying the tracker
  9. store and event broker. (default: endpoints.yml)
  10. --minimum-timestamp MINIMUM_TIMESTAMP
  11. Minimum timestamp of events to be exported. The
  12. constraint is applied in a 'greater than or equal'
  13. comparison. (default: None)
  14. --maximum-timestamp MAXIMUM_TIMESTAMP
  15. Maximum timestamp of events to be exported. The
  16. constraint is applied in a 'less than' comparison.
  17. (default: None)
  18. --conversation-ids CONVERSATION_IDS
  19. Comma-separated list of conversation IDs to migrate.
  20. If unset, all available conversation IDs will be
  21. exported. (default: None)
  22. Python Logging Options:
  23. -v, --verbose Be verbose. Sets logging level to INFO. (default:
  24. None)
  25. -vv, --debug Print lots of debugging statements. Sets logging level
  26. to DEBUG. (default: None)
  27. --quiet Be quiet! Sets logging level to WARNING. (default:
  28. None)

:::info

💡 IMPORT CONVERSATIONS INTO RASA X

This command is most commonly used to import old conversations into Rasa X to annotate them. Read more about importing conversations into Rasa X. :::

16. rasa x

Rasa X is a tool for practicing Conversation-Driven Development. You can find more information about it here. You can start Rasa X in local mode by executing

  1. $ rasa x

To be able to start Rasa X you need to have Rasa X local mode installed and you need to be in a Rasa project directory.

The following arguments are available for rasa x:

  1. usage: rasa x [-h] [-v] [-vv] [--quiet] [-m MODEL] [--data DATA [DATA ...]]
  2. [-c CONFIG] [-d DOMAIN] [--no-prompt] [--production]
  3. [--rasa-x-port RASA_X_PORT] [--config-endpoint CONFIG_ENDPOINT]
  4. [--log-file LOG_FILE] [--endpoints ENDPOINTS] [-p PORT]
  5. [-t AUTH_TOKEN] [--cors [CORS [CORS ...]]] [--enable-api]
  6. [--response-timeout RESPONSE_TIMEOUT]
  7. [--remote-storage REMOTE_STORAGE]
  8. [--ssl-certificate SSL_CERTIFICATE] [--ssl-keyfile SSL_KEYFILE]
  9. [--ssl-ca-file SSL_CA_FILE] [--ssl-password SSL_PASSWORD]
  10. [--credentials CREDENTIALS] [--connector CONNECTOR]
  11. [--jwt-secret JWT_SECRET] [--jwt-method JWT_METHOD]
  12. optional arguments:
  13. -h, --help show this help message and exit
  14. -m MODEL, --model MODEL
  15. Path to a trained Rasa model. If a directory is
  16. specified, it will use the latest model in this
  17. directory. (default: models)
  18. --data DATA [DATA ...]
  19. Paths to the files or directories containing stories
  20. and Rasa NLU data. (default: data)
  21. -c CONFIG, --config CONFIG
  22. The policy and NLU pipeline configuration of your bot.
  23. (default: config.yml)
  24. -d DOMAIN, --domain DOMAIN
  25. Domain specification. This can be a single YAML file,
  26. or a directory that contains several files with domain
  27. specifications in it. The content of these files will
  28. be read and merged together. (default: domain.yml)
  29. --no-prompt Automatic yes or default options to prompts and
  30. oppressed warnings. (default: False)
  31. --production Run Rasa X in a production environment. (default:
  32. False)
  33. --rasa-x-port RASA_X_PORT
  34. Port to run the Rasa X server at. (default: 5002)
  35. --config-endpoint CONFIG_ENDPOINT
  36. Rasa X endpoint URL from which to pull the runtime
  37. config. This URL typically contains the Rasa X token
  38. for authentication. Example:
  39. https://example.com/api/config?token=my_rasa_x_token
  40. (default: None)
  41. --log-file LOG_FILE Store logs in specified file. (default: None)
  42. --endpoints ENDPOINTS
  43. Configuration file for the model server and the
  44. connectors as a yml file. (default: endpoints.yml)
  45. Python Logging Options:
  46. -v, --verbose Be verbose. Sets logging level to INFO. (default:
  47. None)
  48. -vv, --debug Print lots of debugging statements. Sets logging level
  49. to DEBUG. (default: None)
  50. --quiet Be quiet! Sets logging level to WARNING. (default:
  51. None)
  52. Server Settings:
  53. -p PORT, --port PORT Port to run the server at. (default: 5005)
  54. -t AUTH_TOKEN, --auth-token AUTH_TOKEN
  55. Enable token based authentication. Requests need to
  56. provide the token to be accepted. (default: None)
  57. --cors [CORS [CORS ...]]
  58. Enable CORS for the passed origin. Use * to whitelist
  59. all origins. (default: None)
  60. --enable-api Start the web server API in addition to the input
  61. channel. (default: False)
  62. --response-timeout RESPONSE_TIMEOUT
  63. Maximum time a response can take to process (sec).
  64. (default: 3600)
  65. --remote-storage REMOTE_STORAGE
  66. Set the remote location where your Rasa model is
  67. stored, e.g. on AWS. (default: None)
  68. --ssl-certificate SSL_CERTIFICATE
  69. Set the SSL Certificate to create a TLS secured
  70. server. (default: None)
  71. --ssl-keyfile SSL_KEYFILE
  72. Set the SSL Keyfile to create a TLS secured server.
  73. (default: None)
  74. --ssl-ca-file SSL_CA_FILE
  75. If your SSL certificate needs to be verified, you can
  76. specify the CA file using this parameter. (default:
  77. None)
  78. --ssl-password SSL_PASSWORD
  79. If your ssl-keyfile is protected by a password, you
  80. can specify it using this paramer. (default: None)
  81. Channels:
  82. --credentials CREDENTIALS
  83. Authentication credentials for the connector as a yml
  84. file. (default: None)
  85. --connector CONNECTOR
  86. Service to connect to. (default: None)
  87. JWT Authentication:
  88. --jwt-secret JWT_SECRET
  89. Public key for asymmetric JWT methods or shared
  90. secretfor symmetric methods. Please also make sure to
  91. use --jwt-method to select the method of the
  92. signature, otherwise this argument will be
  93. ignored.Note that this key is meant for securing the
  94. HTTP API. (default: None)
  95. --jwt-method JWT_METHOD
  96. Method used for the signature of the JWT
  97. authentication payload. (default: HS256)