In MNN, Interpreter provides three functions for running Session, but in general, runSession is sufficient for most cases.

Run Session

  1. /**
  2.  * @brief run session.
  3.  * @param session   given session.
  4.  * @return result of running.
  5.  */
  6. ErrorCode runSession(Session* session) const;

Just pass in the previously created Session.

The time consumption of the function is not always equal to the time consumption of the inference - for CPU backend, they are equal; for other backends, the function may not wait synchronously for the completion of the inference. For example, the time consumption of the function is equal to the time consumption for shaders encoding and committing for GPU backend.

Run Session with Callbacks

  1. typedef std::function<bool(const std::vector<Tensor*>&, 
  2.                            const std::string& /*opName*/)> TensorCallBack;
  4. /*
  5.  * @brief run session.
  6.  * @param session   given session.
  7.  * @param before    callback before each op. return true to run the op; return false to skip the op.
  8.  * @param after     callback after each op. return true to continue running; return false to interrupt the session.
  9.  * @param sync      synchronously wait for finish of execution or not.
  10.  * @return result of running.
  11.  */
  12. ErrorCode runSessionWithCallBack(const Session* session, 
  13.                                  const TensorCallBack& before, 
  14.                                  const TensorCallBack& end,
  15.                                  bool sync = false) const;

Compared to runSession, runSessionWithCallback provides additional:

  • Callbacks before each op, which could be used to skip the execution;
  • Callback after each op, which could be used to interrupt the inference;
  • Synchronization option, defaults off; when enabled, all backends will wait for inference to complete, ie the function time cost is equal to the inference time cost;


Run Session with Flops

  1. class MNN_PUBLIC OperatorInfo {
  2.     struct Info;
  4. public:
  5.     /** Operator's name*/
  6.     const std::string& name() const;
  8.     /** Operator's type*/
  9.     const std::string& type() const;
  11.     /** Operator's flops, in M*/
  12.     float flops() const;
  14. protected:
  15.     OperatorInfo();
  16.     ~OperatorInfo();
  17.     Info* mContent;
  18. };
  19. typedef std::function<bool(const std::vector<Tensor*>&, const OperatorInfo*)> TensorCallBackWithInfo;
  21. /*
  22.  * @brief run session.
  23.  * @param session   given session.
  24.  * @param before    callback before each op. return true to run the op; return false to skip the op.
  25.  * @param after     callback after each op. return true to continue running; return false to interrupt the session.
  26.  * @param sync      synchronously wait for finish of execution or not.
  27.  * @return result of running.
  28.  */
  29. ErrorCode runSessionWithCallBackInfo(const Session* session, 
  30.                                      const TensorCallBackWithInfo& before,
  31.                                      const TensorCallBackWithInfo& end, 
  32.                                      bool sync = false) const;

In general, runSessionWithCallbackInfo is only used when evaluating the amount of computation. Compared to runSessionWithCallback, the Op type and the calculation amount information are added during the callback.