Get Output Tensor

  1. /**
  2.  * @brief get output tensor for given name.
  3.  * @param session   given session.
  4.  * @param name      given name. if NULL, return first output.
  5.  * @return tensor if found, NULL otherwise.
  6.  */
  7. Tensor* getSessionOutput(const Session* session, const char* name);
  9. /**
  10.  * @brief get all output tensors.
  11.  * @param session   given session.
  12.  * @return all output tensors mapped with name.
  13.  */
  14. const std::map<std::string, Tensor*>& getSessionOutputAll(const Session* session) const;

Interpreter provides two ways to get the output Tensor: getSessionOutput for getting a single output tensor and getSessionOutputAll for getting the output tensor map.

When there is only one output tensor, you can pass NULL to get the tensor when calling getSessionOutput.

Read Data

  1. auto outputTensor = interpreter->getSessionOutput(session, NULL);
  2. auto score = outputTensor->host<float>()[0];
  3. auto index = outputTensor->host<float>()[1];
  4. // ...

The simplest way to read data is to read from host of Tensor directly, but this usage is limited to the CPU backend, other backends need to read the data through deviceid. On the other hand, users need to handle the differences between NC4HW4 and NHWC data layouts.

For non-CPU backends, or users who are not familiar with data layout, copy data interfaces should be used.

Copy Data

NCHW example：

  1. auto outputTensor = interpreter->getSessionOutput(session, NULL);
  2. auto nchwTensor = new Tensor(outputTensor, Tensor::CAFFE);
  3. outputTensor->copyToHostTensor(nchwTensor);
  4. auto score = nchwTensor->host<float>()[0];
  5. auto index = nchwTensor->host<float>()[1];
  6. // ...
  7. delete nchwTensor;

NC4HW4 example：

  1. auto outputTensor = interpreter->getSessionOutput(session, NULL);
  2. auto nc4hw4Tensor = new Tensor(outputTensor, Tensor::CAFFE_C4);
  3. outputTensor->copyToHostTensor(nc4hw4Tensor);
  4. auto score = nc4hw4Tensor->host<float>()[0];
  5. auto index = nc4hw4Tensor->host<float>()[1];
  6. // ...
  7. delete nc4hw4Tensor;

NHWC example：

  1. auto outputTensor = interpreter->getSessionOutput(session, NULL);
  2. auto nhwcTensor = new Tensor(outputTensor, Tensor::TENSORFLOW);
  3. outputTensor->copyToHostTensor(nhwcTensor);
  4. auto score = nhwcTensor->host<float>()[0];
  5. auto index = nhwcTensor->host<float>()[1];
  6. // ...
  7. delete nhwcTensor;

By copying the data in this way, the data layout of tensor created with new is only thing you need to pay attention to. copyToHostTensor is responsible for processing the conversion on the data layout (if needed) and the data copy between the backends (if needed).