Analyze the Image Data

分析图像数据

This lesson teaches you to

本节课您将学习

  1. Initialize Google Cloud Vision

  2. 初始化 Google Cloud Vision

  3. Annotate a captured image

  4. 标记拍摄的图像

Try it out

试一试

A smart doorbell should be able to automatically extract useful data from an image before sending it to the companion app.

作为一个智能的门铃在发送数据到配套的应用之前应该能从图像中自动提取有用的数据。

Useful data could include how many people are at your door (perhaps zero…it’s a prank!) and their emotional state. In this lesson, you will leverage the power of Google’s Cloud Vision API to do image analysis.

有用的数据应该包括几个人在门前（或许没人…可能是个恶作剧！）和他们对应的情绪状态。这节课中，您将利用强大 Google’s Cloud Vision API 去进行图像分析。

Set up the Cloud Vision API

配置 Cloud Vision API

To enable Google Cloud Vision for your project:

在您的工程使用 Google Cloud Vision：

  1. Create a Google Cloud project and enable the API, as described in the Cloud Vision Quickstart guide.

  2. 创建一个 Google Cloud 工程以便使用 API，查看 Cloud Vision 快速入门 指南的介绍。

  3. Generate a new Android API key for your project as described in the Authenticating to a Cloud API Service guide.

  4. 根据 验证 Cloud API 服务 指南来为您的工程生成一个新的 Android API 密钥。

  5. Add the Cloud Vision Java client library dependencies to your app-level build.gradle file:

  6. 在您应用的 build.gradle 文件中添加 Cloud Vision Java 客户端的库依赖：

  1.   dependencies {
  2.     ...
  4.     compile 'com.google.api-client:google-api-client-android:1.22.0' exclude module: 'httpclient'
  5.     compile 'com.google.http-client:google-http-client-gson:1.22.0' exclude module: 'httpclient'
  7.     compile 'com.google.apis:google-api-services-vision:v1-rev22-1.22.0'
  8. }

  1. Add the required permissions to your app’s manifest file:

  2. 在应用的 manifest 文件中添加必要的权限：

  1.   <uses-permission android:name="android.permission.INTERNET" />

Upload the image for processing

上传图像并处理

The Cloud Vision API annotates image data with objects detected in the image, the coordinates of the discovered object, and a score indicating how confident the algorithm is in that object discovery.

Cloud Vision API 利用图像中检测到的对象，对象的坐标，以及对于对象的评分标记算法来标记图像数据。

To send the data to Cloud Vision for processing:

发送图像到 Cloud Vision 进行相关处理，您需要：

  1. Create a new VisionRequestInitializer with your cloud project’s API key.

  2. 使用您的云工程的 API 密钥创建一个 VisionRequestInitializer 对象。

  3. Construct a new Vision instance using the Vision.Builder and the proper HTTP and JSON instances for the Android platform.

  4. 使用 Vision.Builder 创建一个新的 Vision 实例并且为支持 Android 平台提供正确的 HTTP 和 JSON 实例。

Note: The Vision object represents the API endpoint and internally handles the HTTP transport and JSON parsing logic for each request and response.

注意: Vision 对象代表着 API 端对每次请求和响应中的 HTTP 传输和 JSON 解析等相关内部处理逻辑。

  1. Encode the image data into an Image instance. Pass that to an AnnotateImageRequest and activate the LABEL_DETECTION request feature.

  2. 将图像数据转码并传递给 Image 实例。然后将这个实例传递给一个 AnnotateImageRequest 实例并同时设置 LABEL_DETECTION 参数来请求相关特征。

  3. Execute the request as part of a BatchAnnotateImagesRequest and process the response. This is a blocking method call that will take some time to complete, depending on the network conditions.

  4. 使用 BatchAnnotateImagesRequest 执行请求并处理结果。这将是一个耗时的、依赖网络条件的阻塞请求。

  1. import com.google.api.client.extensions.android.http.AndroidHttp;
  2. import com.google.api.client.http.HttpTransport;
  3. import com.google.api.client.json.JsonFactory;
  4. import com.google.api.client.json.gson.GsonFactory;
  5. import com.google.api.services.vision.v1.Vision;
  6. import com.google.api.services.vision.v1.VisionRequestInitializer;
  7. import com.google.api.services.vision.v1.model.AnnotateImageRequest;
  8. import com.google.api.services.vision.v1.model.BatchAnnotateImagesRequest;
  9. import com.google.api.services.vision.v1.model.BatchAnnotateImagesResponse;
  10. import com.google.api.services.vision.v1.model.EntityAnnotation;
  11. import com.google.api.services.vision.v1.model.Feature;
  12. import com.google.api.services.vision.v1.model.Image;
  13. ...
  14. public class CloudVisionUtils {
  15. ...
  16. private static final String CLOUD_VISION_API_KEY = "...";
  17. ...
  19. public Map<String, Float> annotateImage(byte[] imageBytes) throws IOException {
  20.     // Construct the Vision API instance
  21.     HttpTransport httpTransport = AndroidHttp.newCompatibleTransport();
  22.     JsonFactory jsonFactory = GsonFactory.getDefaultInstance();
  23.     VisionRequestInitializer initializer = new VisionRequestInitializer(CLOUD_VISION_API_KEY);
  24.     Vision vision = new Vision.Builder(httpTransport, jsonFactory, null)
  25.             .setVisionRequestInitializer(initializer)
  26.             .build();
  28.     // Create the image request
  29.     AnnotateImageRequest imageRequest = new AnnotateImageRequest();
  30.     Image image = new Image();
  31.     image.encodeContent(imageBytes);
  32.     imageRequest.setImage(image);
  34.     // Add the features we want
  35.     Feature labelDetection = new Feature();
  36.     labelDetection.setType("LABEL_DETECTION");
  37.     labelDetection.setMaxResults(MAX_LABEL_RESULTS);
  38.     imageRequest.setFeatures(Collections.singletonList(labelDetection));
  40.     // Batch and execute the request
  41.     BatchAnnotateImagesRequest requestBatch = new BatchAnnotateImagesRequest();
  42.     requestBatch.setRequests(Collections.singletonList(imageRequest));
  43.     BatchAnnotateImagesResponse response = vision.images()
  44.             .annotate(requestBatch)
  45.             .setDisableGZipContent(true)
  46.             .execute();
  48.     return convertResponseToMap(response);
  49. }

The BatchAnnotateImagesResponse returned from the API wraps the annotation data in a few layers. You may wish to simplify the result by extracting the annotation labels and scores into a simpler collection.

API 所返回的 BatchAnnotateImagesResponse 将标记数据进行了若干层封装。您可能希望将注释标签和分数抽取到一个简单的集合来简化相关操作。

  1. private Map<String, Float> convertResponseToMap(BatchAnnotateImagesResponse response) {
  2.     Map<String, Float> annotations = new HashMap<>();
  4.     // Convert response into a readable collection of annotations
  5.     List<EntityAnnotation> labels = response.getResponses().get(0).getLabelAnnotations();
  6.     if (labels != null) {
  7.         for (EntityAnnotation label : labels) {
  8.             annotations.put(label.getDescription(), label.getScore());
  9.         }
  10.     }
  12.     return annotations;
  13. }

You can now upload the image data from the camera and examine the annotations returned by the Cloud Vision API.

现在您可以上传相机的图像数据然后使用 Cloud Vision API 检查返回的结果。

  1. public class DoorbellActivity extends Activity {
  2.     ...
  4.     private void onPictureTaken(final byte[] imageBytes) {
  5.         if (imageBytes != null) {
  6.             ...
  7.             try {
  8.                 // Process the image using Cloud Vision
  9.                 Map<String, Float> annotations = annotateImage(imageBytes);
  10.                 Log.d(TAG, "cloud vision annotations:" + annotations);
  11.             } catch (IOException e) {
  12.                 Log.e(TAG, "Cloud Vison API error: ", e);
  13.             }
  14.         }
  15.     }
  16. }