Smart Camera

The smart camera can learn and recognize brightly colored objects as well as detecting bar codes and lines, which allows it to be used in various application scenarios, such as garbage sorting, intelligent transportation, object tracking, and intelligent line following.

Connect to mBot or Halocode

Based on the wiring mode, the smart camera can be used as an RJ25 electronic module or mBuild electronic. After connecting it to mBot or Halocode, you can control it through mBot or Halocode.

Connect to mBot

After connecting the smart camera to mBot, you can power it with a 3.7V lithium battery or the mBuild power module.

Method 1 Power through the mBuild power module (recommended)

Method 2 Power through a 3.7V lithium battery

Connect to Halocode

Note: It is recommended to power the smart camera through the mBuild power module. Power only through the USB port may affect the proper use of the block.

Features

Color learning

The smart camera can learn brightly colored objects and identify the color blocks after learning, and then return their coordinates, length, and width.
Learn a colored object as follows:

1. Long press the Learn button until the indicator turns red (orange, yellow, green, blue, or purple, different colors indicate learning different objects), and then release the button.
2. Place the color block to be learned in front of the camera.
3. Observe the indicator on the front or back of the smart camera, and slowly move the object to be learned until the color of the indicator matches the object.
4. Short press the Learn button to record the current learned object. After the learning is successful, when the camera recognizes a learned object, the color of the indicator becomes the same as that of the learned object.

A maximum of seven objects can be learned and recorded in this mode.

After the learning is complete, you can use the following program to track the color block (color block 1).

Note
1. The brighter the color of the learned object, the more accurate the recognition.
Cautionary Example：learning a panda doll

2. The smart camera can learn and record seven objects, which should be easily distinguished in color.
Cautionary Example：learning a yellow ball and a yellow hat

Good example：learning an orange ball and a green ball

3. Switch the smart camera to the color block detection mode to detect the coordinates and size of the color block.
4. The smart camera can prevent light interference. However, changeable light can still affect recognition accuracy. The previous learning results may fail to be used in a new environment. Here are some solutions：

• Reset the white balance
• Turn on the LED before learning or recognizing
• Set the detail parameters in PixyMon

• Use a lampshade to cover the blocks

  Bar codes and lines recognizing

  The smart camera can detect bar codes, lines, and branch roads simultaneously without learning.
  Switch the smart camera to the line/label tracking mode to detect and return the coordinate information of bar codes, lines, and branch roads.
  Bar codes recognizing
  Bar code cards and stickers are included in the package. Click to download the bar code cards.
  You can put the cards on the line-follower map.
  
  Lines and branch roads recognizing
  The smart camera can detect the lines and the number of branch roads, and return the coordinates, directions, and the number of branch roads.
  
  Note
  1. You can set the line tracking mode to dark line on light or light line on dark in the program.
  2. The smart camera will filter out the narrow lines by default. Check PixyMon’s user guide to change the default settings.

  PixyMon

  The smart camera supports PixyMon software. You can use PixyMon to view the camera screen, adjust smart camera functions, and fine-tune some parameters to explore more complex features.
  Follow the steps below to install and use PixyMon:
  1. Click the following links to download PixyMon.
  Download PixyMon for Windows
  Download PixyMon for Mac
  2. Double-click the installation package and follow the prompts to install it.
  3. After the installation is completed, open PixyMon.
  4. Connect Smart Camera to the computer via a USB cable.
  5. After you see the status bar showing connected as shown below, you can start exploring Smart Camara with PixyMon.
  
  Note: For the ease of use of the module, we haven’t developed many functions on the blocks. If you are interested in controlling the smart camera through the blocks to implement more features, use the mBlock 5 Extension Builder.
  mBlock Extension Builder documentation

  Parameters

• Dimensions: 48 × 48 (mm)

• Resolution: 640 x 480
• Field of view: 65.0 degrees
• Effective focal length: 4.65±5% mm
• Recognition speed: 60fps
• Recognition distance: 0.25–1.2 (m)
• Fall resistance: 1m
• Power supply: 3.7V lithium battery or 5V mBuild power module
• Power consumption range: 0.9–1.3W
• Operating temperature: -10℃–55℃

Example programs

Smart camera example programs

Example 1: Color block tracking 1 – color block position tracking

In this example, mBot moves by tracking the specified color block. The motors rotate based on the position of the color block in the image captured by the smart camera. They stop rotating when the color block is detected in the center of the image or no color block is detected.

Note that the way you install the smart camera affects the tracking of the color block and thus affects the movement of mBot. The following figure shows an example of how the smart camera can be installed.

Program details

Example 2: Color block tracking 2 – color block coordinate tracking

In this example, the position (defined as A) of the color block in the image captured is obtained, and the target position is defined as B. Motors rotate based on the difference between A and B, where the difference is calculated based on the coordinates of A and B. In this way, more accurate tracking is performed. The way mBot moves by tracking color block coordinates is similar to that it moves by tracking color block positions.

Program details

Example 3: Line following

The smart camera can recognize line segments. Motors rotate based on the differential speed calculated based on the deviation of the line segment from the center of the image captured.

In line-following mode, it’s recommended that you install the smart camera parallel to the line segment to be recognized.

Program details

Note:

• Kp: Sensitivity in making a turn. When the initial speed is higher or the turn is sharper, a greater Kp is required.
• target: initial speed
• offset: deviation from the lien to be followed. A positive value indicates a deviation to the left of the line, and a negative value indicates one to the right of the line.