证件识别
功能简介
近年来，随着图像处理和模式识别技术的成熟发展，尤其是OCR（光学字符识别) 技术被成功移植到证件行业上来，证件识别成为一门具有较大前景的新兴技术。身份证自动识别技术也就应运而生。
身份证识别，利用OCR（光学字符识别）技术，能将图片中的文字提取出来，并能通过淡化底纹自动分析身份证各栏位信息，从身份证图片中提取性别、籍贯、出生年月、身份证号等信息。因此它有效地解决了身份证信息的录人问题，能够快速，高效地处理数据, 提高了工作效率。
而且随着网络技术和数据库技术的发展，还可以将各部分的识别结果按照不同的需要进行数据管理，如：可以通过网络与公安机关的身份证信息数据库相结合，通过验证可以识别该身份证的真伪，同时完成其它的业务管理。
这使得信息的查询与管理越来越方便、快捷，为身份证自动识别技术的普及提供了前提条件，身份证自动识别技术可以技广泛应用于机场、海关、酒店登记、公民身份核查、暂住人口调查、罪犯追逃等环节中，也为如今实名制提供新兴的技术支持，具有很重要的现实意义。
技术流程
一个典型的身份证识别算法流程图如下：
图像预处理
降噪
滤波、光照处理增强（可选）
灰度拉伸二值化
由灰度图像变成二值图像倾斜校正
Hough变换、投影法
二值化：由于彩色图像所含信息量过于巨大，在对图像中印刷体字符进行识别处理前，需要对图像进行二值化处理，使图像只包含黑色的前景信息和白色的背景信息，提升识别处理的效率和精确度。
图像降噪：由于待识别图像的品质受限于输入设备、环境、以及文档的印刷质量，在对图像中印刷体字符进行识别处理前，需要根据噪声的特征对待识别图像进行去噪处理，提升识别处理的精确度。
倾斜校正：由于扫描和拍摄过程涉及人工操作，输入计算机的待识别图像或多或少都会存在一些倾斜，在对图像中印刷体字符进行识别处理前，就需要进行图像方向检测，并校正图像方向。
图像分割
行分割
身份证图像字符信息分布规则，每行有一定间隙；采用水平投影法进行图像分割
字符分割
垂直投影
字符识别
模板匹配法
对每个字符建立一个标准模板，进行图形匹配、笔画匹配、几何特征匹配。
特点：实现简单，图像质量要求高，计算速度慢，相似字符识别率低人工神经网络字符识别算法
artificial neural network，简称神经网络（neural network），是一种模仿生物神经网络的结构和功能的数学模型或计算模型。
识别结果处理
对各文字识别结果进行后处理纠错
身份证号码验证
①1-2 省级行政区代码②3-4 地级行政区划分代码③5-6 县区行政区分代码④7-14 出生年月日⑤15-17 顺序码，同一地区同年同月同日出生人的编号，奇数是男性，偶数是女性⑥18 校验码，如果是0-9则用0-9表示，如果是10则用X（罗马数字10）表示
- 有效期验证
5年，10年，20年，长期
功能实现
下图为身份证识别系统的系统框图：
图像二值化处理
打开一副图片，也即获取一副彩色身份证图片后，为使得身份证号码颜色与背景色呈现较大差别，故选取R分量作为彩色图像的灰度化，接着对图像进行二值化处理。这里需要获取图像的全局阈值和局部阈值。
首先由Otsu算法（opencv已实现）获得整幅图像的全局阈值T，再由Beresen方法计算得当前像素的领域窗口内的灰度均值Tbn，再利用整个图像的最大灰度值和最小灰度值计算得到一个矫正因素b，则二值化公式可由下式子确定
其中T为Ostu全局阈值，Tbn由下式确定：
b由下式确定：
其中g2为图像中灰度的最大值，g1为图像中灰度的最小值，C为经验系数，通常取0.12算法在函数OstuBeresenThreshold实现，实现的效果如下所示：
身份证号码定位
将二值化得到的图像进行黑白反色处理，即背景为黑色，身份证号码为白色，接着对图像进行闭操作后，使用findContours检测二值图像中的白色像素块的外轮廓，将符合长宽比及面积要求的轮廓提取出来。要找到的轮廓如下所示：
要做汉字识别的话，可以在这部分将汉字区域定位并剪切出来
号码分割
观察到所获得的裁剪出来的身份证号码图像（已经缩放至300*20分辨率大小）如下所示：
在该图像中，显然身份证号码与背景图像颜色区分度高，故进反色后，使用Otsu方法二值化后可得
要将号码分割出来，则只需进行列分割，也即统计
其中f(x,y)为介于两个字符之间的像素
特征提取
提取数字字符的特征向量，也即提取梯度分布特征+灰度分布特征+水平投影直方图+垂直投影直方图，最后每个字符得到一个1*72的特征向量，由detectTextArea函数实现
神经网络训练
所使用的训练图片均由从多张身份证图片上分割得到，之后经过特征提取，获得训练矩阵和标签矩阵保存于ann.xml文件中，由函数getCharText实现，然后由ann_train从中读取训练矩阵和标签矩阵用于神经网络训练，opencv中实现的是多层感知器神经网络。同理，也可以实现SVM模型。
分类器分类
使用训练得的神经网络对所提取的字符特征向量进行分类
校验位计算
由于最后一位有时识别率不高，可能是最后一位的分割结果不是很好，故最后1位校验位直接由前17位数字计算得，由verifySizes函数实现。
结果显示
该身份证号码识别系统所处理的图像必须要求身份证区域尽可能占整幅图像更多的区域，且在纯色背景下拍摄，另外需保证拍摄得的身份证图像尽可能光照均匀，不能有高光存在。
核心代码
ANN
package com.ruoyi.ai.aiModels.idcard.network;
import java.nio.file.Paths;
import java.util.HashMap;
import java.util.Map;
import java.util.Vector;
import com.ruoyi.ai.aiModels.idcard.core.IdCardCoreFunc;
import com.ruoyi.ai.aiModels.idcard.core.FileDeal;
import org.opencv.core.Core;
import org.opencv.core.CvType;
import org.opencv.core.Mat;
import org.opencv.core.TermCriteria;
import org.opencv.imgcodecs.Imgcodecs;
import org.opencv.ml.ANN_MLP;
import org.opencv.ml.Ml;
public class CHAR_ANN {
//public static String trainImages = "res/data/chars2";
public static String trainImages = Paths.get("build","idcard","data","chars2").toAbsolutePath().toString();
//public static String annXml = "res/model/ann.xml";
public static String annXml = Paths.get("build","idcard","model","ann.xml").toAbsolutePath().toString();
private ANN_MLP ann;
private int sizeData = 10;
// 中国车牌
private final char strCharacters[] = { '0', '1', '2', '3', '4', '5', '6', '7', '8', '9', 'X'/*, 'A', 'B', 'C', 'D', 'E',
'F', 'G', 'H', 没有I
'J', 'K', 'L', 'M', 'N', 没有O 'P', 'Q', 'R', 'S', 'T', 'U', 'V', 'W', 'X', 'Y', 'Z'*/ };
private final int numCharacter = 11; /* 没有I和0,10个数字与24个英文字符之和 */
public void annTrain(Mat trainData, Mat classes) {
Mat layerSize=new Mat(1,4,CvType.CV_32SC1);
layerSize.put(0,0, trainData.cols());
layerSize.put(0,1, 40);
layerSize.put(0,2, 20);
layerSize.put(0,3, numCharacter);
getAnn().setLayerSizes(layerSize);
getAnn().setTrainMethod(ANN_MLP.BACKPROP);
/* getAnn().setBackpropWeightScale(1);
getAnn().setBackpropMomentumScale(1);*/
TermCriteria criteria=new TermCriteria(TermCriteria.MAX_ITER|TermCriteria.EPS, 300, 0.001);
getAnn().setTermCriteria(criteria);
getAnn().setActivationFunction(ANN_MLP.SIGMOID_SYM/*,1,1*/);
Mat trainClasses = new Mat(trainData.rows(), numCharacter, CvType.CV_32FC1);
for (int i = 0; i < trainClasses.rows(); i++) {
for (int k = 0; k < trainClasses.cols(); k++) {
if (k == classes.get(0,i)[0])
trainClasses.put(i, k,(float)1.0);
else
trainClasses.put(i, k,(float)0.0);
}
}
boolean r=getAnn().train(trainData, Ml.ROW_SAMPLE, trainClasses);
System.out.println("r:"+r);
getAnn().save(annXml);
//System.out.println("training result: " + success);
}
public Map<String,Mat> saveTrainData() {
Map<String,Mat> result = new HashMap<String,Mat>();
Mat classes = new Mat();
Mat trainingData = new Mat();
Vector<Integer> trainingLabels = new Vector<Integer>();
String path = trainImages;
//int count = 1;
for (int i = 0; i < numCharacter; i++) {
String str = path + '/' + strCharacters[i];
Vector<String> files = new Vector<String>();
FileDeal.getFiles(str, files);
int size = (int) files.size();
for (int j = 0; j < size; j++) {
Mat img = Imgcodecs.imread(files.get(j), 0);
Mat f10 = IdCardCoreFunc.features(img, sizeData);
trainingData.push_back(f10);
trainingLabels.add(i); // 每一幅字符图片所对应的字符类别索引下标
/*System.out.println(count);
count++;*/
}
}
trainingData.convertTo(trainingData, CvType.CV_32FC1);
Mat classTemMat = new Mat(1,trainingLabels.size(),CvType.CV_32FC1);
for (int i = 0; i < trainingLabels.size(); ++i){
classTemMat.put(0, i, trainingLabels.get(i).intValue());
}
classTemMat.copyTo(classes);
result.put("trainingData", trainingData);
result.put("classes", classes);
// System.out.println("End saveTrainData");
return result;
}
public void saveModel( Map<String, Mat> dataMap) {
Mat trainingData =dataMap.get("trainingData");
Mat classes = dataMap.get("classes");
annTrain(trainingData, classes);
}
public static void main(String[] args){
System.loadLibrary(Core.NATIVE_LIBRARY_NAME);
CHAR_ANN annTrain = new CHAR_ANN();
StringBuffer idcar = new StringBuffer();
for (int i = 0; i <= 17; i++) {
//Mat img = Imgcodecs.imread("C:/Users/Administrator/Desktop/tt/test/debug_specMat"+i+".jpg");
Mat img = Imgcodecs.imread(Paths.get("build","idcard","test").toAbsolutePath().toString()+"/debug_specMat"+i+".jpg");
String charText = annTrain.annFind(img);
idcar.append(charText);
}
System.out.println("idcar:\n" + idcar.toString());
}
public String annFind(Mat charMat) {
if(!getAnn().isTrained()){
System.out.println("train....");
Map<String, Mat> resultMap = saveTrainData();
saveModel(resultMap);
}
Mat f = IdCardCoreFunc.features(charMat, sizeData);
//find the nearest neighbours of test data
float result = getAnn().predict(f);
String charText = String.valueOf(strCharacters[(int) result]);
return charText;
}
public ANN_MLP getAnn() {
if(ann==null){
ann =ANN_MLP.create();
}
return ann;
}
public void setAnn(ANN_MLP ann) {
this.ann = ann;
}
}
KNN
package com.ruoyi.ai.aiModels.idcard.network;
import java.nio.file.Paths;
import java.util.HashMap;
import java.util.Map;
import java.util.Vector;
import com.ruoyi.ai.aiModels.idcard.core.IdCardCoreFunc;
import com.ruoyi.ai.aiModels.idcard.core.FileDeal;
import org.opencv.core.Core;
import org.opencv.core.CvType;
import org.opencv.core.Mat;
import org.opencv.imgcodecs.Imgcodecs;
import org.opencv.ml.KNearest;
import org.opencv.ml.Ml;
public class CHAR_KNN {
public static final int K = 5;
public static String trainImages = Paths.get("build","idcard","data","chars2").toAbsolutePath().toString();
private KNearest knn ;
private int sizeData = 10;
// 中国车牌
private final char strCharacters[] = { '0', '1', '2', '3', '4', '5', '6', '7', '8', '9', 'X'/*, 'A', 'B', 'C', 'D', 'E',
'F', 'G', 'H', 没有I
'J', 'K', 'L', 'M', 'N', 没有O 'P', 'Q', 'R', 'S', 'T', 'U', 'V', 'W', 'X', 'Y', 'Z'*/ };
private final int numCharacter = 11; /* 没有I和0,10个数字与24个英文字符之和 */
public void knnTrain(Mat TrainData, Mat classes) {
boolean success = getKnn().train(TrainData, Ml.ROW_SAMPLE,classes);
// getKnn().save(knnXml);
//System.out.println("training result: " + success);
}
public Map<String,Mat> saveTrainData() {
Map<String,Mat> result = new HashMap<String,Mat>();
Mat classes = new Mat();
Mat trainingData = new Mat();
Vector<Integer> trainingLabels = new Vector<Integer>();
String path = trainImages;
for (int i = 0; i < numCharacter; i++) {
String str = path + '/' + strCharacters[i];
Vector<String> files = new Vector<String>();
FileDeal.getFiles(str, files);
int size = (int) files.size();
for (int j = 0; j < size; j++) {
Mat img = Imgcodecs.imread(files.get(j), 0);
Mat f10 = IdCardCoreFunc.features(img, sizeData);
trainingData.push_back(f10);
trainingLabels.add(i); // 每一幅字符图片所对应的字符类别索引下标
}
}
trainingData.convertTo(trainingData, CvType.CV_32FC1);
Mat classTemMat = new Mat(1,trainingLabels.size(),CvType.CV_32FC1);
for (int i = 0; i < trainingLabels.size(); ++i){
classTemMat.put(0, i, trainingLabels.get(i).intValue());
}
classTemMat.copyTo(classes);
result.put("TrainingData", trainingData);
result.put("classes", classes);
// System.out.println("End saveTrainData");
return result;
}
public void saveModel( Map<String, Mat> dataMap) {
String training = "TrainingData";
Mat TrainingData =dataMap.get(training);
Mat Classes = dataMap.get("classes");
knnTrain(TrainingData, Classes);
}
public static void main(String[] args){
System.loadLibrary(Core.NATIVE_LIBRARY_NAME);
CHAR_KNN knnTrain = new CHAR_KNN();
StringBuffer idcar = new StringBuffer();
for (int i = 0; i <= 17; i++) {
Mat img = Imgcodecs.imread(Paths.get("build","idcard","test").toAbsolutePath().toString()+"/debug_specMat"+i+".jpg");
String charText = knnTrain.knnFind(img);
idcar.append(charText);
}
System.out.println("idcar:\n" + idcar.toString());
}
public String knnFind(Mat charMat) {
if(!getKnn().isTrained()){
System.out.println("train....");
Map<String, Mat> resultMap = saveTrainData();
saveModel(resultMap);
}
Mat f = IdCardCoreFunc.features(charMat, 10);
//find the nearest neighbours of test data
Mat results = new Mat();
Mat neighborResponses = new Mat();
Mat dists = new Mat();
getKnn().findNearest(f, K, results, neighborResponses, dists);
String charText = String.valueOf(strCharacters[(int) results.get(0,0)[0]]);
return charText;
}
public KNearest getKnn() {
if(knn==null){
knn = KNearest.create();
}
return knn;
}
public void setKnn(KNearest knn) {
this.knn = knn;
}
}
SVM
package com.ruoyi.ai.aiModels.idcard.network;
import java.nio.file.Paths;
import java.util.HashMap;
import java.util.Map;
import java.util.Vector;
import com.ruoyi.ai.aiModels.idcard.core.IdCardCoreFunc;
import com.ruoyi.ai.aiModels.idcard.core.FileDeal;
import org.opencv.core.Core;
import org.opencv.core.CvType;
import org.opencv.core.Mat;
import org.opencv.core.TermCriteria;
import org.opencv.imgcodecs.Imgcodecs;
import org.opencv.ml.Ml;
import org.opencv.ml.SVM;
public class CHAR_SVM {
public static final int K = 5;
public static String trainImages = Paths.get("build","idcard","data","chars2").toAbsolutePath().toString();
public static String svmXml = Paths.get("build","idcard","model","svm.xml").toAbsolutePath().toString();
private SVM svm =null;
private int sizeData = 10;
// 中国车牌
private final char strCharacters[] = { '0', '1', '2', '3', '4', '5', '6', '7', '8', '9', 'X'/*, 'A', 'B', 'C', 'D', 'E',
'F', 'G', 'H', 没有I
'J', 'K', 'L', 'M', 'N', 没有O 'P', 'Q', 'R', 'S', 'T', 'U', 'V', 'W', 'X', 'Y', 'Z'*/ };
private final int numCharacter = 11; /* 没有I和0,10个数字与24个英文字符之和 */
public void svmTrain(Mat TrainData, Mat classes) {
TermCriteria criteria=new TermCriteria(TermCriteria.MAX_ITER, 100, 1e-6);
getSvm().setKernel(getSvm().LINEAR);
getSvm().setType(getSvm().C_SVC);
getSvm().setGamma(0.5);
getSvm().setNu(0.5);
//1-->97.77778%,2-->100%
getSvm().setC(2);
getSvm().setTermCriteria(criteria);
getSvm().train(TrainData, Ml.ROW_SAMPLE,classes);
getSvm().save(svmXml);
// getSvm().save(svmXml);
//System.out.println("training result: " + success);
}
public Map<String,Mat> saveTrainData() {
Map<String,Mat> result = new HashMap<String,Mat>();
Mat classes = new Mat();
Mat trainingData = new Mat();
Vector<Integer> trainingLabels = new Vector<Integer>();
String path = trainImages;
for (int i = 0; i < numCharacter; i++) {
String str = path + '/' + strCharacters[i];
Vector<String> files = new Vector<String>();
FileDeal.getFiles(str, files);
int size = (int) files.size();
for (int j = 0; j < size; j++) {
Mat img = Imgcodecs.imread(files.get(j), 0);
Mat f10 = IdCardCoreFunc.features(img, sizeData);
trainingData.push_back(f10);
trainingLabels.add(i); // 每一幅字符图片所对应的字符类别索引下标
}
}
trainingData.convertTo(trainingData, CvType.CV_32FC1);
Mat classTemMat = new Mat(1,trainingLabels.size(),CvType.CV_32SC1);//必须这个类型
for (int i = 0; i < trainingLabels.size(); ++i){
classTemMat.put(0, i, trainingLabels.get(i).intValue());
}
classTemMat.copyTo(classes);
result.put("TrainingData", trainingData);
result.put("classes", classes);
// System.out.println("End saveTrainData");
return result;
}
public void saveModel( Map<String, Mat> dataMap) {
String training = "TrainingData";
Mat TrainingData =dataMap.get(training);
Mat Classes = dataMap.get("classes");
svmTrain(TrainingData, Classes);
}
public static void main(String[] args){
System.loadLibrary(Core.NATIVE_LIBRARY_NAME);
CHAR_SVM svmTrain = new CHAR_SVM();
StringBuffer idcar = new StringBuffer();
for (int i = 0; i <= 1; i++) {
Mat img = Imgcodecs.imread(Paths.get("build","idcard","test").toAbsolutePath().toString()+"/debug_specMat"+i+".jpg");
String charText = svmTrain.svmFind(img);
idcar.append(charText);
}
System.out.println("idcar:\n" + idcar.toString());
}
public String svmFind(Mat charMat) {
if(!getSvm().isTrained()){
System.out.println("train....");
Map<String, Mat> resultMap = saveTrainData();
saveModel(resultMap);
}
Mat f = IdCardCoreFunc.features(charMat, 10);
Mat results = new Mat();
getSvm().predict(f, results, 0);
String charText = String.valueOf(strCharacters[(int) results.get(0,0)[0]]);
return charText;
}
public SVM getSvm() {
if(svm==null){
svm = SVM.create();
}
return svm;
}
public void setSvm(SVM svm) {
this.svm = svm;
}
}
Core
package com.ruoyi.ai.aiModels.idcard.core;
import java.util.Vector;
import org.opencv.core.Core;
import org.opencv.core.CvType;
import org.opencv.core.Mat;
import org.opencv.core.Rect;
import org.opencv.core.Size;
import org.opencv.imgproc.Imgproc;
/**
* 核心
*
*/
public class IdCardCoreFunc {
public enum Direction {
UNKNOWN, VERTICAL, HORIZONTAL
}
/**
* Get the Sobel Mat of input image!
*
* @param image
* The input image.
* @return The Sobel Mat image of input image.
*/
public static Mat Sobel(Mat image) {
if (image.empty()) {
System.out.println("Please check the input image!");
return image;
}
Mat gray = image.clone();
if (3 == gray.channels()) {
Imgproc.cvtColor(gray, gray, Imgproc.COLOR_BGR2GRAY);
}
Mat grad = new Mat();
Mat grad_x = new Mat();
Mat grad_y = new Mat();
Mat abs_grad_x = new Mat();
Mat abs_grad_y = new Mat();
final int scharr_scale = 1;
final int scharr_delta = 0;
final int scharr_ddpeth = CvType.CV_16S;
Imgproc.Sobel(gray, grad_x, scharr_ddpeth, 1, 0, 3, scharr_scale, scharr_delta, Core.BORDER_DEFAULT);
Imgproc.Sobel(gray, grad_y, scharr_ddpeth, 0, 1, 3, scharr_scale, scharr_delta, Core.BORDER_DEFAULT);
Core.convertScaleAbs(grad_x, abs_grad_x);
Core.convertScaleAbs(grad_y, abs_grad_y);
Core.addWeighted(abs_grad_x, 0.5, abs_grad_y, 0.5, 0, grad);
return grad;
}
public static double otsu(Mat image) {
int threshold=0;
double maxVariance = 0;
double w0=0,w1=0;//前景与背景像素点所占比例
double u0=0,u1=0;//前景与背景像素值平均灰度
double[] histogram=new double[256];
double Num=image.cols()*image.rows();
//统计256个bin，每个bin像素的个数
for(int i=0;i<image.rows();i++){
for(int j=0;j<image.cols();j++){
double record = image.get(i, j)[0];
histogram[(int)record]++; //cout<<"Histogram[data[i*image.step+j]]++:;"<<histogram[int(*p++)]++<<endl;
}
}
//前景像素统计
for(int i=0;i<255;i++){
w0=0;
w1=0;
u0=0;
u1=0;
for(int j=0;j<=i;j++){
w0=w0+histogram[j];//以i为阈值，统计前景像素个数
u0=u0+j*histogram[j];//以i为阈值，统计前景像素灰度总和
}
w0=w0/Num;u0=u0/w0;
//背景像素统计
for(int j=i+1;j<=255;j++){
w1=w1+histogram[j];//以i为阈值，统计前景像素个数
u1=u1+j*histogram[j];//以i为阈值，统计前景像素灰度总和
}
w1=w1/Num;u1=u1/w1;
double variance=w0*w1*(u1-u0)*(u1-u0); //当前类间方差计算
if(variance > maxVariance)
{
maxVariance = variance;
threshold = i;
}
}
//cout<<"threshold:"<<threshold<<endl;
return threshold;
}
public static float[] projectedHistogram(final Mat img, Direction direction) {
int sz = 0;
switch (direction) {
case HORIZONTAL:
sz = img.rows();
break;
case VERTICAL:
sz = img.cols();
break;
default:
break;
}
// 统计这一行或一列中，非零元素的个数，并保存到nonZeroMat中
float[] nonZeroMat = new float[sz];
Core.extractChannel(img, img, 0);
for (int j = 0; j < sz; j++) {
Mat data = (direction == Direction.HORIZONTAL) ? img.row(j) : img.col(j);
int count = Core.countNonZero(data);
nonZeroMat[j] = count;
}
// Normalize histogram
float max = 0;
for (int j = 0; j < nonZeroMat.length; ++j) {
max = Math.max(max, nonZeroMat[j]);
}
if (max > 0) {
for (int j = 0; j < nonZeroMat.length; ++j) {
nonZeroMat[j] /= max;
}
}
return nonZeroMat;
}
/**
* 将Rect按位置从左到右进行排序
*
* @param vecRect
* @return
*/
public static Vector<Rect> SortRect(final Vector<Rect> vecRect) {
Vector<Rect> out = new Vector<Rect>();
Vector<Integer> orderIndex = new Vector<Integer>();
Vector<Integer> xpositions = new Vector<Integer>();
for (int i = 0; i < vecRect.size(); ++i) {
orderIndex.add(i);
xpositions.add(vecRect.get(i).x);
}
float min = xpositions.get(0);
int minIdx;
for (int i = 0; i < xpositions.size(); ++i) {
min = xpositions.get(i);
minIdx = i;
for (int j = i; j < xpositions.size(); ++j) {
if (xpositions.get(j) < min) {
min = xpositions.get(j);
minIdx = j;
}
}
int aux_i = orderIndex.get(i);
int aux_min = orderIndex.get(minIdx);
orderIndex.remove(i);
orderIndex.insertElementAt(aux_min, i);
orderIndex.remove(minIdx);
orderIndex.insertElementAt(aux_i, minIdx);
float aux_xi = xpositions.get(i);
float aux_xmin = xpositions.get(minIdx);
xpositions.remove(i);
xpositions.insertElementAt((int) aux_xmin, i);
xpositions.remove(minIdx);
xpositions.insertElementAt((int) aux_xi, minIdx);
}
for (int i = 0; i < orderIndex.size(); i++)
out.add(vecRect.get(orderIndex.get(i)));
return out;
}
/**
* Assign values to feature
* <p>
* 样本特征为水平、垂直直方图和低分辨率图像所组成的矢量
*
* @param in
* @param sizeData - 低分辨率图像size = sizeData*sizeData, 可以为0
* @return
*/
public static Mat features(final Mat in, final int sizeData) {
float[] vhist = projectedHistogram(in, Direction.VERTICAL);
float[] hhist = projectedHistogram(in, Direction.HORIZONTAL);
Mat lowData = new Mat();
if (sizeData > 0) {
Imgproc.resize(in, lowData, new Size(sizeData, sizeData));
}
int numCols = vhist.length + hhist.length + lowData.cols() * lowData.rows();
Mat out = Mat.zeros(1, numCols, CvType.CV_32F);
//FloatIndexer idx = out.createIndexer();
int j = 0;
for (int i = 0; i < vhist.length; ++i, ++j) {
out.put(0, j, vhist[i]);
// idx.put(0, j, vhist[i]);
}
for (int i = 0; i < hhist.length; ++i, ++j) {
out.put(0, j, hhist[i]);
// idx.put(0, j, hhist[i]);
}
for (int x = 0; x < lowData.cols(); x++) {
for (int y = 0; y < lowData.rows(); y++, ++j) {
// float val = lowData.ptr(x, y).get() & 0xFF;
float val = (float)(lowData.get(x, y)[0]);
// idx.put(0, j, val);
out.put(0, j, val);
}
}
return out;
}
}
Util
package com.ruoyi.ai.utils;
import java.nio.file.Paths;
import java.util.ArrayList;
import java.util.Map;
import java.util.Vector;
import com.ruoyi.ai.aiModels.idcard.network.CHAR_SVM;
import com.ruoyi.ai.aiModels.idcard.core.IdCardCoreFunc;
import com.ruoyi.common.utils.StringUtils;
import org.opencv.core.Core;
import org.opencv.core.CvType;
import org.opencv.core.Mat;
import org.opencv.core.MatOfPoint;
import org.opencv.core.Rect;
import org.opencv.core.Scalar;
import org.opencv.core.Size;
import org.opencv.imgcodecs.Imgcodecs;
import org.opencv.imgproc.Imgproc;
import org.opencv.ml.SVM;
public class IdCardCvUtil {
public static CHAR_SVM svmTrain;
//public static String svmXml = "res/model/svm.xml";
public static String svmXml = Paths.get("build","idcard","model","svm.xml").toAbsolutePath().toString();
public static void getIdNum(Map<String, Object> map, String filename) {
System.loadLibrary(Core.NATIVE_LIBRARY_NAME);
String text = IdCardCvUtil.getIdCardCode(filename);
if(StringUtils.isEmpty(text)||StringUtils.isNull(text)){
map.put("error", "识别失败");
return;
}
map.put("idNum", text);
}
public static String getIdCardCode(String imagePath) {
Mat rgbMat = Imgcodecs.imread(imagePath); // 原图
Rect rect = detectTextArea(rgbMat);
String text = getCharText(rgbMat, rect);
return text;
}
private static String getCharText(Mat srcMat, Rect rect) {
if (svmTrain == null) {
svmTrain = new CHAR_SVM();
SVM svm = SVM.load(svmXml);
svmTrain.setSvm(svm);
}
Mat effective = new Mat(); // 身份证位置
Mat charsGrayMat = new Mat();
Mat hierarchy = new Mat();
srcMat.copyTo(effective);
Mat charsMat = new Mat(effective, rect);
Imgproc.cvtColor(charsMat, charsGrayMat, Imgproc.COLOR_RGB2GRAY);// 灰度化
Imgproc.GaussianBlur(charsGrayMat, charsGrayMat, new Size(3, 3), 0, 0, Core.BORDER_DEFAULT);
double thresholdValue = IdCardCoreFunc.otsu(charsGrayMat) - 25;//减50是去掉文字周边燥点 阴影覆盖;
Imgproc.threshold(charsGrayMat, charsGrayMat, thresholdValue, 255, Imgproc.THRESH_BINARY_INV);
Imgproc.medianBlur(charsGrayMat, charsGrayMat, 3);
//Imgcodecs.imwrite("temp/charsMat.jpg", charsGrayMat);
Imgcodecs.imwrite(Paths.get("build","idcard","test").toAbsolutePath().toString()+"/charsMat.jpg", charsGrayMat);
ArrayList<MatOfPoint> charContours = new ArrayList<>();
Imgproc.findContours(charsGrayMat, charContours, hierarchy, Imgproc.RETR_EXTERNAL, Imgproc.CHAIN_APPROX_NONE);
Vector<Rect> vecRect = new Vector<Rect>();
for (int k = 0; k < charContours.size(); k++) {
Rect mr = Imgproc.boundingRect(charContours.get(k));
if (verifySizes(mr)) {
vecRect.add(mr);
}
}
Vector<Rect> sortedRect = IdCardCoreFunc.SortRect(vecRect);
int x = 0;
StringBuffer idcar = new StringBuffer();
for (Rect rectSor : sortedRect) {
Mat specMat = new Mat(charsGrayMat, rectSor);
specMat = preprocessChar(specMat);
//Imgcodecs.imwrite("temp/debug_specMat" + x + ".jpg", specMat);
Imgcodecs.imwrite(Paths.get("build","idcard","test").toAbsolutePath().toString()+"/debug_specMat" + x + ".jpg", specMat);
x++;
String charText = svmTrain.svmFind(specMat);
idcar.append(charText);
}
return idcar.toString();
}
private static Rect detectTextArea(Mat srcMat) {
Mat grayMat = new Mat(); // 灰度图
Imgproc.cvtColor(srcMat, grayMat, Imgproc.COLOR_RGB2GRAY);// 灰度化
Imgproc.GaussianBlur(grayMat, grayMat, new Size(3, 3), 0, 0, Core.BORDER_DEFAULT);
grayMat = IdCardCoreFunc.Sobel(grayMat);
//Imgcodecs.imwrite("temp/Sobel.jpg", grayMat);
Imgcodecs.imwrite(Paths.get("build","idcard","test").toAbsolutePath().toString()+"/Sobel.jpg", grayMat);
Imgproc.threshold(grayMat, grayMat, 0, 255, Imgproc.THRESH_OTSU + Imgproc.THRESH_BINARY);
Imgproc.medianBlur(grayMat, grayMat, 3);
//Imgcodecs.imwrite("temp/grayMat.jpg", grayMat);
Imgcodecs.imwrite(Paths.get("build","idcard","test").toAbsolutePath().toString()+"/grayMat.jpg", grayMat);
// 使用闭操作。对图像进行闭操作以后，可以看到车牌区域被连接成一个矩形装的区域。
Rect rect = null;
for (int step = 20; step < 60;) {
//System.out.println(step);
Mat element = Imgproc.getStructuringElement(Imgproc.MORPH_RECT, new Size(step, 1));
Imgproc.morphologyEx(grayMat, grayMat, Imgproc.MORPH_CLOSE, element);
//Imgcodecs.imwrite("temp/MORPH_CLOSE.jpg", grayMat);
Imgcodecs.imwrite(Paths.get("build","idcard","test").toAbsolutePath().toString()+"/MORPH_CLOSE.jpg", grayMat);
/**
* 轮廓提取()
*/
ArrayList<MatOfPoint> contoursList = new ArrayList<>();
Mat hierarchy = new Mat();
Imgproc.findContours(grayMat, contoursList, hierarchy,
Imgproc.RETR_EXTERNAL, Imgproc.CHAIN_APPROX_SIMPLE);
for (int i = 0; i < contoursList.size(); i++) {
Rect tempRect = Imgproc.boundingRect(contoursList.get(i));
if(grayMat.height()/tempRect.height<5){
continue;
}
int r = tempRect.width / tempRect.height;
if (r < 1){
r = tempRect.height/tempRect.width;
}
if (tempRect.width>10 && tempRect.height>10 &&
grayMat.width()!=tempRect.width && r >10 && r<20) {
if (rect == null) {
rect = tempRect;
continue;
}
if (tempRect.y > rect.y) {
rect = tempRect;
}
}
}
if (rect != null) {
//Imgcodecs.imwrite("temp/rectMat.jpg", new Mat(grayMat,rect));
Imgcodecs.imwrite(Paths.get("build","idcard","test").toAbsolutePath().toString()+"/rectMat.jpg", new Mat(grayMat,rect));
return rect;
}
step = step + 5;
}
return rect;
}
/**
* 字符预处理: 统一每个字符的大小
*
* @param in
* @return
*/
private static Mat preprocessChar(Mat in) {
int h = in.rows();
int w = in.cols();
int charSize = 20;
Mat transformMat = Mat.eye(2, 3, CvType.CV_32F);
int m = Math.max(w, h);
transformMat.put(0, 2, ((m - w) / 2f));
transformMat.put(1, 2, ((m - h) / 2f));
Mat warpImage = new Mat(m, m, in.type());
Imgproc.warpAffine(in, warpImage, transformMat, warpImage.size(), Imgproc.INTER_LINEAR, Core.BORDER_CONSTANT,
new Scalar(0));
Mat out = new Mat();
Imgproc.resize(warpImage, out, new Size(charSize, charSize));
return out;
}
private static boolean verifySizes(Rect mr) {
if (mr.size().height < 5 || mr.size().width < 5) {
return false;
}
return true;
}
public static void main(String[] args) throws Exception {
System.loadLibrary(Core.NATIVE_LIBRARY_NAME);
//String text = IdCardCvUtils.getIdCardCode("res/test/xx4.jpg");
String text = IdCardCvUtil.getIdCardCode(Paths.get("build","dataset","idcard","1.jpg").toAbsolutePath().toString());
System.out.println(text);
}
}
若有收获，就点个赞吧
0 人点赞
