川剧是中国最知名的戏曲剧种之一, 变脸是川剧表演的特技之一, 在对象传承上有着严格的师门派别. 有点扯远啦, 回来! 其实主要是我们今天要用 OpenCV 干的事情跟这个有点关系, OpenCV 基于 Landmark 实现人脸关键点提取, 对结果善加利用可以实现人脸交换, 对特定对象施加变脸术. OpenCV 开发者不学川剧也一样可以给各种人变脸, 当然前提是会写代码, 会做 OpenCV. 首先简单说一下原理与流程.
一: 原理与流程
基本原理是利用 OpenCV 的级联检测器实现人脸检测然后基于 Landmak 的 LBF 模型实现人脸 68 个关键点提取, 基于关键点数据实现三角剖分与维诺图计算, 经过几何变换之后得到 mask 区域, 再利用 OpenCV 无缝克隆算法相关 API 实现换脸. 整个工作流程如下:
二: 代码实现
1. 创建 Landmark 关键点检测器
- // 人脸检测与 Landmark68 个关键点检测
- CascadeClassifier face_cascade;
- face_cascade.load(cascade_name);
- FacemarkLBF::Params params;
- params.n_landmarks = 68; // 68 个标注点
- params.initShape_n = 10;
- params.stages_n = 5; // 算法的 5 个强化步骤
- params.tree_n = 6; // 模型中每个标注点结构树 数目
- params.tree_depth = 5; // 决策树深度
- // 创建 LBF landmark 检测器
- PtrFacemarkLBF> facemark = FacemarkLBF::create(params);
- facemark->setFaceDetector((FN_FaceDetector)myDetector, &face_cascade);
- facemark->loadModel(modelfile_name);
- cout 'Loaded model' endl;
2.Landmark 关键点检测
- //vector to store the faces detected in the image
- vectorRect> faces1, faces2;
- vector vectorPoint2f>> shape1, shape2;
- //Detect faces in the current image
- float ratio1 = (float)img1.cols / (float)img1.rows;
- float ratio2 = (float)img2.cols / (float)img2.rows;
- resize(img1, img1, Size((int)(640 * ratio1), (int)(640 * ratio1)), 0, 0, INTER_LINEAR_EXACT);
- resize(img2, img2, Size((int)(640 * ratio2), (int)(640 * ratio2)), 0, 0, INTER_LINEAR_EXACT);
- Mat img1Warped = img2.clone();
- facemark->getFaces(img1, faces1);
- facemark->getFaces(img2, faces2);
- //Initialise the shape of the faces
- facemark->fit(img1, faces1, shape1);
- facemark->fit(img2, faces2, shape2);
3. 三角剖分变换
- vectorPoint2f> points1 = shape1[z];
- vectorPoint2f> points2 = shape2[z];
- img1.convertTo(img1, CV_32F);
- img1Warped.convertTo(img1Warped, CV_32F);
- // Find convex hull
- vectorPoint2f> boundary_image1;
- vectorPoint2f> boundary_image2;
- vector index;
- convexHull(Mat(points2), index, false, false);
- for (size_t i = 0; i index.size(); i++)
- {
- boundary_image1.push_back(points1[index[i]]);
- boundary_image2.push_back(points2[index[i]]);
- }
- // Triangulation for points on the convex hull
- vector vector> triangles;
- Rect rect(0, 0, img1Warped.cols, img1Warped.rows);
- divideIntoTriangles(rect, boundary_image2, triangles);
- // Apply affine transformation to Delaunay triangles
- for (size_t i = 0; i triangles.size(); i++)
- {
- vectorPoint2f> triangle1, triangle2;
- // Get points for img1, img2 corresponding to the triangles
- for (int j = 0; j 3; j++)
- {
- triangle1.push_back(boundary_image1[triangles[i][j]]);
- triangle2.push_back(boundary_image2[triangles[i][j]]);
- }
- warpTriangle(img1, img1Warped, triangle1, triangle2);
- }
4. 计算与模板生成
- // 计算与生成模板
- vectorPoint> hull;
- for (size_t i = 0; i boundary_image2.size(); i++)
- {
- Point pt((int)boundary_image2[i].x, (int)boundary_image2[i].y);
- hull.push_back(pt);
- }
- Mat mask = Mat::zeros(img2.rows, img2.cols, img2.depth());
- fillConvexPoly(mask, &hull[0], (int)hull.size(), Scalar(255, 255, 255));
5. 无缝克隆
- // 无缝克隆
- Rect r = boundingRect(boundary_image2);
- Point center = (r.tl() + r.br()) / 2;
- Mat output;
- img1Warped.convertTo(img1Warped, CV_8UC3);
- seamlessClone(img1Warped, img2, mask, center, output, NORMAL_CLONE);
- imshow('Face_Swapped', output);
- imwrite('D:/face_swap_demo.png', output);
三: 运行效果
原图一
原图二
人脸交换结果
关注公众号, 发送 "源码" 两字获取下载地址
来源: http://www.92to.com/bangong/2018/05-02/33705630.html