评论
打印
Hey Joss, I have a question for you.
乔斯，我有个问题要问你。
Do you know how these Snapchat filters work? like behind the scenes?
你知道Snapchat的这些滤镜是怎么回事吗?就是背后的工作原理那种?
Hmm, I have no idea.
嗯……我也不知道。
Well do you think you can find out?
你觉得你能找到答案吗?
You got it!
得令!
These are what Snapchat calls their lenses, but everyone else calls filters.
这些东西Snapchat他们自己称之为“镜头”，但其他人都叫它们“滤镜”。
They are very silly but the engineering behind them is serious.
虽然看着傻里傻气的，但这些滤镜背后的工程学原理可是十分正经的。
Oh my god.
我的老天鹅哎。
The technology came from a Ukrainian startup called Looksery which Snapchat acquired in September 2015 for a $150 million dollars.
这项技术原本来自乌克兰一家名为 Looksery的创业公司, 该公司于2015年9月被Snapchat以1.5亿美元的价格收购了。
That's reportedly the largest tech acquisition in Ukrainian history.
据报道，该项目已经是乌克兰史上最大的科技公司收购项目了。
Their augmented reality filters tap into the large and rapidly growing field of "computer vision" --
该公司的AR滤镜充分利用了快速发展的“计算机视觉”——
those are applications that use pixel data from a camera in order to identify objects and interpret 3D space.
也即软件使用来自摄像头的像素数据来识别物体，理解三维空间的技术。
Computer vision is how you can deposit checks with your phone,
用手机存支票，
it's how Facebook knows who's in your photos,
Facebook识别你照片里的人物，
how self-driving cars can avoid running over people and how you can give yourself a doggy nose.
自动驾驶的汽车避开行人，给自己安一个狗狗的鼻子用到的都是这种技术。
So how to snapchat filters work?
那么，snapchat的滤镜到底是怎么运作的呢?
They wouldn't let us talk to any of the Looksery engineers but their patents are online.
他们拒绝我们和任何一名Looksery工程师联系，不过网上有他们的专利。
The first step is detection.
第一步是检测。
How does the computer know which part of an image is a face?
计算机是如何知道图像的哪一部分是脸的呢?
This is something that human brains are fantastic at.
识别人脸是人类大脑最擅长的事情。
Too good even.
甚至可以说太擅长了。
But this is what a photo looks like to a computer.
然而，计算机眼中的照片是这个样子的。
If all you have is the data for the color value of each individual pixel, how do you find a face?
如果你只有每个像素的色彩值，那要怎么识别出一张脸呢?
Well the key is looking for areas of contrast, between light and dark parts of the image.
原来，关键就在于在图像的亮部和暗部之间寻找存在色彩对比的区域。
The pioneering facial detection tool is called the Viola-Jones algorithm.
最先进的面部检测工具叫做维奥拉-琼斯算法。
It works by repeatedly scanning through the image data,
它的工作原理是通过反复扫描一张图像的数据，
calculating the difference between the grayscale pixel values underneath the white boxes and the black boxes.
计算白盒和黑盒下灰度像素值的差值。
For instance, the bridge of the nose is usually lighter than the surrounding area on both sides,
例如，鼻翼的色彩通常要比鼻翼两侧区域亮，
the eye sockets are darker than the forehead,
眼窝的色彩要比前额深，
and the middle of the forehead is lighter than the sides of it.
前额的中间比两边亮。
These are crude test for facial features,
这些还只是面部特征的粗略测试，
but if they find enough matches in one area of the image,
但如果检测下来图像的某个区域的匹配度足够高，
it concludes that there is a face there.
计算机就会判定那个部位是脸。
This kind of algorithm won't find your face if you're really tilted or facing sideways,
如果你头偏得太厉害，或者是侧脸面对镜头的，用这种算法就无法识别出你的面部，
but they're really accurate for frontal faces,
但如果你是正对镜头的话，其准确度还是相当高的，
and it's how digital cameras have been putting boxes around faces for years.
数码相机多年来也是用这种算法进行人脸对焦的。
But in order to apply this virtual lipstick, the app needs to do more than just detect my face.
不过，要想应用这种虚拟的口红，除了识别出我的面部，软件还有更多的工作要做。
It has to locate my facial features.
它还必须找出我的各个面部特征。
According to the patents,
根据他们的专利，
it does this with an “active shape model” --
他们在这方面使用了所谓的“活动形状模型”——
a statistical model of a face shape that's been trained by people manually marking the borders of facial features on hundreds, sometimes thousands of sample images.
这是一个关于脸型的统计学模型，是通过对数百名，甚至是数千名采样得到的图像进行人工的面部边缘标记得出的一个模型。
The algorithm takes an average face from that trained data and aligns it with the image from your phone's camera,
算法就从定型的数据中提取一张标准脸，将其与你手机摄像头拍摄的图像进行比对，
scaling it and rotating it according to where it already knows your face is located.
然后根据它已经判定的你的脸的位置进行缩放和旋转。
But it's not a perfect fit, so the model analyzes the pixel data around each of the points,
但这样并不能得到完美的匹配，所以模型会分析每个点周围的像素数据，
looking for edges defined by brightness and darkness.
通过明暗来界定边缘。
From the training images,
从定型的图像来看，
the model has a template for what the bottom of your lips should look like, for example,
比如，你的嘴唇应该是什么样子，模型是有对应的模板的。
so it looks for that pattern in your image and adjust the point to match it.
所以，它会按照这个规律去对应你的照片，然后对个别的点进行相应的调整。
Because some of these individual guesses might be wrong,
因为一开始识别的点可能存在个别错误，
the model can correct and smooth them by taking into account the locations of all the other points.
模型还可以通过对照所有其他点的位置来校正，让识别出的面部线条变得更加流畅。
Once it locates your facial features,
等模型确定了你的面部特征，
those points are used as coordinates to create a mesh.
这些点就会被用作坐标来创建一张网格。
That's a 3D mask that can move, rotate,
这样就得到了一个可以移动还可以旋转的3D蒙版，
and scale along with your face as the video data comes in for every frame
从而随着每一帧视频数据的输入跟着你的脸的变化进行缩放，
and once they've got that, they can do a lot with it.
有了这项技术，他们就能大展拳脚了。
They can deform the mask to change your face shape, change your eye color,
可以对蒙版进行变形来改变你的脸型，改变你眼睛的颜色，
add accessories, and set animations to trigger when you open your mouth or move your eyebrows.
添加饰品，还能在你张开嘴或动眉毛的时候添加动画效果。
And like the IOS app Face Swap Live,
和苹果手机的Face Swap一样，
Snapchat can switch your face with a friend's, although that involves a bunch more data.
Snapchat也能让你和朋友进行换脸游戏，尽管这一技术还需要更多的数据。
The main components of this technology are not new.
这项技术的主要部分其实都不是什么新技术。
What's new is the ability to run them in real time, from a mobile device.
它新就新在能够在移动设备上实时启用。
That level of processing speed is a pretty recent development.
这种处理速度是最近才出现的。
So why go through all this trouble just to give people a virtual flower crown?
那……干嘛为了给大家弄一顶虚拟的花环费这么大劲呢？
Well Snapchats sees a revenue opportunity here.
原来啊，snapchat从这里面看到了巨大的商机。
In a world that's flooded with advertisements,
在这个充斥着广告的世界里，
maybe the best hope that brands have to get us to look at their ads...is to put them on our faces.
也许，品牌投放广告最好的方式……就是把它们戴在我们的脸上。
Facial detection has a creepy side too, particularly when it's used to identify you by name.
面部检测也有令人毛骨悚然的一面，尤其是当它被用来通过一个人的名字来识别这个人的时候。
Both the FBI and private companies like Facebook and Google are massing huge databases of faces
联邦调查局和Facebook、谷歌等私人企业手上都有着庞大的人脸数据库，
and there's currently no federal law regulating it.
但目前还没有对其进行监管的联邦法律。
So some privacy advocates have come up with ways to camouflage your face from facial detection algorithms.
所以，一些倡导保护隐私的人就想出了一些办法来伪装你的脸，避免被面部检测算法到。
It's actually illegal in a lot of places to wear a face mask in public,
不过，在很多地方戴面具都是违法的，
so this project by artist Adam Harvey suggest some things that you can do with your hair and your makeup that can, for now, make your face invisible to computers.
所以，艺术家亚当·哈维就创立了一个项目，建议大家通过发型或者化妆对脸部进行隐形，让当前技术水平的计算机算法检测不出你的面部。
