评论
打印
His recent work on large language models uses classic theory of mind tests that measure the ability of children to attribute false beliefs to other people.
他(科辛斯基博士)最近对于大型语言模型的研究使用了经典的心智理论测试，这些测试衡量儿童理解他人的错误想法的能力。
A famous example is the Sally-Anne test, in which a girl, Anne, moves a marble from a basket to a box when another girl, Sally, isn't looking.
其中一个著名的例子是萨莉-安测试，在这个测试中，一个名叫安的女孩把一个玻璃弹珠从篮子里放到盒子里，而另一个女孩萨莉并没有看到这一过程。
To know where Sally will look for the marble, researchers claimed, a viewer would have to exercise theory of mind, reasoning about Sally's perceptual evidence and belief formation: Sally didn't see Anne move the marble to the box, so she still believes it is where she last left it, in the basket.
研究人员称，要知道萨莉会在哪里找玻璃弹珠，观众必须运用心智理论，推理出萨莉感知到了什么证据并如何形成了她的看法：萨莉没有看到安把玻璃弹珠放到盒子里，所以她认为玻璃弹珠还在之前的地方，而她之前把玻璃弹珠放在篮子里。
Dr. Kosinski presented 10 large language models with 40 unique variations of these theory of mind tests -- descriptions of situations like the Sally-Anne test, in which a person (Sally) forms a false belief.
科辛斯基博士给10个大型语言模型做了40种不同的这类心智理论测试 -- 描述了类似萨莉-安测试的情境，在这种情境下，某个人(萨莉)形成了错误的看法。
Then he asked the models questions about those situations, prodding them to see whether they would attribute false beliefs to the characters involved and accurately predict their behavior.
然后，他向模型提出有关这些情境的问题，试探它们，看看它们是否会认为相关角色产生了错误看法，并准确预测他们的行为。
He found that GPT-3.5, released in November 2022, did so 90 percent of the time, and GPT-4, released in March 2023, did so 95 percent of the time. The conclusion? Machines have theory of mind.
科辛斯基博士发现，2022年11月发布的GPT-3.5在90%的情况下能做到这些，而2023年3月发布的GPT-4在95%的情况下能做到这些。所以结论是什么？就是机器有心智理论。
But soon after these results were released, Tomer Ullman, a psychologist at Harvard University, responded with a set of his own experiments, showing that small adjustments in the prompts could completely change the answers generated by even the most sophisticated large language models.
但这些结果公布后不久，哈佛大学心理学家托默·乌尔曼用他自己的一组实验做出了回应，表示略微调整一下提示词，就可以让哪怕是最复杂精密的大型语言模型完全改变其生成的答案。
If a container was described as transparent, the machines would fail to infer that someone could see into it.
如果说容器是透明的，机器就无法推断出人们可以看到容器里有什么这一情况。
The machines had difficulty taking into account the testimony of people in these situations, and sometimes couldn't distinguish between an object being inside a container and being on top of it.
在这些情境中，机器很难考虑到人们有什么证据，有时甚至无法区分物体在容器内部还是容器顶部。
Maarten Sap, a computer scientist at Carnegie Mellon University, fed more than 1,000 theory of mind tests into large language models and found that the most advanced transformers, like ChatGPT and GPT-4, passed only about 70 percent of the time. (In other words, they were 70 percent successful at attributing false beliefs to the people described in the test situations.)
卡内基梅隆大学的计算机科学家马尔滕·萨普在大型语言模型中输入了1000多项心智理论测试，发现最先进的变换器，如ChatGPT和GPT-4，只在大约70%的情况下通过了这些测试。(换言之，他们理解测试情境所描述的人的错误看法的成功率为70%。)
The discrepancy between his data and Dr. Kosinski's could come down to differences in the testing, but Dr. Sap said that even passing 95 percent of the time would not be evidence of real theory of mind.
这一数据与科辛斯基博士的数据之间的差异可归结为所做的测试不同，但萨普博士表示，即使在95%以上的情况下通过测试也不能证明拥有真正的心智理论。
Machines usually fail in a patterned way, unable to engage in abstract reasoning and often making "spurious correlations," he said.
他说，机器未通过测试是有一种模式的，机器不能进行抽象推理，而且经常在事物之间做出"虚假的关联"。
