That’s thanks to the way it evaluates models not only on the ability to pass arbitrary tests, like most other benchmarks, but also on the ability to execute real-world tasks, which is more unusual.
在測試人工智慧模型時,很難判斷它是在推理,還是僅僅根據訓練資料重複答案。一家中國創投公司開發的新基準或許有助於規避這個問題。
這要歸功於它評估模型的方式,它不僅像大多數其他基準一樣,評估模型通過任意測試的能力,還評估模型執行實際任務的能力,這在實際應用中並不常見。A professional regurgitator is an entertainer whose act consists of swallowing and regurgitating various unusual objects. The objects may consist of anything from live animals (live mice: The Great Waldo), to light bulbs, billiard balls (Stevie Starr) and kerosene (Hadji Ali).
Professional regurgitator - Wikipedia
https://en.wikipedia.org/wiki/Professional_regurgitator
/rɪˈɡəːdʒɪteɪt,ˌriːˈɡəːdʒɪteɪt/
verb
1.
bring (swallowed food) up again to the mouth.
"gulls regurgitate food for the chicks"
Similar:
vomit
bring up
disgorge
regorge
2.
repeat (information) without analysing or comprehending it.
"facts which can then be regurgitated at examinations"
Similar:
repeat
say again
restate
recapitulate
keroseneの意味 - 小学館 プログレッシブ英和中辞典
kerosene, kerosine
音節ker • o • sene
発音kérəsìːn
[名]U((米・豪・NZ))灯油(((英)) paraffin)

沒有留言:
張貼留言