Automated news video production is better with a human touch, study shows

Technology providers like Wochit and Moovly are allowing publishers to mass produce videos at scale. But what do audiences think of the results? Researchers led by the Department’s Professor Neil Thurman have found that only automated videos which have been post-edited by humans were as well liked as fully human-made videos.

“Our research shows that, on average, news consumers liked short-form, automated news videos as much as manually made ones, as long as the automation process involved human supervision”, says Neil Thurman, Professor at Ludwig-Maximilians-University Munich’s Department of Media and Communication.

Together with Dr. Sally Stares (London School of Economic) and Dr. Michael Koliska (Georgetown University), Professor Thurman evaluated the reactions of 4,200 UK news consumers to human-made, highly-automated, and partly-automated videos that covered a variety of topics including Christiano Ronaldo, Donald Trump, and the Wimbledon tennis championships. The partly-automated videos were post-edited by humans after the initial automation process.

The results show that there were no significant differences in how much news audiences liked the human-made and partly-automated videos overall. By contrast, highly-automated videos were liked significantly less. In other words, the results show that news video automation is better with human supervision.

According to Professor Thurman, "one key takeaway of the study is that video automation output may be best when it comes in a hybrid form, meaning a human-machine collaboration. Such hybridity involves more human supervision, ensuring that automated video production maintains quality standards while taking advantage of computers’ strengths, such as speed and scale.”

The full article, entitled “Audience Evaluations of News Videos Made with Various Levels of Automation: A Population-based Survey Experiment” is published in the international peer-reviewed journal Journalism