Nordic Summit 2018: Clark, Hellmuth, Horneland and Jakobsen discuss graphics, AR and AI

From our archives - published in SVG (Sports Video Group, Fergal Ringrose, Editorial Director, Europe) from the NBA's visit to Media City Bergen in Norway. The original story is published here.

Clever creation tools, graphics augmentation and increasingly sophisticated analysis tools are very much at the centre of new developments in sports broadcasting, not least to engage and retain those less-than-dedicated millennial media consumers.

At SVG Europe’s first Nordic Summit, held in partnership with Media City Bergen and NCE Media, Sixty CEO Kjetil Horneland led a panel featuring Doug Clark, IBM Global Cloud & Cognitive Solutions Leader; Vizrt CTO Petter Ole Jakobsen; and Steve Hellmuth, EVP Operations & Technology at the NBA in a discussion about the present and future of this important market segment.

Horneland began by asking Vizrt’s Jakobsen how he sees the broadcast industry changing in the area of sports. What are the biggest challenges ahead? “There are some super-simple trends,” Jakobsen replied. “One is mobile, obviously. Another one is screens, and increased resolution. A third one is AR. And a fourth might be physical things: all this sport has to be a package of entertainment, with something for everybody.

“And whether you end up using visual techniques or other techniques to achieve that, you’re doing it for the same reason. We are so blessed that as long as I’ve been in this industry we’ve been able to continuously produce more advanced and realistic graphics, plus cheaper and and screens – and that trend is just continuing.

“There was a recent study into media companies by Columbia University,” said Jakobsen, “and it showed that some of those organisations are now publishing to 20 different platforms. So if you have some content, and you know that your viewers are fragmented in terms of consumption, you really have an issue there. One of the challenges is to find methods to ensure that enriched content – with graphics I mean – can reach all these devices in an efficient way.”

How the NBA engages with fans across mobile and social

Steve Hellmuth had previously conducted a one-on-one interview with SVG Editorial Director Ken Kerschbaumer earlier in the day, going in-depth into the way the NBA engages with its fans on-court, on television and on mobile devices around the world. Panel moderator Horneland picked up on this interview, asking Hellmuth to talk a little bit more about how the NBA engages with basketball fans across mobile and social media. The NBA has been innovative in many of these areas, observed Horneland, asking Hellmuth to explain his thoughts on how to retain users as we move ahead into the future.

“We have an engine where we are rapidly logging, tagging and capturing the video and it goes to editors who sit on top of different social media channels,” said Hellmuth. “They do what’s then appropriate for that channel – you don’t want to kill someone with 30 clips.

Turning to IBM’s Doug Clark, moderator Kjetil Horneland said, “We’re moving into an abundance of data. Everything can be logged and everything is becoming more and more available. It leads to a point where we will still be telling some stories manually: but when we come to data and AI, there’s a lot of talk about what’s going on but we don’t have many examples of good case studies out there. Doug, do you have some good examples of what IBM has been doing in AI in sports?”

“Sure,” said Clark. “First of all, thank you very much for today – it has been an exceptional day. From an IBM point of view we’ve been marrying three separate components. First of all we’ve investing in cloud for quite a long time. We have also acquired over the last few years some cloud video assets, live streaming and OTT. And we’ve been working for a long time not just in analytics but moving that on to machine learning – basically what we’re all now calling AI.

“Marrying those three things together, the acceleration of the innovation has just been astonishing,” said Clark. “One of the best examples I’ve got is tennis. IBM has been blessed to work with most of the Grand Slams over a good number of years: I think last year was our 27th with Wimbledon.

“I remember the first time I was there, we had guys running film from the on-court still cameramen to motorbikes to be transported to central London to get to the newspapers. Then it became digital; and now it’s all totally wifi and totally interactive live. That whole evolution has gone really fast.

“First of all with Watson Media we do the transcription of the commentary. You can tell a lot from the commentary. Then you’ve got the natural language of the commentary, including the tonality and the emotion. Then you can take the crowd noise – which is different for every match and in every stadium. Then we do facial recognition, so you can see the emotions and the gestures. You can teach Watson to understand the game of tennis – it’s sensitised to the crucial points of the game.

“If you put all that together you can score all of those points, and then that creates an excitement rating and then you can put that into a league table,” said Clark. “And as the games are coming in live, those points are accumulating or moving you up and down. If you only have time for five highlights you can put the top five in and so on. That’s how that highlight reel is generated.

“How do you understand which points were really good?” he asked. “Then think about the camera angles – you can train it [the AI] to recognise that one camera angle is the service and another is the conclusion of the point. So you can put what you can describe as a ‘scene’; each point becomes a scene, and then you can cut and paste those scenes together to create the highlight reel,” said Clark.

Personalised and navigable graphics for NBA fans, with Sixty (edit: now Ease Live)

In concluding the session the NBA’s Steve Hellmuth doubled back to talk about the work the league is engaged in with Norwegian company Sixty. “I would be remiss if I didn’t mention the work we are doing with Sixty, right here in Bergen. Obviously when you get people to your platform, the trick for any sports league is to just hold them longer. If you hold them longer your ratings go up, your fans are happy with your app and your advertisers are happy too with the growing audience. That’s really where we monetise,” said Hellmuth.

“So what we’re working on, particularly with the Mobile View feed, is how things are changing –second screen is now first screen. I’ve designed so many things so many interactive things for PCs over the years … that nobody interacted with! They were leaning back from their PC; their hands weren’t there.

“Now that the mobile phone is the first screen for many, 23% of the people watch the NBA – our League Pass audience, which is about 400,000 strong around the world – on the mobile screen. And that’s growing steeply. Their hands are right there.

“In terms of augmentation we’ve worked with our statistical feeds and our clocks, in order to allow fans define what graphics they want to see and when they want to see them. We’re working with Sixty to make it navigable, so that you can literally live on this device and see what you want to see because your fingers are right there.

“I think it’s really important work, and the delay in the encoding allows us to store graphics to the cloud and deliver them synchronously with time code. We really look forward to doing more,” said Hellmuth.

Steve Hellmuth: "We're working with Sixty (edit: now Ease Live) to make it navigable"

Read the full article on SVG here.