Shazam is world-renowned for providing real-time song-recognition data. Owned by Apple, Shazam has been around since 2002, has over a billion downloads and is used 20 million times a day.
So, can a Croatian startup reach the same levels of success for a video-recognition app? 150sec spoke with Miroslav Zarić, co-founder and CMO of Beyond Seen Screen (BSS), a company developing the ExRey app to identify televised content.
A new technological approach
Zarić explained that the origin of BSS’ project goes back several years. He and his co-founders recognised that there was a growing need for users to identify televised content in the same way Shazam does with music.
The development of the ExRey app was only possible with a completely new approach. Zarić claimed that for the most part, the company built the technology from the ground up, rather than building upon existing technology. The algorithm was developed in-house.
“You can find technologies similar to this but they rely on audio recognition”
Miroslav Zarić, co-founder and CMO, Beyond Seen Screen.
BSS are not alone in developing a product to interact with televised content. However, Zarić maintains that BSS’ approach is unique. He clarified, “you can find technologies similar to this but they rely on audio recognition”.
Zarić went further to say that basing a system on audio is problematic. For example, such tech could result in a song within a movie being identified rather than the movie itself. When it comes to rival projects that don’t rely on audio, Zarić believes BSS still has the upper hand. Other approaches depend on the insertion of QR codes and the need for viewers to scan them in – a feature that makes the user experience less than seamless.
The secret sauce
Interestingly, BSS is reluctant to patent its intellectual property due to the concern that it may reveal too much about how content identification is achieved. As Zarić puts it, “nobody knows the recipe of [our] secret sauce”.
The Croatian entrepreneur described the video identification process as ‘fingerprinting’. For the proprietary system to work, it has to ‘see’ the video and match the fingerprint of the footage with the fingerprint of that specific content in a database.
Over the past year, BSS has grown its fingerprinted database from 1,000 to 30,000 videos. Much more content will need to be added before the app is fully launched but Zarić is confident that can be achieved.
In-app additional content
Beyond recognising the content being shown, ExRey also provides additional information about it.
Additional information could include where the artist is on tour and where tickets can be bought for concerts. When watching a cookery programme, a recipe could feature as additional content. When watching a movie based on real events, the app sends facts about the actual story. It’s worth noting though, that at the moment, viewers have to run a manual search for additional information.
Still, Zarić has come up against criticism that it’s not necessary to show information that users can search for on Google. In response, Zarić explained that the snippets of information serve as a way to keep users engaged in the app. Users can access facts about video content faster than finding the film on Google and searching through pages of results.
Interactive product placement
Product placement is huge in the TV and film industry, and the ExRey app has the ability to add interactive product placements within content. Marketers can include links or additional media to the film or programme being viewed to make brands more identifiable. For example, if a person is drinking a coffee in one scene, ExRey could highlight the place it was bought from and show promotional offers or even the closest location. Not only is the scope for interactive product placements great for user engagement, it has the potential to generate huge sums of revenue.
App status and the road ahead
The ExRey app was released in beta mode in 2019. So far, the app has been successfully trialled against movie trailers and content on Croatia’s DOKU TV.
Future plans at BSS aspire to use blockchain technology to collect data around how much (and what kind) of content people are watching. Such information will be invaluable for the startup and can potentially inform production companies what content to make next. However, developing such functionality is secondary to the BSS’ aim of first perfecting content recognition.