Look behind the curtain at the tech powering Mixer's HypeZone

When a pocket-sized Microsoft Garage team began working on "Sentinel, an AI-powered tool for monitoring live video streams, they had no idea it would eventually end up equally an important function of Microsoft's Mixer streaming service. But, afterward winning the 2022 Microsoft Hackathon, the tool, now known as "Sentry For," became the courage of Mixer's HypeZone.

In a new post on the Microsoft Garage weblog, the visitor takes a expect behind the curtain of HypeZone, offering details on Lookout man For'southward apprehensive journey from Garage project to game-streaming success.

At its inception, Lookout man For was designed past a Microsoft researchers Matthai Philipose, Lenin Sivalingam, Yifan Wu, Peter Bodik, and Victor Bahl. With a background in working on video analytics for the enterprise sector, the team began piece of work on Watch For as a way to translate their feel to use AI to a consumer setting. Over fourth dimension, Microsoft CEO Satya Nadella took discover of the squad, and their project somewhen won Microsoft'southward almanac Hackathon in 2022.

From there, the Mixer team was intrigued by Lookout For and reached out for a potential collaboration. The goal was to use Watch For'due south AI smarts to strop in on the nigh exciting moments from new battle-royale fashion games like PlayerUnknown'southward Battlegrounds (PUBG) and Fortnite, which were surging in popularity. This gave birth to HypeZone.

Lenin says:

Matt already had the idea of HypeZone itself, to switch from stream to stream within a aqueduct – simply the experience of HypeZone evolved very rapidly during our collaboration. We met with Matt and Chad early September. Two weeks later we had a prototype that we showed them. And then nosotros kept improving its accuracy. Past mid-October nosotros had some other prototype that they could use to run their HypeZone experience. We tested it for some other iii weeks. Then, 2 days before release, PUBG changed their UI. i day before release, we had to completely modify all our models.

HypeZone is driven by Watch For's big-scale video analysis of every stream that's coming into Mixer. Every stream we try to understand what's on the screen. We look for diverse metadata that tell us the game is exciting. Text on the screen, icons that tell you lot state of the game, player stats and score. Over fourth dimension nosotros have evolved to understand more and more.

Today, Mixer's HypeZone has expanded to offering 24/vii channels that deliver down-to-the-wire activeness from PUBG, Fortnite, and Rainbow Six Siege.

For more on the story of Watch For'due south journey, bank check out the full web log mail at Microsoft.

We may earn a commission for purchases using our links. Larn more than.

History catches up

The problem for a consumer HoloLens was always the lack of Windows Phone

What is the future for Microsoft (and Windows) mixed reality? The latest report from Business Insider sheds some light, but this all comes dorsum to ane major problem for Microsoft: No mobile Windows Os. Just what about a future headset that is cloud-based? Some thoughts on what could happen for Microsoft mixed reality.