Microsoft Open-Sources MLVC video codec: An AI Codec Beating H.265
Introduction
Video codecs don't usually make headlines outside of engineering circles, but Microsoft's latest release is worth paying attention to. The company has open-sourced MLVC (ML Video Codec) under the permissive MIT license, an AI-powered neural video codec that claims dramatically lower bitrates than the H.264 and H.265 standards most video calls and streams still rely on today. It's already running inside Microsoft Teams. Here's what MLVC actually is, how it works, and what its real-world numbers mean.
Table of Contents
- What Microsoft Actually Announced
- How MLVC's Compression Numbers Stack Up
- What Makes MLVC Different From Traditional Codecs
- The Cross-Platform Problem MLVC Was Built to Solve
- Running on NPUs, Not GPUs
- MLVC Inside Microsoft Teams Today
- What's Actually in the Open-Source Release
- MLVC's Research Lineage: From DCVC to MLVC
- Comparison Table: MLVC vs. H.264 vs. H.265
- Why This Matters Beyond Microsoft
- Common Questions About Neural Video Codecs
- What Developers Should Know Before Trying It
- FAQs
- Conclusion
- Key Takeaways
What Microsoft Actually Announced
Microsoft has released the source code, pretrained model weights, and training scripts for MLVC, its ML Video Codec, under the MIT license, making it freely usable, modifiable, and redistributable, including for commercial purposes. The release accompanies a research paper titled "MLVC: Multi-platform Learned Video Codec for Real-World Deployment," and Microsoft has framed the move explicitly as an invitation for the broader community — researchers, codec engineers, platform vendors, and product teams — to help build the next generation of video coding in the open, rather than behind closed doors.
How MLVC's Compression Numbers Stack Up
According to Microsoft's own published benchmarks, MLVC achieves significant bitrate reductions compared to both major existing video standards, measured at two common resolutions:
| Resolution | Bitrate Reduction vs. H.264 | Bitrate Reduction vs. H.265 |
|---|---|---|
| 360p | 87.8% | 75.5% |
| 540p | 82.7% | 65.4% |
To put the 360p figure in more concrete terms, Microsoft notes that where H.264 might require roughly 1 Mbps for a given quality level at 30 fps, MLVC needs only about 122 kbps for equivalent quality under real-time conditions — around one-eighth the bitrate. These results come from subjective quality testing using the P.910 methodology, run against Microsoft's own Video Conferencing Dataset (VCD), which the company has also separately open-sourced.
What Makes MLVC Different From Traditional Codecs
Traditional codecs like H.264 and H.265 rely on hand-engineered building blocks: motion estimation, mathematical transforms, and entropy modeling, refined incrementally over successive standard generations. Each new generation typically requires enormous engineering effort for comparatively modest gains, and often needs dedicated hardware decoders that can take years to become widely available across consumer devices.
MLVC takes a fundamentally different approach, replacing those hand-engineered components with end-to-end learned neural compression, trained directly against rate-distortion objectives using deep learning rather than manually designed algorithms. This is part of a broader category known as neural video codecs (NVCs), which have shown strong compression efficiency in research settings but have historically struggled with real-world deployment challenges.
The Cross-Platform Problem MLVC Was Built to Solve
That deployment struggle is specifically what MLVC's research paper addresses. Earlier neural video codecs, including quantization-based approaches, often failed to produce consistent, deterministic decoding results across different hardware platforms, sometimes leading to what researchers describe as catastrophic decoding failures when the same compressed video was decoded on different chips.
MLVC's key technical innovation is explicitly transmitting scale parameters through what's called a hyperprior, which guarantees consistent entropy coding results across different devices without requiring bit-exact arithmetic matching between them. This does add some bitrate overhead compared to a theoretically perfect single-platform implementation, but Microsoft's research indicates the team recovered most of that lost efficiency through several architectural improvements, including gated memory mechanisms, ReGLU activation functions, and a long-term reference recovery mechanism designed to maintain quality even across dropped or delayed frames.
Running on NPUs, Not GPUs
Perhaps MLVC's most practically significant design choice is that it's built to run almost entirely on the Neural Processing Units, or NPUs, already shipping inside modern consumer devices — specifically the Apple Neural Engine, and NPUs from Qualcomm and Intel — rather than requiring a dedicated, high-power GPU. Microsoft states the codec operates at no more than 50% NPU utilization on targeted devices, intentionally leaving headroom for the rest of the system to keep running smoothly during a video call or stream.
According to the published research, both the encoder and decoder run at approximately 100 frames per second on average across these commodity NPUs, with Microsoft separately demonstrating real-time performance at 540p and 30 fps across Apple, Intel, and Qualcomm hardware specifically.
MLVC Inside Microsoft Teams Today
MLVC isn't just a research artifact — Microsoft says it's currently being rolled out inside Microsoft Teams, where it's being validated on real peer-to-peer video calls with active telemetry and A/B testing. That's a meaningful detail: it means the bitrate and quality figures Microsoft has published aren't purely lab benchmarks, but are being tested against real-world network conditions and usage patterns at scale, inside one of the most widely used video conferencing platforms in the world.
What's Actually in the Open-Source Release
The MLVC GitHub repository, released under the MIT license, includes the full network architecture source code, pretrained model weights ready to run out of the box, and training scripts, giving developers and researchers the ability to both use MLVC directly and to retrain or fine-tune it for their own specific use cases. This level of openness — full code, weights, and training pipeline together — is notably more comprehensive than many prior neural codec research releases, which have sometimes shared only partial components.
MLVC's Research Lineage: From DCVC to MLVC
MLVC didn't appear out of nowhere. It's described by Microsoft as the latest product iteration of the DCVC (Deep Contextual Video Compression) family of neural video codecs, a research effort Microsoft Research has been developing and open-sourcing since 2021. Earlier entries in that lineage, including DCVC-RT and other variants published on GitHub, established the underlying research foundation around contextual video compression, quadtree-based spatial partitioning, and diverse-context modeling that MLVC builds on, now packaged specifically for practical, cross-platform, real-time deployment rather than primarily as an academic research demonstration.
Comparison Table: MLVC vs. H.264 vs. H.265
| Factor | H.264 | H.265 (HEVC) | MLVC |
|---|---|---|---|
| Compression approach | Hand-engineered blocks | Hand-engineered blocks, refined | End-to-end learned neural compression |
| Hardware requirement | Widely supported dedicated decoders | Dedicated decoders, slower rollout | Runs on existing consumer NPUs |
| Bitrate at 360p (relative) | Baseline | Better than H.264 | Up to 87.8% lower than H.264 |
| Bitrate at 540p (relative) | Baseline | Better than H.264 | Up to 82.7% lower than H.264 |
| Licensing | Patent-encumbered, royalty-bearing | Patent-encumbered, royalty-bearing | MIT License (open source) |
| Deployment status | Universal, mature | Widely supported | Rolling out in Microsoft Teams |
Why This Matters Beyond Microsoft
For an industry that has relied on H.264 and, to a lesser extent, H.265 for the vast majority of video delivery for well over a decade, MLVC's approach represents a genuinely different path forward. Meaningfully lower bitrates at equivalent quality translate directly into lower bandwidth costs for streaming and video-call providers, better call and video quality on constrained or unreliable networks, and reduced storage costs for anyone archiving large volumes of video content.
The permissive MIT license is also notable in an industry where video codec licensing has historically been complicated by patent pools and royalty structures, as seen with both H.264 and H.265. An open, freely usable, high-performance neural codec removes a meaningful barrier to adoption for smaller companies and independent developers who might otherwise be constrained by codec licensing costs.
Common Questions About Neural Video Codecs
- Does this mean H.264 and H.265 are obsolete? Not immediately — those codecs benefit from decades of hardware decoder support built into essentially every device and browser, an advantage MLVC doesn't yet have at the same scale.
- Is MLVC only useful for video calls? While Microsoft's current real-world validation is specifically in Teams video calls, the underlying research addresses general video compression, suggesting broader applicability to streaming and storage use cases over time.
- Does MLVC require special hardware? It's specifically designed to run on NPUs already present in many modern Apple, Intel, and Qualcomm-based devices, rather than requiring new, dedicated hardware.
What Developers Should Know Before Trying It
Developers interested in experimenting with MLVC can access the full source code, pretrained weights, and training scripts from Microsoft's GitHub repository under the MIT license. Given the codec's specific design around NPU-based deployment, developers working on devices without a supported NPU may not see the same real-time performance characteristics Microsoft has demonstrated on Apple, Intel, and Qualcomm hardware specifically. As with any actively developed research-to-production project, keeping an eye on repository updates and Microsoft's ongoing Teams telemetry findings is likely to be useful for tracking how the codec's real-world performance evolves.
FAQs
1. What is MLVC?
MLVC (ML Video Codec) is an AI-powered neural video codec developed by Microsoft Research, open-sourced under the MIT license, designed to significantly reduce video bitrate compared to traditional codecs like H.264 and H.265.
2. How much bandwidth does MLVC actually save?
According to Microsoft's published benchmarks, MLVC achieves up to 87.8% lower bitrate than H.264 and up to 75.5% lower bitrate than H.265 at 360p, with slightly smaller but still substantial gains at 540p.
3. Is MLVC free to use?
Yes, it's released under the MIT license, which permits free use, modification, and redistribution, including for commercial purposes.
4. Is MLVC already being used anywhere?
Yes, Microsoft says it's currently rolling out inside Microsoft Teams, where it's being validated on real peer-to-peer video calls.
5. What hardware does MLVC run on?
It's designed to run primarily on NPUs (Neural Processing Units) already present in many modern devices, specifically citing support for Apple Neural Engine, Qualcomm, and Intel NPUs.
6. How is MLVC different from traditional codecs like H.264?
Traditional codecs use hand-engineered compression techniques refined over years of standards development. MLVC uses end-to-end learned neural compression, trained directly on rate-distortion objectives using deep learning.
7. What's included in the open-source MLVC release?
The GitHub repository includes the full network architecture source code, pretrained model weights, and training scripts.
8. Is MLVC related to any earlier Microsoft video compression research?
Yes, it's described as the latest product iteration of Microsoft Research's DCVC (Deep Contextual Video Compression) family of neural video codecs, which the company has been developing since 2021.
9. Does MLVC work consistently across different devices?
Addressing cross-platform consistency was a core design goal — MLVC explicitly transmits scale parameters through a hyperprior mechanism to ensure consistent decoding results across different hardware, avoiding failures seen in some earlier neural codec approaches.
10. Will MLVC replace H.264 and H.265 entirely?
Not in the near term. While MLVC shows strong compression performance and real-time viability, H.264 and H.265 benefit from near-universal hardware decoder support built up over more than a decade, which MLVC doesn't yet have at the same scale.
Conclusion
Microsoft's decision to open-source MLVC under the MIT license, complete with source code, pretrained models, and training scripts, is a genuinely significant move for an industry that has spent over a decade relying primarily on H.264 and H.265. The bitrate numbers are striking on their own, but what makes MLVC particularly credible is that it's not just a research paper's benchmark table — it's already running inside Microsoft Teams, being validated against real network conditions at scale. Whether MLVC becomes a genuine successor to today's dominant video standards will depend on broader industry and hardware adoption over the coming years, but Microsoft has now handed the entire community the tools to help make that happen.
Key Takeaways
- Microsoft has open-sourced MLVC, a neural video codec, under the MIT license, including full source code, pretrained weights, and training scripts.
- MLVC achieves up to 87.8% lower bitrate than H.264 and up to 75.5% lower bitrate than H.265 at 360p, based on Microsoft's published benchmarks.
- The codec is specifically designed to run on NPUs from Apple, Qualcomm, and Intel, achieving around 100 fps on commodity hardware.
- MLVC is already rolling out inside Microsoft Teams, being validated on real peer-to-peer calls with active telemetry.
- It builds on Microsoft Research's DCVC neural video codec lineage, open-sourced since 2021, now packaged for practical, cross-platform, real-time deployment.
