Сodec War: the show must go on!
Preamble
A minor opus about the current situation in the codec industry with a retrospective and a driver for growth.
The Internet
I am pretty young 25 years old but I still remember the “good old Internet”, which was the Internet of texts and pictures (earlier only text even🙂) but now we have the Internet of video, where ~80% of traffic is video. Why? That is the topic of another article about the humans’ brain and how they picture the world, but I want to mention that since “the Internet of text and pictures”, the industry has acquired:
- much faster Internet connection;
- more powerful client’s and server’s hardware (Moore’s law);
- improved displays.
Even IT-industry has so much hardware progress “the Internet of video” cannot be authentic without the compression algorithms and hardware acceleration. Why? Please read my previous post. Now the world is ruled chiefly by AVC and VP9.
That’s why I want to discuss how the codecs were developed, standardized and entered the market.
Codec industry: how it works
1. So, first of all, the Committee collects technologies (referred to as patents) from various members of a codec group. They harmonize technologies within a single standard — it is called the standardization procedure.
2. How to use standard? The committee forms core technologies, which will be mandatory for the codec because the decoder should decode any stream relevant to the appropriate standard (codec). Such patents are called essential patents. Patents that are outside the essential ones may create another extension to the standard.
3. Develop and publish reference implementation of codec.
4. Publish the distribution and usage policy
So when a chipmaker produces a chip with the support of codec X it should pay to the codec group Y which will share the money between the members of the codec group according to their number and importance of patents for the codec. It can be different according to various license policies.
Codec industry: yesterday
MPEG LA, the company which administrates AVC/H.264 published transparent pricing policy. Usually a chipmaker should pay some cents per chip if it supports AVC and starting from particular number of produced chips. For example:
On August 26, 2010, MPEG LA announced that it wouldn’t charge royalties for H.264 encoded Internet video that is free to end users. All other royalties remain in force, such as royalties for products that decode and encode H.264 video and the operators of free television and subscription channels. The license terms are updated in the 5-year blocks.
See some insights on MPEG LA.
VP8/VP9 was published as open and royalty-free by Google, it is used by YouTube, Wikipedia and Netflix (VP9).
100% sure AVC was industrially successful. One of the proofs of that statement published in 2003 is that it is still widely used (end of 2020). VP8/VP9 is commonly used by Google for streaming and by “fans” of open formats.
In 2013 MPEG LA published HEVC/H.265 but it was not so successful as AVC. Why? A common opinion is because of its pricing and distribution policy. The patent holders understood the popularity of the Internet especially video streaming services like Netflix, Twitch and YouTube, which take 19, 33 and 2 places respectively at “The top 500 sites on the web” by Alexa. They wanted some bigger piece of the cake and published a new rule: chipmakers should pay for streaming services according to their traffic. That was not the last problem: some patent holders did not agree with the rems, so they created HEVC Advance. After that another segment of the patent holders formed the 3rd group. So, if earlier license buyer should pay to MPEG LA and use the codec, now he should go to 2? 3? groups who read all usage rights and payment…
To sum up:
- AVC has transparent pricing;
- VP9 was OK and free but not so good as HEVC;
- HEVC was technically sound and much better than AVC;
- HEVC has a terrible pricing policy.
As a result:
- AVC is still very popular;
- VP9 is used predominantly by Google and Wikipedia;
- HEVC is of very low popularity on the market.
Codec industry: today
Today people consume more and more traffic with video priority, as we can see in Cisco reports.
Internet users acquire new habits and demand, QHD and UHD (4K) display moving to the middle segment of the market so they become more popular. All that increases the requirements for video bitrates and the Internet speed.
Even traditional “good-old” TV companies choose Internet platforms (such as YouTube, Twitch and so on) or develop their own platforms to distribute their content.
They all use a media stack that should be encoded, decoded, packaged, delivered, optimized, recovered and so on.
New applications such as VR, Cloud Gaming and already super popular Game Streaming consume more and more traffic. It should be efficiently coded to save the bitrate but as illustrated earlier the industry still uses codecs from another decade such as AVC, VP9 and even MPEG2.
HEVC should have been “the codec of today” but it did not become popular (How many popular TV models have HEVC support in their chips? How many content providers or streamers use HEVC?). It fell under the onslaught of robber license policy.
Codec industry: tomorrow
If you are still reading this article and thinking about what a naive author he is: yes, that was a naive approach. Except for the technologies and optimizations (Did you read about Jan Ozer’s LCEV x264 Report? :) ), there are also:
- Market inertia and conservatism
- Local government regulations and push
- Organization’s lobby their own technologies or standards
- Company’s business models
Market inertia and conservatism
IP TV providers were already spending millions of dollars and pushed to the market millions of TV boxes with AVC support, so why should they invest more money to push new TV boxes with for example AV1 support? What is their profit if AVC is “OK” for HD TV? Maybe it is better to tune the existing encoder/decoder by software update? Update cycles for such companies could be decades.
Local government regulations and push
Some countries like China push the local market to use specific standards like the AVS-codec family to prevent political risks. It is more related to technological independence and sanctions (which is not the article’s subject).
Organizations lobby their own technologies or standards
Google, Netflix, Amazon invested a lot in the development of the AV-family and already implemented support into their products, so they will continue to promote it on the market. For example Intel has already pushed the hardware and drivers with the AV1 backing: https://www.phoronix.com/scan.php?page=news_item&px=Intel-Media-Driver-2020.3
MPEG organization promotes VVC.
China’s government and companies work with AVS-family.
Company’s business models
There is a big difference between the “hardware companies” like IP TV providers and for example streaming companies (OTT). Streaming companies are far more dynamic. It is easier for them to implement support of the new codec, but they are susceptible to the licensing policy. They cannot pay for each stream.
As mentioned there are many codecs implemented or in development. Let’s sum up most of them in one list:
- AV1
- AV2 (next gen of AV1, already in development)
- VVC
- EVC
- LVC
- AVS family
- MPAI (“motion picture AI” — it is a try of using AI instead of classic compression methods in the codec)
And so on!
It seems like tomorrow’s codec market will be very fragmented by codecs and there is no vision that 1 or 2 codecs will “rule the world”. Companies should support some codecs in their products. Which specifically? It depends on the terms.
Conclusion
As we can see video traffic had made the most significant segment of the Internet a long time ago. However, with the popularization of new applications such as Cloud Gaming, VR, streaming and also reduction of costs for 4K devices the task of high effective bandwidth reduction becomes critical, though at the same time the outdated technologies, distribution policy and legal wars slow down this process.
In addition it seems like “in the new world of tomorrow” each codec will take its place on the market according to the business strategy, model, policy, technologies, environment requirements and government restrictions.
Hard days for the technology enablers? No! If…
No, the lovely days for the industry and technology enablers are ahead if the companies choose optimum investments. Yeah, the challenges are big outstanding such as the fragmentation of the codec industry mentioned above, the problematic 2020 with increased bandwidth and servers loads, increased prices for some of the hardware and so on. Still, there were also new opportunities like new users, new practices and new technologies.
So to be successful tomorrow, we need to choose the right business, scalability models and suitable technologies. As an example: Bitmovin`s report highlights that “51% of respondents indicated that cost control for such things as bandwidth and storage is their biggest challenge” and “taking services and assets faster to market.” How can it be solved?
Companies should choose their right partners to support the variety of codecs or tune the existing one to improve bandwidth efficiency by up to 20%. In such a case, ViСueSoft can help with its services and products.