Start Using
VQ Analyzer - a professional bitstream analysis tool for VVC, AV1, VP9, HEVC/H.265, AVC/H.264 and MPEG2 video codecs

Buy

VQ Analyzer 2020 User Guide Ver 5.4

LEGAL DISCLAIMER

INFORMATION IN THIS DOCUMENT IS PROVIDED IN CONNECTION WITH VICUESOFT PRODUCTS. NO LICENSE, EXPRESS OR IMPLIED, BY ESTOPPEL OR OTHERWISE, TO ANY INTELLECTUAL PROPERTY RIGHTS IS GRANTED BY THIS DOCUMENT. EXCEPT AS PROVIDED IN VICUESOFT'S TERMS AND CONDITIONS OF SALE FOR SUCH PRODUCTS, VICUESOFT ASSUMES NO LIABILITY WHATSOEVER AND VICUESOFT DISCLAIMS ANY EXPRESS OR IMPLIED WARRANTY, RELATING TO SALE AND/OR USE OF VICUESOFT PRODUCTS INCLUDING LIABILITY OR WARRANTIES RELATING TO FITNESS FOR A PARTICULAR PURPOSE, MERCHANTABILITY, OR INFRINGEMENT OF ANY PATENT, COPYRIGHT OR OTHER INTELLECTUAL PROPERTY RIGHT.

UNLESS OTHERWISE AGREED IN WRITING BY VICUESOFT, THE VICUESOFT PRODUCTS ARE NEITHER DESIGNED NOR INTENDED FOR ANY APPLICATION IN WHICH THE FAILURE OF THE VICUESOFT PRODUCT COULD CREATE A SITUATION WHERE PERSONAL INJURY OR DEATH MAY OCCUR.

ViCueSoft may make changes to specifications and product descriptions at any time, without notice. Designers must not rely on the absence or characteristics of any feature or instruction marked "reserved" or "undefined." ViCueSoft reserves these for future definitions and holds no responsibility for conflicts or incompatibilities arising out of future changes to them. The information here is subject to change without notice. Do not finalize a design with this information.

The products described in this document may contain design defects or errors known as errata which may cause the product to deviate from published specifications. Current characterized errata are available on request.

Contact your local ViCueSoft sales office or distributor to obtain the latest specifications before placing your product order.

MPEG is an international standard for video compression/decompression promoted by ISO. Implementations of MPEG CODECs, or MPEG enabled platforms may require licenses from various entities, including ViCueSoft.

*Other names and brands may be claimed as the property of others.

Copyright © 2017-2020, ViCueSoft. All Rights reserved.

Contents

Revision History:

Revision Number Date Description
2.4.1 9/05/17 Initial Draft
2.5.0 10/06/17 Updated options menu
2.5.1 10/14/17 Updated version
3.0.0 11/09/17 Update version, added license activation information
3.1.0 12/04/17 Updated screenshots for AV1
3.2.0 03/01/18 Updated AV1 description
3.2.1 03/20/18 Updated AV1 description for new features
3.3.0 04/18/18 Updated AV1 description
3.4.0 05/30/18 Updated AV1 description
3.5.0 06/28/18 Updated AV1 description
3.5.1 07/12/18 Change version
3.6.0 08/16/18 Change version
3.7.0 10/29/18 Change version
3.8.0 02/10/19 Change version and AV1 description
4.0.0 04/03/19 Change version. Rebranding.
4.1.0 07/07/19 Change version. Refactoring of content.
4.2.0 11/16/19 Change version.
5.0.0 5/21/20 Change version.
5.1.0 6/30/20 Change version.
5.2.0 7/31/20 Change version.
5.3.0 8/27/20 Change version.
5.4.0 10/01/20 Change version and VVC description

Overview

The VQ Analyzer is a graphical coded video bitstream analysis tool, supporting the following coding standards:

  • VVC, SHA-1 (VTM 10.0 tag): 45dfe06835aea43f37456ea54972ef31b765eacf
  • AV1, SHA-1 (av1-normative branch of AOM source): acc3f97753f67e0ce7290b98b7bb71152fe5e264
  • HEVC: (ISO/IEC 23008-2 MPEG-H Part 2 or ITU-T H.265) , 8/10-bit
  • HEVC: RExt extension, 8/10/12-bit, 4:0:0/4:2:0/4:2:2/4:4:4
  • HEVC: SCC extension, conform to HM + SCM 8.5 in reference code
  • HEVC Scalable/Multiview Extension
  • Google’s VP9, profiles 0,1,2,3, 4:2:0/4:2:2/4:4:0/4:4:4, 8/10/12-bit
  • AVC: (H.264/AVC, ISO/IEC 14496-10, MPEG-4 Part 10), except SVC/MVC
  • MPEG2 (ISO/IEC 13818-2 Part 2), 4:2:0/4:2:2, 8-bit
  • MKV container
  • MP4 container
  • MPEG2 TS/PS container
  • AVI container
  • Mpeg Media Transport container

Once a bitstream is loaded, the tool allows the user to inspect each major step of the decoding process visually and numerically, and the structure of the coded image can be explored. This data can be used as a visual reference when learning about HEVC/VP9/AVC/MPEG2/AV1 or when debugging a particular encoder or decoder.

VQ Analyzer is written in C++. There are separate packages for each supported operation system - Windows*, Linux*, macOS*.

Document Conventions, Symbols and Terms

Term Definition
AV1 open, royalty-free video coding format developed by the Alliance for Open Media
AVC Advanced Video Coding
CABAC Context-adaptive binary arithmetic coding
CTB Coding Tree Block
CU Coding Unit
CVS Coded Video Stream
GUI/UI (Graphical) User Interface
HEVC High Efficiency Video Coding
MPEG2 ISO/IEC 13818-2 / ITU-T Rec. H.262 coding standard
NAL Network Abstraction Layer
OBU Open Beatstream Unit
PPS Picture Parameter Set
PU Prediction Unit
QM Quantization Matrix
SAO Sample adaptive offset
SB SuperBlock
SEI Supplemental Enhancement Information
SPS Sequence Parameter Set
TU Transform Unit
VP9 open and royalty free video coding format developed by Google
VPS Video Parameter Set
VVC Versatile Video Coding (MPEG-I Part 3) is a video compression standard
YUV Color space (YUV)

Product Activation

The VQ Analyzer supports several modes of working, depending on the activation information supplied by the user.

If you are running the VQ Analyzer for the first time on the machine licensing system, the license server will connect (over the internet) to register itself and activate the verified trial mode. In this mode you have a trial period to evaluate the product. The only restriction you have is a splash screen that will appear every 5 minutes to remind you that you’re in evaluation mode. There is a grace period of several days during which application can run without internet connection. The verified trial period can be extended by entering the special serial number in the Product run without internet connection. Verified trial period can be extended through the activation dialog. The number of remaining days can be checked in the ‘About’ menu.

If you don’t have internet connection or if the verified trial is over, the application runs in pure trial mode, whereby you can load only sample streams provided in the package.

If you have a single user license and serial number you have to enter them in the ‘Activate’ dialog under ‘Help’ menu. In the ‘Online Activation’ tab, enter your serial number in the ‘Key’ line and click ‘Activate’. If you check ‘Extend Trial,’ the serial number will be used to extend the verified trial.

If that application is already activated, the ‘Deactivate’ button will be shown instead of the ‘Activate’ button. You are then able to deactivate the application on this computer, and reactivate it later, on another or the same computer, as many times as is allowed by your license.

If you have a floating license you have to install the floating server in your network and enter the host and port for it in the ‘Activate’ dialog under the ‘Help’ menu after checking the ‘Floating License’ option.

Also you have the opportunity for Offline Activation. Open the ‘Activate’ dialog under the ‘Help’ menu and choose the ‘Offline Activation’ tab. Follow the instructions.

First, enter your serial number in the ‘Key’ line. Press ‘Activation Request’ button if you want to activate this application, or click the ‘Deactivation Request’ button if you need to deactivate it. Choose the directory for saving the Request file, then click ‘Show in folder’, find your Request file and send it to customersupport@vicuesoft.com. You will receive a response after a short time. The last step is to upload the Response file in the ‘Offline Activation’ tab.

Any choice of parameters in the ‘Activate’ dialog are stored in the system settings and used for both GUI and command line versions.

The same procedures can also be done with the command line tool:

Floating license server configuration

The floating license server is supplied as a separate package when you order a floating license. When you get it, just unpack it and follow the instructions below for the appropriate system where the server will be installed.

Before you can use the float license server it must be activated. To activate the server online, simply pass the product key as follows:

    VQAnalyzerFS -a="ABCD-EFGH-IJKL-MNOP-QRST-UVWX"

Online activation connects to the activation servers using port 80 (HTTP) or port 443 (HTTPS). If you specify a proxy in the configuration then the server will try to connect to the activation servers through that.

If you've already activated and you want to re-activate, then you do not need to pass a new product key. You can just call the server with the "-a" command line argument:

    VQAnalyzerFS -a

If you want to move the server from one computer to another, you have to deactivate it from the first computer before you can activate it on the second computer. To deactivate the server, use the “-deact” command line switch:

    VQAnalyzerFS -deact

Activation and deactivation are possible only if the server is not working at that moment. You need to shut down the server before deactivating. If the server was started with -x or -x -silent commandline arguments, then the active process in the terminal must be terminated. (Interrupting in the terminal can be done with the combination Ctrl+C). If the server started as an “daemon” or service (-i on Windows), then “daemon”/service must be stopped before deactivation.

Server configuration is defined in the Config.xml file (sample files provided in the package). You can load it on server installation with the -config=“Config.xml” option. Options that can be defined inside configuration file are explained inside it.

Check the details in the floating server documentation.

Windows

On Windows you can use a simple command line switch "-i" to set up your server. When successful, the server returns 0, and in the event of a failure it returns non-zero.

  1. It installs the server instance as a Windows Service set to start with the computer and run silently in the background.
  2. It configures the server to allow inbound connections past the Windows Firewall (unless the Windows Firewall service is disabled, in which case it skips this step).
  3. It starts the server immediately.

To uninstall the server simply use the "-u" commandline switch:

    VQAnalyzerFS -u

When successful, the server returns 0, and in the event of a failure it returns non-zero. It stops the server immediately. It removes the Windows Firewall configuration settings for the server if both of the following conditions are met: the Windows Firewall settings were set up during the installation or the Windows Firewall service is running at the time of the uninstallation. If either of those conditions isn't met, then the uninstallation process simply skips this step. It removes the server instance from the Windows Services.

MacOS

On macOS you can use a simple command line switch "-i" to set up your server. When successful, the server returns 0, and in the event of a failure it returns non-zero.

  1. It installs the server instance as a "launchd" daemon set to start with the computer and run silently in the background.
  2. It starts the server immediately.

If you have any firewall software running on macOS then you'll have to set it to allow incoming connections to the server instance.

To uninstall the server simply use the "-u" commandline switch:

    VQAnalyzerFS -u

When successful, the server returns 0, and in the event of a failure it returns non-zero. It stops the server immediately. It removes the server instance from the "launchd" daemons list.

Linux

Since Linux installations vary wildly in how they run "daemons" or "startup services" there is no the "-i" commandline switch for Linux. However, you can build the init script yourself (for systemd, System V init, upstart, or any other init system) to launch the server instance when the computer starts.

When you're running the server instance from your init script there are two command line options you'll want to use:

    /path/to/VQAnalyzerFS -x -silent

That will tell the server instance to run and to run silently (not outputting to "stderr").

Working behind firewall

The VQ Analyzer can be configured to work behind a firewall. Currently, http/https proxies are supported. Normally, proxy settings are detected based on system settings and you don’t need to change them. But if the system settings are wrong or you prefer to use other proxy connection type, you will have to set up the required setting in the ‘File -> Proxy settings’ dialog.

Command Line Features

VQ Analyzer has a separate console application in Windows* to run in command line mode. The GUI application in Windows* also accepts the same command line options but does not provide console output.

VQ Analyzer has a few command line options:

  • Adding -help prints to the available command line options.
  • Adding a file name to the command line will make the VQ Analyzer open that file in the UI right after startup, bypassing the file menu. This can be helpful as part of a debug script.
  • -hevc : force HEVC decoder
  • -vp9 : force vp9 decoder
  • -av1: force av1 decoder
  • -avc : force avc decoder
  • -mpeg2 : force mpeg2 decoder
  • -o <file name>: output file name (default is “output.yuv”). Must be used with -regress
  • -y4m: write the decoded sequence to output.y4m. Must be used with -regress and -dump
  • -n <frame number>: position stream at this frame on startup
  • -debug_yuv <file name>: debug yuv file name to load on startup (for GUI mode) or debug yuv to measure some metrics (for command line)
  • -dependent <file name>: open dependent bitstream
  • -regress: don’t bring up the GUI, just decode the stream
  • -frames : decode specified number of frames in decode order. Must be used with -regress
  • -md5: output pre frame md5 sums
  • -psnr: calculate PSNR for the debug yuv
  • -dump: write the decoded sequence to output.yuv (default is decode order). Must be used with -regress
  • -nocrop: don’t crop the output according to stream parameters. Must be used with -regress and -dump
  • -errors <file name>: write errors to file. Must be used with -regress
  • -stats <file name>: write some stats to file. Must be used with -regress
  • -stream_stats <file name>: write whole stream statistics information to specified file (Only for HEVC streams)
  • -stream_stats_level <level>: statistics level (should be between 1, 2 or 3, default is 1), the higher the level, the more information (Should be used with previous command. Only for HEVC streams)
  • -norm_pix: normalize stats by pixel area
  • -norm_bits: normalize stats by compressed bits
  • -percent: output statistics in percentage
  • -fast <level>: 0 - no fast mode, 1 - skip reconstruction, 2 - skip frame data parsing in addition. Must be used with -regress and -stats, and cannot be used with -dump.
  • -display_order: if used with -dump, the output yuv is written in display order.
  • -dump_bitdepth <bitdepth>: set output bitdepth of yuv file to bitdepth. Should be in the range [8..16]
  • -disable_hevc_ext: disable HEVC extensions
  • -vp9_hw_emul: if specified overflow wrapping is applied to emulate hw behaviour
  • -cpu_max_feature : max set of cpu features to be used, valid args: auto, none, sse2, sse3, ssse3, sse4, avx2
  • -enable_demo_mode: enable demo mode option
  • -fullversion: output full build information
  • -film_grain: combined with -o/-md5 options output two ‘yuv files’/‘md5 hash’ (before film grain process and after)
  • -ext_trial <serial number>: extend trial with specified serial number
  • -activate <serial number>: activate by using specified serial number for single user license
  • -deactivate: deactivate single user license
  • -activated: check current license status
  • -license_server <floating server>: specify floating license server in format host:port

Dual view

Dual view mode allows one to make a direct visual comparison between two bitstreams. This feature is useful for fine tuning codecs with a slight variation in coding parameters. The bitstreams themselves should be coded with the same codec. VQ Analyzer considers the first opened stream to be the master stream. The depended stream is opened just after the master one by choosing “Open depended bitstream…” in File menu.

When the depended bitstream is opened it is synchronized with the master bitstream in frame position, chosen mode, offset and scaling factor in the central information window. Also, the toolbar buttons in central info window work synchronously with the master.

The VQ Analyzer provides a new delta button

in the toolbar of central window.

This button allows for a visual comparison between the master and depended bitstreams. When this mode is activated the blocks with the same structure and the same currently visualized modes will be in gray color, while the blocks with different parameters or blocks with different structure on both view still will be colored. In this way, it’s simple for the user to see a visual difference in the coding parameters. The delta mode can be used for the following modes: Prediction, Transform, Reconstruction, Deblocking, SAO, Info overlays.

For heat map-like modes displaying one parameter, visual difference is shown pre block in three colors: red – positive values, gray - zero difference, blue – negative difference. So you can easily locate any increase or decrease in comparison with the depended bitstream.

Debug YUV

In YUV mode, the YUVDiff feature is exposed. This feature allows for comparison of an external decoded YUV file with the decoded bitstream. Supported formats are 4:2:0 planar and NV12. Also, YUV files that are zipped or gzipped may be opened directly without the need to decompress separately. If a zip file contains more than one YUV file, only the first one is used. When a YUV file is loaded, any mismatches will be indicated with a red dot in the CU containing the mismatch. This allows the user to quickly identify the nature of the mismatch which can assist with debugging.

When a file is opened, four additional buttons become visible on the lower left of the main window:

  • Ref YUV: The original, expected YUV image.
  • Debug YUV: The loaded debug YUV file.
  • Diff YUV: The delta image. Like in Transform mode, areas with zero delta (i.e. Original and Debug image are identical) are flat gray. Areas where the debug YUV has a lower value are darker, and areas where the debug YUV has a higher value are brighter.
  • Reload: Reloads the YUV file, which can be useful as a shortcut to loading via the YUVDiff menu.
  • Rel/Abs - switched between relative and absolute difference

YUV difference plane can be represented as relative between reference and debug pixel

or absolute difference values, that is useful to identify particular pixels with difference.

YUVDiff Menu

  • Open: Brings up a file chooser to open a YUV file. VQ Analyzer considers the YUV stream to have the same frame size, per frame bitdepth and color format as the encoded bitstream.
  • Close: Closes the currently opened YUV file. Any mismatch indicators will disappear.
  • Planar: Causes the VQ Analyzer to interpret the loaded YUV file as Planar.
  • Interleaved: Causes the VQ Analyzer to interpret the loaded YUV file as Y values followed by interleaved U and V values. This option is chosen automatically when the loaded YUV file has the file extension ".nv12".
  • Display order: Causes the VQ Analyzer to use display order picture numbers when determining how far to seek into the YUV file to extract the image to compare with the current picture.
  • Decode order: Causes VQ Analyzer to use decode order picture numbers when determining how far to seek into the YUV file to extract the image to compare with the current picture.
  • Use stream crop values: When checked, YUV files are assumed to contain samples only within the cropping window as defined by the loaded bitstream. Samples outside this window are not compared, and are assumed 0 when viewing the loaded YUV image directly.
  • Set picture offset here: Shortcut for setting the picture offset to the current picture number. See bullet below.
  • Use 16 bit YUV: this option allows one to load a YUV file where both Luma and Chrome written in 16 bit (they could be normalized from actual bitdepth to 16 bit during encoding). This is useful when you have bitdepth changing across the stream. The VQ Analyzer will shift values to the appropriate bitdepth on the current frame.
  • Picture offset: This brings up a dialog allowing the user to enter the picture number of the first picture in the YUV file. For example, if a 100-picture bitstream is loaded but the YUV file only contains pictures 80-99, the user would enter 80 to properly line up the YUV file with the decoded bitstream.
  • Check for file changes: When checked, this option causes VQ Analyzer to periodically check if the loaded YUV file has changed on disk since it was last loaded. If the file has indeed changed, a dialog pops up offering a chance to reload the YUV file.

YUV Detail Mode

In detail mode, the expected YUV values can be inspected. Note that the displayed sample values are the same as those in the previous mode. When a YUV file is loaded, the debug and delta YUV values can be inspected as well. Mismatching values are shown in red.

With “Diff YUV” button difference values switched on/off.

Find first difference with Debug YUV

The feature “Find first difference of reconstructed YUV with reference YUV” works for all codecs VP9, HEVC, AVC, MPEG2, AV1. By clicking the button with a magnifier icon, you can start a sequential search in decode order for the difference between the reconstructed YUV and the reference YUV that should be preloaded via YUVDiff menu. The search runs until the first difference is found or to the end of the stream. If you don’t want to wait for the search to finish, the process can be cancelled at any time with the cancel button in the info modal dialog. The search always starts from the first picture in decoder order.

UI Components

The following sections describe the various components in the UI when a bitstream is loaded.

Options menu

The options menu allows the user to choose between different types of operation modes. The first two options switch between ITU Rec. 601, ITU Rec. 709 and ITU Rec. 2020 color conversion standards, used to map YUV data on an RGB surface. The VQ Analyzer uses the chosen type of YUV conversion on its next picture conversion. Conversions that have already been completed will remain unchanged.

For specific coding modes, like HEVC screen content coding, two additional modes can be used: “YUV as RGB” and “YUV as BGR”. In this case, decoded YUV values are directly mapped onto RGB or BGR values for monitoring.

Color conversion can be achieved with the following transformation (0.0 <= [Y,R,G,B] <= 1.0) ; (-1.0 < [U,V] < 1.0)), where Y,U,V are normalized accordingly:

R = Y + V*(1-Kr)
G = Y - U*(1-Kb)*Kb/Kg - V*(1-Kr)*Kr/Kg
B = Y + U*(1-Kb),

Where Kr, Kb, Kg have the following values:

Kr Kg Kb
ITU Rec. 601 0.299 0.587 0.114
ITU Rec. 709 0.2126 0.7152 0.0722
ITU Rec. 2020 0.2627 0.6780 0.0593

“YUV as RGB” and “YUV as BGR” options are enabled only for YUV 4:4:4 streams. They can be used to represent screen content stream directly coded with RGB channels instead of YUV.

“Enable CPU optimizations []” option enables the use of optimized function for the architecture specified in square brackets. You can disable it if you suspect any odd behaviours from the VQ Analyzer.

“HEVC: Enable HEVC extensions” option enables HEVC RExt extension support (by default). It might be necessary to disable HEVC extensions support if you are decoding stream in HEVC v.1 format with the wrong headers.

“HEVC: Enable HEVC index” option enables HEVC index creation and use (filename.idx in the same directory) for faster access to the furthest frames in the stream. Index size can be huge, so please ensure you have enough disk space.

“HEVC: Allow empty tiles” option allows the VQ Analyzer to decode streams with empty tiles that are not valid cases according to the standard.

“HEVC: Intra mode direction according as in standard” option changes intra mode vectors direction as stated in the HEVC standard.

“Show MV in Prediction Mode” switches on/off MV draw in prediction mode, which is useful for removing MV mess if there are many long MVs in the picture.

“HEVC: Show only visible CTB MV” enables the drawing of MV for CTB in visible area only or draw of all MV in the picture.

“VP9 enable emulation” enables hw emulation in transformation code as in the reference software.

There is a set of options for different cases of hash calculations. These possibilities could cut waiting time for the decoded frame when digest value is not important:

“Force digest calculation” - always calculate digest regardless of the digest presence in the bitstream

“No digest calculation” - forces the VQ Analyzer to skip digest calculations

“As in bitstream” – digest calculation is dependent on the digest presence in the bitstreams. It only occurs when the bitstream has digest.

Stream View

Run full stream analysis

When you click “Run full stream analysis” the VQ Analyzer performs the following for all codecs: - decodes the full stream.

And for HEVC codec it: - creates an index file into the stream directory; - displays stream statistics in the Syntax Info View / Stats tab; - plots Block average QP in the Stream View / Buffer; - shows frames’ errors in the Status View / Errors Tab.

Thumbnails view

The top filmstrip is a horizontally scrolling overview of the pictures in the bitstream. The current picture is highlighted. When a new sequence begins, the top info label is highlighted in green. The current picture is highlighted with a red border. On top is the frame number in decode and display order respectively, and the picture type/structure as indicated by the frame header. Or there is the picture number - the decode order index in the bitstream - and the first slice in the picture type as indicated by the NAL unit types. On the bottom there is the layerId, shown as Lx where is x gives the layerId itself, and the POC value or indicated if the picture should be displayed or not. Referenced pictures for the current picture are indicated with arrows. Green arrows indicate the long-term references. The bar filmstrip has an underlining green band under those frames that begin new CVS. Picture type is colour coded: red – intra picture (I), blue – P-picture, green – B-picture. Red dots show how often the picture is used as a reference picture. The size of the dot is proportional to this usage index. By using this information, you can get an idea which image is most important in predicting other images.

The title bar shows the currently loaded stream and the selected codec for it. Clicking on a picture will cause the VQ Analyzer to decode it and mark it as the current picture, updating the rest of the UI correspondingly. If the new picture requires other pictures to be available as reference that have not yet been decoded, those will automatically be decoded.

Pictures are decoded on demand since it is impractical to store all details of all pictures in memory, especially for HD sequences. The current picture can also be selected by typing in the picture number in the box on the left of the panel. When using this box, the filmstrip is scrolled in such a way that the newly selected current picture is visible.

When a frame from the IVF or WebM/MKV container is a super frame (meaning 1 or more non-displayed picture and one displayable), this collection of pictures has superscription with a line over the frames belonging to the super frame. The bar filmstrip has an underlining green band under the frames belonging to the same super frame.

The application title bar shows the currently loaded stream, its selected codec and the process ID for easier identification of dock widgets. When any dock widget is detached from the main window, its title bar will also show the process ID of the main window.

By right-clicking on the selected picture you can see a popup submenu with a set of extract actions. An action called “Extract minimum CVS to this Picture” (HEVC spec.) will write out a bitstream that contains the minimum number of pictures needed to decode the current picture. Typically, this will consist of the current picture and all previous pictures that are reference pictures up to the nearest IRAP picture. This can be useful for debugging any issues in long sequences. For other codecs this action may be called “Extract pictures form last keyframe to the Picture” or “Extract pictures from last IDR to the Picture”. Other actions allow you to save in the output file statistics and intermediate/final YUV planes.

All separate panels can be detached from the main window or hidden. To reattach panels then, double click on the titlebar. To unhide a panel go to the View menu and select the appropriate panel.

Extended modes

There are a set of extended modes that can be accessed via menus in the Stream View toolbar. In the Thumbnails and Bar mode it is possible to choose between Display/Decode order representation with an additional click.

There are three possible modes with two sub-modes that can then be selected. Sub-modes of the currently selected mode change dynamically depending on the current mode.

Active/DPB refs

Sometimes, it can happen that the VQ Analyzer shows the reference on INTRA frame. It is not actually a mistake, because the VQ Analyzer shows the DPB reference by default - all frames stored in DPB and marked as used for reference. But some of those frames could be inactive and, as a result, not used for motion estimation. For INTRA frame all such references are inactive. To show only active references choose the “ActiveRefs” sub-mode from the popup toolbar.

Filter

Using the filter, you can hide the “Not displayed” frames.

Fields/Frames

The field option gives special visualization capabilities for streaming with fields. Namely, it shows references on a per-field basis. The VQ Analyzer shows the top and bottom fields as they occur in the stream. Their position is shifted according to whether the top or bottom flag is used for the field.

Bars view

Buffer view (Frame sizes view)

All supported codecs have Frame sizes view. When you choose the “Buffer” menu from popup toolbar, the frame sizes view will appear by default. This is a bar chart, plotted with frame sizes. Each bar is colored according to the frame type. When you move the mouse along the bar, the current frame is highlighted and the tooltip shows information about the current frame. By clicking on the selected frame you can decode it. When you press the left mouse button and drag, all plots move along both axes. Double clicking on the canvas area resets the plot back to default settings. You can zoom in and out with mouse wheel on both axis when the cursor is inside the canvas. If the mouse is to the left or below the main x and y axes you can zoom, pan or double click only on one axis (for example, keep the cursor to below the main chart to alter only the x-axis). All actions will then only be applied to that axis.

The movement of the plots is restricted to positive values only, so you can’t move or zoom in a negative direction. The currently decoded frame is shown with a vertical line and the frame number on it.

On the right axis, the average slice QP values are shown. When full stream analysis is complete, the VQ Analyzer adds average block QP plot that is also attached to the right axis. By double clicking on the ‘Moving average’ label item, the sliding window size can be adjusted in the separate dialog.

By double clicking on the ‘[30fps]’ label item, the fps value can be adjusted in the separate dialogue box. Clicking on the legend item dialogue can switch it on and off. The legend panel can also be moved by clicking and dragging.

The “HRD” sub-mode is enabled if a stream has HRD parameters in it, otherwise it is disabled (greyed out).

Blue dashed line shows final arrival time of picture.

General HRD mode (check box is unchecked) builds plot using information only about bitrate and removal time of pictures. So if any gaps between current picture earliest arrival time and previous picture final arrival time exit, then plot can be a little bit wrong. HRD mode with arrival times (check box is checked) builds plot using all available times and inserts points on initial arraival time, final arraival time and on removal time. Navigation is anchored by picture removal time in any mode (so we see rose line at removal picture time (this picture is decoded) and blue dashed line on its picture final arrival time).

A right-click on the central area of the plot in the VQ Analyzer shows additional options for plot adjustment. In this same popup menu you can export HRD timings into a .csv file.

Hierarchy view

The VQ Analyzer shows B-pyramid visualization when the Hierarchy in the Stream View option is selected. It shows the level of mutual B-frame references, frame numbers in decoding /display orders, frame type coded by color and frame references. By clicking on each frame you can highlight particular frame references.

The hierarchy view is available for HEVC and AVC bitstreams only.

Main Panel

The main panel displays the selected picture with visual annotations. The type of annotations and associated interactive behavior depends on the current mode. Clicking and dragging moves the picture around, and the mouse wheel zooms in or out around the mouse cursor.

At the top left of the panel, the current mode is displayed. Along the top right the current scale is shown. In all modes, the CTB row and column values are displayed along the left and top border of the picture, slices are indicated with thick red lines, dependent slice boundaries are dashed red lines and tile boundaries are thick green lines. The screenshot above shows an example of a stream that is 13x8 CTBs, has 6 tiles and a number of slice segments, some of which are dependent. The selected CU is outlined with a pink box.t mode and is displayed. Along the top right the current scale is shown. In all modes, the CTB row and column values are displayed along the left and top border of the picture, slices are indicated with thick red lines, dependent slice boundaries are dashed red lines and tile boundaries are thick green lines. The screenshot above shows an example of a stream that is 13x8 CTBs, has 6 tiles and a number of slice segments, some of which are dependent. The selected CU is outlined with a pink box.

The information in the top right corner shows the current zoom factor and pixel length at selected zoom. By double clicking on zoom factor you can reset it to 1x factor.

The bottom left side has a cluster of buttons:

  • The first submenu has special options to output the currently displayed information to the external image file with different scaling possibilities.
  • FS: Toggles full-screen mode, which hides all UI elements except the main panel. Keyboard shortcut is F.
  • +/-: Zooms in or out, centered on the center of the panel. Note that zooming with the mouse wheel is much easier than using these buttons. Keyboard shortcuts are + and -.
  • R: Resets the current zoom to fit the entire picture in the main panel. Keyboard shortcut is R.
  • YUV, Y, U, V: Toggles between per-component image, or full-color YUV.
  • Pic: Toggles the actual picture underneath the annotations. The picture is mode-dependent. For example, in prediction mode, the picture is comprised of the prediction samples before the transform signal has been added in. It is often helpful to turn off the picture in order to make the annotation easier to read. Keyboard shortcut is P.
  • Info: Toggles the annotations on or off. Keyboard shortcut is I.
  • Details: Switches to the detail mode if it is allowed in the currently selected mode
  • H/D: Toggles between hexadecimal and decimal display of the annotations. This applies to the values in the Syntax panel as well. Keyboard shortcuts are H and D.
  • Delta sign: Switches to the difference mode with slave bitstream if loaded and the difference mode is available.
  • There are five navigation buttons: first frame, previous frame, playback mode, next frame, last frame. The previous/last button will rewind, causing the current picture to be set to the first/last picture in the loaded bitstream. The "Play" button will start playback of the loaded bitstream at a reasonable rate, limited by CPU performance. All normal mode operations are possible during playback. Note that playback mode always operates in currently selected order in the filmstrip.

Modes

With a bitstream loaded, VQ Analyzer can be put into one of available modes using either the F1-F12 keys or using the Mode menu. The mode selection affects only what is displayed in the Main Panel. Most modes can also show details of the current selection (CU, MB …) . This can be toggled with the right mouse button or the main panel’s button strip. Each codec has its own modes, see codec spec.

YUV

YUV mode allows the user to inspect the final decoded sample values, without additional overlay data.

Info Overlays

There are a number of overlays that provide additional information about the picture that you’re currently investigating. When you switch to “Info Overlays” mode, several additional submenus will appear in the bottom-left panel of info window.

You can switch now between additional modes with the drop down menu on the right. PSNR and SSIM modes are enabled if you have loaded a reference YUV file.

Heat Map

The heat map mode gives a visual representation of how the compressed bits of the picture are distributed spatially. Blocks with more bits per pixel are brighter than blocks with fewer. By zooming in, actual CU sizes in bits will be displayed as soon as its size is in the borders of the CU. By default, this mode also shows the CU quadtree boundaries in blue. It may be useful to turn off this overlay using the “Info” button on the lower left of the main panel. Heat Map does not offer a detail mode.

Maximum shown CU depth level can be restricted with the “Depth” on-screen control. Click on ‘+’ and ‘-’ to choose your required depth level.

The gradient used to display the heat map can also be edited:

  • Drag the gradient markers up and down to reposition them.
  • Drag a marker away from the gradient to remove it. A red X indicates removal will take place.
  • Double-click a marker to change the color.
  • Double click the gradient bar to add a new marker.
QP Map

The QP Map shows per-block QP. CUs with bigger QP are brighter than CUs with smaller QP. Coloring of the QP range can be adjusted with a gradient tool on the right top corner. On the ‘skip’ blocks, the QP does not change and they are colored red.

PSNR

The PSNR map shows per-pixel PSNR values, normalized on block size. To enable this mode you have to load the reference YUV file via the DebugYUV menu. The value is calculated per block. A bigger value is shown with a brighter color. The color range can be adjusted with the gradient tool on the right top corner. If the block has no difference to the reference YUV, a letter ‘e’ is shown, standing for ‘exact match’.

SSIM

The SSIM map shows per-pixel PSNR values, normalized on CU size. To enable this mode you have to load a reference YUV file via the DebugYUV menu. The value is calculated per CU. Bigger values have a brighter white color.

Simple Motion

Simple Motion mode offers a way to quickly view the modes and motion of any picture at a glance. No overlays are present except a colored dot showing information about the PU’s mode. Green dots indicate intra prediction, and purple/orange L0/L1 motion vectors for inter prediction are drawn as simple lines. When the motion vector is very small, a dot is drawn instead so that the mode is still easy to identify. When a motion vector does not point to the 0-th index of its respective list, the vector is drawn using a dashed line.

Block info

This panel displays the syntax elements associated with the currently selected block.

CU, TU (HEVC, AVC)

These tabs contain a list of the CABAC syntax elements decoded, in decode order, of the currently selected CU or TU in the Main Panel. Note that only the syntax elements pertaining to the currently selected CU/TU are shown, not any of the syntax elements in a higher or lower nodes in the quadtree. So if a CU is selected that is split into 4 smaller CUs, only the split flag will be shown, not the syntax elements of CUs further down the quadtree.

The left side of the list contains two columns, R and V, which denote the state of the CABAC engine's Range (ivlCurrRange) and Value (ivlOffset) prior to the syntax element decoding process. Any part of the list can be selected and copied for pasting in other programs.

Clicking a syntax element will update the tree in the lower half of this panel. This tree displays the values of all CABAC state variables prior to the decoding of the selected syntax element.

CU (VVC)

These tabs contain a list of the syntax elements decoded of the currently selected CU in the Main Panel.

Block (VP9)

Since the VP9 reference software does not always assign a variable name to each decoded token, a descriptive name has been assigned which conveys the meaning or usage of the token.

Note that only the syntax elements at the currently selected block hierarchy are shown. To see the PART_SPLIT tokens at higher levels in the recursive subdivision process, repeatedly click the same block in the main panel to navigate the hierarchy.

The two left columns R and V represent the value of the bool coder’s range and value prior to decoding the token.

Block (AV1)

Since the AV1 reference software does not always assign a variable name to each decoded token, a descriptive name has been assigned which conveys the meaning or usage of the token.

Note that only the syntax elements at the currently selected block hierarchy are shown. To see the PART_SPLIT tokens at higher levels in the recursive subdivision process, repeatedly click the same block in the main panel to navigate the hierarchy.

The two columns Rng and Dif represent the value of the entropy coder’s range and value prior to decoding the token. The Cnt column corresponds to the internal entropy coder counter.

TX (VP9)

This panel displays the syntax elements associated with the currently selected transform block. Since the VP9 reference software does not always assign a variable name to each decoded token, a descriptive name has been assigned which conveys the meaning or usage of the token.

A particular transform block may be selected in the Main Panel (when in Transform mode), or by clicking in the diagram above the syntax list. Note that when not in detail mode, only the luma transform blocks can be selected from the Main Panel.

The two left columns R and V represent the value of the bool coder’s range and value prior to decoding the token.

MB (MPEG-2)

This panel displays the syntax elements associated with the currently selected macro block.

Stats

This tab displays various statistics extracted from the current picture. The top half shows some picture size and compression stats as well as the image digest information. There are also some min/max/average information about some picture parameters, such as QP value, MVs and so on, in this window.

The bottom half displays pie charts for a number of metrics. Each pie chart can be drawn normalized or un-normalized. Normalized data is weighted by area or compressed bits. For example, there may be a much lower number of 64x64 CUs than 8x8 CUs in the picture (smaller un-normalized pie wedges), but they could still make up the majority of the picture area, making the 64x64 pie wedge large. Normalized numbers in the pie chart are in units of pixels or bits, and un-normalized numbers are raw counts.

The pie chart can be moved by dragging the mouse, and zoomed with the mouse scroll wheel.

By selecting different modes in the stats combo box, you can explore different types of statistics in the bitstream, namely: picture statistics, per syntax element statistics, and overall stream statistics. Overall stream statistics is available only after a full stream analysis and shows accumulated values across the entire stream. Pie charts for stream statistics also show the accumulated values.

Pressing ‘%’ buttons changes the values in the corresponding view to the percentage view. The P/B button switches between Pie and horizontal Bar charts.

The inter-frame motion vector distribution can be viewed by selecting Motion vectors in the right combo box. This widget displays MV with maximum pel accuracy (1/4 pels) as 2D points. The user can switch separate MV values on or off by clicking on the corresponding MV legend.

Syntax panel

The left side of the UI is a resizable panel with several tabs that display information about the current selection in one way or another. The following subsections describe each tab.

NAL/OBU Tab

The NAL/OBU tab lists all units found in the bitstream, in decode order. Above the list the total number of units is displayed. The list has five columns: "Pos" is the byte position of the unit in the file, "Unit Type" is the textual description of the unit's type, "LId" is the Layer ID of the unit, "TID" is the Temporal ID of the unit, and "Size" indicates the total number of bytes in the unit.

Changes in coloring from white to grayscale and back show the Access Unit borders. By clicking on the branch sign you can roll out unit syntax elements as read from the bitstream. For each syntax element, the VQ Analyzer shows the value of it and the number of bits it takes in the bitstream. Red elements will help you to identify error syntax element.

Clicking on any unit will display up to the first 500 raw bytes in the Unit Info panel. If a unit is a VCL unit, meaning it is a coded slice segment, it can be double-clicked in order to bring forward the picture it belongs as the current picture. Conversely, changing the current picture will highlight the unit that contains the first slice segment of that picture. Selecting a CU in the Main Panel will highlight the unit containing the slice that contains the selected CU.

At the bottom of the unit tab is a button called "Extract selected units". Clicking this will write all selected units out to a file.

Header tabs

These tabs contain a list of every syntax element, in decode order, of the VPS, SPS, PPS, slice segment header and SEI messages that apply to the currently selected CU in the Main Panel. Syntax elements that come from a sub-function call are indented accordingly, and the function call itself appears in the list as a syntax element with no value. Any part of the list can be selected and copied for pasting in other programs.

For AVC bitstream SPS, PPS, Slice and SEI tabs are available, Sequence and Frame tabs for AV1 and Slice tab for MPEG-2.

The Frame tab shows all syntax elements decoded in the frame header, both the uncompressed and bool-coded partitions. A few stats about the size of the header are shown below the list.

Since the VP9 (AV1) reference software does not always assign a variable name to each decoded token, a descriptive name has been assigned which conveys the meaning or usage of the token.

The two left-hand columns R and V represent the value of the bool coder’s range and value prior to decoding the token. For the uncompressed part of the frame header, these values are not present.

By pressing the ‘H’ button in the header tab you can see hexadecimal representation of the frame header. This representation shows all bytes of the header, starting position, and number of bytes in the compressed and uncompressed sections. Uncompressed bytes are highlighted in blue.

Container (IVF or MKV)

Depending on the container type of the loaded bitstream, either the MKV or IVF panel will be present in the UI.

  • VP9 or AV1 streams using the WebM or Matroska container will cause the UI to display the container’s EBML document in tree form. Clicking an element causes the raw bytes coding that element to be displayed in the Raw Bytes panel (bottom center of the UI). When decoding a particular frame, the corresponding element will be highlighted in this panel. Other tracks may be present in the MKV file (such as audio), but they are not processed. The raw bytes can still be viewed though.
  • Streams containerized with the IVF format causes the UI to display the IVF information in this left-most tab. Selecting a frame makes the Raw Bytes panel show the raw bytes of that frame, including the IVF frame header. Clicking a frame will causes that frame to be decoded.

Selection Info

Selection Info (HEVC)

This panel gives some details about the current selection at a glance. All values are in decimal:

  • Picture height & width
  • Number of pictures in the bitstream
  • Color format of the current picture
  • Bitdepth values for Luma and Chroma components
  • Tile row/col position
  • Slice segment number of the slice segment that contains the selected CU, and the slice type.
  • CTB address and row/col position
  • Size of the selected CU. Note that it applies to the selection at any point in the quadtree, not necessarily just the leaf nodes.
  • X/Y position of the selected CU in pixels
  • CU prediction mode. Intra, Inter or Skip
  • Depth in the quadtree of the selected CU
  • Pixel X/Y position in the picture of the mouse cursor
  • Z-order index at 4x4 granularity in the CTB at the mouse cursor location
  • PU size and shape. Note that this will be equal to the size of the leaf CU in any mode other than Prediction.
  • TU size and depth. Note that this will be equal to the size of the leaf CU (top of TU tree) in any mode other than Transform.
  • The L0 and L1 motion vectors, if any, of the selected PU. The MVs are shown with an arrow => followed by the reference index.
  • The Luma QP value associated with the selected CU.

Selection Info (VVC)

  • Picture height & width
  • Number of pictures in the bitstream
  • Color format of the current picture
  • Bitdepth values for Luma and Chroma components
  • Tile row/col position
  • Slice segment number of the slice segment that contains the selected CU, and the slice type.
  • CTB address and row/col position
  • Size of the selected CU. Note that it applies to the selection at any point in the quadtree, not necessarily just the leaf nodes.
  • X/Y position of the selected CU in pixels
  • CU prediction mode. Intra, Inter or Skip
  • Depth in the quadtree of the selected CU
  • Pixel X/Y position in the picture of the mouse cursor
  • Z-order index at 4x4 granularity in the CTB at the mouse cursor location
  • TU size and depth. Note that this will be equal to the size of the leaf CU (top of TU tree) in any mode other than Transform.
  • The L0 and L1 motion vectors, if any, of the selected PU. The MVs are shown with an arrow => followed by the reference index.
  • The Luma QP value associated with the selected CU.

Selection Info (VP9)

This panel shows a few details about the current selection at a glance:

  • Frame height & width
  • Number of frames in the bitstream
  • Chroma format of the current picture
  • Bitdepth of the current picture
  • Tile row and column number
  • Current super-block row and column number
  • Segment ID of the selected block and size
  • X/Y position in pixels of the selected block’s upper left corner
  • MI (8x8 granularity) column and row number of the upper left of the selected block
  • Selected block’s partitioning mode
  • X/Y position of the mouse cursor in the frame
  • Size of the selected prediction block
  • Selected block’s prediction mode
  • Transform size used in the selected block (luma)
  • X/Y position in pixels of the selected transform block (Transform Mode)
  • Type of transform used on the selected transform block (Transform Mode)
  • Value of the motion vector(s) used by the current prediction block in 1/8th pel. An arrow and letter indicate the reference frame: L for Last, G for Golden and A for AltRef.

Selection Info (AVC)

This panel shows a few details about the current selection at a glance. All values are in decimal:

  • Picture height & width
  • Number of pictures in the bitstream
  • Chroma format of the current picture
  • Bitdepth of the current picture
  • Slice segment number of the slice segment that contains the selected CU, and the slice type.
  • MB address and row/col position
  • Selected MB type and flags for field MB coding and use 8x8 transform.
  • X/Y position of the selected MB in pixels

    MB prediction mode. Intra, Inter or Skip

  • Pixel X/Y position in the picture of the mouse cursor
  • Prediction size and shape.
  • The L0 and L1 motion vectors, if any, of the selected Prediction block. The MVs are shown with an arrow => followed by the reference index.
  • The Luma and Chroma QP values associated with the selected MB.

Selection Info (MPEG-2)

This panel shows a few details about the current selection at a glance:

  • Frame height & width
  • Number of frames in the bitstream
  • Chroma format (4:2:0, 4:2:2 or 4:4:4)
  • X/Y position of the mouse cursor in the frame
  • Current macro-block row and column number
  • Current macro-block number in slice
  • Macro-block numbers, which belong to current slice (“1st MB number” – “last MB number”)
  • MV list (up to four) of currently selected MB, where FMV0, BMV0 are forward and backward MVs for the first field and FMV1, BMV1 are forward and backward MVs for the second field. If no MV is present in stream, “--” sign is displayed.

Selection Info (AV1)

This panel shows a few details about the current selection at a glance:

  • Frame height & width
  • Number of frames in the bitstream
  • Chroma format of the current picture
  • Bitdepth of the current picture
  • Tile row and column number
  • Current super-block row and column number
  • Segment ID of the selected block and size
  • X/Y position in pixels of the selected block’s upper left corner
  • MI (8x8 granularity) column and row number of the upper left of the selected block
  • Selected block’s partitioning mode
  • X/Y position of the mouse cursor in the frame
  • Selected block’s prediction mode
  • Size of the selected prediction block
  • Transform size used in the selected block (luma)
  • X/Y position in pixels of the selected transform block (Transform Mode)
  • Type of transform used on the selected transform block (Transform Mode)
  • Value of the motion vector(s) used by the current prediction block in 1/8th pel. An arrow and letter indicate the reference frame: L for Last, G for Golden and A for AltRef.

Unit Info

The panel in the bottom center of the UI displays the raw bytes that were used to code the selected block or NAL unit. The current selection can be a Matroska element/frame or an IVF frame (depending on the current bitstream’s container). A few details about the selection are included as well.

NAL unit detail view

When you select an NAL unit in the NAL tab, the VQ Analyzer will immediately display up to the first 500 raw bytes in the Raw Bytes panel, and the controls for expanding NAL unit tree will appear from the left of the “NAL unit” keyword. Control representation is system-dependent.

By clicking on this control you can expand or collapse NAL unit tree items. The structure of this tree mostly replicates NAL unit definitions from the standard. There are items with statements and items with conditions. The items with conditions are greyed out if the condition is false. In this case, only the underlying structure definition from the standard is shown. Otherwise (if the condition is true), you can explore the real syntax values.

HEX View

All supported codecs can be switched to Unit Info window in HEX View mode. This is implemented with a middle mouse double-click inside the Unit Info window. In this mode, the ‘B’ button switches between hex and binary representation. The yellow background highlights the currently selected Unit (e.g. NAL unit, CU, …), showing start bit position on the first selected byte and end bit position on the last selected byte.

Status panel

Status

This tab in the lower left of the status panel displays messages about the decoding process and shows the progress of any actions that may take a while to complete. If a decoded picture’s digest does not match the SEI digest (when present), a warning will be displayed here.

Errors

The errors tab at the bottom left of the panel turns red if the VQ Analyzer detects any errors/warnings in the stream. By clicking on this button, the VQ Analyzer switches to and from the list of detected problems. By double-clicking on the selected error, you can highlight the corresponding block in the main panel window, and the corresponding NAL unit will be highlighted.

Status bar

The 3 numbers in the lower right of this panel represent the memory state of the process. From left to right the 3 numbers are:

  • Amount of memory used
  • Peak used memory
  • Maximum available memory

HEVC

The following sections describe all specific features available when loading an HEVC bitstream. The supported format of a bitstream is the raw bitstream with no surrounding container. Output from the publicly available HEVC reference software HM is in this format and can be opened directly. VQ Analyzer supports Main, Main 10, and Main Still Picture profiles, HEVC RExt profiles (including SCC), SHVC profile and HEVC MultiView profiles. Sequences that go beyond the profile limits maybe be supported as well.

Modes

Use Mode menu to see available modes.

Coding Flow

The coding flow mode gives a visual overview of the ordering of CTBs in the stream, and some information about the decoding process. The blue grid shows the boundaries of the CTBs in the picture. Each CTB contains 3 values:

  • The decode index of the CTB, showing the order of decode in the picture.
  • The CTB address, which is simply the raster scan index of the CTB.
  • The substream that the CTB belongs to. This number will only be greater than 0 if tiles or wavefront tools are employed in the picture.

Operations on the CABAC engine state are displayed as well. A small purple R at the top left of a CTB indicates the CABAC engine is reset before decoding that CTB. A small F indicates a CABAC flush at the end of a substream that is not the end of a slice. Purple arrows indicate CABAC engine state transfer or copy to dependent slices or wavefront rows:

Predictions

This mode shows the PUs in each CU. CU quad tree splits are indicated with blue lines, which get darker as the split depth increases. When a CU split is implied by HEVC and not directly coded in the bitstream, the split lines are dashed.

Intra modes are indicated with green colors, and directional modes also show an arrow indicating the prediction direction. In the lower right corner of an intra PU the chroma mode is indicated in a darker green. When the prediction shape is not the same as the CU size, the prediction unit shapes are shown with green lines for intra blocks, cyan for inter blocks. Dashed green lines in intra blocks means the PU was split along with the transform tree.

Inter PUs are indicated with cyan colors showing the PU splits and mode. Additionally, skipped blocks are shaded with a lined texture. An inter PU is Skip, AMVP or Merge mode. The L0 motion vectors are drawn with an orange color, the L1 motion vectors (B slices only) are drawn with a purple color. The MV value is shown in the lower left corner along with the reference index.

Clicking a PU will select it, and the syntax elements used to code it and the CU it belongs to are displayed in the CU tab on the left. Clicking repeatedly on the same PU will cycle through the CU hierarchy, showing the parent and child relationship.

Below is shown Prediction mode on a zoomed-in selection along the top edge of a picture. The selected CU is surrounded with a pink box while the selected PU is surrounded with a yellow box.

Prediction Detail Mode

In Prediction mode, the sample values of a particular PU can be viewed in detail.

Intra PUs are displayed in green, and the left, upper-left and above prediction arrays are drawn adjacent. Luma blocks show 3 versions of the predictions arrays. From furthest out to in, they represent the arrays in 3 steps:

  1. Initial neighboring samples. Unavailable samples are given with a red "X".
  2. Samples after reference sample substitution process.
  3. Samples after filtering process.

Chroma blocks only show the first two steps, since reference sample filtering is not applied to chroma intra prediction.

Inter PUs are displayed in cyan (turquoise). Each prediction sample can be clicked to display a window that contains the 8x8 or 4x4 sample array for luma or chroma respectively that are the inputs to the motion compensation filter for the selected pixel. In the case of a bi-directional PU, both arrays are shown the L0/L1 colors of orange/purple. Clicking the selected sample again or outside the PU closes this window. Underneath the prediction array values the motion vector predictor lists are displayed. In merge mode, the merge list is shown with the L0 predictor in orange and the L1 predictor in purple. An arrow indicates the merge candidate that is chosen by the bitstream (merge_idx).

When the inter PU does not code motion vectors in merge mode, it is considered AMVP mode (Advance Motion Vector Prediction). The 2-entry list for each motion vector (L0 and/or L1) is shown and the selected predictor is indicated with an arrow.

For HEVC screen content coding stream there is special detail mode exist – Palette mode. In this mode VQ Analyzer shows palette predictor, currently used palette, coded indices, resulting indices and final pixel values.

Transform

In Transform mode the transform trees and accompanying residual signal of the picture can be seen. As in Predictions mode, the CU boundaries are shown with blue lines. The transform splits, when present, are indicated with yellow lines. A dashed yellow line indicates that the TU split was implied. The QP values of each CU are shown in the upper-left corner. The number of non-zero coefficients is shown in the left-bottom corner. When this value is bright yellow, it means that a delta-QP was coded in the bitstream during the decode of that particular CU. Otherwise the number is dark. When DQP is enabled by the PPS, this helps visualize the DQP coding depth (diff_cu_qp_delta_depth). TUs that have transform_skip_flag set are shaded. CUs that are PCM coded are marked with "PCM" in white, and CUs that have cu_transquant_bypass_flag set have a white X through them.

If the "Pic" button is turned on, the raw residual signal is shown in image form. Transform values of 0 result in flat grey, negative values are darker and positive values are brighter.

Clicking on TUs causes that TU to be selected with a blue surrounding box and its syntax elements (when present) to be displayed in the TU tab of the left panel. Clicking a TU repeatedly causes the selection to move up and cycle though the TU, then CU quadtree hierarchy. Note that when the selected TU is a 4x4, the chroma coefficients are coded one level up in the TU hierarchy since HEVC doesn't specify a 2x2 chroma TU. So to view chroma TU syntax when the luma TU is 4x4, click the luma TU twice to move up to the 8x8 level where the chroma syntax is decoded.

Transform Detail Mode

To view the full details of a particular TU or group of TUs, make the TU selection and enter detail mode by right clicking or using the detail mode button at the bottom of the main panel. In this mode the selected TU structure is drawn three times, arranged in a column from top to bottom, showing the 3 major steps in recovering the residual signal:

  1. Coefficient decode. This diagram shows the scan order of the coefficients, the coefficient values and small icons indicating the syntax elements associated with coefficient decode. Refer to the legend shown in the top right of the main panel for the exact details. Coefficient groups are outlined with a green box. This box is bright if the coded_sub_block_flag was equal to 1, dark green if 0. A dashed line means the sub-group was implied to contain coefficients

  1. Inverse quantization. In this diagram the coefficient values are displayed after the inverse quantization process 8.6.3 from the HEVC spec has been applied.

  1. Inverse transformation. This diagram shows the recovered residual signal after the inverse transform process 8.6.4 from the HEVC spec.

Reconstruction

In Reconstruction mode the reconstructed samples can be inspected prior to deblocking. As with Prediction/Transform mode, CU boundaries are shown with blue lines. CUs that have at least one TU with non-zero coefficients are marked with "Non-zero CBF". PCM-coded CUs are indicated with "PCM" in white.

Reconstruction Detail Mode

In detail mode, the reconstructed sample values can be inspected. The selection may be a single CU, or a group of CUs up to and including the entire encompassing CTB.

Loop Filter

Loop Filter mode shows all edges processed by the deblocking filter as described in the MB.  Boundaries are not displayed in this mode, however MB selection still functions as in the other modes. Edges shown are for the luma deblocking process and are color coded in the following manner:

  • Green: Strong luma filter applied
  • Yellow: Weak luma filter applied
  • Red: No filter applied (but edge was evaluated)

Loop Filter Detail Mode

The deblocked sample values can be inspected directly in detail mode. As with Reconstruction Details, the inspected area can consist of one or more CUs. Sample values are indicated in yellow if they were modified by the deblocking process. Sample values that cannot be modified due to pcm_flag or cu_transquant_bypass_flag are indicated in red. Filtered edges belonging to the selected CU are shown in thick dashed lines, color coded the same way as in the non-detail mode. Each edge can be selected and clicked, bringing up a window showing the input values to the edge filter. Note that these values may differ from the reconstructed samples for horizontal edges, as they may have been modified by a vertical edge filter. Additionally, the boundary strength is displayed. Edges that were processed but ultimately not filtered (red colored) will indicate the reason. Typically it is due to the boundary strength being 0, the threshold condition not being met, or a slice/tile edge when those edges shouldn't be processed.

SAO

In SAO mode the SAO filter parameters assigned to each CTB are shown. The mode, and the associated four offsets for each component, are indicated with yellow text, and the merge_up / merge_left flags are indicated with an arrow. The CU quadtree boundaries are shown here as well.

SAO Detail Mode

Entering detail mode on a selected CU will show all samples after being processed by the SAO filter. Sample values that were actually modified by the SAO operation are highlighted in yellow. Each sample can be clicked, bringing up a 3x3 window that shows the input values to the filter process. In the screenshot below, the 45-degree edge filter inputs are the samples to the lower left and upper right. Samples that cannot be modified by the filter are indicated in red.

Info overlays

MV Heat

The MV Map show mv bits per CU. CUs with more mv bits are brighter than CUs with fewer. Coloring of the MV bits range can be adjusted with a gradient tool on the right top corner.

PU Type

The PU Type overlay shows a color map of their types. Intra blocks are colored in red, inter skip is yellow, inter merge is in green, and inter is blue. The legend on the right top corner show color types for blocks. The gradient band lets you adjust transparency when the Pic switch is on.

PU Reference Indices

PU reference indices shows reference indices for each PU. Orange color shows L0 reference index and purple color is for L1 reference index. When a PU has only one reference missing, the reference index is displayed with dash. For blocks with no reference indices, nothing is shown.

Syntax panel

QM

This tab displays the scaling lists, or quantizer matrices, used by the current picture. Scaling lists can be present in the SPS, PPS, or both. PPS scaling lists take precedence over SPS when present. The top of the panel shows in blue text how the various scaling lists are used in this picture. Below that, one of the 20 scaling lists from either the SPS or PPS can be chosen for inspection, and the chosen matrix is displayed in the grid. Matrices for 16x16 and larger TUs have a separate DC coefficient, which is displayed below the grid. Also shown in this panel is the list of syntax elements that is used to code the scaling list set, and is identical to the syntax in the corresponding SPS or PPS tab.

Ref Lists

This tab displays the details of the two reference lists L0 and L1, as well as the Reference Picture Set arrays used to construct the L0 and L1 lists. The L0 and L1 lists have the following columns:

  • Idx: The index associated with the reference picture
  • POC: The Picture Order Count of the reference picture
  • LT: A Yes/No flag indicating if the reference picture is a long-term picture or not
  • W Y: The derived luma weight used in the weighted prediction process of the reference picture
  • Y: The derived luma offset used in the weighted prediction process of the reference picture
  • W Cb: The derived chroma Cb weight used in the weighted prediction process of the reference picture
  • Cb: The derived chroma Cb offset used in the weighted prediction process of the reference picture
  • W Cr: The derived chroma Cr weight used in the weighted prediction process of the reference picture
  • Cr: The derived chroma Cr offset used in the weighted prediction process of the reference picture

The third panel contains, in textual form, the contents of the Reference Picture Set arrays from section 8.3.2 in the HEVC specification.

AVC

The following sections describe all available features when loading an AVC bitstream. The supported format of a bitstream is the raw bitstream with no surrounding container. Output from the publicly available AVC reference software JM is in this format and can be opened directly. VQ Analyzer supports profiles up to High profile. Sequences that go beyond the profile limits maybe be supported as well.

Modes

Use Mode menu to see available modes.

Coding Flow

The coding flow mode gives a visual overview of the ordering of MBs in the stream, and some information of the decoding process. The blue grid shows the boundaries of the MBs in the picture. Each MB contains ?he MB address, which is simply the raster scan index of the CTB.

Predictions

This mode shows the prediction units in each MB. MB splits are indicated with blue lines, which get darker as the split depth increases. When a MB split is implied by AVC (Direct mode for B slices) and not directly coded in the bitstream, the split lines are dashed.

Intra modes are indicated with green colors, and directional modes also show an arrow indicating the prediction direction. In the lower right corner of an intra MB the chroma mode is indicated in a darker green.

Inter Prediction units are indicated with cyan colors showing the Prediction unit splits and mode. Additionally, skipped blocks are shaded with a lined texture. The L0 motion vectors are drawn with an orange color, the L1 motion vectors (B slices only) are drawn with a purple color. The MV value is shown in the lower left corner along with the reference index.

Below is shown Prediction mode on a zoomed-in selection along the top edge of a picture. The selected MB is surrounded with a pink box while the selected Prediction Unit is surrounded with a yellow box.

Prediction Detail Mode

In Prediction mode, the sample values of a particular MB can be viewed in detail.

Intra MBs are displayed in green, and the left, upper-left and above prediction arrays are drawn adjacent.

Inter MBs are displayed in cyan (turquoise).

Transform

In Transform mode, the transforms and accompanying residual signal of the picture can be seen. As in Predictions mode, MB boundaries are shown with blue lines. The transform splits, when present, are indicated with yellow lines. QP values of each MB are shown in the upper-left corner. When this value is bright yellow, it means that a delta-QP was coded in the bitstream during the decode of that particular MB. Otherwise the number is dark. MBs that are PCM coded are marked with “PCM” in white.

If the "Pic" button is turned on, the raw residual signal is shown in image form. Residual values of 0 result in flat grey, negative values are darker and positive values are brighter.

Clicking on MBs causes that transform unit to be selected with a blue surrounding box. Clicking a transform unit repeatedly causes the selection to move up and cycle though the transforms in MB.

Transform Detail Mode

To view the full details of a particular MB, make the MB selection and enter detail mode by right clicking or using the detail mode button at the bottom of the main panel. In this mode the selected TU structure is drawn three times, arranged in a column from top to bottom, showing the 3 major steps in recovering the residual signal:

Reconstruction

In Reconstruction mode, the reconstructed samples prior to deblocking can be inspected. As with Prediction/Transform mode, MB boundaries are shown with blue lines. MBs that have at least one transform with non-zero coefficients are marked with “Non-zero CBP”. PCM-coded MBs are indicated with “PCM” in white.

Reconstruction Detail Mode

In detail mode, the reconstructed sample values can be inspected. The selection may be a single MB.

Loop Filter

Loop Filter mode shows all edges processed by the deblocking filter.MB boundaries are not displayed in this mode, however MB selection still functions as in the other modes. Edges shown are for the luma deblocking process and are color coded in the following manner:

  • Green: Strong luma filter applied
  • Yellow: Weak luma filter applied
  • Red: No filter applied (but edge was evaluated)

Loop Filter Detail Mode

The deblocked sample values can be inspected directly in detail mode. As with Reconstruction Details, Sample values are indicated in yellow if they were modified by the deblocking process. Sample values that cannot be modified due to PCM MB or bypass_flag are indicated in red. Filtered edges belonging to the selected CU are shown in thick dashed lines, color coded the same way as in the non-detail mode. Each edge can be selected and clicked, bringing up a window showing the input values to the edge filter. Note that these values may differ from the reconstructed samples for horizontal edges, as they may have been modified by a vertical edge filter. Additionally, the boundary strength is displayed. Edges that were processed but ultimately not filtered (red colored) will indicate the reason. Typically, it is due to the boundary strength being 0, the threshold condition not being met, or a slice/tile edge when those edges shouldn't be processed.

Info Overlays

MB Type

MB Type overlays show color map of their types. Intra blocks are colored in red, inter skips are yellow colored, B-Direct is in green and inter is in blue.

MB Reference Indices

MB reference indices show reference indices for each subblock 8x8. Orange color shows L0 reference index and purple color is for L1 reference index. When PU has only one reference missing reference index is displayed with a dash. For blocks with no reference indices nothing is shown.

Syntax panel

QM

This tab displays the scaling lists, or quantizer matrices, used by the current picture. Scaling lists can be present in the SPS, PPS, or both. PPS scaling lists take precedence over SPS when present. The top of the panel shows in blue text how the various scaling lists are used in this picture. Below that, one of the 20 scaling lists from either the SPS or PPS can be chosen for inspection, and the chosen matrix is displayed in the grid. Also shown in this panel is the list of syntax elements that is used to code the scaling list set, and is identical to the syntax in the corresponding SPS or PPS tab.

Ref Lists

This tab displays the details of the two reference lists L0 and L1, as well as the Reference Picture Set arrays used to construct the L0 and L1 lists. The L0 and L1 lists have the following columns:

  • Idx: The index associated with the reference picture
  • POC: The Picture Order Count of the reference picture
  • LT: A Yes/No flag indicating if the reference picture is a long-term picture or not
  • W Y: The derived luma weight used in the weighted prediction process of the reference picture
  • Y: The derived luma offset used in the weighted prediction process of the reference picture
  • W Cb: The derived chroma Cb weight used in the weighted prediction process of the reference picture
  • Cb: The derived chroma Cb offset used in the weighted prediction process of the reference picture
  • W Cr: The derived chroma Cr weight used in the weighted prediction process of the reference picture
  • Cr: The derived chroma Cr offset used in the weighted prediction process of the reference picture

The third panel contains, in textual form, the contents of the Reference Picture Set arrays from section 8.3.2 in the HEVC specification.

By clicking the ‘Reordered’ button you will see reference lists state before reordering.

VP9

The following sections describe all specific available features when loading a VP9 bitstream. The bitstream must be containerized with either the WebM or IVF container format. The publicly available VP9 reference software can output both formats. Raw uncontainerized bitstreams are presently not supported due to the nature of the VP9 standard; there is no way to determine where each frame starts without doing a full decode of the sequence.

Modes

Use the Mode menu to see available modes.

Coding Flow

Coding flow mode gives an overview of the way the frame is constructed. Each superblock’s decode index is shown in yellow and block-partioning is indicated with blue boundaries. Tile boundaries, when present, are shown with thick green lines and the tile size as coded in the bitstream is shown at the top left of each tile that has a size coded. When segmentation is enabled, each segment is shaded with a unique color. No detail mode is available.

Predictions

This mode shows the details of the prediction blocks in the frame. Block splits are indicated with blue lines, the selected block is surrounded with a pink box and the selected prediction block is surrounded with a yellow box. When a prediction block is selected, the syntax used to code it is displayed in the Block tab of the Left Panel. Clicking a block repeatedly will navigate the recursive block hierarchy.

Intra prediction block contains the intra luma mode in bright green, and the chroma intra mode in dark green near the bottom of the block. When the intra prediction operation is split due to the size of the transform blocks, the intra split boundaries are indicated with dashed green lines. Directional intra luma modes also contain a turquoise arrow indicating the prediction direction.

Inter prediction blocks show the inter mode in cyan, the first motion vector in orange and the second motion vector in purple. The motion vector value is in units of 1/8th pixels. After the motion vector value the reference frame it points to is indicated with a single letter: L for Last, G for Golden and A for AltRef. The motion compensation filter type that applies to the prediction block is indicated in yellow near the top right corner of the block.

Prediction Detail Mode

Entering detail mode on a selected prediction block allows the sample values of the prediction process to be viewed directly.

Intra-predicted blocks display the intra prediction process of each component. The left, above-left and above reference sample arrays are shown as well.

Inter-predicted blocks show the filtered prediction samples arrays of each component, the predictor list used for motion vector prediction (for each motion vector) and the filter type applied. To the left of the prediction sample arrays the impulse response of the applied filter is drawn (mostly decoratively). Each prediction sample can be clicked to display a window that contains the 8x8 or 4x4 sample array for luma or chroma respectively that are the inputs to the motion compensation filter for the selected pixel. The applicable filter phases are also indicated. In the case of compound prediction, both arrays & phase sets are shown with the first/second colors of orange/purple. Clicking the selected sample again or outside the block closes this window.

Transform

Transform mode allows for the inspection of the transform structure of each block. Block boundaries are displayed in blue, the currently selected block is outlined with a pink box and the selected transform block is outlined with a blue box. Blocks that code no residual signal are indicated with “Skip” in yellow. Blocks that use a transform size that is smaller than the block’s own size show the transform boundaries within the block in yellow.

Clicking a particular transform block causes the syntax elements used to decode it to be visible in the “TX” tab of the Left Panel.

The residual image itself can be shown using the “Pic” button on the bottom button strip. Residual values of 0 are flat gray, negative values are darker and positive values are brighter.

The VQ Analyzer shows the number of non-zero coefficients with “Coeffs: 2” label on the corresponding block.

Transform Detail Mode

To view the full details of the transform blocks belonging to a particular block, select a transform block and enter detail mode by right clicking or using the detail mode button at the bottom of the main panel. The currently selected transform block is highlighted with the sample grid. The transform structure of the selected block is drawn three times, and arranged in a column from top to bottom, showing the 3 major steps in recovering the residual signal:

  1. Decode and inverse scan process. In this diagram the scan pattern is drawn for each transform block. In an attempt to reduce clutter, larger scan pattern jumps are drawn with a darker color. The currently selected transform block is highlighted with the sample grid. Moving the mouse over a particular coefficient will highlight the previous (red) and next coefficient (cyan) in scan order.

  1. Inverse scaling process. The scaled values and the scaling factors used to obtain them are shown in this diagram.

  1. Inverse transform process. This recovered residual values and the transform type used are displayed in this diagram.

Reconstruction

In Reconstruction mode, the reconstructed samples can be inspected prior to loop filtering. As with Prediction/Transform mode, block boundaries are shown with blue lines. Blocks with no coefficients are marked with “Skip” in white.

Reconstruction Detail Mode

In detail mode, the reconstructed sample values can be inspected. The selection may be a single block, or a group of blocks up to and including the entire encompassing super-block.

Loop Filter

Loop filter mode shows all processed edges in the frame. Edges are 1 sample high/tall, and are color coded as follows:

  • Green - 16-tap filter
  • Yellow - 8-tap filter
  • Orange - 4-tap filter
  • Red - Not filtered, threshold not met (but processed).

Loop Filter Detail Mode

The filtered sample values can be inspected directly in detail mode. As with Reconstruction Details, the inspected area can consist of one or more blocks. Sample values are indicated in yellow if they were modified by the deblocking process. Samples to the left and above the current block needed for the filter are shown as well. Filtered edges belonging to the selected blocks are shown in thick dashed lines, color coded the same way as in non-detail mode. Each edge can be selected and clicked, bringing up a window showing the input values to the edge filter. Note that these values may differ from the reconstructed samples, as they may have been modified by an earlier edge filter.

Info Overlays

Block Type

Block Type overlay shows a color map of block types. Intra blocks are colored in red, inter skip are yellow colored and inter are in blue.

Efficiency Map

Efficiency map mode shows visually how efficient the arithmetic coding of each block is. Blocks with more bools per bits are brighter than blocks with less. By default this mode also shows the block boundaries in blue. It may be useful to turn off this overlay using the "Info" button on the lower left of the main panel. Efficiency Map does not offer a detail mode.

Syntax panel

Probabilities

The probabilities used to decode the current frame are visible in this panel. They are organized hierarchically in the same way as the VP9 reference software. Each probability is given as three numbers. From left to right:

  1. Probability as it was at the start of the frame decode process. For keyframes or when the reset_frame_context flag is equal to 1, this is the VP9 default value.
  2. Probability value after modification by the bool-coded section of the frame header.
  3. Probability value after modification by the adaptation process after frame decode.

When the probability adaptation process causes the value to actually change, the probability is shown in blue. The total number of probability values that were actually changed is given below the tree.

Counts

This tab shows the values of all the counters used for probability adaptation at the end of the frame decode. The counters are organized in hierarchical fashion in the same way as the VP9 reference decoder. Color bars give an indication of relative size of each counter in its group, and the parent icon contains the sum of the counts.

Refs

This panel displays the data that persists in the decoder between frames:

  • The 8-entry pool of reference frames and which of each of these is considered the Last, Golden and AltRef frame. If any of the frames should be replaced with the current frame after decode (as specified by refresh_frame_flags), the right most column will indicate that.
  • Segmentation data for each segment.
  • The 4-entry frame context buffer as it applies to the current frame. The center column indicates the frame that generated that context, and the right column indicates whether or not it was the result of adaptation at that time.
  • Loop filter deltas for all four references and both modes.

AV1

The following sections describe all available features when loading a AV1 bitstream. The bitstream must be containerized with either the WebM or IVF container format. The publicly available AV1 reference software can output both formats. Raw, uncontainerized bitstreams are presently not supported due to the nature of the AV1 standard; there is no way to determine where each frame starts without doing a full decode of the sequence.

Modes

Use the Mode menu to see available modes.

Coding Flow

Coding flow mode gives an overview of the way the frame is constructed. Each superblock’s decode index is shown in yellow and block-partioning is indicated with blue boundaries. Tile boundaries, when present, are shown with thick green lines and the tile size, as coded in the bitstream, is shown at the top left of each tile that has a size coded. When segmentation is enabled, each segment is shaded with a unique color. No detail mode is available.

Predictions

This mode shows the details of the prediction blocks in the frame. Block splits are indicated with blue lines, the selected block is surrounded with a pink box and the selected prediction block is surrounded with a yellow box. When a prediction block is selected, the syntax used to code it is displayed in the Block tab of the Left Panel. Clicking a block repeatedly will navigate the recursive block hierarchy.

Intra prediction blocks contain the intra luma mode in bright green, and the chroma intra mode in dark green near the bottom of the block. When the intra prediction operation is split due to the size of the transform blocks, the intra split boundaries are indicated with dashed green lines. Directional intra luma modes also contain a turquoise arrow indicating the prediction direction.

Inter prediction blocks show the inter mode in cyan, the first motion vector in orange and the second motion vector in purple. The motion vector value is in units of 1/8th pixels. After the motion vector value, the reference frame it points to is indicated with a single letter: L for Last, G for Golden and A for AltRef. The motion compensation filter type that applies to the prediction block is indicated in yellow near the top right corner of the block. If inter mode contains intrainter prediction intra mode, this will be written under inter mode in bright green.

The VQ Analyzer shows horizontal and vertical filtering separately for each block on the left and right top corners accordingly. Motion type is shown in the bottom right corner of the block.

Prediction Detail Mode

Entering detail mode on a selected prediction block allows the sample values of the prediction process to be viewed directly.

Intra-predicted blocks display the intra prediction process of each component. The left, above-left and above reference sample arrays are shown as well.

Inter-predicted blocks show the filtered prediction samples arrays of each component, the predictor list used for motion vector prediction (for each motion vector) and the filter type applied.

Zero MV - motion vector mv representing global motion for this block

MV Stack: motion vectors after Find MV Stack Process (This process constructs an array RefStackMv containing motion vector candidates).

MV Stack Sorted: motion vectors from MV Stack after sorting, extra search and clamping processes.

Transform

Transform mode allows inspection of the transform structure of each block. Block boundaries are displayed in blue, the currently selected block is outlined with a pink box and the selected transform block is outlined with a blue box. Blocks that code no residual signal are indicated with “Skip” in yellow. Blocks that use a transform size that is smaller than the block’s own size show the transform boundaries within the block in yellow.

The residual image itself can be shown using the “Pic” button on the bottom button strip. Residual values of 0 are flat gray, negative values are darker and positive values are brighter.

Transform Detail Mode

To view the full details of the transform blocks belonging to a particular block, select a transform block and enter detail mode by right clicking or using the detail mode button at the bottom of the main panel. The currently selected transform block is highlighted with the sample grid. The transform structure of the selected block is drawn three times, arranged in a column from top to bottom, showing the 3 major steps in recovering the residual signal:

  1. Decode and inverse scan process. In this diagram, the scan pattern is drawn for each transform block. In an attempt to reduce clutter, larger scan pattern jumps are drawn with a darker color. The currently selected transform block is highlighted with the sample grid. Moving the mouse over a particular coefficient will highlight the previous and the next coefficient in scan order.

  1. Inverse scaling process. The scaled values and the scaling factors used to obtain them are shown in this diagram.

  1. Inverse transform process. This recovered residual values and the transform type used are displayed in this diagram.

Reconstruction

In Reconstruction mode the reconstructed samples prior to loop filtering can be inspected. As with Prediction/Residual mode, block boundaries are shown with blue lines. Blocks with no coefficients are marked with “Skip” in white.

Reconstruction Detail Mode

In detail mode, the reconstructed sample values can be inspected. The selection may be a single block, or a group of blocks up to and including the entire encompassing super-block.

Loop Filter

Loop filter mode shows all the processed edges in the frame. Edges are 1 sample high/tall, and are color coded as follows:

  • Green - 16-tap filter
  • Yellow - 8-tap filter
  • Orange - 4-tap filter
  • Red - Not filtered, threshold not met (but processed).

Loop Filter Detail Mode

The filtered sample values can be inspected directly in detail mode. As with Reconstruction Details, the inspected area can consist of one or more blocks. Sample values are indicated in yellow if they were modified by the deblocking process. Samples to the left and above the current block need for the filter are shown as well. Filtered edges belonging to the selected blocks are shown in thick dashed lines, color coded the same way as in non-detail mode. Each edge can be selected and clicked, bringing up a window showing the input values to the edge filter. Note that these values may differ from the reconstructed samples, as they may have been modified by an earlier edge filter.

CDEF Filter

CDEF mode shows all the processed blocks in the frame. There are three parameters shown for each block: level, strength and CDEF direction.

Detail CDEF mode

The filtered sample values can be inspected directly in detail mode. As with Reconstruction Details, the inspected area can consist of one or more blocks. Sample values are indicated in yellow if they were modified by the CDEF process.

SuperRes Filter

If superres is enabled, in SuperRes Filter mode upscaled bloсks can be expected. Scaling factor is shown in top left. Upscaled blocks splits are indicated with grey lines. By left mouse click, block 8x8 or 64x64 pixels for viewing in detail can be selected.

Detail SuperRes Mode

If superres is enabled, in SuperRes Filter detail mode upscaled pixels can be expected. By click on a pixel, superres params are shown. For 64x64 blocks above and bottom stripes are present.

Loop restoration Filter

Loop restoration filter shows all processed blocks in the frame. The information on each block shows the filter type and the size of the block itself.

The analyzer shows more filter parameters in detail mode.

Film Grain Pixels

If film grain is enabled, in Film Grain Pixels mode samples after film grain synthesis process can be expected. 32x32 blocks can be selected for detailed view.

Detail Film Grain Mode

Entering detail mode on a selected 32x32 block will show all samples after the film grain synthesis process. Sample values that were actually modified by the film grain synthesis process are highlighted in yellow. Each sample can be clicked, bringing up a grey window with formula for calculating value.

In the bottom by click on “Grain info details” LumaGrain, CbGrain and CrGrain arrays can be expected. Each sample can be clicked, bringing up a gray window with formula for calculating value.

Info Overlays

Block Type

Block Type overlay shows a color map of the various types. Intra blocks are colored in red, inter skip are yellow colored and inter are in blue.

Efficiency Map

Efficiency map mode shows visually how efficient the arithmetic coding of each block is. Blocks with more bools per bits are brighter than those with fewer. By default, this mode also shows the block boundaries in blue. It may be useful to turn off this overlay using the "Info" button on the lower left of the main panel. Efficiency Map does not offer a detail mode.

Syntax panel

Refs

This panel displays the data that persists in the decoder between frames:

  • The 8-entry pool of reference frames and where each of these is considered the Last, Golden and AltRef frame. If any of the frames should be replaced with the current frame after decode (as specified by refresh_frame_flags), the right most column will indicate that.
  • Segmentation data for each segment.
  • The 4-entry frame context buffer as it applies to the current frame. The center column indicates the frame that generated that context, and the right column indicates whether or not it was the result of adaptation at that time.
  • Loop filter deltas for all four references and both modes.

MPEG-2

The following sections describe all available features when loading a MPEG-2 bitstream. The bitstream must be containerized with mpeg-2 container format.

Modes

Use the Mode menu to see available modes.

Predictions

This mode shows the details of the prediction blocks in the frame. The selected block is surrounded with a pink box. When a prediction block is selected, the syntax used to code it is displayed in the Block tab of the Left Panel.

Intra prediction block contains the intra luma mode in bright green, and can contain so-called “concealment” motion vectors if they are present in bitstream.

Inter prediction blocks show the inter mode in cyan. Up to two pairs of MVs are drawn in a block. The left pair corresponds to the first field. The right one – to the second one (if any). The forward motion vector in orange and the backward one in purple. The motion vector value is in units of 1/8th pixels.

Prediction Detail Mode

Entering detail mode on a selected prediction block allows the sample values of the prediction process to be viewed directly. No details mode is available for intra blocks.

Inter-predicted blocks display contains predicted values for luma and chroma components.

Transform

Transform mode allows the user to inspect the transform structure of each block. Block boundaries are displayed in blue, and the currently selected block is outlined with a pink box. Blocks that code no residual signal are indicated with “Skip” label.

Clicking a particular transform block causes the syntax elements used to decode it to be visible in the “MB” tab of the Left Panel.

The residual image itself can be shown using the “Pic” button on the bottom button strip. Residual values of 0 are flat gray, negative values are darker and positive values are brighter.

Transform Detail Mode

To view the full details of the transform blocks belonging to a particular block, selected a transform block and enter detail mode by right clicking or using the detail mode button at the bottom of the main panel. The currently selected transform block is highlighted with the sample grid. The transform structure of the selected block is drawn two times, arranged in a column from top to bottom, showing the 2 major steps in recovering the residual signal:

  1. Inverse scaling process. The scaled values and the scaling factors used to obtain them are shown in this diagram.
  2. Inverse transform process. This recovered residual values and the transform type used are displayed in this diagram.

Qp Map

Efficiency map mode gives a visual representation of how QP values are distributed in the image.

Syntax panel

Pictures (bitstream)

In the “Pictures” tab, the VQ Analyzer presents the raw picture sequence with type, offset and size.

Sequence

The Sequence tab shows all syntax elements decoded in the sequence header and sequence extension header, both the uncompressed and bool-coded partitions.

GOP

The GOP tab shows all syntax element decoded in GOP header, both the uncompressed and bool-coded partitions.

Picture

The Picture tab shows all syntax element decoded in picture header, picture coding extension, picture display extension and picture spatial scalable extension, all of them are uncompressed and bool-coded.

User

This panel displays the data that is decoded in user extension.

VVC

The following sections describe all specific features available when loading a VVC bitstream. The supported format of a bitstream is the raw bitstream with no surrounding container. Output from the publicly available VVC reference software HM is in this format and can be opened directly. VQ Analyzer supports Main, Main 10, and Main Still Picture profiles, VVC RExt profiles (including SCC), SHVC profile and VVC MultiView profiles. Sequences that go beyond the profile limits maybe be supported as well.

Dual Tree

For I slices if separate coding tree syntax structure is used luma or chroma coding trees are displayed in main window. Use the check box in lower right corner for switching between luma and chroma.

In this case luma and chroma samples in detail mode will be presented separatly acording to check box.

Modes

Use Mode menu to see available modes.

Coding Flow

The coding flow mode gives a visual overview of CTBs. The blue grid shows the boundaries of the CTBs in the picture. Tile boundaries, when present, are shown with thick green lines, slice boundaries - with thick red lines and subpicture boundaries - with thick beige lines.

Predictions

Intra modes are indicated with green colors, and directional modes also show an arrow indicating the prediction direction. In the lower right corner of an intra CU the chroma mode is indicated in a darker green.

Inter CUs are indicated with cyan colors showing the CU splits and mode. Additionally, skipped blocks are shaded with a lined texture. An inter CU is AMVP, Merge Regular, Merge GEO, Merge CIIP, AFFINE 4Pram/6Param. The L0 motion vectors are drawn with an orange color, the L1 motion vectors are drawn with a purple color. The MV value is shown in the lower left corner along with the reference index.

Clicking a CU will select it, and the syntax elements used to code it are displayed in the CU tab on the left. Clicking repeatedly on the same CU will cycle through the CU hierarchy, showing the parent and child relationship.

Below is shown Prediction mode on a zoomed-in selection along the top edge of a picture. The selected CU is surrounded with a pink box while the selected CU is surrounded with a yellow box.

For affine prediction blocks L0 and L1 motion vectors are drawn in the corners. Affine motion model is presented as dashed parallelograms.

Prediction Detail Mode

In prediction mode the sample values of a particular PU can be viewed in detail by right mouse click or “Details” button on main panel.

Intra PUs are displayed in green. The left, left-above and above prediction arrays are drawn next to the detailed samples.

If sps_mrl_enabled_flag is equal to 1 for intra blocks in details three reference lines of pixels (0, 1 and 3) is printed. The line selected for prediction is marked with an asterisk and label with this information.

Inter PUs are displayed in cyan (turquoise).

For inter blocks detailed information is drawn:

  • displacements along vectors from references;
  • reference blocks arrays got by 1/16 pixel accuracy.

For IBC, AMVP and Affine modes coded motion vectors differences are displayed. For IBC, AMVP and Affine modes motion vectors candidates are displayed with a sign for the chosen one. For IBC blocks block vector coordinates are displayed. For Merge modes merge index is displayed. For GEO Merge blocks split direction and two merge indices is printed. For MMVD blocks MVD info is displayed. For Merge Affine blocks also merge type is displayed. For CIIP blocks inter and intra weights values are printed.

Transform

In Transform mode the transform trees and accompanying residual signal of the picture can be seen. As in Predictions mode, the CU boundaries are shown with blue lines. The QP values of each CU are shown in the left-lower corner. The number of coefficients is shown in the left-upper corner. Transform types are printed in the center. CUs that are BDPCM coded are marked with “PCM” in white.

If the "Pic" button is turned on, the raw residual signal is shown in image form. Transform values of 0 result in flat grey, negative values are darker and positive values are brighter.

Clicking on TUs causes that TU to be selected with a blue surrounding box. Clicking a TU repeatedly causes the selection to move up and cycle though the TU, then CU quadtree hierarchy.

Transform Detail Mode

To view the details of a particular TU, make the TU selection and enter detail mode by right clicking or using the detail mode button at the bottom of the main panel.

Reconstruction

Reconstruction Detail Mode

Inverse Map

Inverse Map Detail Mode

Loop Filter

Loop Filter mode shows all edges processed by the deblocking filter. Edges shown are for the luma deblocking process and are color coded in the following manner:

  • Green: Strong luma filter applied
  • Yellow: Weak luma filter applied
  • Red: No filter applied (but edge was evaluated)

Loop Filter Detail Mode

The deblocked sample values can be inspected directly in detail mode. Sample values are indicated in yellow if they were modified by the deblocking process. Filtered edges belonging to the selected CU are shown in thick dashed lines, color coded the same way as in the non-detail mode. Each edge can be selected and clicked, bringing up a window showing the input values to the edge filter. Additionally, the boundary strength is displayed. Edges that were processed but ultimately not filtered (red colored) will indicate the reason.

SAO

In SAO mode the SAO filter parameters are shown. The mode, and the associated four offsets for each component, are indicated with yellow text, and the merge_up / merge_left flags are indicated with an arrow.

SAO Detail Mode

Entering detail mode on a selected CU will show all samples after being processed by the SAO filter. Sample values that were actually modified by the SAO operation are highlighted in yellow. Each sample can be clicked, bringing up a 3x3 window that shows the input values to the filter process. In the screenshot below, the 45-degree edge filter inputs are the samples to the lower left and upper right. Samples that cannot be modified by the filter are indicated in red.

Info overlays

Inter Memory Reads

Syntax panel

Ref Lists

This tab displays the details of the two reference lists L0 and L1. The L0 and L1 lists have the following columns:

  • Idx: The index associated with the reference picture
  • Layer id
  • POC: The Picture Order Count of the reference picture
  • LT: A Yes/No flag indicating if the reference picture is a long-term picture or not

YUV

The following sections describe all available features when loading a YUV file. YUV allows the user to view different types of raw YUV data.

On loading a YUV file, the VQ Analyzer will try to guess the YUV width height and bitdepth based on file name and internal data. Files with special naming like file_widthxheight.yuv will skip internal sizes detection and use sizes from file name.

In any case, the VQ Analyzer will show a loading dialog for the YUV file to adjust loading parameters like width, height, chroma format, plane order, bitdepth, bits per value, and chroma order.

Width and Height labels are clickable. You can input values directly in it or use ‘Select predifined value’ combobox to select predifined values, or you can use the slider to choose sizes.

The bottom slider will change the frame and allows you to quickly and easily check the bitstream.

Miscellaneous

A few other features to keep in mind:

  • VQ Analyzer saves various settings upon exit. These settings include window size, recent files etc., and are restored on the next launch.
  • If the loaded bitstream is modified or deleted while loaded, a dialog will pop-up asking to reload or close. This protects against unintended changes.

Attributions

Qt

Qt licensed under the GNU Lesser General Public License (LGPL) version 3. It is appropriate for the development of Qt applications provided you can comply with the terms and conditions of the GNU LGPL version 3. For details, please refer to http://qt-project.org/doc/qt-5/licensing.html for specific details.

Qwt

Qwt License Version 1.0, January 1, 2003

The Qwt library and included programs are provided under the terms of the GNU LESSER GENERAL PUBLIC LICENSE (LGPL) with the following exceptions:

  1. Widgets that are subclassed from Qwt widgets do not constitute a derivative work.
  2. Static linking of applications and widgets to the Qwt library does not constitute a derivative work and does not require the author to provide source code for the application or widget, use the shared Qwt libraries, or link their applications or widgets against a user-supplied version of Qwt. If you link the application or widget to a modified version of Qwt, then the changes to Qwt must be provided under the terms of the LGPL in sections 1, 2, and 4.
  3. You do not have to provide a copy of the Qwt license with programs that are linked to the Qwt library, nor do you have to identify the Qwt license in your program or documentation as required by section 6 of the LGPL.

However, programs must still identify their use of Qwt.

The following example statement can be included in user documentation to satisfy this requirement: [program/widget] is based in part on the work of the Qwt project (http://qwt.sf.net).

p. 28961