372 lines
17 KiB
HTML
372 lines
17 KiB
HTML
<HTML><HEAD><TITLE>xiph.org: Ogg Vorbis documentation</TITLE>
|
|
<BODY bgcolor="#ffffff" text="#202020" link="#006666" vlink="#000000">
|
|
<nobr><img src="white-ogg.png"><img src="vorbisword2.png"></nobr><p>
|
|
|
|
|
|
<h1><font color=#000070>
|
|
Ogg Vorbis stereo-specific channel coupling discussion
|
|
</font></h1>
|
|
|
|
<em>Last update to this document: July 16, 2002</em><br>
|
|
|
|
<h2>Abstract</h2> The Vorbis audio CODEC provides a channel coupling
|
|
mechanisms designed to reduce effective bitrate by both eliminating
|
|
interchannel redundancy and eliminating stereo image information
|
|
labeled inaudible or undesirable according to spatial psychoacoustic
|
|
models. This document describes both the mechanical coupling
|
|
mechanisms available within the Vorbis specification, as well as the
|
|
specific stereo coupling models used by the reference
|
|
<tt>libvorbis</tt> codec provided by xiph.org.
|
|
|
|
<h2>Mechanisms</h2>
|
|
|
|
In encoder release beta 4 and earlier, Vorbis supported multiple
|
|
channel encoding, but the channels were encoded entirely separately
|
|
with no cross-analysis or redundancy elimination between channels.
|
|
This multichannel strategy is very similar to the mp3's <em>dual
|
|
stereo</em> mode and Vorbis uses the same name for its analogous
|
|
uncoupled multichannel modes.<p>
|
|
|
|
However, the Vorbis spec provides for, and Vorbis release 1.0 rc1 and
|
|
later implement a coupled channel strategy. Vorbis has two specific
|
|
mechanisms that may be used alone or in conjunction to implement
|
|
channel coupling. The first is <em>channel interleaving</em> via
|
|
residue backend type 2, and the second is <em>square polar
|
|
mapping</em>. These two general mechanisms are particularly well
|
|
suited to coupling due to the structure of Vorbis encoding, as we'll
|
|
explore below, and using both we can implement both totally
|
|
<em>lossless stereo image coupling</em> [bit-for-bit decode-identical
|
|
to uncoupled modes], as well as various lossy models that seek to
|
|
eliminate inaudible or unimportant aspects of the stereo image in
|
|
order to enhance bitrate. The exact coupling implementation is
|
|
generalized to allow the encoder a great deal of flexibility in
|
|
implementation of a stereo or surround model without requiring any
|
|
significant complexity increase over the combinatorially simpler
|
|
mid/side joint stereo of mp3 and other current audio codecs.<p>
|
|
|
|
A particular Vorbis bitstream may apply channel coupling directly to
|
|
more than a pair of channels; polar mapping is hierarchical such that
|
|
polar coupling may be extrapolated to an arbitrary number of channels
|
|
and is not restricted to only stereo, quadraphonics, ambisonics or 5.1
|
|
surround. However, the scope of this document restricts itself to the
|
|
stereo coupling case.<p>
|
|
|
|
<h3>Square Polar Mapping</h3>
|
|
|
|
<h4>maximal correlation</h4>
|
|
|
|
Recall that the basic structure of a a Vorbis I stream first generates
|
|
from input audio a spectral 'floor' function that serves as an
|
|
MDCT-domain whitening filter. This floor is meant to represent the
|
|
rough envelope of the frequency spectrum, using whatever metric the
|
|
encoder cares to define. This floor is subtracted from the log
|
|
frequency spectrum, effectively normalizing the spectrum by frequency.
|
|
Each input channel is associated with a unique floor function.<p>
|
|
|
|
The basic idea behind any stereo coupling is that the left and right
|
|
channels usually correlate. This correlation is even stronger if one
|
|
first accounts for energy differences in any given frequency band
|
|
across left and right; think for example of individual instruments
|
|
mixed into different portions of the stereo image, or a stereo
|
|
recording with a dominant feature not perfectly in the center. The
|
|
floor functions, each specific to a channel, provide the perfect means
|
|
of normalizing left and right energies across the spectrum to maximize
|
|
correlation before coupling. This feature of the Vorbis format is not
|
|
a convenient accident.<p>
|
|
|
|
Because we strive to maximally correlate the left and right channels
|
|
and generally succeed in doing so, left and right residue is typically
|
|
nearly identical. We could use channel interleaving (discussed below)
|
|
alone to efficiently remove the redundancy between the left and right
|
|
channels as a side effect of entropy encoding, but a polar
|
|
representation gives benefits when left/right correlation is
|
|
strong. <p>
|
|
|
|
<h4>point and diffuse imaging</h4>
|
|
|
|
The first advantage of a polar representation is that it effectively
|
|
separates the spatial audio information into a 'point image'
|
|
(magnitude) at a given frequency and located somewhere in the sound
|
|
field, and a 'diffuse image' (angle) that fills a large amount of
|
|
space simultaneously. Even if we preserve only the magnitude (point)
|
|
data, a detailed and carefully chosen floor function in each channel
|
|
provides us with a free, fine-grained, frequency relative intensity
|
|
stereo*. Angle information represents diffuse sound fields, such as
|
|
reverberation that fills the entire space simultaneously.<p>
|
|
|
|
*<em>Because the Vorbis model supports a number of different possible
|
|
stereo models and these models may be mixed, we do not use the term
|
|
'intensity stereo' talking about Vorbis; instead we use the terms
|
|
'point stereo', 'phase stereo' and subcategories of each.</em><p>
|
|
|
|
The majority of a stereo image is representable by polar magnitude
|
|
alone, as strong sounds tend to be produced at near-point sources;
|
|
even non-diffuse, fast, sharp echoes track very accurately using
|
|
magnitude representation almost alone (for those experimenting with
|
|
Vorbis tuning, this strategy works much better with the precise,
|
|
piecewise control of floor 1; the continuous approximation of floor 0
|
|
results in unstable imaging). Reverberation and diffuse sounds tend
|
|
to contain less energy and be psychoacoustically dominated by the
|
|
point sources embedded in them. Thus, we again tend to concentrate
|
|
more represented energy into a predictably smaller number of numbers.
|
|
Separating representation of point and diffuse imaging also allows us
|
|
to model and manipulate point and diffuse qualities separately.<p>
|
|
|
|
<h4>controlling bit leakage and symbol crosstalk</h4> Because polar
|
|
representation concentrates represented energy into fewer large
|
|
values, we reduce bit 'leakage' during cascading (multistage VQ
|
|
encoding) as a secondary benefit. A single large, monolithic VQ
|
|
codebook is more efficient than a cascaded book due to entropy
|
|
'crosstalk' among symbols between different stages of a multistage cascade.
|
|
Polar representation is a way of further concentrating entropy into
|
|
predictable locations so that codebook design can take steps to
|
|
improve multistage codebook efficiency. It also allows us to cascade
|
|
various elements of the stereo image independently.<p>
|
|
|
|
<h4>eliminating trigonometry and rounding</h4>
|
|
|
|
Rounding and computational complexity are potential problems with a
|
|
polar representation. As our encoding process involves quantization,
|
|
mixing a polar representation and quantization makes it potentially
|
|
impossible, depending on implementation, to construct a coupled stereo
|
|
mechanism that results in bit-identical decompressed output compared
|
|
to an uncoupled encoding should the encoder desire it.<p>
|
|
|
|
Vorbis uses a mapping that preserves the most useful qualities of
|
|
polar representation, relies only on addition/subtraction (during
|
|
decode; high quality encoding still requires some trig), and makes it
|
|
trivial before or after quantization to represent an angle/magnitude
|
|
through a one-to-one mapping from possible left/right value
|
|
permutations. We do this by basing our polar representation on the
|
|
unit square rather than the unit-circle.<p>
|
|
|
|
Given a magnitude and angle, we recover left and right using the
|
|
following function (note that A/B may be left/right or right/left
|
|
depending on the coupling definition used by the encoder):<p>
|
|
|
|
<pre>
|
|
if(magnitude>0)
|
|
if(angle>0){
|
|
A=magnitude;
|
|
B=magnitude-angle;
|
|
}else{
|
|
B=magnitude;
|
|
A=magnitude+angle;
|
|
}
|
|
else
|
|
if(angle>0){
|
|
A=magnitude;
|
|
B=magnitude+angle;
|
|
}else{
|
|
B=magnitude;
|
|
A=magnitude-angle;
|
|
}
|
|
}
|
|
</pre>
|
|
|
|
The function is antisymmetric for positive and negative magnitudes in
|
|
order to eliminate a redundant value when quantizing. For example, if
|
|
we're quantizing to integer values, we can visualize a magnitude of 5
|
|
and an angle of -2 as follows:<p>
|
|
|
|
<img src="squarepolar.png">
|
|
|
|
<p>
|
|
This representation loses or replicates no values; if the range of A
|
|
and B are integral -5 through 5, the number of possible Cartesian
|
|
permutations is 121. Represented in square polar notation, the
|
|
possible values are:
|
|
|
|
<pre>
|
|
0, 0
|
|
|
|
-1,-2 -1,-1 -1, 0 -1, 1
|
|
|
|
1,-2 1,-1 1, 0 1, 1
|
|
|
|
-2,-4 -2,-3 -2,-2 -2,-1 -2, 0 -2, 1 -2, 2 -2, 3
|
|
|
|
2,-4 2,-3 ... following the pattern ...
|
|
|
|
... 5, 1 5, 2 5, 3 5, 4 5, 5 5, 6 5, 7 5, 8 5, 9
|
|
|
|
</pre>
|
|
|
|
...for a grand total of 121 possible values, the same number as in
|
|
Cartesian representation (note that, for example, <tt>5,-10</tt> is
|
|
the same as <tt>-5,10</tt>, so there's no reason to represent
|
|
both. 2,10 cannot happen, and there's no reason to account for it.)
|
|
It's also obvious that this mapping is exactly reversible.<p>
|
|
|
|
<h3>Channel interleaving</h3>
|
|
|
|
We can remap and A/B vector using polar mapping into a magnitude/angle
|
|
vector, and it's clear that, in general, this concentrates energy in
|
|
the magnitude vector and reduces the amount of information to encode
|
|
in the angle vector. Encoding these vectors independently with
|
|
residue backend #0 or residue backend #1 will result in bitrate
|
|
savings. However, there are still implicit correlations between the
|
|
magnitude and angle vectors. The most obvious is that the amplitude
|
|
of the angle is bounded by its corresponding magnitude value.<p>
|
|
|
|
Entropy coding the results, then, further benefits from the entropy
|
|
model being able to compress magnitude and angle simultaneously. For
|
|
this reason, Vorbis implements residue backend #2 which pre-interleaves
|
|
a number of input vectors (in the stereo case, two, A and B) into a
|
|
single output vector (with the elements in the order of
|
|
A_0, B_0, A_1, B_1, A_2 ... A_n-1, B_n-1) before entropy encoding. Thus
|
|
each vector to be coded by the vector quantization backend consists of
|
|
matching magnitude and angle values.<p>
|
|
|
|
The astute reader, at this point, will notice that in the theoretical
|
|
case in which we can use monolithic codebooks of arbitrarily large
|
|
size, we can directly interleave and encode left and right without
|
|
polar mapping; in fact, the polar mapping does not appear to lend any
|
|
benefit whatsoever to the efficiency of the entropy coding. In fact,
|
|
it is perfectly possible and reasonable to build a Vorbis encoder that
|
|
dispenses with polar mapping entirely and merely interleaves the
|
|
channel. Libvorbis based encoders may configure such an encoding and
|
|
it will work as intended.<p>
|
|
|
|
However, when we leave the ideal/theoretical domain, we notice that
|
|
polar mapping does give additional practical benefits, as discussed in
|
|
the above section on polar mapping and summarized again here:<p>
|
|
<ul>
|
|
<li>Polar mapping aids in controlling entropy 'leakage' between stages
|
|
of a cascaded codebook. <li>Polar mapping separates the stereo image
|
|
into point and diffuse components which may be analyzed and handled
|
|
differently.
|
|
</ul>
|
|
|
|
<h2>Stereo Models</h2>
|
|
|
|
<h3>Dual Stereo</h3>
|
|
|
|
Dual stereo refers to stereo encoding where the channels are entirely
|
|
separate; they are analyzed and encoded as entirely distinct entities.
|
|
This terminology is familiar from mp3.<p>
|
|
|
|
<h3>Lossless Stereo</h3>
|
|
|
|
Using polar mapping and/or channel interleaving, it's possible to
|
|
couple Vorbis channels losslessly, that is, construct a stereo
|
|
coupling encoding that both saves space but also decodes
|
|
bit-identically to dual stereo. OggEnc 1.0 and later uses this
|
|
mode in all high-bitrate encoding.<p>
|
|
|
|
Overall, this stereo mode is overkill; however, it offers a safe
|
|
alternative to users concerned about the slightest possible
|
|
degradation to the stereo image or archival quality audio.<p>
|
|
|
|
<h3>Phase Stereo</h3>
|
|
|
|
Phase stereo is the least aggressive means of gracefully dropping
|
|
resolution from the stereo image; it affects only diffuse imaging.<p>
|
|
|
|
It's often quoted that the human ear is deaf to signal phase above
|
|
about 4kHz; this is nearly true and a passable rule of thumb, but it
|
|
can be demonstrated that even an average user can tell the difference
|
|
between high frequency in-phase and out-of-phase noise. Obviously
|
|
then, the statement is not entirely true. However, it's also the case
|
|
that one must resort to nearly such an extreme demonstration before
|
|
finding the counterexample.<p>
|
|
|
|
'Phase stereo' is simply a more aggressive quantization of the polar
|
|
angle vector; above 4kHz it's generally quite safe to quantize noise
|
|
and noisy elements to only a handful of allowed phases, or to thin the
|
|
phase with respect to the magnitude. The phases of high amplitude
|
|
pure tones may or may not be preserved more carefully (they are
|
|
relatively rare and L/R tend to be in phase, so there is generally
|
|
little reason not to spend a few more bits on them) <p>
|
|
|
|
<h4>example: eight phase stereo</h4>
|
|
|
|
Vorbis may implement phase stereo coupling by preserving the entirety
|
|
of the magnitude vector (essential to fine amplitude and energy
|
|
resolution overall) and quantizing the angle vector to one of only
|
|
four possible values. Given that the magnitude vector may be positive
|
|
or negative, this results in left and right phase having eight
|
|
possible permutation, thus 'eight phase stereo':<p>
|
|
|
|
<img src="eightphase.png"><p>
|
|
|
|
Left and right may be in phase (positive or negative), the most common
|
|
case by far, or out of phase by 90 or 180 degrees.<p>
|
|
|
|
<h4>example: four phase stereo</h4>
|
|
|
|
Similarly, four phase stereo takes the quantization one step further;
|
|
it allows only in-phase and 180 degree out-out-phase signals:<p>
|
|
|
|
<img src="fourphase.png"><p>
|
|
|
|
<h3>example: point stereo</h3>
|
|
|
|
Point stereo eliminates the possibility of out-of-phase signal
|
|
entirely. Any diffuse quality to a sound source tends to collapse
|
|
inward to a point somewhere within the stereo image. A practical
|
|
example would be balanced reverberations within a large, live space;
|
|
normally the sound is diffuse and soft, giving a sonic impression of
|
|
volume. In point-stereo, the reverberations would still exist, but
|
|
sound fairly firmly centered within the image (assuming the
|
|
reverberation was centered overall; if the reverberation is stronger
|
|
to the left, then the point of localization in point stereo would be
|
|
to the left). This effect is most noticeable at low and mid
|
|
frequencies and using headphones (which grant perfect stereo
|
|
separation). Point stereo is is a graceful but generally easy to
|
|
detect degradation to the sound quality and is thus used in frequency
|
|
ranges where it is least noticeable.<p>
|
|
|
|
<h3>Mixed Stereo</h3>
|
|
|
|
Mixed stereo is the simultaneous use of more than one of the above
|
|
stereo encoding models, generally using more aggressive modes in
|
|
higher frequencies, lower amplitudes or 'nearly' in-phase sound.<p>
|
|
|
|
It is also the case that near-DC frequencies should be encoded using
|
|
lossless coupling to avoid frame blocking artifacts.<p>
|
|
|
|
<h3>Vorbis Stereo Modes</h3>
|
|
|
|
Vorbis, as of 1.0, uses lossless stereo and a number of mixed modes
|
|
constructed out of lossless and point stereo. Phase stereo was used
|
|
in the rc2 encoder, but is not currently used for simplicity's sake. It
|
|
will likely be re-added to the stereo model in the future.
|
|
|
|
<p>
|
|
<hr>
|
|
<a href="http://www.xiph.org/">
|
|
<img src="white-xifish.png" align=left border=0>
|
|
</a>
|
|
<font size=-2 color=#505050>
|
|
|
|
Ogg is a <a href="http://www.xiph.org">Xiph.org Foundation</a> effort
|
|
to protect essential tenets of Internet multimedia from corporate
|
|
hostage-taking; Open Source is the net's greatest tool to keep
|
|
everyone honest. See <a href="http://www.xiph.org/about.html">About
|
|
the Xiph.org Foundation</a> for details.
|
|
<p>
|
|
|
|
Ogg Vorbis is the first Ogg audio CODEC. Anyone may freely use and
|
|
distribute the Ogg and Vorbis specification, whether in a private,
|
|
public or corporate capacity. However, the Xiph.org Foundation and
|
|
the Ogg project (xiph.org) reserve the right to set the Ogg Vorbis
|
|
specification and certify specification compliance.<p>
|
|
|
|
Xiph.org's Vorbis software CODEC implementation is distributed under a
|
|
BSD-like license. This does not restrict third parties from
|
|
distributing independent implementations of Vorbis software under
|
|
other licenses.<p>
|
|
|
|
Ogg, Vorbis, Xiph.org Foundation and their logos are trademarks (tm)
|
|
of the <a href="http://www.xiph.org/">Xiph.org Foundation</a>. These
|
|
pages are copyright (C) 1994-2002 Xiph.org Foundation. All rights
|
|
reserved.<p>
|
|
</body>
|
|
|
|
|
|
|
|
|
|
|
|
|