[Clam-devel] residual spectrum line segment approximation?
roumbaba
roumbaba at gmail.com
Thu Jul 17 12:48:59 PDT 2008
Thanks Xavier! That helps a lot.
So I'll try to apply a BH92 to my spectrum. I will do it in Matlab
for now to see if it work. I will let you know.
- In order to do it properly I need to understand why the analysis
windows are forced to odd size windows? ( the sms tutorial says <
(Res)AnalysisWindowSize>: Note: if the value entered is not odd, the
program will internally add +1 to it ) ... Probably for taking
advantage of symetry somewhere?
- Anyhow the real question is this: the size of the 1STF Matrix which
holds the complex spectrums is 513. How shall I go to apply the bh92
window on it?
I am thinking of this:
- Take the 513 bins complex spectrum
- build a corresponding 1024 (1025?) symmetric spectrum with the axis
of symmetry at bin 512 (513?)
- take the ifft of this spectrum
- multiply the result by the bh92 1024 window (value by value
multiply: value 1 of the bh92 window with bin 1 of the ifft obtain at
the previous step etc )
- take the fft of the resulting windowed signal
- Store this result in the 1STF frames for noise
Does that seem correct to you?
Thanks again,
Rumbaba
On 15 juil. 08, at 23:55, Xavier Amatriain wrote:
> Hi Roumbaba, and congrats for your progress!
>
> You are right on the source of your problem: SMSSynthesis expects
> your residual to come with an analysis window and if not things are
> likely to mess up.
>
> The lines that are "guilty" for that are around SMSSynthesis.cxx:252
>
> http://clam.iua.upf.edu/doc/CLAM-doxygen/SMSSynthesis_8cxx-
> source.html#l00252
>
> First the peaks are synthesized into a sinusoidal spectrum. Then
> the two spectrums are added. Already at that point the spectrums
> are supposed to have the same analysis window (BH92) and size. The
> effect of that window is undone in line 261 when the global
> spectral synthesis is performed.
>
> The issue here is that you need to guarantee that both spectrum
> come from a similar place before adding them... The sinusoidal
> peaks are reconstructed by convolving by the transform of the main
> lobe of the window (BH92) but you are reconstructing the residual
> in a different way. So.... you either apply the BH92 transform to
> your spectrum or avoid doing that in the peak synthesis (and then
> avoid multiplying by the inverse in the global spectral synthesis).
> None of the two options are immediate but I'd say the first one
> should be easier to work out.
>
> Hope it helps... and if you get it to work don't forget to report
> back.
>
> roumbaba wrote:
>>
>>
>> Hello all and thanks again for your previous help,
>>
>> So I have written some matlab script to perform noise spectrum
>> line segment approximation.
>>
>> - As input the script takes an sdif file generated by analysis
>> with SMSConsole.
>> - It then reads all sdif frames, in particular the 1STF frames
>> containing the noise spectrums in complex form.
>> - It converts these complex spectrums into magPhase form
>> - It performs line segment approximation on the amplitudes.
>>
>> To check the impact of the approximation on the quality of
>> resynthesis the script does the following:
>> - It reconstructs full noise magnitude spectrums from the line
>> approximations (by linear interpolation)
>> - It randomizes the phases
>> - It converts the new "smoothed" magPhase spectrums back to
>> complex spectrums
>> - It writes back the sdif file with these new "smoothed"
>> spectrums instead of the original raw noise spectrums.
>>
>> Then I run SMSConsole to synthesize that sdif file with the exact
>> same parameters than for the original sdif file.
>> My problem is that the resulting synthesised noise sounds like
>> something is wrong in the synthesis overlap-add (like lots of
>> discontinuites in the resynthesis)
>> I think that this might be due to what is described in the Serra/
>> Smith 1990 CMJ paper concerning line segment approximation noise
>> resynthesis:
>>
>> " ...Since the [new] phase spectrum used is not the result of an
>> analysis process (with windowing of a waveform, zero padding, and
>> FFT computation), the resulting signal does not tapper to 0 at the
>> boundaries. This is because a phase spectrum with random values
>> corresponds to a phase spectrum of a rectangular-windowed noise
>> waveform of size N. In order to succeed in the overlap-add
>> resynthesis (ie, to obtain smooth transitions between frames) we
>> need a smoothly windowed waveform of size M, where M is the
>> synthesis-window length. ....
>> "
>>
>> So what might be happening is that by default SMSConsole assumes
>> that the 1STF frames are *NOT* line segment approximation and
>> therefore does *NOT* perform that last windowing at synthesis
>> time. I have gone a little bit through SMS/Clam code but I cannot
>> find where I can change this behavior or even if that is the
>> default behavior. Where shoud I look in the SMS/Clam code?
>>
>>
>> Thanks,
>>
>> Roumbaba
>>
>>
>>
>> On 27 mai 08, at 23:25, Xavier Amatriain wrote:
>>
>>> Hi Roumbaba,
>>>
>>> In the paper you cite it says "you can", which does not mean "you
>>> have to" :-) Doing an approximation of the residual model is indeed
>>> an interesting thing to do, especially if you want to reduce the
>>> amount of data in your transformed signal, however it is not a must.
>>> Note that there are many other ways to model the residual apart
>>> from the one mentioned in that paper.
>>>
>>> So far, in CLAM we are using the residual as is, with no modeling
>>> or approximation. The "only" downside is that the transformed
>>> signal (SMS Data) is in fact larger than the original audio when
>>> it could be much smaller with not much loss in quality. If for
>>> whatever reason you do need to do the residual modeling you can
>>> look at the SpectralEnvelopeExtract processing. This processing
>>> generates a spectral approximation (spectrum in bpf format) but
>>> from an array of peaks, it would not be hard to modify it to work
>>> with an input spectrum.
>>>
>>> X
>>>
>>>
>>> roumbaba wrote:
>>>> Hi all,
>>>>
>>>> I am trying to understand how the residual spectrum gets modeled
>>>> in clam/SMS. I have read the Serra/Smith 1990 CMJ paper and as I
>>>> understand it it describes two steps:
>>>> 1- substract the harmonic spectrum from the original spectrum
>>>> 2- perform a line-segment approximation of the residual spectrum
>>>> obtained in 1
>>>>
>>>> I have stepped through clam and SMS code and I think I can see
>>>> where step 1 gets performed:
>>>>
>>>> SMSAnalysisCore::Do()
>>>> {
>>>>
>>>> mSinSpectralAnalysis.Do();
>>>> mResSpectralAnalysis.Do();
>>>> ...
>>>> ...
>>>> ...
>>>> mSynthSineSpectrum.Do();
>>>> mSpecSubstracter.Do(); /* step 1 gets performed here I think*/
>>>>
>>>> }
>>>>
>>>>
>>>> but I cannot find where step 2 (line approximation) gets
>>>> performed. Where should I look in the code?
>>>>
>>>> Thank you very much,
>>>> Cheers,
>>>>
>>>> Roumbaba
>>>>
>>>> ps:
>>>>
>>>> Here is a quote from the paper I mentionned above:
>>>>
>>>> "Approximation of the Spectral Residual
>>>>
>>>> Assuming the the residual signal is quasi-stochastic, each
>>>> magnitude-spectrum residual can be approximated by its envelope
>>>> since only its shape contributes to the sound characteristics.
>>>> [...] The particular line-segment approximation performed here
>>>> is done by stepping through the magnitude spectrum and finding
>>>> local maxima in every section, ..."
>>>>
>>>>
>>>> _______________________________________________
>>>> Clam-devel mailing list
>>>> Clam-devel at llistes.projectes.lafarga.org
>>>> https://llistes.projectes.lafarga.org/cgi-bin/mailman/listinfo/
>>>> clam-devel
>>>
>>>
>>> _______________________________________________
>>> Clam-devel mailing list
>>> Clam-devel at llistes.projectes.lafarga.org
>>> https://llistes.projectes.lafarga.org/cgi-bin/mailman/listinfo/
>>> clam-devel
>>
>>
>> _______________________________________________
>> Clam-devel mailing list
>> Clam-devel at llistes.projectes.lafarga.org
>> https://llistes.projectes.lafarga.org/cgi-bin/mailman/listinfo/
>> clam-devel
>
>
> _______________________________________________
> Clam-devel mailing list
> Clam-devel at llistes.projectes.lafarga.org
> https://llistes.projectes.lafarga.org/cgi-bin/mailman/listinfo/clam-
> devel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.clam-project.org/pipermail/clam-devel-clam-project.org/attachments/20080717/019d3b44/attachment-0003.htm>
More information about the clam-devel
mailing list