[Clam-devel] residual spectrum line segment approximation?

roumbaba roumbaba at gmail.com
Wed Aug 6 15:07:55 PDT 2008


So I have *not* managed to correctly apply the bh92 window to my  
modified residual spectrum and thus I have *not* eliminate phase  
discontinuities at resynth time.

One thing i still do not understand is why SMS need odd analysis  
window sizes and how I should handle this. I specify analysis window  
size to be 1024 and internally it seems to become 1025 and my 1STF  
frames are 513 in size. The fact that i do not understand that issue  
might be one of the source of what I do not do right.

Here is where I am at so far. Any hint on what I do wrong or should  
do otherwise is welcome of course:

- For testing purpose the only modification I do to the original 513  
values of the noise spectrum is to randomize phases.
- Then I expand the 513 spectrum to a 1026 spectrum by an even  
symetry across the 513.5 axis and complex conjugate of the last 513  
values.

- Then I do a circular convolution of my 1026 spectrum with the FFT  
of a  1026 bh92time window.
	the way I compute the bh92  time window is (matlab code for now):

		w1Length = 1026;
		fConst=2*pi/(w1Length+1-1);
		w1=[1:w1Length];
		w1=.35875 -.48829*cos(fConst*w1)+.14128*cos(fConst*2*w1) -.01168*cos 
(fConst*3*w1);

- When I check the real  part (and the magnitude)  of the ifft of the  
resulting 1026 values spectrum resulting of the convolutiong, I do  
see that the windowing worked and that the resulting time signal  
smoothes to 0 at begining and end.

- Then I take the first 513 values of the resulting spectrum and  
replace the corresponding 1STF frame in the original sdif analysis file

Still I get phase discontinuites in the resynth signal.

What am i missing?

Thanks,

Baba






On 15 juil. 08, at 14:55, Xavier Amatriain wrote:

> Hi Roumbaba, and congrats for your progress!
>
> You are right on the source of your problem: SMSSynthesis expects  
> your residual to come with an analysis window and if not things are  
> likely to mess up.
>
> The lines that are "guilty" for that are around SMSSynthesis.cxx:252
>
> http://clam.iua.upf.edu/doc/CLAM-doxygen/SMSSynthesis_8cxx- 
> source.html#l00252
>
> First the peaks are synthesized into a sinusoidal spectrum. Then  
> the two spectrums are added. Already at that point the spectrums  
> are supposed to have the same analysis window (BH92) and size. The  
> effect of that window is undone in line 261 when the global  
> spectral synthesis is performed.
>
> The issue here is that you need to guarantee that both spectrum  
> come from a similar place before adding them... The sinusoidal  
> peaks are reconstructed by convolving by the transform of the main  
> lobe of the window (BH92) but you are reconstructing the residual  
> in a different way. So.... you either apply the BH92 transform to  
> your spectrum or avoid doing that in the peak synthesis (and then  
> avoid multiplying by the inverse in the global spectral synthesis).  
> None of the two options are immediate but I'd say the first one  
> should be easier to work out.
>
> Hope it helps... and if you get it to work don't forget to report  
> back.
>
> roumbaba wrote:
>>
>>
>> Hello all and thanks again for your previous help,
>>
>> So I have written some matlab script to perform noise spectrum  
>> line segment approximation.
>>
>> - As input the script  takes  an sdif file generated by analysis  
>> with  SMSConsole.
>> - It then reads all sdif frames, in particular the 1STF frames  
>> containing the noise spectrums in complex form.
>> - It converts these complex spectrums into magPhase form
>> - It performs line segment approximation on the amplitudes.
>>
>> To check the impact of the approximation on the quality of   
>> resynthesis the script does the following:
>> - It  reconstructs  full noise magnitude spectrums from the line  
>> approximations  (by linear interpolation)
>> - It randomizes the phases
>> - It converts the new "smoothed" magPhase spectrums back to  
>> complex spectrums
>> - It writes back  the sdif file with these new "smoothed"  
>> spectrums instead of the original raw noise spectrums.
>>
>> Then I run SMSConsole to synthesize that sdif file with the exact  
>> same parameters than for the original sdif file.
>> My problem is that the resulting synthesised noise sounds like  
>> something is wrong in the synthesis overlap-add (like lots of  
>> discontinuites in the resynthesis)
>> I think that this might be due to what is described in  the Serra/ 
>> Smith 1990 CMJ paper concerning line segment approximation noise  
>> resynthesis:
>>
>> " ...Since the [new] phase spectrum used is not the result of an  
>> analysis process (with windowing of a waveform, zero padding, and  
>> FFT computation), the resulting signal does not tapper to 0 at the  
>> boundaries. This is because a phase spectrum with random values  
>> corresponds to a phase spectrum of a rectangular-windowed noise  
>> waveform of size N. In order to succeed in the overlap-add  
>> resynthesis (ie, to obtain smooth transitions between frames) we  
>> need a smoothly windowed waveform of size M, where M is the  
>> synthesis-window length. ....
>> "
>>
>> So what might be happening is that by default SMSConsole assumes  
>> that the 1STF frames are *NOT* line segment approximation and  
>> therefore does *NOT* perform that last windowing at synthesis  
>> time. I have gone a little bit through SMS/Clam code but I cannot  
>> find where I can change this behavior or even if that is the  
>> default behavior. Where shoud I look in the SMS/Clam code?
>>
>>
>> Thanks,
>>
>> Roumbaba
>>
>>
>>
>> On 27 mai 08, at 23:25, Xavier Amatriain wrote:
>>
>>> Hi Roumbaba,
>>>
>>> In the paper you cite it says "you can", which does not mean "you  
>>> have to" :-) Doing an approximation of the residual model is indeed
>>> an interesting thing to do, especially if you want to reduce the  
>>> amount of data in your transformed signal, however it is not a must.
>>> Note that there are many other ways to model the residual apart  
>>> from the one mentioned in that paper.
>>>
>>> So far, in CLAM we are using the residual as is, with no modeling  
>>> or approximation. The "only" downside is that the transformed
>>> signal (SMS Data) is in fact larger than the original audio when  
>>> it could be much smaller with not much loss in quality. If for
>>> whatever reason you do need to do the residual modeling you can  
>>> look at the SpectralEnvelopeExtract processing. This processing
>>> generates a spectral approximation (spectrum in bpf format) but  
>>> from an array of peaks, it would not be hard to modify it to work
>>> with an input spectrum.
>>>
>>> X
>>>
>>>
>>> roumbaba wrote:
>>>> Hi all,
>>>>
>>>> I am trying to understand how the residual spectrum gets modeled  
>>>> in clam/SMS. I have read the Serra/Smith 1990 CMJ paper and as I  
>>>> understand it  it describes two steps:
>>>> 1- substract the harmonic spectrum from the original spectrum
>>>> 2- perform a line-segment approximation of the residual spectrum  
>>>> obtained in 1
>>>>
>>>> I have stepped through clam and SMS code and I think I can see  
>>>> where step 1 gets performed:
>>>>
>>>> SMSAnalysisCore::Do()
>>>> {
>>>>
>>>> mSinSpectralAnalysis.Do();
>>>> mResSpectralAnalysis.Do();
>>>> ...
>>>> ...
>>>> ...
>>>> mSynthSineSpectrum.Do();
>>>> mSpecSubstracter.Do(); /* step 1 gets performed here I think*/
>>>>
>>>> }
>>>>
>>>>
>>>> but I cannot find where step 2 (line approximation) gets  
>>>> performed. Where should I look in the code?
>>>>
>>>> Thank you very much,
>>>> Cheers,
>>>>
>>>> Roumbaba
>>>>
>>>> ps:
>>>>
>>>> Here is a quote from the paper I mentionned above:
>>>>
>>>> "Approximation of the Spectral Residual
>>>>
>>>> Assuming the the residual signal is quasi-stochastic, each  
>>>> magnitude-spectrum residual can be approximated by its envelope  
>>>> since only its shape contributes to the sound characteristics.  
>>>> [...] The particular line-segment approximation performed here  
>>>> is done by stepping through the magnitude spectrum and finding  
>>>> local maxima in every section, ..."
>>>>
>>>>
>>>> _______________________________________________
>>>> Clam-devel mailing list
>>>> Clam-devel at llistes.projectes.lafarga.org
>>>> https://llistes.projectes.lafarga.org/cgi-bin/mailman/listinfo/ 
>>>> clam-devel
>>>
>>>
>>> _______________________________________________
>>> Clam-devel mailing list
>>> Clam-devel at llistes.projectes.lafarga.org
>>> https://llistes.projectes.lafarga.org/cgi-bin/mailman/listinfo/ 
>>> clam-devel
>>
>>
>> _______________________________________________
>> Clam-devel mailing list
>> Clam-devel at llistes.projectes.lafarga.org
>> https://llistes.projectes.lafarga.org/cgi-bin/mailman/listinfo/ 
>> clam-devel
>
>
> _______________________________________________
> Clam-devel mailing list
> Clam-devel at llistes.projectes.lafarga.org
> https://llistes.projectes.lafarga.org/cgi-bin/mailman/listinfo/clam- 
> devel





More information about the clam-devel mailing list