[Clam-devel] residual spectrum line segment approximation?
roumbaba
roumbaba at gmail.com
Sun Jun 8 16:07:59 PDT 2008
Thanks for your answer. I am not sure if my approach is right. I was
actually trying to follow your original suggestion as I understood it:
>>> If for whatever reason you do need to do the residual modeling
>>> you can look at the SpectralEnvelopeExtract processing. This
>>> processing
>>> generates a spectral approximation (spectrum in bpf format) but
>>> from an array of peaks, it would not be hard to modify it to work
>>> with an input spectrum.
(see full message below)
The thing is that i am trying to get a model for the residual
spectrum. Do you think that a peak dectection on a residual would
actually work?
Also can you point me to sample code for visulalizing spectrums,
bpfs, peakarrays? That would be a good way to check that
peakdetection is actually acceptable on my residuals ...
Thanks again,
R
On 8 juin 08, at 15:56, Xavier Amatriain wrote:
> Hi Roumbaba,
>
> I am not sure what is causing the crash. However I can tell that
> the approach you are taking makes little sense (If I am reading it
> right). If you want to approximate a spectrum you cannot take all
> the bins in it. My recommendation would be to feed the input
> spectrum to a SpectralPeakDetect object and then use the existing Do
> () to output the spectrum BPF. This way you would be picking the
> peaks in the spectrum and doing the line approximation from it.
>
> Hope it helps!
>
> X
>
> roumbaba wrote:
>> Thanks, so I am trying to add a Do method in
>> SpectralEnvelopeExtract that takes a spectrum an not an array of
>> peaks.
>> I have something working but I seem to be able to only call my Do
>> function function once. It crashes on the second time I call it. I
>> don't see what it is I am doing wrong or not doing.
>> Here is my code so far:
>>
>>
>> in spectralEnvelopeExtract.cxx (based on the original Do function
>> that takes a peak array as input):
>>
>> bool SpectralEnvelopeExtract::Do(const Spectrum& input,
>> Spectrum& output)
>> {
>> output.SetScale(input.GetScale());
>> TSize nBins=input.GetSize();
>>
>>
>>
>> if (nBins<4) return false; //cannot extract an envelope out of
>> only 3 peaks!
>>
>> CheckOutputType(output);
>>
>> DataArray& magBuffer=input.GetMagBuffer();
>> DataArray& phaseBuffer=input.GetPhaseBuffer();
>> // DataArray& freqBuffer=input.GetFreqBuffer();
>>
>> Array<Point>& magPointArray=output.GetMagBPF().GetPointArray();
>> Array<Point>& phasePointArray=output.GetPhaseBPF().GetPointArray();
>>
>>
>> //Max number of points allowed: should be a config param
>> magPointArray.Resize(mConfig.GetMaxPeaks());
>> magPointArray.SetSize(mConfig.GetMaxPeaks());
>> phasePointArray.Resize(mConfig.GetMaxPeaks());
>> phasePointArray.SetSize(mConfig.GetMaxPeaks());
>>
>>
>> TData curBinFreq = 0;
>>
>> for(int i=0;i<nBins;i++)
>> {
>> curBinFreq = input.GetSpectralRange()*i/
>> (float)nBins;
>>
>> magPointArray[i+1].SetX(curBinFreq);
>> magPointArray[i+1].SetY(magBuffer[i]);
>>
>> phasePointArray[i+1].SetX(curBinFreq);
>> phasePointArray[i+1].SetY(phaseBuffer[i]);
>>
>> }
>>
>>
>> //todo: a lot of duplicated code, should extract in different
>> functions
>> if(input.GetScale()==EScale::eLog)
>> {
>>
>>
>> //we now set first point (maybe we should do the same as with last
>> point?)
>> magPointArray[0].SetX(0);
>> magPointArray[0].SetY(magBuffer[0]-3);
>> phasePointArray[0].SetX(0);
>> phasePointArray[0].SetY(0);
>> nBins++;
>>
>>
>> /* we keep adding points to bpf until magnitude is insignificant
>> (note that we add points outside the spectral range) */
>> curBinFreq = (nBins-2)
>> *input.GetSpectralRange()/(float)nBins; /*note: I compute the
>> current bin's frequency instead of the peakArray'sfreqBuffer */
>> TData lastFreq = curBinFreq; //freqBuffer[nBins-2];
>> curBinFreq = (nBins-3)
>> *input.GetSpectralRange()/(float)nBins;
>> TData freqGap = lastFreq-curBinFreq; //freqBuffer[nBins-3];
>> TData currentFreq=lastFreq+freqGap;
>> TData currentMag=magBuffer[nBins-2];
>>
>>
>> while(currentMag>-200)
>> {
>> currentMag-=(currentFreq/lastFreq-1)*12;
>>
>>
>> magPointArray[nBins].SetY(currentMag);
>> magPointArray[nBins].SetX(currentFreq);
>> phasePointArray[nBins].SetY(0);
>> phasePointArray[nBins].SetX(currentFreq);
>>
>>
>> currentFreq+=freqGap;
>> nBins++;
>> if(nBins==mConfig.GetMaxPeaks()) break;
>>
>>
>> }
>> //we resize arrays to final size
>> magPointArray.Resize(nBins);
>> magPointArray.SetSize(nBins);
>> phasePointArray.Resize(nBins);
>> phasePointArray.SetSize(nBins);
>>
>>
>>
>> }
>> else
>> {
>> //we now set first point (maybe we should do the same as with last
>> point?)
>> magPointArray[0].SetX(0);
>> magPointArray[0].SetY(magBuffer[0]*0.5);
>> phasePointArray[0].SetX(0);
>> phasePointArray[0].SetY(0);
>> nBins++;
>>
>>
>> /* we keep adding points to bpf until magnitude is insignificant
>> (note that we add points outside the spectral range) */
>> curBinFreq = (nBins-2)
>> *input.GetSpectralRange()/(float)nBins;
>> TData lastFreq = curBinFreq;//freqBuffer[nBins-2];
>> curBinFreq = (nBins-3)
>> *input.GetSpectralRange()/(float)nBins;
>> TData freqGap=lastFreq-curBinFreq;//freqBuffer[nBins-3];
>> TData currentFreq=lastFreq+freqGap;
>> TData currentMag=magBuffer[nBins-2];
>>
>>
>>
>> while(currentMag<0.0000000001)
>> {
>> currentMag*=CLAM_pow(0.06,(double)(currentFreq/lastFreq-1.0));
>>
>>
>> magPointArray[nBins].SetY(currentMag);
>> magPointArray[nBins].SetX(currentFreq);
>> phasePointArray[nBins].SetY(0);
>> phasePointArray[nBins].SetX(currentFreq);
>>
>>
>> currentFreq+=freqGap;
>> nBins++;
>> if(nBins==mConfig.GetMaxPeaks()) break;
>> }
>>
>>
>> //we resize arrays to final size
>> magPointArray.Resize(nBins); //!!! (note 1) This is where the
>> crash seem to happen on the second time I call this Do method (see
>> error messsage below)
>> magPointArray.SetSize(nBins);
>> phasePointArray.Resize(nBins);
>> phasePointArray.SetSize(nBins);
>>
>>
>> }
>>
>>
>> output.SetSize(nBins);
>> output.GetMagBPF().UpdateSplineTable();
>>
>>
>> return true;
>> }
>>
>> and this is how I try to use it in my program:
>>
>> void SMSStdio::PrintSpectrum(std::string inputXMLFileName)
>> {
>>
>> TIndex i,j;
>> TData m1, m2; int nbOfFrames = 0;
>>
>> LoadConfig(inputXMLFileName);
>> LoadAnalysis(mGlobalConfig.GetInputAnalysisFile());
>>
>>
>> if(!GetState().GetHasTransformation())
>> {
>> CopySegmentExceptAudio(mOriginalSegment,mTransformedSegment);
>> GetState().SetHasTransformation(true);
>> }
>>
>>
>> mTransformedSegment.mCurrentFrameIndex=0;
>> nbOfFrames = mTransformedSegment.GetnFrames();
>>
>> SpectralEnvelopeExtract spectralEnvelopeExtract;
>> Spectrum outputEnvSpec;
>> Spectrum inputSpec;
>> SpecTypeFlags f; Frame inputFrame;
>>
>>
>>
>> for( i=0; i < nbOfFrames; i++)
>> {
>> inputFrame = mTransformedSegment.GetFrame
>> ( mTransformedSegment.mCurrentFrameIndex++ );
>> inputSpec = inputFrame.GetResidualSpec();
>>
>>
>>
>> //Change inputSpec's type from Complex to MagPhase
>> inputSpec.GetType(f);
>> if( !f.bMagPhase )
>> { f.bMagPhase = true;
>> inputSpec.SetTypeSynchronize(f);
>> }
>>
>>
>> //Set outputSpectrum's type To BPF
>> outputEnvSpec.GetType(f);
>> if( !f.bMagPhaseBPF )
>> { f.bMagPhaseBPF = true;
>> outputEnvSpec.SetType(f); //set spectrum to be a BPF
>> }
>>
>>
>> spectralEnvelopeExtract.Do( inputSpec,
>> outputEnvSpec ); // !!!!!this is where I always get the crash on
>> the second time i hit this line see error message below and note 1
>> above
>>
>> }
>>
>>
>>
>> // outputEnvSpec.BPF2MagPhase();
>> // outputEnvSpec.MagPhase2Complex();
>> /* for ( j = 0; j < 10; j++ )
>> {
>> m1 = inputSpec.GetMag( j );//myFrame.GetResMag(j);
>> m2 = outputEnvSpec.GetMag( j );
>>
>>
>> fprintf( stderr, "m1 = %f m2 = %f", m1 , m2);
>> }
>> fprintf( stderr, "\n");
>> */
>>
>>
>> return;
>> }
>>
>> This is the error I get:
>>
>> SMSConsole(18690,0xa000ed88) malloc: *** error for object
>> 0xd98d600: incorrect checksum for freed object - object was
>> probably modified after being freed, break at szone_error to debug
>> SMSConsole(18690,0xa000ed88) malloc: *** set a breakpoint in
>> szone_error to debug
>>
>> Program received signal EXC_BAD_ACCESS, Could not access memory.
>> Reason: KERN_INVALID_ADDRESS at address: 0x3fe7984a
>> 0x900074d8 in szone_free ()
>>
>>
>> Any idea of what I am doing wrong?
>>
>> Thanks.
>>
>>
>>
>>
>>
>> On 27 mai 08, at 14:25, Xavier Amatriain wrote:
>>
>>> Hi Roumbaba,
>>>
>>> In the paper you cite it says "you can", which does not mean "you
>>> have to" :-) Doing an approximation of the residual model is indeed
>>> an interesting thing to do, especially if you want to reduce the
>>> amount of data in your transformed signal, however it is not a must.
>>> Note that there are many other ways to model the residual apart
>>> from the one mentioned in that paper.
>>>
>>> So far, in CLAM we are using the residual as is, with no modeling
>>> or approximation. The "only" downside is that the transformed
>>> signal (SMS Data) is in fact larger than the original audio when
>>> it could be much smaller with not much loss in quality. If for
>>> whatever reason you do need to do the residual modeling you can
>>> look at the SpectralEnvelopeExtract processing. This processing
>>> generates a spectral approximation (spectrum in bpf format) but
>>> from an array of peaks, it would not be hard to modify it to work
>>> with an input spectrum.
>>>
>>> X
>>>
>>>
>>> roumbaba wrote:
>>>> Hi all,
>>>>
>>>> I am trying to understand how the residual spectrum gets modeled
>>>> in clam/SMS. I have read the Serra/Smith 1990 CMJ paper and as I
>>>> understand it it describes two steps:
>>>> 1- substract the harmonic spectrum from the original spectrum
>>>> 2- perform a line-segment approximation of the residual spectrum
>>>> obtained in 1
>>>>
>>>> I have stepped through clam and SMS code and I think I can see
>>>> where step 1 gets performed:
>>>>
>>>> SMSAnalysisCore::Do()
>>>> {
>>>>
>>>> mSinSpectralAnalysis.Do();
>>>> mResSpectralAnalysis.Do();
>>>> ...
>>>> ...
>>>> ...
>>>> mSynthSineSpectrum.Do();
>>>> mSpecSubstracter.Do(); /* step 1 gets performed here I think*/
>>>>
>>>> }
>>>>
>>>>
>>>> but I cannot find where step 2 (line approximation) gets
>>>> performed. Where should I look in the code?
>>>>
>>>> Thank you very much,
>>>> Cheers,
>>>>
>>>> Roumbaba
>>>>
>>>> ps:
>>>>
>>>> Here is a quote from the paper I mentionned above:
>>>>
>>>> "Approximation of the Spectral Residual
>>>>
>>>> Assuming the the residual signal is quasi-stochastic, each
>>>> magnitude-spectrum residual can be approximated by its envelope
>>>> since only its shape contributes to the sound characteristics.
>>>> [...] The particular line-segment approximation performed here
>>>> is done by stepping through the magnitude spectrum and finding
>>>> local maxima in every section, ..."
>>>>
>>>>
>>>> _______________________________________________
>>>> Clam-devel mailing list
>>>> Clam-devel at llistes.projectes.lafarga.org <mailto:Clam-
>>>> devel at llistes.projectes.lafarga.org>
>>>> https://llistes.projectes.lafarga.org/cgi-bin/mailman/listinfo/
>>>> clam-devel
>>>
>>>
>>> _______________________________________________
>>> Clam-devel mailing list
>>> Clam-devel at llistes.projectes.lafarga.org <mailto:Clam-
>>> devel at llistes.projectes.lafarga.org>
>>> https://llistes.projectes.lafarga.org/cgi-bin/mailman/listinfo/
>>> clam-devel
>>
>> ---------------------------------------------------------------------
>> ---
>>
>> _______________________________________________
>> Clam-devel mailing list
>> Clam-devel at llistes.projectes.lafarga.org
>> https://llistes.projectes.lafarga.org/cgi-bin/mailman/listinfo/
>> clam-devel
>>
>
>
> _______________________________________________
> Clam-devel mailing list
> Clam-devel at llistes.projectes.lafarga.org
> https://llistes.projectes.lafarga.org/cgi-bin/mailman/listinfo/clam-
> devel
More information about the clam-devel
mailing list