[Clam-devel] residual spectrum line segment approximation?
Xavier Amatriain
xavier at amatriain.net
Sun Jun 8 15:56:46 PDT 2008
Hi Roumbaba,
I am not sure what is causing the crash. However I can tell that the
approach you are taking makes little sense (If I am reading it right).
If you want to approximate a spectrum you cannot take all the bins in
it. My recommendation would be to feed the input spectrum to a
SpectralPeakDetect object and then use the existing Do() to output the
spectrum BPF. This way you would be picking the peaks in the spectrum
and doing the line approximation from it.
Hope it helps!
X
roumbaba wrote:
> Thanks, so I am trying to add a Do method in SpectralEnvelopeExtract
> that takes a spectrum an not an array of peaks.
> I have something working but I seem to be able to only call my Do
> function function once. It crashes on the second time I call it. I
> don't see what it is I am doing wrong or not doing.
> Here is my code so far:
>
>
> in spectralEnvelopeExtract.cxx (based on the original Do function that
> takes a peak array as input):
>
> bool SpectralEnvelopeExtract::Do(const Spectrum& input, Spectrum&
> output)
> {
> output.SetScale(input.GetScale());
> TSize nBins=input.GetSize();
>
>
>
>
> if (nBins<4) return false; //cannot extract an envelope out of only 3
> peaks!
>
> CheckOutputType(output);
>
> DataArray& magBuffer=input.GetMagBuffer();
> DataArray& phaseBuffer=input.GetPhaseBuffer();
> // DataArray& freqBuffer=input.GetFreqBuffer();
>
> Array<Point>& magPointArray=output.GetMagBPF().GetPointArray();
> Array<Point>& phasePointArray=output.GetPhaseBPF().GetPointArray();
>
>
> //Max number of points allowed: should be a config param
> magPointArray.Resize(mConfig.GetMaxPeaks());
> magPointArray.SetSize(mConfig.GetMaxPeaks());
> phasePointArray.Resize(mConfig.GetMaxPeaks());
> phasePointArray.SetSize(mConfig.GetMaxPeaks());
>
>
> TData curBinFreq = 0;
>
> for(int i=0;i<nBins;i++)
> {
> curBinFreq =
> input.GetSpectralRange()*i/(float)nBins;
>
> magPointArray[i+1].SetX(curBinFreq);
> magPointArray[i+1].SetY(magBuffer[i]);
>
> phasePointArray[i+1].SetX(curBinFreq);
> phasePointArray[i+1].SetY(phaseBuffer[i]);
>
> }
>
>
> //todo: a lot of duplicated code, should extract in different functions
> if(input.GetScale()==EScale::eLog)
> {
>
>
> //we now set first point (maybe we should do the same as with last point?)
> magPointArray[0].SetX(0);
> magPointArray[0].SetY(magBuffer[0]-3);
> phasePointArray[0].SetX(0);
> phasePointArray[0].SetY(0);
> nBins++;
>
>
> /* we keep adding points to bpf until magnitude is insignificant
> (note that we add points outside the spectral range) */
> curBinFreq =
> (nBins-2)*input.GetSpectralRange()/(float)nBins; /*note: I compute the
> current bin's frequency instead of the peakArray'sfreqBuffer */
> TData lastFreq = curBinFreq; //freqBuffer[nBins-2];
> curBinFreq =
> (nBins-3)*input.GetSpectralRange()/(float)nBins;
> TData freqGap = lastFreq-curBinFreq; //freqBuffer[nBins-3];
> TData currentFreq=lastFreq+freqGap;
> TData currentMag=magBuffer[nBins-2];
>
>
> while(currentMag>-200)
> {
> currentMag-=(currentFreq/lastFreq-1)*12;
>
>
> magPointArray[nBins].SetY(currentMag);
> magPointArray[nBins].SetX(currentFreq);
> phasePointArray[nBins].SetY(0);
> phasePointArray[nBins].SetX(currentFreq);
>
>
> currentFreq+=freqGap;
> nBins++;
> if(nBins==mConfig.GetMaxPeaks()) break;
>
>
> }
> //we resize arrays to final size
> magPointArray.Resize(nBins);
> magPointArray.SetSize(nBins);
> phasePointArray.Resize(nBins);
> phasePointArray.SetSize(nBins);
>
>
>
> }
> else
> {
> //we now set first point (maybe we should do the same as with last point?)
> magPointArray[0].SetX(0);
> magPointArray[0].SetY(magBuffer[0]*0.5);
> phasePointArray[0].SetX(0);
> phasePointArray[0].SetY(0);
> nBins++;
>
>
> /* we keep adding points to bpf until magnitude is insignificant
> (note that we add points outside the spectral range) */
> curBinFreq =
> (nBins-2)*input.GetSpectralRange()/(float)nBins;
> TData lastFreq = curBinFreq;//freqBuffer[nBins-2];
> curBinFreq =
> (nBins-3)*input.GetSpectralRange()/(float)nBins;
> TData freqGap=lastFreq-curBinFreq;//freqBuffer[nBins-3];
> TData currentFreq=lastFreq+freqGap;
> TData currentMag=magBuffer[nBins-2];
>
>
>
> while(currentMag<0.0000000001)
> {
> currentMag*=CLAM_pow(0.06,(double)(currentFreq/lastFreq-1.0));
>
>
> magPointArray[nBins].SetY(currentMag);
> magPointArray[nBins].SetX(currentFreq);
> phasePointArray[nBins].SetY(0);
> phasePointArray[nBins].SetX(currentFreq);
>
>
> currentFreq+=freqGap;
> nBins++;
> if(nBins==mConfig.GetMaxPeaks()) break;
> }
>
>
> //we resize arrays to final size
> magPointArray.Resize(nBins); //!!! (note 1) This is where the crash
> seem to happen on the second time I call this Do method (see error
> messsage below)
> magPointArray.SetSize(nBins);
> phasePointArray.Resize(nBins);
> phasePointArray.SetSize(nBins);
>
>
> }
>
>
> output.SetSize(nBins);
> output.GetMagBPF().UpdateSplineTable();
>
>
>
> return true;
> }
>
> and this is how I try to use it in my program:
>
> void SMSStdio::PrintSpectrum(std::string inputXMLFileName)
> {
>
> TIndex i,j;
> TData m1, m2;
> int nbOfFrames = 0;
>
> LoadConfig(inputXMLFileName);
> LoadAnalysis(mGlobalConfig.GetInputAnalysisFile());
>
>
>
> if(!GetState().GetHasTransformation())
> {
> CopySegmentExceptAudio(mOriginalSegment,mTransformedSegment);
> GetState().SetHasTransformation(true);
> }
>
>
>
> mTransformedSegment.mCurrentFrameIndex=0;
> nbOfFrames = mTransformedSegment.GetnFrames();
>
> SpectralEnvelopeExtract spectralEnvelopeExtract;
> Spectrum outputEnvSpec;
> Spectrum inputSpec;
> SpecTypeFlags f;
> Frame inputFrame;
>
>
>
>
>
> for( i=0; i < nbOfFrames; i++)
> {
> inputFrame = mTransformedSegment.GetFrame(
> mTransformedSegment.mCurrentFrameIndex++ );
> inputSpec = inputFrame.GetResidualSpec();
>
>
>
>
>
> //Change inputSpec's type from Complex to MagPhase
> inputSpec.GetType(f);
> if( !f.bMagPhase )
> {
> f.bMagPhase = true;
> inputSpec.SetTypeSynchronize(f);
> }
>
>
>
> //Set outputSpectrum's type To BPF
> outputEnvSpec.GetType(f);
> if( !f.bMagPhaseBPF )
> {
> f.bMagPhaseBPF = true;
> outputEnvSpec.SetType(f); //set spectrum to be a BPF
> }
>
>
>
> spectralEnvelopeExtract.Do( inputSpec, outputEnvSpec ); //
> !!!!!this is where I always get the crash on the second time i hit
> this line see error message below and note 1 above
>
>
>
> }
>
>
>
>
>
> // outputEnvSpec.BPF2MagPhase();
> // outputEnvSpec.MagPhase2Complex();
> /*
> for ( j = 0; j < 10; j++ )
> {
> m1 = inputSpec.GetMag( j );//myFrame.GetResMag(j);
> m2 = outputEnvSpec.GetMag( j );
>
>
>
> fprintf( stderr, "m1 = %f m2 = %f", m1 , m2);
> }
> fprintf( stderr, "\n");
> */
>
>
>
> return;
> }
>
> This is the error I get:
>
> SMSConsole(18690,0xa000ed88) malloc: *** error for object 0xd98d600:
> incorrect checksum for freed object - object was probably modified
> after being freed, break at szone_error to debug
> SMSConsole(18690,0xa000ed88) malloc: *** set a breakpoint in
> szone_error to debug
>
> Program received signal EXC_BAD_ACCESS, Could not access memory.
> Reason: KERN_INVALID_ADDRESS at address: 0x3fe7984a
> 0x900074d8 in szone_free ()
>
>
> Any idea of what I am doing wrong?
>
> Thanks.
>
>
>
>
>
> On 27 mai 08, at 14:25, Xavier Amatriain wrote:
>
>> Hi Roumbaba,
>>
>> In the paper you cite it says "you can", which does not mean "you
>> have to" :-) Doing an approximation of the residual model is indeed
>> an interesting thing to do, especially if you want to reduce the
>> amount of data in your transformed signal, however it is not a must.
>> Note that there are many other ways to model the residual apart from
>> the one mentioned in that paper.
>>
>> So far, in CLAM we are using the residual as is, with no modeling or
>> approximation. The "only" downside is that the transformed
>> signal (SMS Data) is in fact larger than the original audio when it
>> could be much smaller with not much loss in quality. If for
>> whatever reason you do need to do the residual modeling you can look
>> at the SpectralEnvelopeExtract processing. This processing
>> generates a spectral approximation (spectrum in bpf format) but from
>> an array of peaks, it would not be hard to modify it to work
>> with an input spectrum.
>>
>> X
>>
>>
>> roumbaba wrote:
>>> Hi all,
>>>
>>> I am trying to understand how the residual spectrum gets modeled in
>>> clam/SMS. I have read the Serra/Smith 1990 CMJ paper and as I
>>> understand it it describes two steps:
>>> 1- substract the harmonic spectrum from the original spectrum
>>> 2- perform a line-segment approximation of the residual spectrum
>>> obtained in 1
>>>
>>> I have stepped through clam and SMS code and I think I can see where
>>> step 1 gets performed:
>>>
>>> SMSAnalysisCore::Do()
>>> {
>>>
>>> mSinSpectralAnalysis.Do();
>>> mResSpectralAnalysis.Do();
>>> ...
>>> ...
>>> ...
>>> mSynthSineSpectrum.Do();
>>> mSpecSubstracter.Do(); /* step 1 gets performed here I think*/
>>>
>>> }
>>>
>>>
>>> but I cannot find where step 2 (line approximation) gets performed.
>>> Where should I look in the code?
>>>
>>> Thank you very much,
>>> Cheers,
>>>
>>> Roumbaba
>>>
>>> ps:
>>>
>>> Here is a quote from the paper I mentionned above:
>>>
>>> "Approximation of the Spectral Residual
>>>
>>> Assuming the the residual signal is quasi-stochastic, each
>>> magnitude-spectrum residual can be approximated by its envelope
>>> since only its shape contributes to the sound characteristics. [...]
>>> The particular line-segment approximation performed here is done by
>>> stepping through the magnitude spectrum and finding local maxima in
>>> every section, ..."
>>>
>>>
>>> _______________________________________________
>>> Clam-devel mailing list
>>> Clam-devel at llistes.projectes.lafarga.org
>>> <mailto:Clam-devel at llistes.projectes.lafarga.org>
>>> https://llistes.projectes.lafarga.org/cgi-bin/mailman/listinfo/clam-devel
>>
>>
>> _______________________________________________
>> Clam-devel mailing list
>> Clam-devel at llistes.projectes.lafarga.org
>> <mailto:Clam-devel at llistes.projectes.lafarga.org>
>> https://llistes.projectes.lafarga.org/cgi-bin/mailman/listinfo/clam-devel
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> Clam-devel mailing list
> Clam-devel at llistes.projectes.lafarga.org
> https://llistes.projectes.lafarga.org/cgi-bin/mailman/listinfo/clam-devel
>
More information about the clam-devel
mailing list