[Clam-devel] Spectral Analysis/Synthesis

[Pau Arumí]

On dj, 2008-02-07 at 20:22 +0100, Karsten Krispin wrote:
> Hello!
> 
> Well, I'm new to this list, so let me first introduce a bit.. ;)
> 
> Currently I'm implementing an own synthesizer for our own reactable.
> I'm using Qt4 for the graphical stuff while I chose CLAM for the DSP stuff.
> 
> The problem I have got, is that when I try to implement the spectral-domain 
> stuff, I get quite disorted outputs when my signals has been pushed through a 
> spectral analysis/synthesis-chain. 
> 
> My current setup in short is like the following:
> 
>  Samplerate: 48000
>  Buffersize (Hopsize) : 512 ( I guess that's default)
> 
> Derived class from CLAM::Frame for Audio-Transport: I've just implemented some 
> flags of whether the spectrum or audio array still holds the latest 
> transformed data etc.. (Spectrum or Audio transforms are setting those flags 
> automaticly, so that they know whether they need to synthesize the audio from 
> spectrum or vice-versa).
> 
> Then, for my Synthesizer Items, I've derived QGraphicItems a dozent of times, 
> but that should't matter. How ever, all Items who can perform such a 
> conversation between audio and spectrum are having their own private 
> SpectralAnalysis/SpectralSynthesis and OverlapAdd. That's due to the fact, 
> that they run in different threads, for parallelism. And FFTW3 seems to need 
> this, to become thread-safe. 
> 
> So, the init stuff for those three CLAM-Processing looks like the following 
> (that's within the class-constructor):
> 
> 	int samplingRate = 48000, frameSize = 512;
> 	
> 	CLAM::SpectralAnalysisConfig specAnalConf;
> 	CLAM::SpectralSynthesisConfig specSynthConf;
> 	CLAM::OverlapAddConfig	overlapConf;
> 	
> 
> 	specAnalConf.SetSamplingRate(samplingRate);
> 	specAnalConf.SetHopSize(frameSize);
> 	this->_specAnal.Configure(specAnalConf);
> 	this->_specAnal.Start();
> 	
> 
> 	specSynthConf.SetHopSize(frameSize);
> 	specSynthConf.SetSamplingRate(samplingRate);
> 	this->_specSynth.Configure(specSynthConf);
> 	this->_specSynth.Start();
> 	
> 	overlapConf.SetFrameSize(frameSize);
> 	this->_overlapAdd.Configure(overlapConf);
> 	this->_overlapAdd.Start();
> 
> 
> 
> Two functions of that class are responsible for the conversation:
> 
> AudioFlowItem::convertAudioToSpectrum(AudioPackage *package)
> {
> [...] this->_specAnal.Do(package->GetAudioFrame(), package->GetSpectrum())
> }
> 
> AudioFlowItem::convertSpectrumToAudio(AudioPackage *package)
> {
> 
> 	[...]
> 	CLAM::Audio tmpAudio;
> 	tmpAudio.SetSize(package->frameSize()*2);
> 	this->_specSynth.Do(package->GetSpectrum(), tmpAudio);
> 	this->_overlapAdd.Do(tmpAudio, package->GetAudioFrame());
> }
> 
> 
> Whereas AudioPackage is the derived class of CLAM::Frame.
> I've just included the relevant parts here, of course.
> 
> 
> I'm sorry that I ask like a kind of "please code my application". But that 
> what you see here is actually anything what fails within my app and I don't 
> have any clue why... From what I understand, using the config-default's 
> should work pretty well. I've also read the thread "Spectral 
> Analysis/Synthesis problem" of globot on that list here. He seems to have/had 
> the same problem. How ever, I could not get any hints out of that thread. 
> 
> To ramp up this mess, I've constructed the same part in NetworkEditor 
> (SimpleOsc -> SpecAnal -> SpecSynth -> OverlapAdd -> AudioSink) and compared 
> the config-values of Analysis and Synth  with those within my app. They are 
> absolutely the same. But within NetworkEditor everything works fine.
> 
> If I leave out the spectrum stuff and just push out a pure sine-wave without 
> anal/synth in between the generator and audio-sink, it will be outputted 
> pretty smooth. So it is nothing driver/hw-related, like x-runs...


Hi Karsten,
your system seems fairly complex and I think that having an isolated
example of spectralAnalysis/Synthesis working would help a lot.

The code doesn't rings a bell on me about the problem. But it is likely
that it's is related to the fact that you are not using ports.
SpectralAnalysis ports consumes a given HopSize of samples while access
a given WindowSize.
So my advice is to take a look at this test, and maybe start from this
code:
CLAM/test/FunctionalTests/ProcessingTests/TestStreamingSMSAnalysisSynthesis.cxx

In case the problem is because of that: you can still use the
processings without ports (as you do now) but buffers would require some
extra cooking 

Cheers from Barcelona!

Pau



> Thanks in advance and greetings from germany,
> 
> Karsten Krispin
> 
> Ps: Please excuse my bad english but it became late ;)
> 
> _______________________________________________
> Clam-devel mailing list
> Clam-devel at llistes.projectes.lafarga.org
> https://llistes.projectes.lafarga.org/cgi-bin/mailman/listinfo/clam-devel