[Clam-devel] SMSSynthesis

[Xavier Amatriain]

Hi Greg,

Greg Kellum wrote:
> I made the aforementioned changes in the SMSSynthesis class, or I made 
> some of the aforementioned changes...  I agree that it would be 
> possible to both check to see whether all ports were connected and to 
> make it configurable to remove the out ports for the residual and 
> sinusoidal audio, but it struck me as both easier and more important 
> to do the first rather than the second. 
>
> My reasons for thinking so are: 
> One, I think one should program with the users in mind that don't know 
> very much about how the application works, and such users won't know 
> to look into the configuration dialog to disable two unused ports.  
> And these two ports can make a significant difference in an 
> application's performance.  My own application runs 10% with those two 
> unneeded IFFTs disabled.
> Two, we have to keep in mind that CLAM is first and foremost a C++ 
> library for audio and music, and it's unlikely that a programmer 
> looking at the API for SMSSynthesis would know to navigate through 
> SMSSynthesis's nested configuration elements to set the right flags to 
> disable those two out ports. 
I agree with you. I'd like to hear from Pau if he has a different 
opinion but I had this same feeling when I replied to you: doing port 
checking would not only  be
easier to implement but also result in a more user friendly interface.
> Anyway, while I was working on this, I compared my profiling results 
> with a pure SinusoidalSynthesis.  One would think that the CPU usage 
> of the SinusoidalSynthesis and the SMSSynthesis with one IFFT would be 
> nearly identical, but they're not.  The SinusoidalSynthesis is 30% 
> faster.  I spent some time thinking about why this might be, and all I 
> could come up with is that the residual spectrum objects are very 
> large and copying around their data must eating up a lot of CPU 
> cycles.  So, I wanted to ask whether there is any code in CLAM for 
> analyzing and resynthesizing residuals stored as envelopes rather than 
> as the complete FFT bin values.  Using envelopes would take much less 
> memory...
You hit right on the nail here. This is a missing feature in CLAM that I 
have longed for a long time but have not implemented for a lack of time. 
Residual
approximation is not trivial and might take some thought. However a 
trivial implementation would be "easy" to do using the 
SpectrumEnvelopeExtractor in CLAM
and applying the resulting envelope to shape a flat broadband spectrum.

Btw, in terms of optimization I am wondering, are you compiling with the 
CLAM_OPTIMIZE activated? Do you  have any idea of whether this makes
a big difference in your processes (it does in my computer for the SMS 
Analysis Synthesis procedure).