[Clam-devel] Re: gsoc project - real time synthesizer for a woodwind instrument

Thu May 22 01:49:37 PDT 2008

Hi Yushen,

I thought a little bit more about why you were only hearing a single  
sample being used in your database, and what I wrote in my last email  
was wrong...  Um...  Basically, once the synthesizer chooses an SDIF  
file for a particular note it sticks with that file.   This wasn't  
what I originally intended to do when I started working on the  
synthesizer, but I never got to the point where switching SDIF files  
made the synthesis results sound better;  it always made them sound  
worse... In any case if you want to hear all your audio files being  
used you need to make sure a new voice is being generated for each new  
note.  The algorithm for creating new voices that's being used in the  
command line application is really very primitive.  I was just using  
it for testing, and my intention was that the control source would be  
responsible for creating new voices by sending a new MIDI note on  
message or an OSC control message.  What the command line application  
is doing is checking to see whether the pitch has changed by about a  
semitone, and if it has, then it allocates a new voice.  (See the file  
ControlStreamSegmentator)  If in your tests only one or two files were  
being used, I would think it must be because the test score you were  
using doesn't contain jumps in pitch of a semitone.  Could you send me  
the test score that you were using?

With regards to the loopMaker application, I just tried running that  
myself, and as I couldn't remember how it worked, at first I had the  
same problems as you did.  But then I thought about why it wasn't  
finding any loop points, and I remembered that if the loop points were  
too close the loop sounded bad.  And in the file LoopPointExtractor I  
had made a rule that frames that were too close together couldn't be  
chosen as loop points.  The method that actually does this is called  
isCloseToFrameAlreadyReported, and as you can see, at the moment it's  
not disallowing loops of less than a second:

bool  
LoopPointExtractor::isCloseToFrameAlreadyReported(DistanceMeasure&  
currentDistanceMeasure, std::vector<CLAM::DistanceMeasure>&  
reportedDistanceVector)
{
	int minimumFrameSeparation = 20;
	int oneSecondFrameSeparation = 173;
... and so on...

You might want to change this to a smaller value, or alternatively,  
you could just use sound files with a steady state portion that is  
longer than a second.  If you try to use samples with a very short  
steady state portion, then you're going to need to build some sort of  
jitter algorithm into your synthesizer so that the produced note  
doesn't sound too static -- just like they did in Salto...

Best,
Greg

On May 22, 2008, at 9:11 AM, Han, Yushen wrote:

> Hi, Greg and Pau
>
> I did some tests on SDIF files out of wave files and so far it works  
> fine.
> Now I am trying to find out the loop of a single note and store the  
> loop in .xml file.
> Here I have questions for each of you separately:
>
> (1) for Greg
> I compiled your makeLoop (from the examples folder) successfully.
> But if I specify the start time and the end time of the steady  
> state, makeLoopExe always "calculate 0 EarthMover's distance".
> If I do not specify the time, makeLoopExe would never finish. ( it  
> gives dots ".................." )
>
> Did you notice this problem before? Maybe it only happened to me.
>
> Also, in your previous message you said you "turned off the database  
> lookup function".
> Can you please explain more about what "the lookup function" is  
> supposed to do and why you turned it off?
>
> ( I could see that you disabled the residual part in the  
> SDIFInConfig. But I am not sure if this is what you were talking  
> about... )
>
> (2) for Pau:
> So far we have not decide which woodwind instrument we should work  
> with.
> Should I work on Saxophone or Oboe?
> Do we have annotated saxophone notes from SALTO with all loop  
> identified?
>
> I cut and labeled 72 oboe notes recorded in an anechoic chamber by  
> hand (24 pitches C4-B5 * 3 different dynamics).
> It will take a while to find the loop for those notes. Before I  
> proceed, I would like to make sure I am working on the right  
> instrument :-)
>
>
> Best regards,
> Han, Yushen
>
> On Sun, May 18, 2008 at 10:35 AM, Greg Kellum  
> <greg.kellum at gmail.com> wrote:
> Hi,
>
> In response to point 3, I turned off the database lookup function at
> some point, because of an exception that kept occurring...  I can't
> remember whether I eventually fixed this or not.  I'll have to
> check...
>
> Greg
>
> On Sat, May 17, 2008 at 6:04 PM, Han, Yushen <yushen.han at gmail.com>  
> wrote:
> > Hello, Pau
> >
> > Thanks for your concerns. Here is some update from me.
> >
> > 1. Based on Greg's last update, I am able to compile the command- 
> line
> > version of continuousExcitationSynthesizer.
> > Also, I can read the XML file where the looping information is  
> stored with
> > his metadata reader.
> > As for SDIF, I am able to use CLAM/examples/SDIF2Wav and Wav2SDIF  
> to convert
> > between SDIF and WAV.
> >
> > 2. In the NetworkEditor, I still could not have  
> "SDIFDatabaseProcessing".
> > (Greg had reported that he fixed this problem.)
> > Right now I don't really use the network editor but the command-line
> > executable version.
> >
> > To resume our remote debugging, last time you concluded
> > "It seems your OS is not loading some of the processings. To confirm
> > this: how many processings do you have in the 3D Audio category?
> > You should see 14."
> > However, I only had 1 processing - VectorBasedArrayPanning  
> available after
> > compiling that 3D plugin.
> >
> > 3. I did some experiment with 12 oboe notes in one octave and  
> concluded that
> > if there is only one source sounding,
> > the first 4 harmonics are quite in phase ( I only observe the  
> first 4
> > harmonics). Therefore I was thinking that we may disregard the  
> instantaneous
> > phase in the SMS. However, this is not true for an orchestra.
> >
> > "- [2+ days] To investigate the effect of slowly-changing phase  
> changes
> > between different partials
> >     on the perception of the harmonic sound. If the slowly-change  
> phase did
> > not affect the sound
> >     very much ( suppose all harmonics are all in phase ), further
> > simplification in synthesis
> >     could be done. This experiment is open to suggestion from the  
> mentor."
> > I will write to you separately about that and more raw ideas.
> >
> > 4. Right now I am trying to modify Greg's code and Wav2SDIF  
> example for:
> > "- [1-2 days] To synthesize with SMS with the analysis result (in  
> a simple
> > setting).
> >   To produce some audible result to evaluate the performance of SMS
> > analysis/transformation."
> >
> > It seemed that the Wav2SDIF example had done the majority of the  
> work.
> > I am studying the code but I don't know if I could improve this  
> part.
> > Do you recommend me to read some updated documents on SDIF?
> > http://www.cnmat.berkeley.edu/SDIF/ does not seem to be very recent.
> >
> > 5. I apologize for my late updating.
> > I am thinking of keeping a blog on this project...
> >
> > Best regards,
> > Han, Yushen
> >
> >
> >
> >
> > On Sat, May 17, 2008 at 7:40 AM, Pau Arumí <parumi at iua.upf.edu>  
> wrote:
> >>
> >> Hello Yushen,
> >>
> >> how is all doing? I'd like to hear in what you are up regarding  
> your
> >> project.
> >>
> >> are you still blocked with the plugin installation? in that case  
> is it
> >> possible for you have also a linux box?
> >>
> >> i'd like you to write some raw ideas about what to be implemented  
> and
> >> some planning, and then i'll comment and keep refining the  
> planning.
> >> is that ok?
> >>
> >> cheers!
> >> pau
> >>
> >>
> >
> >
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.clam-project.org/pipermail/clam-devel-clam-project.org/attachments/20080522/43695331/attachment-0003.htm>