[Clam-devel] Re: gsoc project - real time synthesizer for a woodwind instrument

Greg Kellum greg.kellum at gmail.com
Fri May 30 01:47:00 PDT 2008


Hi Yushen,

I was looking at the problem the wrong way.  I was looking at it as a  
choice between looping at the zero crossings or cross fading.  But of  
course you can do both, and I guess it will sound better if you do do  
both...

Best,
Greg



On May 29, 2008, at 7:04 AM, Han, Yushen wrote:

> Greg,
>
> I see. That makes sense.
> I would not expect to have good loops without crossfading.
> It seems that one can't say much about the quality of the loop until  
> he could hear the loop with crossfading.
> I need to look into LoopingSDIFFileReader before I can tell more  
> about the loops that the loopMaker found.
>
> I did some test with my oboe samples and the loop (precise to  
> sample) sound very good with zero crossings/no crossfading.
> Thanks for your reminder.
>
> Best regards,
> Han, Yushen
>
> On Thu, May 29, 2008 at 12:41 AM, Greg Kellum  
> <greg.kellum at gmail.com> wrote:
> Hi Yushen,
>
> The loop making algorithm is not trying to match up the start and end
> points exactly so that you could just loop there;  it's intended to be
> used together with crossfading.  In my own experience trying to match
> up the zero crossings rather than cross fading doesn't work well;
> Trying that in Kontakt always sounded like crap, but I suppose if
> you're trying to loop very harmonic material with a stable period it
> could work...  In any case the looping algorithm is looking for
> spectrally similar frames, and then in my synth, they are crossfaded.
> This happens in the class LoopingSDIFFileReader...  Before you
> re-implement the algorithm you might want to try looping your material
> in Kontakt or some other sampler to see if you can get a good loop
> using only zero crossings and no crossfading...
>
> Best,
> Greg
>
>
> On Wed, May 28, 2008 at 10:10 PM, Han, Yushen <yushen.han at gmail.com>  
> wrote:
> > Hi,
> >
> > I forget to mention that the most of my samples are 1 - 1.5 sec  
> long. (oboe)
> > The stable state is as long as 1 sec usually, which is way too  
> short for
> > Greg's algorithm.
> >
> > I could get some good loops if I specify the startSample and  
> endSample.
> > But startTime endTime (in ms) did not generate good loops.
> >
> > Best regards,
> > Han, Yushen
> >
> > On Wed, May 28, 2008 at 4:07 PM, Han, Yushen  
> <yushen.han at gmail.com> wrote:
> >>
> >> Hi, Pau and Greg
> >>
> >> Thanks to Greg's reply, I am able to covert my audio samples to  
> SDIF and
> >> identify loops ( with startTime/endTime in ms specified in a .XML  
> file).
> >> But If I extract the loop with startTime/endTime from the SDIF  
> file, the
> >> loops contain "clicks" and hence sound bad.
> >>
> >> Here I suspect hat a ms may not be the best unit to specify a loop.
> >> e.g. suppose SR = 44100 Hz, HOP = 256 samples (as used in Greg's  
> code),
> >> for a note C4 (f = 261.6Hz), we will have 3.8ms per period i.e  
> 168.8
> >> samples per period.
> >> The fraction will lead to round error if we specify the loop by ms.
> >>
> >> In the hope that a sample-by-sample approach will avoid the click,
> >> I am trying to develop my algorithm to identify an optimal loop  
> based on
> >> the best f0 given by
> >> segment.GetFrame(counter).GetFundamental();
> >>
> >> However, I check out the reference for segment, frame, and audio  
> but did
> >> not know how to specify the unit by sample.
> >>
> >>
> >> http://www.clam.iua.upf.edu/doc/CLAM-devel-doxygen/classCLAM_1_1Segment.html
> >> http://www.clam.iua.upf.edu/doc/CLAM-devel-doxygen/classCLAM_1_1Audio.html
> >>
> >> Do you agree that it is the reason why the loop sounds like  
> "clicking"?
> >> Do you know if there is a way to refer to a chunk of audio by  
> sample?
> >> Thanks!
> >>
> >> Best regards,
> >> Han, Yushen
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >> On Thu, May 22, 2008 at 6:51 AM, Pau Arumí <parumi at iua.upf.edu>  
> wrote:
> >>>
> >>> On dj, 2008-05-22 at 03:11 -0400, Han, Yushen wrote:
> >>> > Hi, Greg and Pau
> >>> >
> >>> > I did some tests on SDIF files out of wave files and so far it  
> works
> >>> > fine.
> >>> > Now I am trying to find out the loop of a single note and  
> store the
> >>> > loop in .xml file.
> >>> > Here I have questions for each of you separately:
> >>> >
> >>> > (1) for Greg
> >>> > I compiled your makeLoop (from the examples folder)  
> successfully.
> >>> > But if I specify the start time and the end time of the steady  
> state,
> >>> > makeLoopExe always "calculate 0 EarthMover's distance".
> >>> > If I do not specify the time, makeLoopExe would never finish.  
> ( it
> >>> > gives dots ".................." )
> >>> >
> >>> > Did you notice this problem before? Maybe it only happened to  
> me.
> >>> >
> >>> > Also, in your previous message you said you "turned off the  
> database
> >>> > lookup function".
> >>> > Can you please explain more about what "the lookup function" is
> >>> > supposed to do and why you turned it off?
> >>> >
> >>> > ( I could see that you disabled the residual part in the  
> SDIFInConfig.
> >>> > But I am not sure if this is what you were talking about... )
> >>> >
> >>> > (2) for Pau:
> >>> > So far we have not decide which woodwind instrument we should  
> work
> >>> > with.
> >>> > Should I work on Saxophone or Oboe?
> >>> > Do we have annotated saxophone notes from SALTO with all loop
> >>> > identified?
> >>> >
> >>> > I cut and labeled 72 oboe notes recorded in an anechoic  
> chamber by
> >>> > hand (24 pitches C4-B5 * 3 different dynamics).
> >>> > It will take a while to find the loop for those notes. Before I
> >>> > proceed, I would like to make sure I am working on the right
> >>> > instrument :-)
> >>>
> >>> Hi Yushen,
> >>>
> >>> Yes please, proceed with your oboe notes.
> >>> I'll also search for the old-salto saxophoe database, but there  
> are
> >>> chances they are not available.
> >>> It would be interesting to work with a couple (or more)  
> instruments,
> >>> though we can concentrate on the sound quality for only one of  
> them.
> >>> I think that working with multiple instruments will force us to  
> improve
> >>> the tools for creating instruments, which should be one of the  
> goals.
> >>> Another goal i'd like to propose is offering transformations not  
> typical
> >>> in time-domain based samples, taking advantage of the underlying
> >>> spectral model. Timber morphing from one instrument to another  
> is the
> >>> first that comes to my mind.
> >>> Of course, the first milestone is having a usable instrument.
> >>>
> >>> Pau
> >>>
> >>>
> >>> >
> >>> > Best regards,
> >>> > Han, Yushen
> >>> >
> >>> > On Sun, May 18, 2008 at 10:35 AM, Greg Kellum <greg.kellum at gmail.com 
> >
> >>> > wrote:
> >>> >         Hi,
> >>> >
> >>> >         In response to point 3, I turned off the database lookup
> >>> >         function at
> >>> >         some point, because of an exception that kept  
> occurring...  I
> >>> >         can't
> >>> >         remember whether I eventually fixed this or not.  I'll  
> have to
> >>> >         check...
> >>> >
> >>> >         Greg
> >>> >
> >>> >
> >>> >         On Sat, May 17, 2008 at 6:04 PM, Han, Yushen
> >>> >         <yushen.han at gmail.com> wrote:
> >>> >         > Hello, Pau
> >>> >         >
> >>> >         > Thanks for your concerns. Here is some update from me.
> >>> >         >
> >>> >         > 1. Based on Greg's last update, I am able to compile  
> the
> >>> >         command-line
> >>> >         > version of continuousExcitationSynthesizer.
> >>> >         > Also, I can read the XML file where the looping  
> information
> >>> >         is stored with
> >>> >         > his metadata reader.
> >>> >         > As for SDIF, I am able to use CLAM/examples/SDIF2Wav  
> and
> >>> >         Wav2SDIF to convert
> >>> >         > between SDIF and WAV.
> >>> >         >
> >>> >         > 2. In the NetworkEditor, I still could not have
> >>> >         "SDIFDatabaseProcessing".
> >>> >         > (Greg had reported that he fixed this problem.)
> >>> >         > Right now I don't really use the network editor but  
> the
> >>> >         command-line
> >>> >         > executable version.
> >>> >         >
> >>> >         > To resume our remote debugging, last time you  
> concluded
> >>> >         > "It seems your OS is not loading some of the  
> processings. To
> >>> >         confirm
> >>> >         > this: how many processings do you have in the 3D Audio
> >>> >         category?
> >>> >         > You should see 14."
> >>> >         > However, I only had 1 processing -  
> VectorBasedArrayPanning
> >>> >         available after
> >>> >         > compiling that 3D plugin.
> >>> >         >
> >>> >         > 3. I did some experiment with 12 oboe notes in one  
> octave
> >>> >         and concluded that
> >>> >         > if there is only one source sounding,
> >>> >         > the first 4 harmonics are quite in phase ( I only  
> observe
> >>> >         the first 4
> >>> >         > harmonics). Therefore I was thinking that we may  
> disregard
> >>> >         the instantaneous
> >>> >         > phase in the SMS. However, this is not true for an
> >>> >         orchestra.
> >>> >         >
> >>> >         > "- [2+ days] To investigate the effect of slowly- 
> changing
> >>> >         phase changes
> >>> >         > between different partials
> >>> >         >     on the perception of the harmonic sound. If the
> >>> >         slowly-change phase did
> >>> >         > not affect the sound
> >>> >         >     very much ( suppose all harmonics are all in  
> phase ),
> >>> >         further
> >>> >         > simplification in synthesis
> >>> >         >     could be done. This experiment is open to  
> suggestion
> >>> >         from the mentor."
> >>> >         > I will write to you separately about that and more raw
> >>> >         ideas.
> >>> >         >
> >>> >         > 4. Right now I am trying to modify Greg's code and  
> Wav2SDIF
> >>> >         example for:
> >>> >         > "- [1-2 days] To synthesize with SMS with the analysis
> >>> >         result (in a simple
> >>> >         > setting).
> >>> >         >   To produce some audible result to evaluate the  
> performance
> >>> >         of SMS
> >>> >         > analysis/transformation."
> >>> >         >
> >>> >         > It seemed that the Wav2SDIF example had done the  
> majority of
> >>> >         the work.
> >>> >         > I am studying the code but I don't know if I could  
> improve
> >>> >         this part.
> >>> >         > Do you recommend me to read some updated documents  
> on SDIF?
> >>> >         > http://www.cnmat.berkeley.edu/SDIF/ does not seem to  
> be very
> >>> >         recent.
> >>> >         >
> >>> >         > 5. I apologize for my late updating.
> >>> >         > I am thinking of keeping a blog on this project...
> >>> >         >
> >>> >         > Best regards,
> >>> >         > Han, Yushen
> >>> >         >
> >>> >         >
> >>> >         >
> >>> >         >
> >>> >         > On Sat, May 17, 2008 at 7:40 AM, Pau Arumí
> >>> >         <parumi at iua.upf.edu> wrote:
> >>> >         >>
> >>> >         >> Hello Yushen,
> >>> >         >>
> >>> >         >> how is all doing? I'd like to hear in what you are up
> >>> >         regarding your
> >>> >         >> project.
> >>> >         >>
> >>> >         >> are you still blocked with the plugin installation?  
> in that
> >>> >         case is it
> >>> >         >> possible for you have also a linux box?
> >>> >         >>
> >>> >         >> i'd like you to write some raw ideas about what to be
> >>> >         implemented and
> >>> >         >> some planning, and then i'll comment and keep  
> refining the
> >>> >         planning.
> >>> >         >> is that ok?
> >>> >         >>
> >>> >         >> cheers!
> >>> >         >> pau
> >>> >         >>
> >>> >         >>
> >>> >         >
> >>> >         >
> >>> >
> >>> >
> >>>
> >>
> >
> >
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.clam-project.org/pipermail/clam-devel-clam-project.org/attachments/20080530/c4f0ceea/attachment-0004.htm>


More information about the clam-devel mailing list