[Clam-devel] Re: gsoc project - real time synthesizer for a woodwind instrument

Greg Kellum greg.kellum at gmail.com
Wed May 28 21:41:30 PDT 2008


Hi Yushen,

The loop making algorithm is not trying to match up the start and end
points exactly so that you could just loop there;  it's intended to be
used together with crossfading.  In my own experience trying to match
up the zero crossings rather than cross fading doesn't work well;
Trying that in Kontakt always sounded like crap, but I suppose if
you're trying to loop very harmonic material with a stable period it
could work...  In any case the looping algorithm is looking for
spectrally similar frames, and then in my synth, they are crossfaded.
This happens in the class LoopingSDIFFileReader...  Before you
re-implement the algorithm you might want to try looping your material
in Kontakt or some other sampler to see if you can get a good loop
using only zero crossings and no crossfading...

Best,
Greg


On Wed, May 28, 2008 at 10:10 PM, Han, Yushen <yushen.han at gmail.com> wrote:
> Hi,
>
> I forget to mention that the most of my samples are 1 - 1.5 sec long. (oboe)
> The stable state is as long as 1 sec usually, which is way too short for
> Greg's algorithm.
>
> I could get some good loops if I specify the startSample and endSample.
> But startTime endTime (in ms) did not generate good loops.
>
> Best regards,
> Han, Yushen
>
> On Wed, May 28, 2008 at 4:07 PM, Han, Yushen <yushen.han at gmail.com> wrote:
>>
>> Hi, Pau and Greg
>>
>> Thanks to Greg's reply, I am able to covert my audio samples to SDIF and
>> identify loops ( with startTime/endTime in ms specified in a .XML file).
>> But If I extract the loop with startTime/endTime from the SDIF file, the
>> loops contain "clicks" and hence sound bad.
>>
>> Here I suspect hat a ms may not be the best unit to specify a loop.
>> e.g. suppose SR = 44100 Hz, HOP = 256 samples (as used in Greg's code),
>> for a note C4 (f = 261.6Hz), we will have 3.8ms per period i.e 168.8
>> samples per period.
>> The fraction will lead to round error if we specify the loop by ms.
>>
>> In the hope that a sample-by-sample approach will avoid the click,
>> I am trying to develop my algorithm to identify an optimal loop based on
>> the best f0 given by
>> segment.GetFrame(counter).GetFundamental();
>>
>> However, I check out the reference for segment, frame, and audio but did
>> not know how to specify the unit by sample.
>>
>>
>> http://www.clam.iua.upf.edu/doc/CLAM-devel-doxygen/classCLAM_1_1Segment.html
>> http://www.clam.iua.upf.edu/doc/CLAM-devel-doxygen/classCLAM_1_1Audio.html
>>
>> Do you agree that it is the reason why the loop sounds like "clicking"?
>> Do you know if there is a way to refer to a chunk of audio by sample?
>> Thanks!
>>
>> Best regards,
>> Han, Yushen
>>
>>
>>
>>
>>
>>
>>
>>
>> On Thu, May 22, 2008 at 6:51 AM, Pau Arumí <parumi at iua.upf.edu> wrote:
>>>
>>> On dj, 2008-05-22 at 03:11 -0400, Han, Yushen wrote:
>>> > Hi, Greg and Pau
>>> >
>>> > I did some tests on SDIF files out of wave files and so far it works
>>> > fine.
>>> > Now I am trying to find out the loop of a single note and store the
>>> > loop in .xml file.
>>> > Here I have questions for each of you separately:
>>> >
>>> > (1) for Greg
>>> > I compiled your makeLoop (from the examples folder) successfully.
>>> > But if I specify the start time and the end time of the steady state,
>>> > makeLoopExe always "calculate 0 EarthMover's distance".
>>> > If I do not specify the time, makeLoopExe would never finish. ( it
>>> > gives dots ".................." )
>>> >
>>> > Did you notice this problem before? Maybe it only happened to me.
>>> >
>>> > Also, in your previous message you said you "turned off the database
>>> > lookup function".
>>> > Can you please explain more about what "the lookup function" is
>>> > supposed to do and why you turned it off?
>>> >
>>> > ( I could see that you disabled the residual part in the SDIFInConfig.
>>> > But I am not sure if this is what you were talking about... )
>>> >
>>> > (2) for Pau:
>>> > So far we have not decide which woodwind instrument we should work
>>> > with.
>>> > Should I work on Saxophone or Oboe?
>>> > Do we have annotated saxophone notes from SALTO with all loop
>>> > identified?
>>> >
>>> > I cut and labeled 72 oboe notes recorded in an anechoic chamber by
>>> > hand (24 pitches C4-B5 * 3 different dynamics).
>>> > It will take a while to find the loop for those notes. Before I
>>> > proceed, I would like to make sure I am working on the right
>>> > instrument :-)
>>>
>>> Hi Yushen,
>>>
>>> Yes please, proceed with your oboe notes.
>>> I'll also search for the old-salto saxophoe database, but there are
>>> chances they are not available.
>>> It would be interesting to work with a couple (or more) instruments,
>>> though we can concentrate on the sound quality for only one of them.
>>> I think that working with multiple instruments will force us to improve
>>> the tools for creating instruments, which should be one of the goals.
>>> Another goal i'd like to propose is offering transformations not typical
>>> in time-domain based samples, taking advantage of the underlying
>>> spectral model. Timber morphing from one instrument to another is the
>>> first that comes to my mind.
>>> Of course, the first milestone is having a usable instrument.
>>>
>>> Pau
>>>
>>>
>>> >
>>> > Best regards,
>>> > Han, Yushen
>>> >
>>> > On Sun, May 18, 2008 at 10:35 AM, Greg Kellum <greg.kellum at gmail.com>
>>> > wrote:
>>> >         Hi,
>>> >
>>> >         In response to point 3, I turned off the database lookup
>>> >         function at
>>> >         some point, because of an exception that kept occurring...  I
>>> >         can't
>>> >         remember whether I eventually fixed this or not.  I'll have to
>>> >         check...
>>> >
>>> >         Greg
>>> >
>>> >
>>> >         On Sat, May 17, 2008 at 6:04 PM, Han, Yushen
>>> >         <yushen.han at gmail.com> wrote:
>>> >         > Hello, Pau
>>> >         >
>>> >         > Thanks for your concerns. Here is some update from me.
>>> >         >
>>> >         > 1. Based on Greg's last update, I am able to compile the
>>> >         command-line
>>> >         > version of continuousExcitationSynthesizer.
>>> >         > Also, I can read the XML file where the looping information
>>> >         is stored with
>>> >         > his metadata reader.
>>> >         > As for SDIF, I am able to use CLAM/examples/SDIF2Wav and
>>> >         Wav2SDIF to convert
>>> >         > between SDIF and WAV.
>>> >         >
>>> >         > 2. In the NetworkEditor, I still could not have
>>> >         "SDIFDatabaseProcessing".
>>> >         > (Greg had reported that he fixed this problem.)
>>> >         > Right now I don't really use the network editor but the
>>> >         command-line
>>> >         > executable version.
>>> >         >
>>> >         > To resume our remote debugging, last time you concluded
>>> >         > "It seems your OS is not loading some of the processings. To
>>> >         confirm
>>> >         > this: how many processings do you have in the 3D Audio
>>> >         category?
>>> >         > You should see 14."
>>> >         > However, I only had 1 processing - VectorBasedArrayPanning
>>> >         available after
>>> >         > compiling that 3D plugin.
>>> >         >
>>> >         > 3. I did some experiment with 12 oboe notes in one octave
>>> >         and concluded that
>>> >         > if there is only one source sounding,
>>> >         > the first 4 harmonics are quite in phase ( I only observe
>>> >         the first 4
>>> >         > harmonics). Therefore I was thinking that we may disregard
>>> >         the instantaneous
>>> >         > phase in the SMS. However, this is not true for an
>>> >         orchestra.
>>> >         >
>>> >         > "- [2+ days] To investigate the effect of slowly-changing
>>> >         phase changes
>>> >         > between different partials
>>> >         >     on the perception of the harmonic sound. If the
>>> >         slowly-change phase did
>>> >         > not affect the sound
>>> >         >     very much ( suppose all harmonics are all in phase ),
>>> >         further
>>> >         > simplification in synthesis
>>> >         >     could be done. This experiment is open to suggestion
>>> >         from the mentor."
>>> >         > I will write to you separately about that and more raw
>>> >         ideas.
>>> >         >
>>> >         > 4. Right now I am trying to modify Greg's code and Wav2SDIF
>>> >         example for:
>>> >         > "- [1-2 days] To synthesize with SMS with the analysis
>>> >         result (in a simple
>>> >         > setting).
>>> >         >   To produce some audible result to evaluate the performance
>>> >         of SMS
>>> >         > analysis/transformation."
>>> >         >
>>> >         > It seemed that the Wav2SDIF example had done the majority of
>>> >         the work.
>>> >         > I am studying the code but I don't know if I could improve
>>> >         this part.
>>> >         > Do you recommend me to read some updated documents on SDIF?
>>> >         > http://www.cnmat.berkeley.edu/SDIF/ does not seem to be very
>>> >         recent.
>>> >         >
>>> >         > 5. I apologize for my late updating.
>>> >         > I am thinking of keeping a blog on this project...
>>> >         >
>>> >         > Best regards,
>>> >         > Han, Yushen
>>> >         >
>>> >         >
>>> >         >
>>> >         >
>>> >         > On Sat, May 17, 2008 at 7:40 AM, Pau Arumí
>>> >         <parumi at iua.upf.edu> wrote:
>>> >         >>
>>> >         >> Hello Yushen,
>>> >         >>
>>> >         >> how is all doing? I'd like to hear in what you are up
>>> >         regarding your
>>> >         >> project.
>>> >         >>
>>> >         >> are you still blocked with the plugin installation? in that
>>> >         case is it
>>> >         >> possible for you have also a linux box?
>>> >         >>
>>> >         >> i'd like you to write some raw ideas about what to be
>>> >         implemented and
>>> >         >> some planning, and then i'll comment and keep refining the
>>> >         planning.
>>> >         >> is that ok?
>>> >         >>
>>> >         >> cheers!
>>> >         >> pau
>>> >         >>
>>> >         >>
>>> >         >
>>> >         >
>>> >
>>> >
>>>
>>
>
>




More information about the clam-devel mailing list