[Clam-devel] Re: TODOs

[Han, Yushen]

Hi, Pau  and Greg

Thanks! Here I try to answer Pau's questions briefly.

1. the segment error
>From your error message It seems that the score file was not read properly.

> TestScore::LoadScore. number of tokens: 5   // here we should have only 2 tokens every line
> scoreVetor [0][0] = floatToken =0  // this should be the frequency
> scoreVetor [1][0] = floatToken =0  // this should be the loudness/brightness
> Segmentation fault

To make sure that the synthesizer read the right score, here is the
syntax to use the synthesizer from the command lines
./SimpleOboeSynthesizer <where the SDIF files are> <the text score
file> testOutput.wav synthesis.xml

The text score files could be found in testScore.tar, C#D.txt is an
easy start where OboeDMajorAt5.txt is a challenge.

As I mentioned before, my new plugin could read score text from 3
major platforms
( the delimiter is '\r' in Mac, '\n' in Unix and \n\r in DOS/Windows ).
To test, I created score text files for 3 different platforms from my
Mac (by TextWrangler),
they are all fine for me. You can see testScore.cxx for detail. If
that is the problem for you, I can simply specify one format.

2. To integrate the two plugin is not a difficult job to do.
Greg's plugin was referring to some system header files that were only
useful for the plugin at the moment.
However, I tried to make those components ( I renamed them as well ) a
part of the plugin, not the system.

e.g. The following hierarchy of classes are re-organized by me based
on Greg's code. All of them are plugin headers instead of system
files.

SDIFDatabase.cxx
->
SDIFFileReaderWithBuffer.cxx
->
SDIFFileReaderWithLoop.cxx

Do you think I should keep his structure or this design when I merge
our plugins?
I would like to hear Greg's comments too.

3. To shrink the database, the quickest way is to leave only a subset
of Oboe samples. It did not have to cover all pitches from A#3 - F6 at
the moment but only those pitches present in the test score.

4. To make it plugin, I think I should be able to get the score from a
MIDI controller and be able to load the samples in another thread. (Do
you think the latter is a must?)
Now I tried to compile the osc plugin but it still did not work.

g++ -o LibloSource.os -c -fPIC -DCLAM_FLOAT -D_DEBUG -DUSE_PTHREADS=1
-DUSE_XERCES=1 -DCLAM_USE_XML -DUSE_SNDFILE=1 -DUSE_OGGVORBIS=1
-DWITH_VORBIS=1 -DUSE_MAD=1 -DWITH_MAD=1 -DUSE_ID3=1 -DUSE_PORTAUDIO=1
-D__MACOSX_CORE__ -DUSE_RTAUDIO=1 -DUSE_FFTW3=1 -I/usr/local/include
-I/opt/local/include -I/usr/local/include -I/usr/local/include/CLAM
LibloSource.cxx
In file included from LibloSource.cxx:1:
LibloSource.hxx:8:19: error: lo/lo.h: No such file or directory
LibloSource.hxx:95: error: 'lo_arg' has not been declared

I search my hard drive but there is no lo.h.
Do I need more dependencies? Thank in advance for your suggestion.

5. I think a blog instead of the wikipage that I am writing will be very good.
Do you have any spot for me? If not, I will use my google page to post
the information for potential testers.



Best regards,
Han, Yushen




On Tue, Jul 22, 2008 at 5:37 AM, Pau Arumí <parumi at iua.upf.edu> wrote:
> Congrats Yushen, finally you have some code commited  :-)
>
> commit 11673
>
>      * Yushen's patch to SpectralPeakArray: Added Resize(..) before
>        SetSize(..).  Thanks!
>
> commit 11671
>
>      * Yushen patch: New plugin "temporal_oboeSynthesizer". As the name
>        says, it's temporal be cause it should be integrated with the
>        existing continuous...Synthesizer, since they share  most of the
>        code.
>        But it's now commited to help Yushen with the integration
>
>
> Some quick observations:
>      * The duplication between your plugin dir and continuous..Synth
>        should be removed (which is the vast part of your code). I think
>        the reasonable thing to do is integrate your changes in the
>        original continous..Synth dir. Yushen, can you evaluate if this
>        is simple to do? (and maybe start sending patches)
>      * Why you have array.hxx??
>      * It is important to have as many testers as possible. At least to
>        synthesize a score file: So having step-by-step how-to somewhere
>        in the wiki should be very good. This should include all steps
>        (compilation, database set up, etc).
>      * I've executed SimpleOboeSynthesizer and got a segmentation
>        fault, see below
>      * Your new DB package is 61 MB ! I'm reluctant to add this load to
>        the data repository, specially being already commited a 30 MB
>        DB. (We could leave big DBs at the web to download instead of
>        the data repo)
>      * While I understand that big DB can be necessary when developing
>        I think we should prepare a small but usable DB for normal users
>        and testers. This should be less than 30 MB -- using less
>        samples per octave. Do you think it's a good idea?
>
> Finally, let me insist that we should put top priority in a) integrating
> code to svn, removing duplication with continuous..Synth, ensuring it's
> multiplatform, etc and b) having the real-time version. And doing all
> development in the repository commiting small patches.
>
> Cheers
> P
>
>
> $ ./SimpleOboeSynthesizer ~/clam/testdata/OboeSDIF/ synthesis.xml
> ##########################################################
> ######################## WARNING #########################
> ##########################################################
> At file /home/parumi/local/include/CLAM/Factory.hxx line 280
> WARNING. While adding a creator method in the factory, id
> 'SDIFDatabaseProcessing' was already registered.
> TestScore::LoadScore. filename: synthesis.xml
> getline(myfile, line, '\r' ).eof() == true. && rowNumber == 0 ch <- '\n'
> TestScore::LoadScore. line: <?xml version="1.0" encoding="UTF-8"
> standalone="no" ?> delimiters==\
> \
> TestScore::LoadScore. number of tokens: 5
> scoreVetor [0][0] = floatToken =0
> scoreVetor [1][0] = floatToken =0
> Segmentation fault
>
>
> On dt, 2008-07-22 at 03:34 -0400, Han, Yushen wrote:
>> Pau,
>>
>> Here is a .tar that only contains source code of the my plugin.
>> It has all .cxx .hxx that belong to the *plugin* (with SConstruct,
>> synthesis.xml, and an .xcodeproj file).
>> To use this plugin, a previous patch that fixed a bug in
>> SpectralPeakArray should be applied.
>>
>> Major change:
>> - enable the residual part of the SMS in playback. The residual part
>> has a simple crossfading scheme between the current frame and the
>> *next* loop point.
>> - A new set of oboe note samples (from the McGill Master Samples) has
>> been added to the database.
>> Now we have 3 sets of samples of different timbre that could be
>> selected according to the brightness.
>> The set (SDIF files with an external converter) could be downloaded at
>> https://www.slashtmp.iu.edu/public/download.php?FILE=yushan/613526r0rxg
>>
>> Some utilities to work with the plugin:
>> - a pitch tracker to get the score from a real performance ( this is
>> not a part of the CLAM yet and it works only for Mac )
>> A example of oboe D Major scale (audio and score) is given (see
>> testScore.tar) as well.
>> - a WavInfo to extract note onset/offset time from note samples and to
>> pass the information to the loopMaker, which has been modified to skip
>> the silence.
>>
>> For the past few days, I was trying different ways to improve the
>> quality of sound.
>> The factors that may affect the sound may include:
>> (1) the lack of the non-harmonic (mostly associated with the attack) part.
>> This has been solved and we can listen to either the residual or the
>> sinusoidal part after synthesis,
>>
>> (2) the fluctuation in the frequency/brightness score
>> This was an important when I used a score extracted from string instrument.
>> The oboe requires explicit note onset information and has a physical
>> capacity in gradual pitch change over time.
>> I tried to incorporate those things in my pitch tracker - this could
>> be another post.
>>
>> (3) the lack of voice morphing and reverberation. I will add this
>> feature in my next patch.
>>
>> (4) the limited harmonic template - apparently 3 sets for each pitch
>> is not enough, but we may not need to load more samples but to model
>> the peakArray.
>>
>> (5) some of the "cross-fading" loop points don't sound good...
>>
>> Also, I have some problem in using the osc... I may need some help
>> from you if the problem persist tomorrow.
>>
>> Best regards,
>> Han, Yushen
>>
>> On Thu, Jul 17, 2008 at 6:15 AM, Pau Arumí <parumi at iua.upf.edu> wrote:
>> > On dj, 2008-07-17 at 05:58 -0400, Han, Yushen wrote:
>> >> Pau,
>> >>
>> >> Sure, I would write to you more.
>> >>
>> >> Please refer to my blog:
>> >> http://iua-share.upf.edu/wikis/clam/index.php/GSoC/Real-time_Synthesizer_Using_SMS_Models_for_a_Woodwind_Instrument
>> >> for what I am doing these days.
>> >
>> > Why not using a proper blog instead of writing the progress in a wiki
>> > page? With a blog is easier to track updates, and we can syndicate it to
>> > the planet.
>> >
>> >
>> >> Still, the sound quality needs some more improvement.
>> >> One of the reasons is that I did not have the explicit information of
>> >> note onset in my score.
>> >> So I just made a pitch tracker to extract the pitch in Hz of a certain
>> >> voice (among different voices).
>> >>
>> >> Should I make the MIDI connection work first in a realtime scenario?
>> >> My version is able to deal with the score on the fly ( it pretended
>> >> that it did not receive the score in advance) now.
>> >
>> > MIDI is not ready at the moment, we should use OSC by now. (see
>> > plugins/osc)
>> > But we should put higher priority to make the synthesis work on
>> > NE/real-time, regardless on how we generate the score events. We can
>> > deal with that later.
>> >
>> > Also, I'd like to shift the discussion more on terms of code. You should
>> > be sending several patches each week, so we can do often commits, give
>> > feedback and improve.
>> >
>> > I hope to receive the code tarball soon...
>> >
>> > Best regards
>> > Pau
>> >
>> >
>> >
>> >
>> >
>> >> Best regards,
>> >> Han, Yushen
>> >>
>> >>
>> >> On Wed, Jul 16, 2008 at 9:55 AM, Pau Arumí <parumi at iua.upf.edu> wrote:
>> >> > Hi Yushen,
>> >> > I'd like to have often feedback to you.
>> >> > Remember to prepare a tarball with your code so I can commit it and give
>> >> > feedback.
>> >> > Also, have you NE compiled from source? We need to work on the real-time
>> >> > version of the synth ASAP...
>> >> >
>> >> > Regards
>> >> > Pau
>> >> >
>> >> >
>> >
>> >
>
>