[Clam-devel] Re: A YES/NO question about the capability of the synthesizer & validity of the score

[Pau Arumí]

On dj, 2008-07-10 at 17:35 -0400, Han, Yushen wrote:
> Pau and Greg,
> 
> I was talking to my professor who was an oboe player.
> He suggested that a new note should be identified explicitly in the score.
> This is because the length of the pipe in an oboe is changing as a new
> note is attacking.
> The musician made an explicit effort at the moment this transition.
> 
> I am thinking of having explicit information over frames in the score:
> MIDI frequency brightness
> 
> In case that the music is driven by an instrument that does not given
> pitch explicitly or the program is taking input from the
> pitch-tracking of an audio, the program should be able to make a
> decision on the fly.
> 
> Does this make sense to you?

Yes. All sorts of instrument modeling techniques like the articulations
you mention are worth being considered and maybe implemented. Specially
the techniques that can take advantage of the underlying SMS spectral
model -- and thus can be differentiated from conventional samplers.d

Referring to the gsoc project, I'd give higher priority to the real-time
version -- that is, working in NE responding to osc events.  This should
not be very hard, hopefully. 

P


> 
> Best regards,
> Han, Yushen
> 
> On Thu, Jul 3, 2008 at 1:48 PM, Han, Yushen <yushen.han at gmail.com> wrote:
> > Greg,
> >
> > I did not use a breath controller before. But it seems that the breath
> > controller will collaborate with a MIDI keyboard so I know the note on
> > / off time. The breath controller could continually either change the
> > velocity or do pitch bend.
> >
> > I was thinking about holding a few frames in a realtime simulated
> > situation of reading the score.
> > By simulation, I mean that we pretend the score is known to us only up
> > to the current frame.
> > Actually "a few frames" did not help me identify a new note much. So I
> > will have to give up this idea.
> >
> >
> > From our conversation, I think your assumption is that the score
> > information is all available to the synthesizer, so we can look ahead.
> > Is that correct?
> >
> > Best regards,
> > Han, Yushen
> >
> > On Thu, Jul 3, 2008 at 1:36 PM, Greg Kellum <greg.kellum at gmail.com> wrote:
> >> Hi Yushen,
> >>
> >> Oh...  At first I thought you were talking about when reading from a score
> >> file.  Um...  In what cases would holding a couple of frames help you decide
> >> whether it's a new note?  I've never used a breathe controller before, but I
> >> would have thought that when the performer presses a different button that
> >> causes a new MIDI note on message which should create a new note on the
> >> synthesizer.  Are you thinking about cases where he accidentally presses a
> >> wrong note?
> >>
> >> Best,
> >> Greg
> >>
> >> On Jul 3, 2008, at 7:28 PM, Han, Yushen wrote:
> >>
> >>> Hi, Greg
> >>>
> >>> In my mind it was "having the synthesizer controlled in real-time by a
> >>> performer".
> >>> The performer is the musician who will be using the MIDI controller.
> >>>
> >>> What's in your mind?
> >>>
> >>> Best regards,
> >>> Han, Yushen
> >>>
> >>>
> >>> On Thu, Jul 3, 2008 at 1:02 PM, Greg Kellum <greg.kellum at gmail.com> wrote:
> >>>>
> >>>> Hi Yushen,
> >>>>
> >>>>> I also thought about holding 2 (or more) frames in synthesis to make a
> >>>>> decision on whether it is a new note. A delay of 2 frame (0.011s)
> >>>>> would not be noticed by human but neither it is enough for us to be
> >>>>> sure about whether a new note is coming. You did not hold any frames
> >>>>> in synthesis, did you?
> >>>>
> >>>> Which input modality do you have in mind here?  Are you talking about
> >>>> reading from a score file or having the synthesizer controlled in
> >>>> real-time
> >>>> by a performer?  And if it's with a performer, you mean then someone
> >>>> playing
> >>>> a MIDI controller or someone working with audio driven synthesis?
> >>>>
> >>>> Best,
> >>>> Greg
> >>>>
> >>>>>
> >>>>> Best regards,
> >>>>> Han, Yushen
> >>>>>
> >>>>> On Thu, Jul 3, 2008 at 6:08 AM, Greg Kellum <greg.kellum at gmail.com>
> >>>>> wrote:
> >>>>>>
> >>>>>> Hi Yushen,
> >>>>>>
> >>>>>> It sounds like you're having problems with transients...  When I
> >>>>>> generated that score for the violin sonata, I used a pitch tracker I
> >>>>>> had been working on earlier to generate the initial score, but then, I
> >>>>>> went back over it and made corrections to the score by hand.
> >>>>>> Sometimes when there were transitions between notes, there simply
> >>>>>> isn't any unambiguous pitch value, and if you force a pitch tracker to
> >>>>>> give you a value for those frames, it will make a mistake, right?
> >>>>>>
> >>>>>> I later added transient detection to the pitch tracker so if there was
> >>>>>> a big change in the pitch value and the aperiodicity of the signal
> >>>>>> jumped as well, then the new pitch value would be marked as a
> >>>>>> transient and suppressed.  You could do something like this in the
> >>>>>> synthesizer instead.  But the score generator has access to the
> >>>>>> original signal and can make a much better guess as to whether a frame
> >>>>>> is a transient than the synthesizer can.
> >>>>>>
> >>>>>> Best,
> >>>>>> Greg
> >>>>>>
> >>>>>>
> >>>>>> On Thu, Jul 3, 2008 at 12:23 AM, Han, Yushen <yushen.han at gmail.com>
> >>>>>> wrote:
> >>>>>>>
> >>>>>>> Pau,
> >>>>>>>
> >>>>>>> So it is YES - my synthesizer should be able to play ANYTHING
> >>>>>>> indicated from the score.
> >>>>>>>
> >>>>>>> A difficulty is that I did not know how something that is arbitrarily
> >>>>>>> given in the score file should sound like.
> >>>>>>> The concept of a "note" which should last at least for 50ms, may not
> >>>>>>> apply anymore.
> >>>>>>>
> >>>>>>> But I can still make it work by SMS transformation. Let's hear how it
> >>>>>>> will sound...
> >>>>>>>
> >>>>>>> Han, Yushen
> >>>>>>>
> >>>>>>>
> >>>>>>> On Wed, Jul 2, 2008 at 6:17 PM, Pau Arumí Albó <parumi at iua.upf.edu>
> >>>>>>> wrote:
> >>>>>>>>
> >>>>>>>> El dc 02 de 07 de 2008 a les 18:12 -0400, en/na Han, Yushen wrote:
> >>>>>>>>
> >>>>>>>>> Here I have a fundamental question about the purpose of this
> >>>>>>>>> synthesizer:
> >>>>>>>>> Should the synthesizer be able to produce whatever the score
> >>>>>>>>> indicates,
> >>>>>>>>> even if it is something a real instrument / human musician would not
> >>>>>>>>> be able to do?
> >>>>>>>>
> >>>>>>>> sure
> >>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>
> >>>>
> >>>>
> >>
> >>
> >