[Clam-devel] Re: GSoC: Enhancing chord detection

[Roman Goj]

David García Garzón wrote:
> Xavier, i just asked Roman to do a quick spike using the implementation he has 
> (matlab) to see whether is worth or not to do a new implementation of the 
> chord extraction by using such algorithm. Matlab just for the spike.
>
> I fully agree with you that having MP in clam would be nice but that having 
> another algorithm not that fast, not realtime and with nearly the same 
> results is not worth a GSoC project. So our plan is to quickly explore the MP 
> algorithm and if the result are not that precise then go on with the original 
> plan.
> 
> The spike should have ended today but talking with Roman we have seen several 
> things that have not been taken into account for the spike.
> 
> In summary, we are on spike and we'll work on MP just if we really get better 
> results, at least significantly on accuracy.

Hi!

I agree, Xavier, that I should start coding in CLAM... frankly I'm not a
big fan of Matlab, I'd love to be doing all this research in a nice C++
working environment ;)

But for now - I made my implementation a little faster, so that I could
produce bigger results for more polyphonic signals (we were wondering
how the algorithm would perform with such a signal) and the results I
have are here: ro-baczek.4shared.com (files: poly300.jpg, etc. 100,300
meaning the number of iterations shown on the plot, a hundred iterations
took about  6 minutes).

I guess these results do not differ much from the ones in the current
implementation (actually even the false notes appear to be the same in
both the algorithms). (this for iterations below 1000, what happens
later is that the algorithm tries to decompose every last bit of the
signal and just keeps getting more and more less and less "energetic" atoms)

But this - getting the actual decomposition is just the beginning. The
best thing about mp is that the results are so granular - after
obtaining them I can select isolated atoms and simply throw them out -
on the basis that most of the notes take longer than just one atom. The
long bass notes in the signal are also very thought-provoking - every
bass note has at least 20 -100 atoms (depending on the number of
performed iterations), but all those iterations are probably useless -
if the algorithm finds one note it should be able to extend it to both
sides quite quickly (this should be faster then a single iteration, I
guess) thus eliminating (or at least reducing) the note in just one
iteration (or maybe 3,4).

Also - mp has the possibility of adjusting to one's needs I guess - like
taking a difficult part of the signal and using a bigger atom dictionary
to decompose that... and so on and so forth...

Also - I've been talking to people at my faculty today and perhaps I'll
be doing some more research soon in things that could be useful for
extracting a single note from the signal... (i.e. the
instrument-specific-atom... from a polyphonic signal - adjusting the
dictionary of atoms, while doing the decomposition)

So - the algorithm is great - ... but perhaps you're right - these are
still a little bit more research ideas than something that would allow
CLAM to compete with the software mentioned a few days ago - iChords...

So - perhaps I should start trying to improve what's in CLAM already
rather then implementing mp? But if you think the current algorithm has
reached it's limits, then I'd say mp should be the way to go, when
choosing another algorithm, simply because it's so easily extendible :)

Maybe I can just start now focusing on the inner workings of the
algorithm(s) in CLAM, try to improve the current one and, if time allows
(or simply after the "official summer" ;) ) try to implement MP, which
should be waaay easier then - after I know better the ways of doing
things in CLAM? :)

Also - can I have a question... one I probably should've asked a long,
long time ago? What is the chord detection used for right now? :)

I guess this affects the way I should work on my project - because I
guess mp could do lots of nice things, which might turn out to be not
that useful (for now, I guess :) ). i.e. does the current algorithm
recognise only 12 notes as opposed to i.e. for the piano 88? So is
getting all the actual notes that important or just the chords that matter?

Also - if it's just the chords that matter - one thing I haven't tried -
are chord specific atoms... this could maybe improve the speed really
greatly, perhaps this could be used in real-time (I'd say it could
be)...? Then basically finding only one atom for a given short time
window should get the best chord (or maybe doing three iterations - if
all three give the same chord, the chances that there is a different
chord in the sound would be slim)

OK, sorry for flooding you with ideas (I really think it should be
relatively easy to implement them during the summer - the bottleneck
would be getting mp to work, then testing different versions of it
should be easy and take little time (like right now with Matlab) - huh,
well I guess what I have in mind is more of a research platform than a
stable, quick and useful single algorithm :( though - a little research
should lead to a good algorithm with all the ideas above, I suppose ;)
). I should've probably discussed all this with you way before now... :(
I just didn't feel up to that before writing a semi-working
implementation and I started doing that not that long ago...

But - to summarize - I get knee deep in CLAM code now, try to see if I
get an idea what should be done to improve the chord detection algorithm
(suggestions welcome! ;) ), then try to do all I can to improve it...
and then try implementing mp (as a cherry on top of the summer ;) ) ...?
Does this sound like an OK work plan? :)

Sorry I got carried away in writing *again*... (goes away to at last set
up that blog and the talking head for those long, long "I have a
vision!" entries... ;) )

Roman