[Clam-devel] Re: recall/precision for a simple song, and the road to getting more and better results...

[Roman Goj]

Roman Goj wrote:
(...)
> current svn ChordExtractor:
> RECALL:  0.804825351098
> PRECISION:  0.813022917425
> 
> my "improved" segmentation algorithm:
> RECALL:  0.741447605329
> PRECISION:  0.748999636231
> 
> The bigger the values the better of course... Heh... hooray for
> progress! :-/

Hullo!

Well, progress has finally come! :) Not "progress :-/" or " \"progress\"
" or "well, huh, at least I tried..." sort of progress, but progress:

For the same file as in the above examples:
RECALL:  0.824630896651
PRECISION:  0.833030192797

So 2% better then current svn and at 30% less segments :) Of course
there's still so much room for improvement, we could all have a very
nice party there and there'd still be room for a few grand pianos and a
pool (to be exact: 18%), but still... :-)

These are results achieved by a refactored version of the improved chord
similarity algorithm I had in my sandbox (but with a slight change in
one threshold value), which I am (finally!) committing to svn :)

Of course the previous segmentation method is still the default one, but
to test this one through ChordExtractor you can now issue the command
like this:
* some small refactorings in ChordSegmentator - cleaning the code
* added a second segmentation algorithm, based on chord similarity
** a new segment is not opened if the new chord is "too similar" to the
chord in the currently opened segment
** method choice via a constructor parameter
** the previous algorithm stays the default choice
ChordExtractor -m 1 Debaser-WoodenHouse.mp3

(configurable use through the TonalAnalysis processing coming ... with
realtime segmentation in NetEdit)

The results on the Cohen song of this svn committed version of the
algorithm can now be found here:

http://www.box.net/shared/ng9zp058dk

> http://www.box.net/shared/f79cpka57y
> Cohen.poolGROUNDTRUTH  - the ground truth (or rather: more or less the
> way I and cohenchords.com percieve it)
> 
> http://www.box.net/shared/zv1j28gkir
> Cohen.poolCOMPUTEDOLD - the current simple svn algorithm results
> 
> http://www.box.net/shared/ocovpkykve
> Cohen.poolCOMPUTEDNEW - the new "in my sandbox" "sort of improved"
> algorithm results

OK, now that the algorithm is in svn there's some simple improvements
I'm eager to test... :)

Official changelog:
* some small refactorings in ChordSegmentator - cleaning the code
* added a second segmentation algorithm, based on chord similarity
** a new segment is not opened if the new chord is "too similar" to the
chord in the currently opened segment
** method choice via a constructor parameter
** the previous algorithm stays the default choice

Ouch! forgot to add in the svn commit!
* uses the estimateChord method found in ChordExtractor

and also would've been good to add
* the chord index for the segment is now calculated as max of mean
correlation over all frames of the segment
...perhaps this is actually overkill, something simpler could be in
place... but I think it doesn't introduce much overhead, so maybe it can
stay there for now... also the estimateChord method can be shortened a
lot for this particular use...

cheers!
roman
-------------- next part --------------
A non-text attachment was scrubbed...
Name: ChordSimilarityBasedSegmentationAlgorithm.patch
Type: text/x-diff
Size: 5584 bytes
Desc: not available
URL: <http://lists.clam-project.org/pipermail/clam-devel-clam-project.org/attachments/20070817/5202a4db/attachment-0004.patch>