Last modified: October 27, 2015
In module gamera.toolkits.musicstaves.musicstaves_rl_roach_tatem
This code is a clean-room reimplementation of the algorithm described in
Roach, J. W., and J. E. Tatem. 1988. Using domain knowledge in low-level visual processing to interpret handwritten music: an experiment. Pattern Recognition 21 (1): 33-44.
For a general overview of the algorithm, readers are referred to the paper. In addition to the algorithm described in this paper, we have added two improvements described below.
Implementation notes:
When computing the vector field, only the angle is stored for each pixel. The distance and thickness values are used by higher-level parts of their system and are not used for raw staff removal, so to save run- and programming-time, this was not implemented.
Roach and Tatem suggest a radius for the vector field window of "at least half the width of one staves in the image." I assume the authors meant "width in the vertical direction" as the horizontal direction would be so large as to take hours to compute a vector field.
The original algorithm given by Roach and Tatem for computing the vector field was in our experiments computationally too expensive for practical use. Hence we have used a brute force approach instead, which picks the longest chord for a predefined range of angles (see compute_longest_chord_vectors).
The rules described with bullet points in Sections 3.4 and 3.5 are quite clear, but exactly how those rules are applied is not entirely clear to me. For instance, are the rules applied in parallel or in sequence? Should the side effect of one rule be immediately applied? Adding to the confusion, it appears the first rule can not be applied in sequence before the second. Through experimentation, I decided to apply each rule to the entire image, one at a time. The side effects are applied all at once before applying the next rule.
The algorithm has a concept of "questionable pixels", which are pixels that likely belong to both a staffline and a musical figure. Nothing is made of this information at present.
Roach and Tatem claim it is a simple matter to find stafflines because they are long horizontal connected components. In our experiments, however, the stafflines were so broken up that we were unable to find a threshold to separate them from other noise using horizontal width alone.
As the application of Roach and Tatem was handwritten music, their method simply removes all perfectly straight horizontal shapes. As this yields very poor results in printed music (it also removes beams and parts of lyrics) we have added two improvements:
- Vertical black runlengths longer than 2*staffline_height are not removed. This should leave most beams intact.
- On the horizontal lines detected by Roach and Tatem's algorithm we apply the staff finding algorithm StaffFinder_dalitz and only remove staff segments that cross the detected staff lines.
| Author: | Michael Droettboom, Thomas Karsten and Christoph Dalitz, based on Roach, J. W. and J. E. Tatem |
|---|
Detects and removes staff lines from a music/tablature image.
Signature:
remove_staves(crossing_symbols='all', num_lines=0, resolution=3.0, debug=0)
with
- crossing_symbols:
- This algorithm only supports the value 'all'.
- num_lines:
- Number of lines within one staff. A value of zero for automatic detection is supported, but if you give the actual number, staff line detection will improve.
- resolution:
- the resolution parameter passed to compute_longest_chord_vectors
- debug:
- Set the debug level verbosity. For debug >= 1, tracing information is printed to stdout, for debug >= 2 images with prefix debug are written into the current working directory.
Note
The property image is the result of our "improved" algorithm. For comparison, the result of Roach and Tatem's original method is stored in the property image_original.
Returns the y-positions of all staff lines at a given x-position. Can only be called after remove_staves.
Signature:
get_staffpos(x=0)
with
- x:
- This parameter has no effect, because the staffline is assumed to be horizontal.
The return value is a list of StaffObj.
Note that a more accurate way to obtain the staff lines for this class is by accessing self.linelist, because this does not just yield the average y-position, but the full staff line skeletons. Example:
ms = MusicStaves_skeleton(image)
ms.remove_staves()
sf = StaffFinder(image, ms.staffline_height, ms.staffspace_height)
sf.linelist = ms.linelist
rgb = sf.show_result()
rgb.save_PNG("foundstaves.png")