We've been putting together test cases and experimenting with techniques and actually developing code for actually scanning ballots - in other words, the 'simple' problem of taking a digital and figuring out if it is 'correctly' filled out by a voter, and who got the votes. Here is a progress report on this work so far. In that short document you will see, step by step, how one approach would work, the intermediate steps required, the supplementary information which would define the ballot style: how it would be detected and then applied. (The term ballot style, in this context means the information that specifies where to expect each vote oval on a particular ballot and what it means.)
We've made very good progress in this area. While we've worked so far only with one common look for ballots, so-called (by some) 'office style' ballots as generated by Premier. We believe that many if not all of our techniques would work equally well with other graphically designed ballots.
One interesting preliminary result is that it seems possible (and worthwhile) to get rid of the bulk of the computerish marks that decorate most ballots today and reduce that visual clutter to a set of 4 black squares in each corner of a ballot. That would be enough to allow the image analyzer to line itself up reliably. Read the progress report for the rest of the details.