Merge Git Repo to New Git Rep.

Here's the problem I needed to solve: a group needed their own Git area to work in. We wanted to be able to work in our own Git trees without breaking each other's tree.

We are running gitolite on our corporate Git servers. We can't add repos without going through an administrator.

We have an automated build server that fetches the entire Git repo. The more branches in the repo, the longer it takes to fetch and the longer each automated build takes.

The solution was to create a new set of repos, parallel to the original "main" repos. (Our gitolite admin created those repos for us.) Now, how to get the current code from the main repo to the second group's new repos?

https://stackoverflow.com/questions/17371150/moving-git-repository-content-to-another-repository-preserving-history

# get the new repo (currently empty)
git clone git@gitolite:new-group-dev/reponame reponame

# should now have an empty tree called "reponame"
# cd reponame

# add the secondary repo
git remote add r1remote git@gitolite:old-group-dev/other-repo-name

# fetch the secondary repo code
git fetch r1remote

# and at this point the command diverge from the Stack Overflow. I'm pulling in the branch I created
# that the other group is going to start from.
git merge r1remote/b/dpoole/davesbranch

# all my code should now be in the current directory

# do NOT 'git remote rm' the r1remote; we want to be able to merge changes from it

# push code to new-group-dev/reponame on the gitolite server
git push origin master

# we should have two remotes.
% git remote -v
origin    git@gitolite:new-group-dev/reponame (fetch)
origin    git@gitolite:new-group-dev/reponame (push)
r1remote    git@gitolite:old-group-dev/other-repo-name (fetch)
r1remote    git@gitolite:old-group-dev/other-repo-name (push)

# TODO update this post when we start cross merging. That should be fun!

Adding CUPS to Raspberry Pi

In order to use localhost:631 to manage CUPS, the pi user needs to be in the lp and lpadmin groups.

% sudo usernmod -a -G lp,lpadmin pi

Probably need to logout/login again.

The CUPS web login will prompt user username/password. Use the pi user and your pi password.

Adding New Scanner to Raspberry Pi

More notes on getting scan working on Raspberry Pi running NOOBs (Debian).

A"permission denied" problem is usually the first stumbling block using libusb to directly talk to scanner.  SANE will automatically configure the scanner to be read/writeable by the user.

SANE (http://www.sane-project.org/) is awesome sauce. But it doesn't support the weird scanners I get to handle.

To add a new scanner, add the vid/pid to /lib/udev/rules.d/60-libsane.rules after LABEL="libsane_usb_rules_begin" (towards start of file) and before LABEL="libsane_rules_end".

SANE's udev rules finish by calling /bin/setfacl to set permissions on the USB devnode. Just adding the scanner's vid/pid to the sane udev rules used to be enough (Ubuntu 12.04). Not so anymore.

The Raspberry Pi Debian has a "scanner" group. The scanner dev node is created rw on that group. So I needed to add the user 'pi' to the scanner group with usermod.

I don't know if the scanner group is a new feature of SANE or something in the newer Debians. (I'm running an older Ubuntu on my laptop). But I can now talk to my scanner without being root.

Changing Console Keyboard on Raspberry Pi

I'm ramping up on using Raspberry Pi as a scanner development machine. As such, I'm installing all the usual suspects: sane, libusb, PIL, matplotlib, and numpy.

The default console keyboard isn't "us". As a US keyboard user, I found things like | (pipe) weren't working for me.

The console keyboard is configured in /etc/default/keyboard.  See also keyboard(5).  I changed KBDLAYOUT="gb" to KBDLAYOUT="us" and rebooted. Voila! My keyboard now works.

Scanning from an eSCL Device Using Command Line

eSCL is HP's and Apple's scan protocol. (IETF standards track even?) Uses XML.

xmllint is from the libxml2-utils package.

Get Scanner Status

%  curl -s http://localhost:8080/eSCL/ScannerStatus | xmllint -format -

Get Scanner Capabilities

% curl -s http://localhost:8080/eSCL/ScannerCapabilities | xmllint -format -

Start a scan job

%  curl -v -X POST -d @scansettings.xml  http://localhost:8080/eSCL/ScanJobs

Device should respond with a 201 + Location of the new job. The Location will have a jobid (integer) (or a UUID).

Retrieve the scan job (in this example, 208 is the jobid from the 201 response to the POST)

% curl -s http://localhost:8080/eSCL/ScanJobs/208/NextDocument > out.dat

The 'out.dat' file should be the scanned image. Should be a jpeg or pdf or some other image. Jpeg is most likely.

% file out.dat

out.dat: JPEG image data, JFIF standard 1.01

Simple(ish) scansettings.xml

<?xml version="1.0" encoding="UTF-8"?>
<scan:ScanSettings xmlns:pwg="http://www.pwg.org/schemas/2010/12/sm" xmlns:scan="http://schemas.hp.com/imaging/escl/2011/05/03">
  <pwg:Version>2.0</pwg:Version>
  <pwg:ScanRegions>
    <pwg:ScanRegion>
      <pwg:Height>3300</pwg:Height>
      <pwg:ContentRegionUnits>escl:ThreeHundredthsOfInches</pwg:ContentRegionUnits>
      <pwg:Width>2550</pwg:Width>
      <pwg:XOffset>0</pwg:XOffset>
      <pwg:YOffset>0</pwg:YOffset>
    </pwg:ScanRegion>
  </pwg:ScanRegions>
  <pwg:InputSource>Platen</pwg:InputSource>
  <scan:ColorMode>Grayscale8</scan:ColorMode>
</scan:ScanSettings>

Scanning from Apple AirPrint + AirScan

TL;DR. AirPrint/AirScan with Image Capture + AirScanScanner requires pdl=application/octet-string in mDNS TXT record. Otherwise, AirScanScanner will not work.

AirPrint Requires AirScan.

The Apple AirPrint specification 1.4 requires MFP (Multi-Function Printers; printers with an attached scanner) to also allow scanning over the AirPrint connection. The required protocol is eSCL, an XML over HTTP originally created by HP.

AirScan/eSCL devices advertise themselves over mDNS as _uscan._utcp.  One of the fields in the TXT record is "pdl".

The documentation shows an example: "pdl=application/pdf,image/jpeg"

The AirPrint documentation says PDF and JPEG scanning are required. That's all that's mentioned.  (The pdl field becomes critically important later.)

OSX Image Capture finds network scanners through the mDNS. The actual scanning is done through an executable called AirScanScanner.
/System/Library/Image Capture/Devices/AirScanScanner.app

Only Mavericks can successfully scan from an AirScan/eSCL device. MtnLion connects but fails.

Adding AirScan to Existing AirPrint Device.

I was tasked with adding AirScan support for our scanners.

We have an HP X576 that supports eSCL. Image Capture successfully scans gray/rgb JPEG from HP through eSCL. However, when scanning from my code, Image Capture would only scan rgb. Grayscale would silently fail. No output image.

Only clue was a log message:

Mar 14 16:22:24 latches.local Image Capture[5359]: ImageIO: CGImageSourceCreateWithURL url parameter is nil

The Plot Thickens.

After mimicing as much of the HTTP+XML eSCL as possible, I attacked AirScanScanner with the debugger and dtruss. AirScanScanner writes its temporary images to /var/folders/hw/<longname>/T. The incoming image is written to a temporary file then moved to the user's Pictures folder.

A color scan (for both HP and me) :
rename("/var/folders/hw/vb001gqj3zv8vrlbr4tn8cqw0000gp/T/temp.awDE7YxI\0", "/Users/davep/Pictures/Scan 93.jpeg\0")         = 0 0

In the case of a grayscale scan, AirScanScanner's behavior deviates. From the HP, AirScanScanner writes a .ica file (??? Image Capture Application intermediate format perhaps?) 

rename("/var/folders/hw/vb001gqj3zv8vrlbr4tn8cqw0000gp/T/temp.K7RsxSBk\0", "/var/folders/hw/vb001gqj3zv8vrlbr4tn8cqw0000gp/T/Image Capture_TempScan.LLCnftRz/Scan 91.ica\0")         = 0 0

However, The dtruss traces showed AirScanScanner writing my incoming JPEG image as a TIFF.

rename("/var/folders/hw/vb001gqj3zv8vrlbr4tn8cqw0000gp/T/temp.jlIExPds\0", "/var/folders/hw/vb001gqj3zv8vrlbr4tn8cqw0000gp/T/Image Capture_TempScan.N8yDdG7e/Scan 96.tiff\0")         = 0 0

The .tiff is never moved to Pictures. The scan's entire temp directory is never deleted (leaks).

The PDL.

Eventually I found I was advertising pdl slightly differently than the HP. The HP response is shown below:

% dns-sd -L "HP Officejet Pro X576dw MFP [F7816C]" _uscan._tcp
Lookup HP Officejet Pro X576dw MFP [F7816C]._uscan._tcp.local
DATE: ---Fri 21 Mar 2014---
 9:41:34.872  ...STARTING...
 9:41:35.026  HP\032Officejet\032Pro\032X576dw\032MFP\032[F7816C]._uscan._tcp.local. can be reached at HP843497F7816C.local.:8080 (interface 4)
 txtvers=1 vers=2.0 pdl=application/octet-stream,application/pdf,image/jpeg ty=HP\ Officejet\ Pro\ X576dw\ MFP adminurl=http://HP843497F7816C.local. note= UUID=1c852a4d-b800-1f08-abcd-843497f7816c representation= rs=/eSCL cs=binary,color,grayscale is=platen,adf duplex=T

The "pdl=application/octet-stream,application/pdf,image/jpeg" turns out is the key.

When I changed my advertisement to match the HP's, monochrome scanning suddenly began working.

If I changed my advertisement to only "pdl=application/octet-string", scanning still worked.

For reasons unknown, "application/octent-string" is requred in the pdl field of the mDNS advertisement.

EnableLogging and SaveIntermediateFiles.

The AirScanScanner executable contains two very interesting strings: EnableLogging and SaveIntermediate Files.  I would love to be able to enable those debug features. I tried a few tricks with the AirScanScanner.plist but nothing happened. The Image Capture utility launches AirScanScanner so I wasn't able to successfully inject environment variables into it.

In Conclusion.

I'm mostly writing this to help the next firmware engineer tasked with adding Scan support to an AirPrint device. The eSCL is a beautiful simple protocol that simplifies scanning.

Restoring Protected Files on OSX ("chmod operation not permitted")

Work bought me a new MacBookPro last month. The laptop had issues with crashing from the beginning. Finally took it to the Apple Store. They reported they could find no issues and re-imaged the disk.

I've been restoring files from all my backups (DropBox, Super Duper images, and Time Machine image). I've been trying to cherry-pick the restored files using Finder and command line rather than the Time Machine utility. Has been a problem.

When the Apple Store re-imaged the machine, they created a user 'test' to run their diagnostic tools. Once I got the machine back, I added myself 'davep'. The 'test' account was uid 501. The 'davep' account was uid 502. As davep was the first user added before, the Time Machine backups were all owned by 501.

I was unable to chdir into several of the Time Machine disk directories. I couldn't even sudo chmod the trees.

http://superuser.com/questions/326645/sudo-chown-fails-with-operation-not-permitted
http://superuser.com/questions/279235/mac-os-x-why-does-chown-report-operation-not-permitted

The chmod was explicitly forbidden by ACLs on the dirs. Makes sense. OSX wants to protect the backup files as much as possible.

An "ls -led Documents" for example shows:

drwx------+ 36 501  staff  11220 Dec 20 09:48 Documents//
 0: group:everyone deny add_file,delete,add_subdirectory,delete_child,writeattr,writeextattr,chown
 1: group:everyone deny delete

I highlighted the problem id and attribute. The owner is no longer me (I'm 502, the test account was created at 501). The ACL has forbidden anyone from changing the ownership.

After digging through Stackoverflow and Google, I've been using the following commands to clear up the problem:
  
sudo chmod -R -a "everyone deny chown" dirname
sudo chown -R davep dirname

The first chmod will remove the "deny chown" on the tree dirname. The second chown will change the rightful owner ship to me.

Restoring usblp and /dev/usb/lp0

In our firmware development, we use /dev/usb/lp? to talk with our printers. For example:

% python pjltest.py /dev/usb/lp0  # run some PJL tests

Whenever a USB printer is attached, a new char dev node is populated in /dev/usb. The dev node can be read/written just like a file so it's quite easy to communicate with the printer's USB without having to write C code on libusb.

Ubuntu 11.10 did away with the usblp driver. From what I understand, CUPS has stopped using /dev/usb/lp? files to talk with the USB print interface. However, the usblp driver will grab the print interface, effectively blocking CUPS.

The solution in Ubuntu 11.10 (11.04 still had /dev/usb/lp?) was to add usblp to a modeprob blacklist. To restore the /dev/usb/lp?, edit /etc/modprobe.d/blacklist-cups-usblp.conf and comment out the line:

blacklist usblp

For example
#blacklist usblp

On the next USB plug, modprobe is once again free to load usblp.

Note: this will probably break CUPS which is why usblp was added to the blacklist in the first place.

Vess Cola, Part 5.

I had to tinker a while but I think I found a nice way to draw the pieces with ImageMagick's '-draw' operators.

Using the 'floodfill' didn't turn out too well. The line's anti-aliasing caused weird artifacts. (See previous post.) Instead of a square plus two lines, I switched to four triangles.Zooming in looks nicer.

Ha!  Fun.

#!/bin/sh

# draw Vess Cola puzzle bottles
# davep 10-Jul-2011

SIZE=300
HALF=$(( ${SIZE}/2 ))
FOURTH=$(( ${SIZE}/4 ))
EIGHTH=$(( ${SIZE}/8 ))
THREEFOURTH=$(( ${FOURTH}/3 ))

function north {
    if [ $1 == "bottom" ] ; then
        echo "rectangle $(($HALF-$EIGHTH)),0 $(($HALF+$EIGHTH)),$FOURTH"
    else
        echo "polygon $(($HALF-$EIGHTH)),0 $(($HALF+$EIGHTH)),0 $HALF,$FOURTH"
    fi
}

function east {
    if [ $1 == "bottom" ] ; then
        echo "rectangle $(($SIZE-$FOURTH)),$(($HALF-$EIGHTH))  $SIZE,$(($HALF+$EIGHTH))"
    else
        echo "polygon $SIZE,$(($HALF-$EIGHTH)) $SIZE,$(($HALF+$EIGHTH)) $(($SIZE-$FOURTH)),$HALF"
    fi
}

function south {
    if [ $1 == "bottom" ] ; then
        echo "rectangle $(($HALF-$EIGHTH)),$(($SIZE-$FOURTH)) $(($HALF+$EIGHTH)),$SIZE"
    else
        echo "polygon $HALF,$(($SIZE-$FOURTH)) $(($HALF+$EIGHTH)),$SIZE $(($HALF-$EIGHTH)),$SIZE"
    fi
}

function west {
    if [ $1 == "bottom" ] ; then
        echo "rectangle 0,$(($HALF-$EIGHTH)) $FOURTH,$(($HALF+$EIGHTH))"
    else
        echo "polygon 0,$(($HALF-$EIGHTH)) $FOURTH,$HALF 0,$(($HALF+$EIGHTH))"
    fi
}

function piece {
    num=$1
    shift
    convert -size $((${SIZE}+1))x$((${SIZE}+1)) xc:gray -fill white -stroke black \
        -draw "rectangle 0,0 $SIZE,$SIZE" \
        -fill $1 -draw "polygon 0,0 $SIZE,$0 $HALF,$HALF"\
        -fill $3 -draw "polygon $SIZE,0 $SIZE,$SIZE $HALF,$HALF" \
        -fill $5 -draw "polygon 0,$SIZE $SIZE,$SIZE $HALF,$HALF"\
        -fill $7 -draw "polygon 0,0 0,$SIZE $HALF,$HALF"\
        -fill black -draw "$(north $2)"\
        -fill black -draw "$(east $4)"\
        -fill black -draw "$(south $6)"\
        -fill black -draw "$(west $8)"\
        puzzlepiece${num}.png
}

piece 0 red bottom green top blue top yellow bottom
piece 1 red bottom blue bottom green top yellow top
piece 2 red top yellow top blue bottom green bottom
piece 3 green bottom blue bottom green top yellow top
piece 4 yellow bottom red top blue top red bottom
piece 5 green top yellow top red bottom blue bottom
piece 6 red top yellow top green bottom blue bottom
piece 7 blue top green bottom yellow bottom red top
piece 8 red top blue bottom yellow bottom green top

Vess Cola, Part 4.

I started writing a post on how I was arrive at the 4**9 calculation. I wanted a way to illustrate the 4x pattern of the puzzle pieces.

My original drawings were done with OmniGraffle. OmniGraffle is an amazing piece of software. Its ease-of-use gives me something to strive for in my own software.

However, OmniGraffle is quite good at diagramming, drawing squares, shapes, connecting them, etc, but I wanted a nicely colored red/green/yellow/blue square. Flood fill isn't something in OmniGraffle's design.

So how to draw a puzzle piece? I don't want to use my Tkinter GUI app yet (saving that for a future post) but I needed a way to get a simple puzzle piece into a post. And taking a screen shot felt like cheating. Too easy. Yes, I like to find the hard way to do things. I learn a lot that way.

Several years ago, I wrote a little Python+ImageMagick script to do the familiar "Forbidden!" red crossed-out circle. ImageMagick has some very nice drawing primitives. (I'll post the cross.py at some point. I'd like to stay remotely close to the Vess Cola topic for a while.)

First shot was something like:

% convert -size 100x100 xc:gray -fill white -stroke black -draw "rectangle 10,10 90,90" -draw "line 10,10 90,90" -draw "line 90,10 10,90" puzzlepiece.png

Well. It's square with a cross through it. Missing color. Needs some fill. Needs some flood fill. Easy with a paint program. But what's the fun in that? ImageMagick can do flood fill.


% convert -size 100x100 xc:gray -fill white -stroke black \
    -draw "rectangle 10,10 90,90" \
    -draw "line 10,10 90,90" \
    -draw "line 90,10 10,90" \
    -fill blue -draw 'color 20,50 floodfill' \
    -fill red -draw 'color 50,20 floodfill' \
    -fill green -draw 'color 75,50 floodfill' \
    -fill yellow -draw 'color 50,75 floodfill' puzzlepiece2.png

Flood fill is a bit tricky. There can be strange effects around the edges. I'm getting weird white borders. The diagonals are because of anti-aliasing (ImageMagick's web page warns about this). Not sure what's causing the white along the red, green, and blue triangles. Not the yellow, though.

Zooming in on the image via an ImageMagick scaling operation:

% convert -resize 500% -filter Point puzzlepiece2.png p3.png

 

I'm using the 'point' filter to avoid interpolating the result. I didn't want a soft scaled up image. I wanted to see pixel-by-pixel.

(I could have scaled up with OSX's Preview which does a good job of not interpolating pixels. I would have had to take a dreaded screen shot. Nothing against screen shots but I'm having fun looking for a good way to create these images programatically.)

My Tkinter app I used filled triangles. I wonder how that would look in ImageMagick? Could I do it even as an SVG to avoid the jaggies?

Vess Cola, Part 3.

Graphs can be made with pointers or an array configuration.

Speed is of the essence in this application. Why? The sheer number of permutations.

Flexibility isn't important. I don't need the full range of capabilities of a directed graph. Each vertex needs a right and a down edge.

But how many permutations are there? Curious.

There are nine items that can be arranged.

Permutations http://en.wikipedia.org/wiki/Permutation. 9! == 362880

But each card has four positions. Crud. (How did I calculate this again?)  4**9 comes to mind.

Vess Cola, Part 2.

The problem is how to represent the 3x3 grid of Vess Cola cards in such a way that I can compare the color and bottle top/bottom. Red Top must match Red Bottom, and so forth.

At first glance, a good data structure to represent the cards' grid would be a directed graph.

A directed graph would be simpler than an undirected graph. The compare only needs to work one way.

For example, Card 0 only has to compare itself to Cards 1 and 3. The relationship is symmetrical. If Card 0 matches Card 1 then certainly Card 1 matches Card 0. With a directed graph, I can connect the cards together to quickly traverse the pattern.

So how to represent in Python?

There are numerous graph libraries. There are numerous graph libraries in Python.

Vess Cola, Part 1.

When I was growing up, I had a wonderful window+grandmotherly next door neighbor--Gerri (Jeri? Geraldine?) LeVeaux. She was the kind of next door neighbor who always had cookies for us kids and paid us $1 to mow her tiny lawn even though she was perfectly able to mow it herself.

Some time, when I was a kid, she gave me a puzzle she said belonged to her son. I became the proud caretaker of the Vess Puzzle.

Hmmm... I need to scan the back of the pieces, too.

I've long since lost the envelope the puzzle came in. But the puzzle is simple enough. Match up the same color top and bottom of the Vess Cola bottles.

During college, I'd get the puzzle out of my desk, solve it to avoid doing homework.

A little while ago, I started talking with my father-in-law about puzzles. I hadn't played with Vess for quite a while so I brought out the puzzle. And couldn't solve it.

Being a programmer nerd, I thought Vess would be a fun programming problem. So I set about solving the puzzle with a Python script.

Installing 32-bit Support to 64-bit Ubuntu

If you are running Ubuntu 64-bit, install the ia32-libs

# sudo apt-get install ia32-libs

(Or use Synaptic.)

References:
http://ubuntuforums.org/showthread.php?t=720732
http://www.debian-administration.org/articles/534

Linux USB Printer/Scanner Hacks.

I develop scanner/printer firmware. I develop under Linux.

Here are a few handy things I use to make my life easier.

Disable the Ubuntu “New Printer” dialog.
From Jeremy Ward, a coworker.

“One issue I was having was the printer driver install prompts displayed when plugging the dev board in/loading code on the dev board. I remember you mentioning that you had the same problem on Ubuntu. I was able to fix this issue by modifying the rules in /lib/udev/rules.d/70-printers.rules. I commented out all of the rules since that is what tells the system to launch the prompts. I don’t think there are any negative consequences from doing so.”

Allow user read/write to /dev/usb/lp*.
Add yourself to the ‘lp’ group with usermod. See usermod(8) man page.

Fedora : run system-config-users script (tested on FC14)
Ubuntu 11.04 : (from Desktop) System -> Administration -> Users and Groups

Allow user to scan without using root.
Either of the following methods should work. The following examples assume a USB VID of 0×8086 and USB PID of 0×1234.

From Eric Huang, a coworker.

- Add a new file called /etc/udev/rules.d/mrvl-printer.rules
- Add a line to the new file:
ATTRS{idVendor}==”8086” MODE:=”0666”

From David Poole.

- Edit /etc/udev/rules.d/40-libsane.d
- Add the following before the first ATTRS{idVendor}:
ATTRS{idVendor}==”8086″, ATTRS{idProduct}==”1234″, ENV{libsane_matched}=”yes”

I've also found being logged in to the console (the PC itself) allows me to scan on Fedora. There is some magic going on in udev where, if the user is on the console, access to the USB devices is allowed.

Installing 32-bit Support to 64-bit Fedora 14

I'm moving to a new Linux build box. My original Linux box is a 32-bit Fedora FC14. My new Linux box is a 64-bit Fedora FC14.

Our compiler toolchain is a 32-bit app. The compiler didn't run on the new box, giving a cryptic error about /ld/ld-linux.so.2

Dug around Google. There are two postfixes (?) on FC14 packages: .i686 and .x86_64 (FC13 used .i586).

Short story: as root,
rpm -i bzip2-libs.i686

The dependency pulled in the 32-bit glibc. Probably could have installed just glibc.i686

The two glibc versions are installable side-by-side.

Products I've Worked on.

HP LaserJet m1005. First scanner/copier.
HP LaserJet m1120. First network scan.
HP LaserJet m1319. First ADF, stepper motor.
HP LaserJet m1130. Low memory. Digital sensor.
HP LaserJet m1212nf. First combo ADF,flatbed. Digital sensor.

Document Recognition and Retrieval. Electronic Imaging Conference. 27-Jan-2011

"Example centric document design and development"

Wilcoxon-Mann-Whitney statistic
http://en.wikipedia.org/wiki/Mann%E2%80%93Whitney_U

Spearman Correlation Coefficient
http://en.wikipedia.org/wiki/Spearman%27s_rank_correlation_coefficient

Blueprint - Adobe Labs

Perceptron
http://en.wikipedia.org/wiki/Perceptron

---

"Feature relevance analysis for writer identification"

Freeman chain codes
http://en.wikipedia.org/wiki/Chain_code

Grapheme
http://en.wikipedia.org/wiki/Grapheme

Chi-squared
http://en.wikipedia.org/wiki/Chi-square_test

---

"Using perturbed handwriting to support writer identification in the presence of data constraints"

MGH - Model Generated Handwriting

MPG - Model Perturbed Handwriting

"Text-Independent Writer Identification and Verification on Offline Arabic Handwriting"
Bulacu, Schomaker, Brink 2007

SVM (Support Vector Machine) with radius basis kernel

McNemar's Test
http://en.wikipedia.org/wiki/McNemar%27s_test

---

"Statistical characterization of handwriting characteristics using automated tools"

Probabalistic Graphical Model
http://en.wikipedia.org/wiki/Graphical_model

Document Recognition and Retrieval. Electronic Imaging Conference. 26-Jan-2011

Capture: Image to Archive (conference)

Scene Analysis "Functional Role Labeling"

Image Template -> template management

Trainable pattern classifiers. Features + Classifiers.
Features
--------
Haar
runlength
Fourier
word counts

Classifiers
-----------
Decision tree
nearest neighbor
SVM (Support Vector Machine)
generative probability
density

"Learning Image Anchor Templates for Document Classification and Data Extraction"
Sarkar. http://www.icpr2010.org/pdfs/icpr2010_ThAT7.5.pdf

Constellation Model
http://en.wikipedia.org/wiki/Constellation_model

Information extraction by finding repeated structure
Evgeniy Bart, Prateek Sarkar
http://dx.doi.org/10.1145/1815330.1815353

Best First Leaf Search (from aforementioned paper)

NIST tax form data sets
http://www.nist.gov/srd/nistsd2.cfm

---

"Introduction of Statistical Information in a Syntactic Analyzer for Document Image Recognition"

Sayre's Paradox
http://dx.doi.org/10.1016/0031-3203(73)90044-7

Hidden Markov Model
http://en.wikipedia.org/wiki/Hidden_Markov_model

---

"MRF Model w/ Parameter Optimization by CRF for online recognition of handwritten Japanese characters"

MRF - Markov Random Field
http://en.wikipedia.org/wiki/Markov_random_field

CRF - Conditional Random Field
http://en.wikipedia.org/wiki/Conditional_random_field

Extract feature points using Ramner method
U. Ramer “An Iterative Procedure for the Polygonal Approximation of Plan Closed Curves” Computer Graphics and Image Processing, vol.1, pp244-256, 1972.
http://dx.doi.org/10.1016/S0146-664X(72)80017-0

Stochastic Gradient Descent
http://en.wikipedia.org/wiki/Stochastic_gradient_descent

Viterbi algorithm
http://en.wikipedia.org/wiki/Viterbi_algorithm

Baum-Welch algorithm
http://en.wikipedia.org/wiki/Baum%E2%80%93Welch_algorithm

Elastic matching
http://en.wikipedia.org/wiki/Elastic_Matching

---

"Improving an HMM based offline handwriting recognition system using MME-PSO optimization"

MME -

PSO - Particle Swarm Optimization
http://en.wikipedia.org/wiki/Particle_swarm_optimization

MD-LSTM
http://en.wikipedia.org/wiki/Long_short_term_memory

HTK - toolkit for building HMMs (Cambridge)
http://htk.eng.cam.ac.uk/

---

"Segmenting text from outdoor images taken by mobile phones using color features"

Preprocessing:
RGB -> HSI
histogram equalization Intensity channel
HSI -> RGB

Image binarization

Noise removal

Image Segmentation.
http://people.cs.uchicago.edu/~pff/

"Font and Background Color Independent Text Binarization"
Kasar edge cue based algorithm
http://www.imlab.jp/cbdar2007/proceedings/papers/O1-1.pdf

Levenshtein Distance
http://en.wikipedia.org/wiki/Levenshtein_distance

Bag of Words
http://en.wikipedia.org/wiki/Bag_of_words_model_in_computer_vision

Local Adaptive Binarization

---

"Perceptive Method for Handwritten Text Segmentation"

Kalman Filtering
http://en.wikipedia.org/wiki/Kalman_filter

Delaunay graph for distance computation
http://en.wikipedia.org/wiki/Delaunay_triangulation

DMOSp

---

"A masked based enhancement method for historical documents"

Filtering (noise reduction)
- Wiener
http://en.wikipedia.org/wiki/Wiener_filter
- Median
http://en.wikipedia.org/wiki/Median_filter

Markov Random Fields
http://en.wikipedia.org/wiki/Markov_random_field

Local Binarization [Gatos 2006]
"Adaptive degraded document image binarization"
http://dx.doi.org/10.1016/j.patcog.2005.09.010

OCR - Tesseract
http://en.wikipedia.org/wiki/Tesseract_%28software%29

Electronic Imaging Conference. 26-Jan-2011

Keynote: "Problems in Biological Imaging: Opportunities for Signal Processing"

PSF - Point Spread Function
http://en.wikipedia.org/wiki/Point_spread_function

Restoration: Denoise, Deconvolution.

http://en.wikipedia.org/wiki/Deconvolution

Hysteresis
http://en.wikipedia.org/wiki/Hysteresis

Segmentation
- thresholding
http://en.wikipedia.org/wiki/Thresholding_%28image_processing%29

- watershed
http://en.wikipedia.org/wiki/Watershed_%28image_processing%29