## Optimal Alphabetical Order

### May 14, 2013

Today’s exercise comes from the book *Making the Alphabet Dance: Recreational Wordplay* by Ross Eckler, but I found it at http://www.math.cmu.edu/~bkell/alpha-order/:

Assume we have a list of all the words in the English language. Under the normal ordering of the alphabet in English (A, B, C, …, Z), some number of these words have all their letters in alphabetical order (for example, DEEP, FLUX, and CHIMPS). However, if we change the ordering of the alphabet, we will change the number of words having their letters in “alphabetical” order. What arrangement of the letters of the alphabet maximizes this number?

Your task is to write a program to find such an arrangement. When you are finished, you are welcome to read or run a suggested solution, or to post your own solution or discuss the exercise in the comments below.

This Python version gives a score of 4038 words in right order after about 3.5 minutes.

Sorry for not solving the actual quiz, but reading this problem I’ve become curious, what would be the result if we interpret the word list as an ordered character preference vote, and run a Concorcet (in this case Schulze) method on them. http://en.wikipedia.org/wiki/Schulze_method

If my Haskell program is correct (I haven’t tested it… :() the result is:

“EAIORSLNTUCDGPMHBYKFWVZJXQ”

(in this particular case I’ve simply removed duplicate characters from all word, maybe other ways of handling duplicates is possible to)

Cleared up the code a bit (using pairs instead of lists, correcting typos, etc…) here: http://hpaste.org/87967

Still offtopic :$ but intresetnig this Schulze voting was much ado about nothing, the result:

EAIORSLNTUCDGPMHBYKFWVZJXQ

is nearly the identical to the one we get from character frequencies

ESAIORLNTUDCGPMHBYFKWVZJXQ

If we don’t follow the requirement about handling the missing characters (they get quite much down-vote when they are not there) the result is:

JQVFWZBPHUMKCOALTIRNESGDYX

This should represent the usual character order in words better – I guess.

The process can be speeded up enormously using a Trie to store all the words. Then counting the words in the right order is about a factor 10 faster. I will post code later. I get now 4046 words in correct order with key=CWBFJHLOAQUMPVXINTKGZERDYS after 134 seconds. I do not get this answer with every run, as it it a random optimization.

Interesting. I actually worked out this problem about four months ago for one of the Hard challenges on the /r/DailyProgrammer subreddit: [01/25/13] Challenge #118 [Hard] Alphabetizing cipher (albeit with a smaller dictionary).

Given the appropriate helper functions (see blog post below), here’s my basic Racket solution (it uses hill climbing and randomization on finding a local maximum):

Blog post (with the above solution and a few more zany matrix based things that never ended up going anywhere): An optimal alphabetizing cipher

Full source (on GitHub): alphabetizing-cipher.rkt on GitHub

(While writing this post, I’ve found a few solutions just under 4000. I’ll let it run overnight and see what happens.)

By the way I think that this problem is equivalent with the NP-hard maximum clique problem: if we make the graph by connecting two words if there exist an alphabet such that both words have all their letters in alphabetical order, then from the cliques (where each word is connected with each other) would be easy to generate an appropriate alphabet

my pythons: http://pastebin.com/XgxQmF7W

best = “JQBPCOUFVWMHAXZTRLIKNGEDSY” # 3038 words

@szemet: Yes, a maximal clique would give an optimal answer. But the runtime is so ridiculously bad… If you check my link(s) above, I implemented the Bron-Kerbosch algorithm to try and solve it that way, but after about two months the largest clique it had found was only 46 words (on the smaller dictionary). This is definitely one of the classes of problems where a statistical approach will be better. It may be possible to speed up the problem a bit more with some domain specific knowledge, but I’m not sure how.

Here is the Python code using a trie to represent te words. The fastest solution sofar was 26 seconds with a score of 4046 and key BPCHFLJOWAQUXMVINKGTZERDYS.