<?xml version="1.0" encoding="UTF-8"?><rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
	xmlns:georss="http://www.georss.org/georss" xmlns:geo="http://www.w3.org/2003/01/geo/wgs84_pos#" xmlns:media="http://search.yahoo.com/mrss/"
		>
<channel>
	<title>Comments on: The Daily Cryptogram</title>
	<atom:link href="http://programmingpraxis.com/2009/07/14/the-daily-cryptogram/feed/" rel="self" type="application/rss+xml" />
	<link>http://programmingpraxis.com/2009/07/14/the-daily-cryptogram/</link>
	<description>A collection of etudes, updated weekly, for the education and enjoyment of the savvy programmer</description>
	<lastBuildDate>Mon, 28 May 2012 03:30:45 +0000</lastBuildDate>
	<sy:updatePeriod>hourly</sy:updatePeriod>
	<sy:updateFrequency>1</sy:updateFrequency>
	<generator>http://wordpress.com/</generator>
	<item>
		<title>By: Pete Bevin</title>
		<link>http://programmingpraxis.com/2009/07/14/the-daily-cryptogram/#comment-340</link>
		<dc:creator><![CDATA[Pete Bevin]]></dc:creator>
		<pubDate>Tue, 14 Jul 2009 17:09:09 +0000</pubDate>
		<guid isPermaLink="false">http://programmingpraxis.com/?p=944#comment-340</guid>
		<description><![CDATA[FYI, words 4 and 5 (AEFGAD and FZGPEAG) aren&#039;t in /usr/share/dict/words on Mac.  Only word 4 is missing on Ubuntu.]]></description>
		<content:encoded><![CDATA[<p>FYI, words 4 and 5 (AEFGAD and FZGPEAG) aren&#8217;t in /usr/share/dict/words on Mac.  Only word 4 is missing on Ubuntu.</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: Mark VandeWettering</title>
		<link>http://programmingpraxis.com/2009/07/14/the-daily-cryptogram/#comment-338</link>
		<dc:creator><![CDATA[Mark VandeWettering]]></dc:creator>
		<pubDate>Tue, 14 Jul 2009 15:23:40 +0000</pubDate>
		<guid isPermaLink="false">http://programmingpraxis.com/?p=944#comment-338</guid>
		<description><![CDATA[Several years ago, I got involved in Simon Singh&#039;s Code Challenge, and wrote a number of programs to try to crack classic ciphers.  As part of this, I wrote a program which used a genetic algorithm to crack cryptograms.  In many ways, it&#039;s kind of crude: it doesn&#039;t use a dictionary of english words, and ignored punctuation and word spacing.  What it does use is a table containing the logarithm of the probability of each trigram for a corpus of text I analyzed (don&#039;t remember what I used, maybe Sherlock Holmes or the Bible or something from Project Gutenberg).   For longer cryptograms, it almost always works perfectly.  It mucked up a couple of letters on this one, which is short and includes a fairly diverse set of trigrams, but the message was very easy to read even with these few mistakes in assigning low probability letters.

Here&#039;s the Python source.  For some reason, I can&#039;t find the code which generated the pickled &quot;corpus&quot;, but it shouldn&#039;t be too hard to figure out.  Perhaps I&#039;ll rewrite one during lunch.   The code ia also strictly speaking not an &quot;algorithm&quot;, since doesn&#039;t actually stop: it just keeps running, trying to maximize the score, and does nothing to detect stasis.   But you might find it amusing.

[ EDIT: I put the word &#039;python&#039; in single quotes in the sourcecode tag; without the single quotes, the tag is not recognized and the formatting is incorrect. I also fixed some ampersand-coded html characters.  I hope this is correct now. ProgPrax ]

[sourcecode lang=&#039;python&#039;]
#!/u0/markv/my-python/bin/python
#                   _                            
#  __ _ _ _  _ _ __&#124; &#124;_ ___  __ _ _ _ __ _ _ __  
# / _&#124; &#039;_&#124; &#124;&#124; &#124; &#039;_ \  _/ _ \/ _` &#124; &#039;_/ _` &#124; &#039;  \ 
# \__&#124;_&#124;  \_, &#124; .__/\__\___/\__, &#124;_&#124; \__,_&#124;_&#124;_&#124;_&#124;
#         &#124;__/&#124;_&#124;           &#124;___/                
# 
# A simple program for automatically solving simple substitution ciphers
# 
# Written by Mark VandeWettering


import pickle
import random
import string
import re
import sys

ldict = pickle.load(open(&#039;newcorpus.p&#039;))

letters = &quot;ABCDEFGHIJKLMNOPQRSTUVWXYZ&quot;

sorig = open(sys.argv[1]).read().upper()

swork = re.sub(r&quot;[^A-Z\!?.,&#039;]+&quot;, &quot; &quot;, sorig) 
swork = filter(lambda x : x in r&quot;ABCDEFGHIJKLMNOPQRSTUVWXYZ!?.,&#039; &quot;, swork)

print &quot;SWORK: &quot;, swork

POOLSIZE = 1000

pool = []

for p in range(POOLSIZE):
    a = list(letters)
    random.shuffle(a)
    pool.append(&#039;&#039;.join(a))

def calculatescore(x):
    s = 0 
    for i in range(len(x)-3):
	s = s + ldict.get(x[i:i+3], 0)
    return s 

class BogusError(Exception):
	pass

def mutate(str):
    if random.uniform(0, 1) &lt; 0.01:
	a = list(letters)
	random.shuffle(a)
	return &#039;&#039;.join(a)
    s = list(str)
    c0 = random.choice(range(26))
    c1 = random.choice(range(26))
    s[c0], s[c1] = s[c1], s[c0] 
    return &#039;&#039;.join(s)

def crossover(pop, mom):
    pop = list(pop)
    mom = list(mom)
    c0 = random.randint(0, 13)
    c1 = random.randint(1, 13)
    c1 = c0 + c1 
    child = pop[:]
    for idx in range(c0, c1):
	idx2 = pop.index(mom[idx])
	child[idx], child[idx2] = child[idx2], child[idx]
    child = &#039;&#039;.join(child)

    for c in string.uppercase:
	if c not in child:
	    raise BogusError

    return child


def sumpool(pool):
    total = 0 
    sums = []

    for score, key in pool:
	total = total + score
	sums.append(total)

    return total, sums

import bisect

def pick(total, l):
    idx = bisect.bisect_right(l, random.randint(0, total))
    return idx 

while True:
    newpool = []
    for p in pool:
	twork = string.translate(swork, string.maketrans(letters, p))
	score = calculatescore(twork)
	newpool.append((score, p))
    newpool.sort(lambda x, y: cmp(y,x))

    newpool = newpool[:POOLSIZE/2]

    # now, generate the distribution 

    total, sums = sumpool(newpool)
    newpool = map(lambda x: x[1], newpool)

    torig = string.translate(sorig, string.maketrans(letters, newpool[0]))
    
    print &quot;Best Decode score = %d&quot; % sums[0]
    print torig
    
    for x in range(POOLSIZE/2):

	idx = pick(total, sums)
	if (random.uniform(0, 1) &lt; 0.15):
	    newpool.append(mutate(newpool[idx]))
	else:
	    idx2 = pick(total, sums)
	    newpool.append(crossover(newpool[idx], newpool[idx2]))

    pool = newpool 
[/sourcecode]]]></description>
		<content:encoded><![CDATA[<p>Several years ago, I got involved in Simon Singh&#8217;s Code Challenge, and wrote a number of programs to try to crack classic ciphers.  As part of this, I wrote a program which used a genetic algorithm to crack cryptograms.  In many ways, it&#8217;s kind of crude: it doesn&#8217;t use a dictionary of english words, and ignored punctuation and word spacing.  What it does use is a table containing the logarithm of the probability of each trigram for a corpus of text I analyzed (don&#8217;t remember what I used, maybe Sherlock Holmes or the Bible or something from Project Gutenberg).   For longer cryptograms, it almost always works perfectly.  It mucked up a couple of letters on this one, which is short and includes a fairly diverse set of trigrams, but the message was very easy to read even with these few mistakes in assigning low probability letters.</p>
<p>Here&#8217;s the Python source.  For some reason, I can&#8217;t find the code which generated the pickled &#8220;corpus&#8221;, but it shouldn&#8217;t be too hard to figure out.  Perhaps I&#8217;ll rewrite one during lunch.   The code ia also strictly speaking not an &#8220;algorithm&#8221;, since doesn&#8217;t actually stop: it just keeps running, trying to maximize the score, and does nothing to detect stasis.   But you might find it amusing.</p>
<p>[ EDIT: I put the word 'python' in single quotes in the sourcecode tag; without the single quotes, the tag is not recognized and the formatting is incorrect. I also fixed some ampersand-coded html characters.  I hope this is correct now. ProgPrax ]</p>
<pre class="brush: python;">
#!/u0/markv/my-python/bin/python
#                   _                            
#  __ _ _ _  _ _ __| |_ ___  __ _ _ _ __ _ _ __  
# / _| '_| || | '_ \  _/ _ \/ _` | '_/ _` | '  \ 
# \__|_|  \_, | .__/\__\___/\__, |_| \__,_|_|_|_|
#         |__/|_|           |___/                
# 
# A simple program for automatically solving simple substitution ciphers
# 
# Written by Mark VandeWettering


import pickle
import random
import string
import re
import sys

ldict = pickle.load(open('newcorpus.p'))

letters = &quot;ABCDEFGHIJKLMNOPQRSTUVWXYZ&quot;

sorig = open(sys.argv[1]).read().upper()

swork = re.sub(r&quot;[^A-Z\!?.,']+&quot;, &quot; &quot;, sorig) 
swork = filter(lambda x : x in r&quot;ABCDEFGHIJKLMNOPQRSTUVWXYZ!?.,' &quot;, swork)

print &quot;SWORK: &quot;, swork

POOLSIZE = 1000

pool = []

for p in range(POOLSIZE):
    a = list(letters)
    random.shuffle(a)
    pool.append(''.join(a))

def calculatescore(x):
    s = 0 
    for i in range(len(x)-3):
	s = s + ldict.get(x[i:i+3], 0)
    return s 

class BogusError(Exception):
	pass

def mutate(str):
    if random.uniform(0, 1) &lt; 0.01:
	a = list(letters)
	random.shuffle(a)
	return ''.join(a)
    s = list(str)
    c0 = random.choice(range(26))
    c1 = random.choice(range(26))
    s[c0], s[c1] = s[c1], s[c0] 
    return ''.join(s)

def crossover(pop, mom):
    pop = list(pop)
    mom = list(mom)
    c0 = random.randint(0, 13)
    c1 = random.randint(1, 13)
    c1 = c0 + c1 
    child = pop[:]
    for idx in range(c0, c1):
	idx2 = pop.index(mom[idx])
	child[idx], child[idx2] = child[idx2], child[idx]
    child = ''.join(child)

    for c in string.uppercase:
	if c not in child:
	    raise BogusError

    return child


def sumpool(pool):
    total = 0 
    sums = []

    for score, key in pool:
	total = total + score
	sums.append(total)

    return total, sums

import bisect

def pick(total, l):
    idx = bisect.bisect_right(l, random.randint(0, total))
    return idx 

while True:
    newpool = []
    for p in pool:
	twork = string.translate(swork, string.maketrans(letters, p))
	score = calculatescore(twork)
	newpool.append((score, p))
    newpool.sort(lambda x, y: cmp(y,x))

    newpool = newpool[:POOLSIZE/2]

    # now, generate the distribution 

    total, sums = sumpool(newpool)
    newpool = map(lambda x: x[1], newpool)

    torig = string.translate(sorig, string.maketrans(letters, newpool[0]))
    
    print &quot;Best Decode score = %d&quot; % sums[0]
    print torig
    
    for x in range(POOLSIZE/2):

	idx = pick(total, sums)
	if (random.uniform(0, 1) &lt; 0.15):
	    newpool.append(mutate(newpool[idx]))
	else:
	    idx2 = pick(total, sums)
	    newpool.append(crossover(newpool[idx], newpool[idx2]))

    pool = newpool 
</pre>
]]></content:encoded>
	</item>
	<item>
		<title>By: Remco Niemeijer</title>
		<link>http://programmingpraxis.com/2009/07/14/the-daily-cryptogram/#comment-337</link>
		<dc:creator><![CDATA[Remco Niemeijer]]></dc:creator>
		<pubDate>Tue, 14 Jul 2009 14:49:28 +0000</pubDate>
		<guid isPermaLink="false">http://programmingpraxis.com/?p=944#comment-337</guid>
		<description><![CDATA[First a small notification: I&#039;ll be on vacation the next 4 weeks, so I won&#039;t be able to post any solutions in that period. I should be back for puzzle 59.

My Haskell solution (see http://bonsaicode.wordpress.com/2009/07/14/programming-praxis-the-daily-cryptogram/ for a version with comments):

[sourcecode lang=&#039;css&#039;]
import Data.Char
import Data.List
import qualified Data.Map as M
import GHC.Exts

match :: M.Map Char String -&gt; String -&gt; String -&gt; Bool
match _ []     []     = True
match k (c:cs) (p:ps) = M.findWithDefault [p] c k == [p] &amp;&amp;
                        match (M.insert c [p] k) cs ps
match _ _      _      = False

substitute :: M.Map Char String -&gt; String -&gt; String
substitute key = concatMap (\c -&gt; M.findWithDefault [c] c key)

getKeys :: M.Map Char String -&gt; [String] -&gt; String -&gt; [M.Map Char String]
getKeys k dict c = [M.fromList .
    unionBy (\(a,x) (b,y) -&gt; a == b &#124;&#124; x == y) (M.assocs k) .
    zip w $ map return p &#124; w &lt;- words c,
    let ps = filter (match k w) dict, length ps &lt; 100, p &lt;- ps]

score :: M.Map Char String -&gt; String -&gt; Int
score k = negate . length . filter isLower . substitute k

findBestKeys :: Int -&gt; [String] -&gt; String -&gt; [M.Map Char String]
findBestKeys n dict c = iterate (take 10 . sortWith (`score` c) .
    concatMap (\k -&gt; getKeys k dict c)) [M.empty] !! n

solve :: Int -&gt; [String] -&gt; String -&gt; [String]
solve n dict c = map (`substitute` c) $ findBestKeys n dict $
                 filter (\x -&gt; isAlpha x &#124;&#124; isSpace x) c

crypto = &quot;P OYUUAOEXYW YM AEFGAD, FZGPEAG JAATUL, MYC ENA &quot; ++
         &quot;AGFOPEXYW PWG AWSYLVAWE YM ENA DPHHL ZCYICPVVAC&quot;

main :: IO ()
main = do dict &lt;- fmap lines $ readFile &quot;english-words.10&quot;
          mapM_ print $ solve 7 dict crypto
[/sourcecode]]]></description>
		<content:encoded><![CDATA[<p>First a small notification: I&#8217;ll be on vacation the next 4 weeks, so I won&#8217;t be able to post any solutions in that period. I should be back for puzzle 59.</p>
<p>My Haskell solution (see <a href="http://bonsaicode.wordpress.com/2009/07/14/programming-praxis-the-daily-cryptogram/" rel="nofollow">http://bonsaicode.wordpress.com/2009/07/14/programming-praxis-the-daily-cryptogram/</a> for a version with comments):</p>
<pre class="brush: css;">
import Data.Char
import Data.List
import qualified Data.Map as M
import GHC.Exts

match :: M.Map Char String -&gt; String -&gt; String -&gt; Bool
match _ []     []     = True
match k (c:cs) (p:ps) = M.findWithDefault [p] c k == [p] &amp;&amp;
                        match (M.insert c [p] k) cs ps
match _ _      _      = False

substitute :: M.Map Char String -&gt; String -&gt; String
substitute key = concatMap (\c -&gt; M.findWithDefault [c] c key)

getKeys :: M.Map Char String -&gt; [String] -&gt; String -&gt; [M.Map Char String]
getKeys k dict c = [M.fromList .
    unionBy (\(a,x) (b,y) -&gt; a == b || x == y) (M.assocs k) .
    zip w $ map return p | w &lt;- words c,
    let ps = filter (match k w) dict, length ps &lt; 100, p &lt;- ps]

score :: M.Map Char String -&gt; String -&gt; Int
score k = negate . length . filter isLower . substitute k

findBestKeys :: Int -&gt; [String] -&gt; String -&gt; [M.Map Char String]
findBestKeys n dict c = iterate (take 10 . sortWith (`score` c) .
    concatMap (\k -&gt; getKeys k dict c)) [M.empty] !! n

solve :: Int -&gt; [String] -&gt; String -&gt; [String]
solve n dict c = map (`substitute` c) $ findBestKeys n dict $
                 filter (\x -&gt; isAlpha x || isSpace x) c

crypto = &quot;P OYUUAOEXYW YM AEFGAD, FZGPEAG JAATUL, MYC ENA &quot; ++
         &quot;AGFOPEXYW PWG AWSYLVAWE YM ENA DPHHL ZCYICPVVAC&quot;

main :: IO ()
main = do dict &lt;- fmap lines $ readFile &quot;english-words.10&quot;
          mapM_ print $ solve 7 dict crypto
</pre>
]]></content:encoded>
	</item>
	<item>
		<title>By: Programming Praxis &#8211; The Daily Cryptogram &#171; Bonsai Code</title>
		<link>http://programmingpraxis.com/2009/07/14/the-daily-cryptogram/#comment-336</link>
		<dc:creator><![CDATA[Programming Praxis &#8211; The Daily Cryptogram &#171; Bonsai Code]]></dc:creator>
		<pubDate>Tue, 14 Jul 2009 14:49:15 +0000</pubDate>
		<guid isPermaLink="false">http://programmingpraxis.com/?p=944#comment-336</guid>
		<description><![CDATA[[...] Praxis &#8211; The Daily&#160;Cryptogram By Remco Niemeijer  Today’s Programming Praxis problem is an interesting one: we have to write a program to solve [...]]]></description>
		<content:encoded><![CDATA[<p>[...] Praxis &#8211; The Daily&nbsp;Cryptogram By Remco Niemeijer  Today’s Programming Praxis problem is an interesting one: we have to write a program to solve [...]</p>
]]></content:encoded>
	</item>
</channel>
</rss>

