Write a Playfair encryption program

J I*, 536 431 417 380 263 218 203 197 186 167

p=:4 :0
a=.u:65+9-.~i.26
,_2(5|,:~@|.@(=/)+$$,A.~5*1-0{=/)&.(5 5#:(~.n x,a)&i.)\(,'X'#~2|#)(({.,'X',}.)~1+2*1{&I._2{.\2=/\]) ::]^:_(n=:a(e.~#])'JI'charsub toupper)y
)

(with extensive suggestions from @algorithmshark)

example use:

   'Stack Overflow' p 'The cat crept into the crypt, crapped, and crept out again.'
SIRACARDFMVUICVSMORDZNAKECMZMFBCYNRDFMSVTVKBTMMY

splits input correctly:

   d=:(({.,'X',}.)~1+2*1{&I._2{.\2=/\]) ::]
   d^:_ 'MASSACHUSETTS'
MASXSACHUSETTS

---

*_{^{replace every J with an I, right?}}

Ruby, 461 411 366 359 352 346 330 characters

k,m=$<.map{|x|x.tr(?j,?i).upcase.tr('^A-Z','').chars}
t=[*((k&k)|[*?A..?Z]-[?J]).each_slice(5)]
m=(m*'').gsub(/(.)\1/,'\1X\1').chars
c=->n{[t.index{|r|k=r.index n},k]}
$><<(m.size%2<1?m:m+[?X]).each_slice(2).map{|p,q|a,b,d,e=*c[p],*c[q]
a==d ?[t[a][(b+1)%5],t[d][(e+1)%5]]:b==e ?[t[(a+1)%5][b],t[(d+1)%5][e]]:[t[a][e],t[d][b]]}*''

Thanks to @daniero for saving... err, a lot of bytes. \o/

Here's the ungolfed code:

key = gets.chomp
msg = gets.chomp
transform = ->str{
    str.gsub! 'j', 'i'
    str.upcase!
    str.gsub! /[^A-Z]/, ''
    str.split('')
}

# 1. Generate a key table
key = transform[key]
chars = key.uniq + ([*?A..?Z] - key - ['J'])
tbl = Array.new(5) {
    Array.new(5) {
        chars.shift
    }
}

# 2. Prepare the message
msg = transform[msg]
msg = msg.join('').gsub(/(.)\1/){ "#{$1}X#{$1}" }.split('')
msg = (msg.length % 2 == 0 ? msg : msg + ['X']).each_slice(2).to_a

# 3. Encryption
coords = ->chr{
    i = -1
    [tbl.index{|row| i = row.index chr}, i]
}
msg.map! do |c1, c2|
    c1, c2 = coords[c1], coords[c2]
    if c1[0] == c2[0]
        # same row
        [tbl[c1[0]][(c1[1] + 1) % 5], tbl[c2[0]][(c2[1] + 1) % 5]]
    elsif c1[1] == c2[1]
        # same column
        [tbl[(c1[0] + 1) % 5][c1[1]], tbl[(c2[0] + 1) % 5][c2[1]]]
    else
        # neither
        [tbl[c1[0]][c2[1]], tbl[c2[0]][c1[1]]]
    end
end

# Output!
puts msg.join

Here's some sample outputs:

llama@llama:...code/ruby/ppcg23276playfair$ printf 'Stack Overflow\nThe cat crept into the crypt, crapped, and crept out again.\n' | ./playfair.rb; printf 'This is a password!\nProgramming Puzzles and Code Golf is a Stack Exchange site.\n' | ./playfair.rb
SIRAVXRDFMVUUYVSBLRDZNYVECMZMFBCYNRDFMSVTVKBVBMY
WDDEDSXIXOQFBTUYVQFISQWGRPFBWMESATAHHGMBVEITQFFISHMI

C: 495 401 355 341 characters

It's just a rough sketch as of now. I should be able to shave off at least a hundred characters.

Goal accomplished: more than a hundred characters (154 as of now) have mysteriously vanished from the code.

p[25],l[96],a=1,b,c;o(x,y){putchar(p[x%5^y%5?x/5*5+(x/5^y/5?y:x+1)%5:(x+5)%25]);}main(){for(;a&&((a=(b=getchar())>31)||(b=65))||b++<90;c=0)for(b&=-33;b/65-b/91&&p[c]^b-(b==74);p[c++]||(p[--c]=b-(b==74),l[b]=c));for(;b=getchar(),b=b>31?b&-33:(c=88),b=b/65-b/91?a?a^b?(c*=c==88,b):(c=b,88):(a=b,0):0,a&b&&(o(a=l[a],b=l[b]),o(b,a),a=c),c^88;);}

With some pleasant whitespace:

p[25],l[96],a=1,b,c;
o(x,y){
    putchar(p[
        x%5^y%5
            ?x/5*5+(x/5^y/5?y:x+1)%5
            :(x+5)%25
    ]);
}
main(){
    for(;
        a&&(
            (a=(b=getchar())>31)||
            (b=65)
        )||b++<90;
        c=0
    )for(
        b&=-33;
        b/65-b/91&&
        p[c]^b-(b==74);
        p[c++]||(
            p[--c]=b-(b==74),
            l[b]=c
        )
    );
    for(;
        b=getchar(),
        b=b>31
            ?b&-33
            :(c=88),
        b=b/65-b/91
            ?a
                ?a^b
                    ?(c*=c==88,b)
                    :(c=b,88)
                :(a=b,0)
            :0,
        a&b&&(
            o(a=l[a],b=l[b]),
            o(b,a),
            a=c
        ),
        c^88;
    );
}

I wrote the first iteration of the program on the verge of falling asleep, so it had a lot of superfluous meaningless statements and such. Most of that is rectified, but there are quite a few areas where improvement is most definitely possible.