Give Me a Ladder!

Python 2, 316 bytes

def f(m):
 r=[list(' '*61)for y in range(31)]
 (y,x),(b,a)=[[(109-p)/10*3,1+min(p%20,19-p%20)*6]for p in[max(m)-1,min(m)-1]]
 while y<b:c=cmp(a,x);r[y][x:x+4]=r"  /|\___/|\ "[c%3::3];x+=c;y+=1
 if x==a:r[y-1][x+2-(r[y-1][x]>' ')]=" "
 while x-a:c=cmp(a,x)+1;r[y-2][x+c:x+2+c]="__";r[y-1][x+c*3/2]="_";x+=c-1
 print r

Try it online! (footer paints over grid for pretty output)

Function that inputs a list of two numbers and outputs a list of lists of characters.

How it works

We find character coordinates of the higher point (e.g. 96), then translate it along diagonal, vertical, or horizontal movements to the coordinates of the lower point (e.g. 54), modifying the character matrix r as we go.

def f(m):
    r = [list('.'*61) for y in range(31)]
    m = [max(m)-1,min(m)-1]
    # (x,y) is the higher point
    # (a,b) is the lower point
    # x increases right; y increases down
    (y,x),(b,a) = [[(109-p)/10*3,1+min(p%20,19-p%20)*6] for p in m]
    # move (x,y) to (a,b) via 1-unit translations
    # \ diagonal
    while x < a and y < b:
        r[y][x+1:x+4] = "\\_\\"
        x += 1
        y += 1
    # / diagonal
    while x > a and y < b:
        r[y][x:x+3] = "/_/"
        x -= 1
        y += 1
    # straight down
    while x == a and y < b:
        r[y][x+1:x+4] = "|_|"
        y += 1
    # remove space at the bottom of the ladder
    # (r[y-1][x]!='.' deals with off-by-one error in the case of
    # \ diagonals with no straight segment)
    if x == a:
        r[y-1][x+2-(r[y-1][x]!='.')] = " "
    # left
    while x > a:
        r[y-2][x+1] = "_"
        r[y-2][x] = "_"
        r[y-1][x] = "_"
        x -= 1
    # right
    while x < a:
        r[y-2][x+2] = "_"
        r[y-2][x+3] = "_"
        r[y-1][x+3] = "_"
        x += 1
    return r

Charcoal, 130 bytes

FχFχ⊞υ×⁶⎇﹪ι²⁻⁹κκ≔⁻LυNθ≔⁻LυNη≔⊕§υηζ≔׳÷ηχηJ§υθ׳÷θχ F›ζⅈ«≦⊕ζ »F⁻ζ⁺ⅈ⁻ⅉηG←←↑→↘→²_F⁻ⅈ⁺ζ⁻ⅉηG↑←↙→²_W›ⅉη«≔⎇›⁻ⅉη↔⁻ζⅈ²∨›ζⅈ³ιP✳ι¹←←✳ι¹→¿›ⅉη_

Try it online! Link is to verbose version of full 198-byte program that includes drawing the entire background. Explanation:

FχFχ⊞υ×⁶⎇﹪ι²⁻⁹κκ

Get the X-coordinates of all 100 squares (since each row reverses direction).

≔⁻LυNθ≔⁻LυNη

Input the starting and ending square, but subtract from 100.

≔⊕§υηζ≔׳÷ηχη

Calculate the destination character.

J§υθ׳÷θχ

Jump to the start character.

 F›ζⅈ«≦⊕ζ »

Erase the start character, and if this is a right-leaning ladder, then erase the next character too (this is because right-leaning ladders start 1 character further right than left-leaning ladders for some reason).

F⁻ζ⁺ⅈ⁻ⅉηG←←↑→↘→²_

Draw the horizontal part of a right-leaning ladder, if applicable. This is complicated slightly because the horizontal part does not start 1 character further right.

F⁻ⅈ⁺ζ⁻ⅉηG↑←↙→²_

Draw the horizontal part of a left-leaning ladder, if applicable.

W›ⅉη«

Repeat until the top of the ladder is reached.

≔⎇›⁻ⅉη↔⁻ζⅈ²∨›ζⅈ³ι

Work out which direction we're going in (3 = up left, 2 = up, 1 (default) = up right).

P✳ι¹←←✳ι¹→

Draw the next section of ladder.

¿›ⅉη_

Draw the next rung, unless we've got to the top of the ladder.