Orthogonal Orientation

MATL, 10 6 bytes

4 bytes saved thanks to Martin!

19\qX!

Try it online!

19\            % implicitly take input (a character) and compute mod-19 of its ASCII code
   q           % subtract 1. Gives 17, 2, 3, 4 for the characters '^<v>' respectively.
               % These numbers correspond to 1, 2, 3, 4 modulo 4, and so are the numbers
               % of 90-degree rotations required by each character
    X!         % implicitly take input (string). Rotate the computed number of times
               % in steps of 90 degrees. Implicitly display

---

Old version, without modulo operations: 10 bytes

'^<v>'=fX!

Try it online!

'^<v>'         % push string
      =        % implicitly take input (a char) and test for equality
       f       % find index of matching character
        X!     % implicitly take input (string). Rotate that number of times
               % in steps of 90 degrees. Implicitly display

Python 3, 64 51 48 bytes

Saved 6 bytes thanks to xnor.

Saved 7 bytes thanks to Lynn.

Saved 3 bytes thanks to DSM and Morgan from sopython.

lambda c,s:'\n'[c<'?':].join(s[::1|-(c in'<^')])

The function accepts one of the characters from <>^v as the first argument and the string that needs to be rotated as the second argument.

---

Here's a more readable version:

lambda c, s: ('\n' if c in '^v' else '').join(s[::-1 if c in'<^' else 1])

Haskell, 57 bytes

f">"=id
f"<"=reverse
f"v"=init.((:"\n")=<<)
f _=f"<".f"v"

Usage example: f "v" "ABC" -> "A\nB\nC".

Direction > is the idendity function, < reverses it's argument, v appends a newline to each character in the string and drops the last one and ^ is v followed by <.