Derivative with respect to a vector is a gradient?
It's purely notation.
Given a real-valued function $f(\mathbf r) = f(x^1, \dots, x^n)$ of $n$ real variables, one defines the derivative with respect to $\mathbf r$ as follows: \begin{align} \frac{\partial f}{\partial \mathbf r}(\mathbf r) = \left(\frac{\partial f}{\partial x^1}(\mathbf r), \dots, \frac{\partial f}{\partial x^n}(\mathbf r)\right) \end{align} so, by definition, $\partial f/\partial \mathbf r$ is a vector of functions that precisely equals $\nabla f$. You may also run into the notation $\nabla_{\mathbf r}f$ which means precisely the same thing.
An advantage of the notations $\partial f/\partial\mathbf r$ and $\nabla_\mathbf r$ is that they make explicit the symbol used to label the argument of the function, and this can sometimes stave off confusion.
Addendum. You may also come across the following notation. Let $\mathbf f(\mathbf r) = \mathbf f(x^1, \dots, x^n)$ be a real, $m$-component vector-valued function of $n$ real variables, then define its derivative with respect to $\mathbf r$ as the following matrix, often referred to as the Jacobian matrix: \begin{align} \frac{\partial\mathbf f}{\partial \mathbf r}(\mathbf r) = \begin{pmatrix} \frac{\partial f^1}{\partial x^1}(\mathbf r) & \cdots & \frac{\partial f^1}{\partial x^n}(\mathbf r) \\ \vdots & \ddots & \vdots \\ \frac{\partial f^m}{\partial x^1}(\mathbf r) & \cdots & \frac{\partial f^m}{\partial x^n}(\mathbf r) \\ \end{pmatrix} \end{align} Consider, for example, the function $\mathbf g(\mathbf r) = \mathbf r$. In this case, you can convince yourself that its derivative $\partial/\partial\mathbf r$ is the identity matrix; \begin{align} \frac{\partial\mathbf g}{\partial \mathbf r}(\mathbf r) = \begin{pmatrix} 1 & \cdots & 0 \\ \vdots & \ddots & \vdots \\ 0 & \cdots & 1 \\ \end{pmatrix} \end{align}
No, not really.
A gradient is the derivative of a scalar. It is not actually a vector, but a dual vector or 1-form.
http://en.wikipedia.org/wiki/Gradient
Vectors and 1-forms have different transformation properties, and used to be called contra-variant and co-variant vectors, but the language of exterior calculus makes this much cleaner. Intuitively, a 1-form operating on a vector gives you a scalar, and vice versa, and is essentially the definition of the inner product.
In flat space or Cartesian coordinates these are the same thing, but once you go to curved space or curvilinear coordinates they are quite different. So, the relation:
$$\frac{\partial}{\partial \mathbf{r} }=\nabla$$
Really only holds in flat space/Cartesian coordinates.
The partial derivative of a vector is not the gradient! This is because the partial derivative operator does not in fact operate in a coordinate independent way, but scalars, vectors, and tensors are coordinate independent.
Instead, without any further knowledge we can define something called the Lie derivative, which operates on vector and tensor fields and tells you the change of the tensor field along the flow of another tensor field (remember that tensors can be written without reference to coordinates):
http://en.wikipedia.org/wiki/Lie_derivative
Note we have suddenly started talking about vector fields, which are vectors defined at every point on your system.
Parenthetically, Lie derivatives are useful because if you take the Lie derivative of some tensor along a vector and find that it is zero, that vector is called a Killing vector and is a symmetry of the system.
If you have a manifold with a metric tensor defined, you can use it to define the covariant derivative, which is essentially the coordinate-independent partial derivative.
http://en.wikipedia.org/wiki/Manifold
http://en.wikipedia.org/wiki/Covariant_derivative
(And the covariant derivative is useful in defining parallel transport and geodesics.)
http://en.wikipedia.org/wiki/Parallel_transport
http://en.wikipedia.org/wiki/Geodesic
So your notion of oddness is spot on, because you've secretly entered the world of curved space! A very good general overview of these concepts can be found here:
http://preposterousuniverse.com/grnotes/grnotes-two.pdf
http://preposterousuniverse.com/grnotes/grnotes-three.pdf
And now you've gone down the rabbit hole! :-)