Vibrating string, free end boundary condition

The open boundary condition means, as stated in the question, that at the boundary no force acts on the end of the string in the direction of elongation.

As the tip of the string has infinitesimal mass, we can argue as if we were considering conditions for static equilibrium (if the forces caused by the string would differ from the forces caused by the wall by a finite amount, the end of the string would be accelerated infinitely which is unphysical).

The forces acting on the end of the string can easily be analysed: The force acts in the direction of the string (because an ideal string has, by defintion, no resistance to bending) and is equal in magnitude to the tension of the string.

This means that there is no force parallel to the boundary if the direction of the string is perpendicular to the surface. This condition can obviously be encoded in the requirement that $\partial_x u = 0$ (as $u$ is the $y$ coordinate of the string at position $x$, so the string is perpendicular to the boundary if the slope of the graph is zero).

I think that Sebastian Riese's answer captures the essence of the argument. Just to add to it for those who might be interested, here is a mathematical version of the argument:

Let a string extend from $x = 0$ to $x = L$ and let $u(x)$ be the $y$ coordinate of each point of the string. Consider a portion of the string from $x = 0$ to $x = b$ and apply Newton's second law to it, which states that the sum of external forces on this portion of the string equals the sum of the mass times acceleration of the elements composing the portion of string. Consider the $y$ component of this equation, then the string being considered as a continuous medium, the latter quantity can be expressed as the integral of the string's density times the second order derivative of $u(x)$ with respect to time, $\int_0^b \rho u_{tt}(x) dx$.

Now consider the external forces on the portion of string in the $y$ direction. Internal forces due to the tension of the string do not contribute a net force because of the action-reaction law, so that the net forces only occur at the end points of the string. At $x = 0$, because the end of the string is constrained to remain at $x = 0$ but is otherwise free to move along $y$, the net force along $y$ is null. At $x = b$, a net force will result from the tension in the string (the atoms on the left and right of $x = b$ are pulling on each other, but only the atom on the left are part of the portion of string considered). Tension being a force aligned along the string, the $y$ component of the force at $x = b$ is $T \frac{u_x}{\sqrt{1+u_x^2}} \approx T u_x$ where $T$ is the tension magnitude. Therefore Newton's second law along $y$ reads

$T u_x(b) = \int_0^b \rho u_{tt}(x) dx$

Now assume that $u_x(x)$ is continuous and that $u_{tt}(x)$ is continuous and bounded, taking the limit $b \rightarrow 0$ yields $u_x(0) = 0$, which is the boundary condition.