What is the smallest distance possible between two stars?

You can look at databases of binary stars to tell you what the range of orbital periods/separations of stars currently are. Unfortunately that isn't going to answer your question because many short-period binary systems have evolved to be that way - e.g. the short-period cataclysmic variable stars or the "contact" binaries known as W UMa systems, where the stars are actually touching each other and have a common envelope. Binaries can also be made "harder" by interactions with third bodies, especially during their early lives if they are born in dense star clusters, or the orbits may be shrunk or the stars may even merge due to gas-induced orbital decay during the early embedded phase of the formation of a star cluster (Korntreff et al. 2012) . Hence there are many older short period binary systems with periods shorter than one day, where the components are barely separated, or are not separated at all.

What you need to do is look for a catalogue of binary objects in the youngest star forming regions or star clusters, where you might reasonably assume there has been little time for interaction (to some extent it depends whether you include early life in a star forming environment as part of the birth process or not).

It is actually quite hard to do surveys of this kind. You need repeated, high resolution spectroscopic measurements of quite faint objects in order to find the doppler shifts due to binary motion. But some work is out there. Meibom et al. (2006) search for spectroscopic binaries in the young(ish) clusters M34 and M35 (250 and 150 Ma respectively). They find 6 binaries in these clusters which have separations smaller than 0.12 au and periods below 13 days. The shortest period binary has a $0.9M_{\odot}$ primary with a period of 2.25 days. Nevertheless, the authors concede that these binaries may have been hardened by close third-body encounters.

Morales-Calderon et al. (2012) pursued a different approach; performing photometric monitoring in the infrared to search for eclipsing binaries in the very young Orion Nebula cluster (age about 2 Ma). They found six binaries with periods between 3.9 and 20.5 days.

There are then quite a few other individual results and studies. I can't immediately locate any good compilation, but this is the closest binary that a brief search uncovered: Bakis et al. (2011) analysed the binary IM Mon, probably a 10 Ma old system in Orion. They find an orbital period of only 1.19 days and primary and secondary masses of 5.5 and 3.3$M_{\odot}$. Their radii are about 30% of the separation between the stellar centres (estimated to be $a=9.77R_{\odot} = 0.045$ au). The comparative youth and mass of this binary suggest it was probably "born this way".

There is a database of visual binaries - that's a good place to start. It includes the following plot of period and eccentricity:

enter image description here

The bottom left corner represents a system with a period of $10^{-1.6}\approx 0.025$ years or just over 9 days.

Now from Kepler's Laws, we have that the square of the period scales as the cube of the average distance. If the two stars have the same mass as our Sun, we can estimate the distance using the correction for reduced mass given on hyperphysics:

$$T^2 = \frac{a^3}{m_1+m_2}$$

where $a$ is expressed in a.u., $m$ in mass of the sun, and $T$ is calculated in years.

Using this, we end up with an estimated distance of $a = 0.11 a.u.$. That is "very close" - much closer than the orbit of Mercury. For reference, the radius of the Sun is 0.0046 a.u. Tidal forces would be enormous.

There may be better ways to estimate all of this - I will try to ping a few people who know much more about these things.

There's already an accepted answer, but I think it would be interesting to briefly look at the physical processes that go on in these systems.

Let's consider the mechanics going on first. We have two stars, in orbit around their centre of mass, and we work in the centre of mass frame. Any orbiting test particle will be subject to a potential due to the mass of the first star $M_1$, the mass the second star $M_2$, and a term for conservation of angular momentum. This gives us a potential: $$\phi = -\frac{GM_1}{r_1}-\frac{GM_2}{r_2} - \frac{1}{2}\omega^2[(x-x_{c.o.m})^2+y^2].$$ The distance from the centre of mass to each star is given as $r_1$ and $r_2$. This is known as the Roche potential, and it is analytically a very challenging potential to work with. However we can work with it numerically, and that is shown below. Roche Potential evaluated numerically, with its rather cheery contour projection shown below. What can we notice about this potential, apart from the fact it looks fairly happy. First of all, the force felt is given by the gradient of the potential $\nabla \phi$. It could be interesting to look at points where there is no force felt by a test particle, areas where the gradient is zero. There are 5 of these points, and they are called Lagrange points. On the diagram, 3 of these are shown, and marked as $L_1$, $L_2$ and $L_3$. For the purpose of this post we're most interested in the inner Lagrange point $L_1$. Although for your interest, as force-free points, they are very useful points for positioning a spacecraft, e.g. if you have a probe to observe the Sun, placing it at the inner Lagrange point between Earth and the Sun allows you to always face the Sun, whilst spending relatively little fuel.

The fact I said relatively little brings me onto the next point. The Lagrange points from 1-3 are unstable minima. This means if you slightly perturb your test-mass (or your probe), it will leave the stable point. (The points $L_4$ and $L_5$ are stable minima, and even after a slight perturbation, the probe would return to the minimum point). You can understand this in terms of a small ball in a cereal bowl. If the cereal bowl is the right way up, even if you slightly move the ball from the centre, it will return to the centre. Whereas if the bowl is upside-down, moving the ball even slightly will lead to it falling off the bowl, and leaving the stable point.

So between our two stars, we have an upside-down bowl point, where if mass from one star reaches it, it can quite easily fall into the potential of the second star. This is such an important point, we define the "Roche-lobes", as the surfaces of constant $\phi$, which touch at the inner Lagrange point $L_1$. On the contour plot, these are the bold lined contours. As a star reaches its Roche-lobe, it begins to distort to that tear drop shape. (This distorted shape can actually be observed in binary systems, as the "amount" of star that we see depends on the orbital phase, resulting in a brightness modulation.) Once the star reaches $L_1$, it is able to begin mass-transfer onto the other star. At this point, you have reached the maximum size for your star, without it becoming a contact binary. A rough idea of what happens is shown in the next diagram:

enter image description here

The star which fills its Roche-lobe is able to transfer mass onto the second star, and it is this Roche potential that provides a physical limit on two stars remaining distinct. This is still a simplistic picture. There can be stable and unstable mass-transfer, depending on $q=M_2/M_1$, the mass ratio. In one case, as the mass is transferred, the donor Roche-lobe grows, meaning the star is no-longer in contact with its $L_1$ point, preventing mass transfer. In the other case, the donor Roche-lobe shrinks, which accelerates mass-transfer. This unstable mass-transfer is then a runaway process.

The shape of the Roche potential depends purely on $q$, and the orbital period $q$. It is then scaled by $a$, the binary separation. That means that the limit to the distance between two distinct stars, will depend on the binary separation distance, and the mass of the two stars. There are various numerical fits to the Roche potential, valid for different ranges of of $q$ in your system. By looking at these, and picking some parameters for your stars, you could calculate the actual physical limit of distance between your stars, before they could no longer be considered distinct.