Musical notes and colors of a rainbow
On the most basic level, the answer is a flat no. The seven primary notes in an octave is specific to the western musical tradition. It's not entirely arbitrary as you say, but there are many other choices that could have been made, and there are other cultures who use fewer notes (e.g. pentatonic scales in blues music) or more (e.g. Indian classical music). The seven colours in the rainbow are also somewhat arbitrary. (Are indigo and violet really different colours? Why don't we count aquamarine, right between green and blue?)
Having said that, it does happen to be the case that the range of frequencies we can see is just a little short of an octave, ranging from about 440-770 THz. This is really more or less a coincidence, but because of it, I can point out a relationship between light and colours, just for fun.
The A above middle C is defined, for modern instruments, as 440Hz. The A an octave above is 880Hz, and in general if we go $n$ octaves up we get a frequency of $440\times 2^n$. If we go forty octaves up from A we get a note of 483 THz. This can't be played as a sound wave (air can't vibrate at frequencies that are too high) but as an electromagnetic wave it's a slightly reddish orange.
If we go down a note to G we get $392\times 2^{40}$ Hz $= 431$ THz, which is just into the infra-red. (It might be possible to see it as a very deep red colour, but I'm not sure.) However, moving up from there we get the following colours:
- G - 431 THz - infra-red
- A - 483 THz - orange
- B - 543 THz - yellow-green
- C - 576 THz - green
- D - 646 THz - blue
- E - 724 THz - indigo
- F - 768 THz - violet (barely visible)
- G - 862 THz - ultra-violet
(I leave the sharps and flats as an exercise to the reader.) So you can't see G (or F#), but the other notes do actually have colours.
However, as I said this is just a bit of fun and does not in any way have any practical implications, since sounds at those frequencies can't be transmitted through air.
As requested in comments:
There is a connection in the sense that Isaac Newton regarded both musical harmony and optical physics as branches of mathematics (Kepler did the same with harmony and astronomy, and this kind of thing was not original to them), and deliberately chose 7 rainbow colours to match the common Western scale, despite his poor eyesight initially only spotting 5 colours; he later added orange and indigo
Wikipedia's article on the rainbow says
Newton chose to divide the visible spectrum into seven colours out of a belief derived from the beliefs of the ancient Greek sophists, who thought there was a connection between the colours, the musical notes, the known objects in the Solar System, and the days of the week.
and includes a reference to a 2004 article by Niels Hutchison, MUSIC FOR MEASURE, On the 300th Anniversary of Newton's "Opticks"
Answer to this question varies significantly based on point of view.
Physics & Physiology: No, there is no connection. The mechanisms are quite different (EM vs. acoustic spectrum, eyes vs. ears etc.) and number 7 is arbitrary.
Musicology & Aesthetics: Number 7 isn't that important, because in an octave there is actually 12 notes if we adopt western model and it is not the only option, vision does not have an octave-principle etc. On the other hand, there are numerous theories (but in context of this site let's label them "analogies") about visible color and tone (e.g. from Newton himself). This notion (part of a bigger phaenomenon called synesthesia) is psychological and experience-based. It lies on no physical principle - well, other than: "Waves! It all fits together, ya know?!"