Abbreviate list of names until they fit

Jelly, 25 bytes

ṖṖ;⁽÷ẓỌµ€µL€MḢµ¦µj⁴L>⁵ð¿j

Try it online!

This unfortunately does not contain the error-if-invalid feature.

Explanation

This is what I call the Jelly syndrome: the easy part is explaining the algorithm, the hard part is explaining how you found the exact order to put the code in to make it execute correctly.

ṖṖ;⁽÷ẓỌµ€µL€MḢµ¦µj⁴L>⁵ð¿j
                       ¿     While
                 j           joining the input list
                  ⁴          with the glue produces a string
                   L         whose length
                    >        is larger than
                     ⁵       the maximum,
               ¦             in the input array at the
             Ḣ               first of the positions that
            M                have maximal
          L€                 length,
ṖṖ                           remove the last two characters,
  ;                          append
   ⁽÷ẓ                       the number 8230
      Ọ                      and cast it to a character.
                        j    Finally, join again with the glue.

05AB1E, 25 bytes

[гýDg²›≠#\Déθ©k®¨¨"…"«sǝ

Try it online!

Python 3, 102 bytes

def f(l,n,g):
 while len(g.join(l))>n:i=l.index(max(l,key=len));l[i]=l[i][:-2]+'…'
 return g.join(l)

Try it online!

Alternatively if you want to see the result being shortened over time:

def f(l,n,g):
 while len(g.join(l))>n:i=l.index(max(l,key=len));l[i]=l[i][:-2]+'…';yield g.join(l)