Displaying vertex coordinates of a polygon or line without creating a new layer

There is a way to display the coordinates... though not as a label.

One can add style with geometry generator to extract the vertices, then add a font marker style that will be overridden to display the vertices coordinates..

1. add the vertices

Add a style of type geometry generator, geometry type point computed with the expression nodes_to_points( $geometry)

2. Style it as a Font Marker

3. Instruct to display the coordinates

Click the epsilon beside character(s) to edit the data-defined override. Enter the following expression

round(x( geometry_n( $geometry, @geometry_part_num )),4) || ' - ' || round(y( geometry_n( $geometry, @geometry_part_num )),4)

Let's break it down a bit:

geometry_n( $geometry, @geometry_part_num )) --> get the nth geometry from the one being rendered. From step 1, the geometry being rendered is an array of point (each vertices). Because we use @geometry_part_num, it is applied on every part (= on every vertex).
x(..) and y(...) -> the x/y coordinates of the point
round(..) --> let's not display 18 decimals :-)
|| ' - ' || --> Concatenate the X and Y coordinates, separate them with a -

There is still something to copy from OpenJUMP (and improve). In OpenJUMP vertex XY is available as a decoration. Unfortunately the font or the numbers is tiny and there is no user interface for changing it.

EDIT

I was reading the question too fast and used wrong decorations first.