How to teach a kid geometry
I've assisted in teaching kids aged 10–14 mathematical art, and found a lot of success with the following project:
Invent a polyhedron and try to make it! Basically, take construction paper, and have them use straightedge and compass to make shapes and subsequently cut them out. The most "appealing" are regular shapes. Since your child is 4, you should probably cut the shapes for them.
Here were some especially nice moments:
One student tries to make a vertex with $6$ equilateral triangles and asks me why it wasn't working.
Multiple students worked on constructing pentagons, which was a bit of a challenge (for me too!)
Some students finished early and started coloring their polyhedra, with $4$ or $5$ colors. Of course I bothered them by asking if they could have done it with fewer.
Some students couldn't "See" the shape, so we would use projections to form planar graphs, or use "nets" to help. This was often a bit of a brain cramp but a lot of fun.
here is a blog post chronicling a collection of activities we did (some were more advanced than others: triangle inequalities, pythagorean theorem, finding a way to approximate pi with arbitrary precision etc.) that might have some nice ideas for you!
If it were me, (assuming a child in elementary school) I would start heading into drawing with a ruler and compass.
Working on accurately measuring lengths and angles, drawing circles, lines, and perhaps other shapes, depending on the tools you add, would introduce a visual-art aspect that might help fuel further interest.
For instance, you could work on constructing an equilateral triangle, then adding a few iterations to make it a Sierpinski triangle.
Or you could construct the various centers on a triangle and show they're collinear.
Or you can work on drawing some simple fractal patterns.
Color and pattern goes hand in hand with art and mathematics.
Origami is also a good way to go, and is a good option for children who can't be trusted with sharp implements. There's a book called "Geometric exercises in paper folding" by T. Sundara Row that was passable, but I imagine there's newer and better books like that out now.
I highly recommend the Keys to Geometry workbooks, which are how I fell in love with geometry and construction myself as a kid.
They require no preparation at all. If your kid can draw a straight line with a ruler (straightedge), he/she is ready.
The books are almost entirely interactive. Get a good compass, and a straightedge. (A ruler is okay only if your kid understands not to use the markings on it at all.) By book seven your kid will be able to construct a perfect regular pentagon on a blank sheet of paper, unassisted.
I have also supervised several students through them (graded their exercises) and I recommend reading the material in the teacher's manual, if possible. (Or you can just get the workbooks; you'll be able to check the work fairly easily.) The teacher's manual contains a section on how to instruct beginners to use a straightedge correctly and precisely, how to hold a compass, etc.—which is likely easy for you to do but is more difficult to put in words so that a child can actually learn it based on your description.
I will also note that it is important that the work be checked for precision. Don't tolerate sloppiness even a little bit, because if the ability to do the instruction "Draw an arc with center X" is missed (slightly off from X, or a slightly imperfect arc) in book 1, then the later books will be impossible. These books have extremely little repetition. That's one of the reasons I loved them so much.
These books very definitely encourage thinking.
I've seen teachers of higher math criticize them because they don't contain actual proofs. Instead, they have the student work through some concrete examples ("draw a triangle"; "construct the perpendicular bisector of each side"; "do they meet in one point?") and after working through several such examples, ask the student to choose the right word: "The perpendicular bisectors of the sides of a triangle (always/sometimes/never) meet at one point." All basic rules are demonstrated in such a fashion.
"That's not a proof!" some say. But this is missing the forest for the trees; other such examples where the answer is "sometimes" are conclusively proven by example, and having the student find the answer for himself or herself is far, far better than reciting a litany of "rules" and then claiming this "proves" something.
(Newsflash for higher math teachers: the only purpose for a proof is to make someone realize something true. If you convince only yourself but not the student, you have failed. Okay, off my soapbox.)
Point being, this is an accessible approach to the elements, that kids as young as 8 can do. And high school students benefit as well. (Most modern geometry texts have far more text telling the student what is true, and very little requirement that the student work it out himself or herself with an actual compass.)