Position a SceneKit object in front of SCNCamera's current orientation

You don't need this much math — it's all math that SceneKit is effectively already doing, so you just need to ask it for the right results.

[How do I] produce the SCNVector3 that puts the node in "front" of the camera at a given distance?

In the local coordinate system of a node hosting a camera, "front" is the -Z direction. If you put something at (0, 0, -10) relative to the camera, it's directly centered in the camera's line of sight and 10 units away. There's two ways to do that:

1. Just make the text node a child node of the camera.

   cameraNode.addChildNode(textNode)
   textNode.position = SCNVector3(x: 0, y: 0, z: -10)
   

   Note that this also makes it move with the camera: if the camera changes orientation or position, textNode comes along for the ride. (Imagine a selfie stick in reverse — pivot the phone at one end, and the other end swings around with it.)

   Since the text node is in camera-node space, you probably don't need anything more to make it face the camera, either. (The default orientation of a SCNText relative to its parent space makes the text face the camera.)

2. (New in iOS 11 / macOS 10.13 / Xcode 9 / etc.) SceneKit now includes some convenience accessors and methods for doing various relative-geometry math in fewer steps. Most of those properties and methods are available in new flavors that support the system-wide SIMD library instead of using SCN vector/matrix types — and the SIMD library has nice features like overloaded operators built in. Putting that together, operations like this become a one-liner:

   textNode.simdPosition = camera.simdWorldFront * distance
   

   (simdWorldFront is a vector of length 1.0, so you can change the distance just by scaling it.)

3. If you're still supporting iOS 10 / macOS 10.12 / etc, you can still use the node hierarchy to do coordinate conversions. The text node can be a child of the scene (the scene's rootNode, actually), but you can find the scene-coordinate-space position corresponding to the camera-space coordinates you want. You don't really need a function for it, but this might do the trick:

   func sceneSpacePosition(inFrontOf node: SCNNode, atDistance distance: Float) -> SCNVector3 {
       let localPosition = SCNVector3(x: 0, y: 0, z: CGFloat(-distance))
       let scenePosition = node.convertPosition(localPosition, to: nil) 
            // to: nil is automatically scene space
       return scenePosition
   }
   textNode.position = sceneSpacePosition(inFrontOf: camera, atDistance: 3)
   

   In this case, the text node is initially positioned relative to the camera, but doesn't stay "connected" to it. If you move the camera, the node stays where it is. Since the text node is in scene space, its orientation isn't connected to the camera, either.

[How do I] produce the SCNVector4 (or SCNQuaternion) that orients the node so it is "facing" the camera?

This part you either don't need to do much for or can hand over entirely to SceneKit, depending on which answer to the first question you used.

1. If you did (1) above — making the text node a child of the camera node — making it face the camera is a one-step thing. And as I noted, with SCNText as your test geometry that step is already done. In its local coordinate space, an SCNText is upright and readable from the default camera orientation (that is, looking in the -Z direction, with +Y up and +X to the right).

   If you have some other geometry you want to face the camera, you need orient it only once, but you'll have to work out the appropriate orientation. For example, an SCNPyramid has its point in the +Y direction, so if you want the point to face toward the camera (watch out, it's sharp), you'll want to rotate it around its X-axis by .pi/2. (You can do this with eulerAngles, rotation, orientation or pivot.)

   Since it's a child of the camera, any rotation you give it will be relative to the camera, so if the camera moves it'll stay pointed at the camera.

2. If you did (2) above, you could try to work out in scene-space terms what rotation is needed to face the camera, and apply that whenever the node or the camera move. But there's an API to do that for you. Actually, two APIs: SCNLookAtConstraint makes any node "face towards" any other node (as determined by the -Z direction of the constrained node), and SCNBillboardConstraint is sort of the same thing but with some extra considerations for the assumption that you want to face not just some other node, but specifically the camera.

   It works pretty well just with default options:

   textNode.constraints = [ SCNBillboardConstraint() ]
   

An extra caveat

If you're using the sceneSpacePosition function above, and your camera node's orientation is set at render time — that is, you don't set transform, eulerAngles, rotation, or orientation on it directly, but instead use constraints, actions, animations, or physics to move/orient the camera — there's one more thing to consider.

SceneKit actually has two node hierarchies in play at any given time: the "model" tree and the "presentation" tree. The model tree is what directly setting transform (etc) does. All those other ways of moving/orienting a node use the presentation tree. So, if you want to calculate positions using camera-relative space, you'll need the camera's presentation node:

textNode.position = sceneSpacePosition(inFrontOf: camera.presentation, atDistance: 3)
                                                        ~~~~~~~~~~~~~

(Why is it this way? Among other things, it's so you can have implicit animation: set position inside a transaction, and it'll animate from the old position to the new. During that animation, the model position is what you set it to, and the presentation node is continuously changing its position.)

(Swift 4)

Hey, you can use this simpler function if you want to put the object a certain position relative to another node (e.g. the camera node) and also in the same orientation as the reference node:

func updatePositionAndOrientationOf(_ node: SCNNode, withPosition position: SCNVector3, relativeTo referenceNode: SCNNode) {
    let referenceNodeTransform = matrix_float4x4(referenceNode.transform)

    // Setup a translation matrix with the desired position
    var translationMatrix = matrix_identity_float4x4
    translationMatrix.columns.3.x = position.x
    translationMatrix.columns.3.y = position.y
    translationMatrix.columns.3.z = position.z

    // Combine the configured translation matrix with the referenceNode's transform to get the desired position AND orientation
    let updatedTransform = matrix_multiply(referenceNodeTransform, translationMatrix)
    node.transform = SCNMatrix4(updatedTransform)
}

If you'd like to put 'node' 2m right in front of a certain 'cameraNode', you'd call it like:

let position = SCNVector3(x: 0, y: 0, z: -2)
updatePositionAndOrientationOf(node, withPosition: position, relativeTo: cameraNode)

Edit: Getting the camera node

To get the camera node, it depends if you're using SCNKit, ARKit, or other framework. Below are examples for ARKit and SceneKit.

With ARKit, you have ARSCNView to render the 3D objects of an SCNScene overlapping the camera content. You can get the camera node from ARSCNView's pointOfView property:

let cameraNode = sceneView.pointOfView

For SceneKit, you have an SCNView that renders the 3D objects of an SCNScene. You can create camera nodes and position them wherever you want, so you'd do something like:

let scnScene = SCNScene()
// (Configure scnScene here if necessary)
scnView.scene = scnScene
let cameraNode = SCNNode()
cameraNode.camera = SCNCamera()
cameraNode.position = SCNVector3(0, 5, 10)  // For example
scnScene.rootNode.addChildNode(cameraNode)

Once a camera node has been setup, you can access the current camera in the same way as ARKit:

let cameraNode = scnView.pointOfView

And I got it by piecing together several other answers... start with these methods:

func pointInFrontOfPoint(point: SCNVector3, direction: SCNVector3, distance: Float) -> SCNVector3 {
    var x = Float()
    var y = Float()
    var z = Float()

    x = point.x + distance * direction.x
    y = point.y + distance * direction.y
    z = point.z + distance * direction.z

    let result = SCNVector3Make(x, y, z)
    return result
}

func calculateCameraDirection(cameraNode: SCNNode) -> SCNVector3 {
    let x = -cameraNode.rotation.x
    let y = -cameraNode.rotation.y
    let z = -cameraNode.rotation.z
    let w = cameraNode.rotation.w
    let cameraRotationMatrix = GLKMatrix3Make(cos(w) + pow(x, 2) * (1 - cos(w)),
                                              x * y * (1 - cos(w)) - z * sin(w),
                                              x * z * (1 - cos(w)) + y*sin(w),

                                              y*x*(1-cos(w)) + z*sin(w),
                                              cos(w) + pow(y, 2) * (1 - cos(w)),
                                              y*z*(1-cos(w)) - x*sin(w),

                                              z*x*(1 - cos(w)) - y*sin(w),
                                              z*y*(1 - cos(w)) + x*sin(w),
                                              cos(w) + pow(z, 2) * ( 1 - cos(w)))

    let cameraDirection = GLKMatrix3MultiplyVector3(cameraRotationMatrix, GLKVector3Make(0.0, 0.0, -1.0))
    return SCNVector3FromGLKVector3(cameraDirection)
}

Now you can call it like this:

let rectnode = SCNNode(geometry: rectangle)
let dir = calculateCameraDirection(cameraNode: self.camera)
let pos = pointInFrontOfPoint(point: self.camera.position, direction:dir, distance: 18)
rectnode.position = pos
rectnode.orientation = self.camera.orientation
sceneView.scene?.rootNode.addChildNode(rectnode)

The distance 18 positions it between a grid on a sphere at radius 10 and a background image at 25. It looks really good and the performance on my 5s is surprisingly good.