How can I specify a custom Cargo output directory?

Use the CARGO_TARGET_DIR environment variable:

CARGO_TARGET_DIR=../my-target cargo run --bin my_project

(This is stated in the question, but I wanted to highlight it for anyone that skips over that)

[build] is a Cargo-level configuration rather than for the project:

This document will explain how Cargo’s configuration system works, as well as available keys or configuration. For configuration of a project through its manifest, see the manifest format.

Put your [build] inside $PROJECT_DIR/.cargo/config or even $HOME/.cargo/config. See the above link for all the options.