Docker environment variables in multi-stage builds

The problem is not about multi-stage specifically.

It's about differences between Dockerfile ARG & docker-compose YAML build args ("build arguments"); and Dockerfile ENV & docker-compose YAML environment/.env.

The docs were updated (more recently than the original post), and it is fairly clear now:

args

Add build arguments, which are environment variables accessible only during the build process.

Example from the docker-compose docs

Starting simple, just showing the interaction between Dockerfile and the YAML:

ARG buildno
ARG gitcommithash

RUN echo "Build number: $buildno"
RUN echo "Based on commit: $gitcommithash"

build:
  context: .
  args:
    buildno: 1
    gitcommithash: cdc3b19

build:
  context: .
  args:
    - buildno=1
    - gitcommithash=cdc3b19

Example to tie it back to the question:

See the other answer in this thread.

---

Docs & deepening your understanding

Learn one layer of abstraction at a time

I recommend to go from the Dockerfile level of abstraction, upward. Making sure you understand each layer before you add the next layer of abstraction.

1. Dockerfile (and then play with running containers from your Dockerfile ... using default ENV, then playing with --env, then playing with ARG and --build-arg)

2. Then add docker-compose details in, and play with those.

3. Then loop back to Dockerfiles and understanding multi-stage builds.

Dockerfile

A helpful blog-post -- focuses on the Dockerfile but in all cases, it's best to understand Dockerfiles alone before adding the extra layers of abstraction on top of that, such as docker-compose YAML.

https://vsupalov.com/docker-arg-env-variable-guide/

docker-compose

Then docker-compose official docs:

• https://docs.docker.com/compose/environment-variables/
• https://docs.docker.com/compose/env-file/
• https://docs.docker.com/compose/compose-file/#environment
• https://docs.docker.com/compose/compose-file/#env_file

multi-stage Dockerfiles

• https://docs.bitnami.com/containers/how-to/optimize-docker-images-multistage-builds/
• https://medium.com/@tonistiigi/advanced-multi-stage-build-patterns-6f741b852fae
• https://github.com/garethr/multi-stage-build-example

So this is not a multi-stage issue.

It appears ENV variables are only used when running containers (docker-compose up). Not at build time (docker-compose build). So you have to use arguments:

.env:

TEST=11111

docker-compose.yaml:

version: '3'
services:
  test:
    build:
      context: .
      args:
        TEST: ${TEST}

Dockerfile:

FROM nginx:alpine
ARG TEST
ENV TEST ${TEST}
CMD ["sh", "-c", "echo $TEST"]

test command:

docker rmi test_test:latest ; docker-compose build && docker run -it --rm test_test:latest

Seriously the documentation is somewhat lacking.

Reference: https://github.com/docker/compose/issues/1837