webpack4 does not compile all html files?

You shall use html-loader with interpolate flag enabled:

       {
            test: /\.(html)$/,
            include: [
                path.resolve(__dirname, './frontend/pages/')
            ],
            use: [
                {
                    loader: 'file-loader'
                },
                {
                    loader: 'extract-loader'
                },
                {
                    loader: 'html-loader',
                    options: {
                        interpolate: true,
                    }
                }
            ],
            exclude: [
                path.resolve(__dirname, 'frontend/src/', 'template.html')
            ]
        }

I have create pr to fix your problem: https://github.com/sarnor/club/pull/1.

Also noting that interpolation is ugly when you can just write image source like: <img src="../../image/icons/iconfinder_tech_0001_4023871.png".

Also noting that your relative url was pointing to wrong directory, you were missing ../ in your url.