Maximize $\log(2)+\log(3/2)x+\log(2)y+\log(5/2)z$ if $x+y+z\leq 1$ and $(y+z)^2+2x-x^2-2xy\leq 1-2\gamma$, $0.24 \leq \gamma \leq 0.25$
In this answer we solve a particular case of the problem, when $z=0$.
Now the last constraint becomes $g(x,y)=y^2+2x-x^2-2xy\le 1-2\gamma$. Since $\frac{\partial g}{\partial y}=2y(1-x)\le 0$, for $x$ fixed we can increase $y$ increasing $f$ by this, until $y$ will be bounded by a constraint $x+y\le 1$ or $g(x,y)= 1-2\gamma $. Let’s consider these cases.
1) If $x+y=1$ then the constraint $g(x,y)\le 1-2\gamma$ becomes $x^2-x+\gamma\le 0$. In order to maximize $f(x,y,z)=\log 2+\log\frac 32+\left(\log 2-\log\frac 32\right)y$ we have to maximize $y$, that is to minimize $x$. This happens when $x=\tfrac{1-\sqrt{1-4\gamma}}2$. So it seems we have to swap $x$ and $y$ in your claim.
2) If $g(x,y)= 1-2\gamma$ then $y=x\pm\sqrt{D}$, where $$D=2x^2-2x+1-2\gamma=2\left(x-\tfrac 12\right)^2+\tfrac 34-2\gamma\ge\tfrac 14.$$ Fix $x$ and look for the constrained $y$ maximizing $f$.
Let’s check when we can take the plus sign in the formula for $y$. This is allowed iff $2x+\sqrt{D}\le 1$, that is when $x\le\tfrac 12$ and $2x^2-2x+\gamma\ge 0$, that is if $x\le x_1=\tfrac{1-\sqrt{1-2\gamma}}2$. Since $\frac{\partial D}{\partial x}>0$ when $x<\tfrac 12$, $y$ increases when $x$ increases from $x$ to $x_1$. So in this case the maximum of $f$ is attained when $x=x_1$. Then $y=1-x_1$ and this is Case 1.
If $x>x_1$ then we have $y=x-\sqrt{D}$. The constraint $x+y\le 1$ becomes $x\le\tfrac 12$ or $2x^2-2x+\gamma\le 0$, that is $x_1<x\le \tfrac{1+\sqrt{1-2\gamma}}2=x_2$. We have
$$f(x,y,z)=\log 2+x\log\frac 32+(x-\sqrt{D})\log 2=h(x).$$
Then $h’(x)=\log 2+\log\tfrac 32-\tfrac{2x-1}{\sqrt{D}}\log 2$. We claim that $h’(x)>0$. This is clear when $x\le\tfrac 12$. If $x\ge\tfrac 12$ then we have to show that $(1+c)\sqrt{D}>2x-1$, where $c=\frac{\log\tfrac 32}{\log 2}$. Let’s do this.
$(1+c) \sqrt{D}>2x-1$
$(1+c)^2(2x^2-2x+1-2\gamma)>4x^2-4x+1$
Since $(1+c)^2>2.5$, it suffices to show that
$2.5(2x^2-2x+1-2\gamma)\ge 4x^2-4x+1$
$x^2-x+1.5-5\gamma\ge 0$
$\left(x-\frac 12\right)^2+1.25-5\gamma\ge 0$, which is true because $\gamma\le 0.25$.
Thus $h$ increases when $x$ increases so a maximum of $f$ is attained when $x=x_2$. Then $y=1-x_2$ and this is Case 1 again.
(New solution)
The maximum is $$\ln 2 + \tfrac{1-\sqrt{1-4\gamma}}{2}\ln \tfrac{3}{2} + \tfrac{1+\sqrt{1-4\gamma}}{2}\ln 2$$ at $x = \frac{1-\sqrt{1-4\gamma}}{2}$, $y = \frac{1+\sqrt{1-4\gamma}}{2}$, and $z = 0$. Let us prove it.
We first give the following auxiliary result. The proof is given at the end.
Fact 1: At optimum, either $x+y+z = 1$ or $z = 0$.
Let us proceed. Let $f = f(x, y, z) = x\ln \frac{3}{2} + y\ln 2 + z\ln \frac{5}{2}$. From Fact 1, we split into two cases:
1) $x + y + z = 1$: The constraint $(y+z)^2 + 2x - x^2 - 2xy \le 1 - 2\gamma$ becomes $xy \ge \gamma$. By using $z = 1 - x - y$, we have $$f = f(x, y) = x\ln \tfrac{3}{2} + y\ln 2 + (1-x-y)\ln \tfrac{5}{2} = \ln \tfrac{5}{2} - x\ln \tfrac{5}{3} - y\ln \tfrac{5}{4}.$$ The constraints are: $x\ge 0, y\ge 0, x + y \ge \frac{4}{5}, x+y \le 1$, and $xy \ge \gamma$.
We claim that, at optimum, $xy = \gamma$. Indeed, suppose $xy > \gamma$ at optimum (clearly, $x>\gamma$ and $\gamma < y < 1 - \gamma$), there exists $0 < \epsilon$ such that $(x - \epsilon, y + \epsilon)$ is feasible and $f(x - \epsilon, y + \epsilon) > f(x, y)$, which contradicts the optimality of $(x,y)$.
From $xy = \gamma$, we have $x + y \ge 2\sqrt{xy} = 2\sqrt{\gamma} \ge 2\sqrt{\frac{6}{25}} > \frac{4}{5}$. From $y = \frac{\gamma}{x}$ and $x + y \le 1$, we have $\frac{1-\sqrt{1-4\gamma}}{2}\le x \le \frac{1+\sqrt{1-4\gamma}}{2}$. Then, we have $$f = f(x) = \ln \tfrac{5}{2} - x\ln \tfrac{5}{3} - \tfrac{\gamma}{x}\ln \tfrac{5}{4} .$$ The constraints are: $\frac{1-\sqrt{1-4\gamma}}{2}\le x \le \frac{1+\sqrt{1-4\gamma}}{2}$. We have \begin{align} f'(x) &= - \ln \tfrac{5}{3} + \tfrac{\gamma}{x^2}\ln \tfrac{5}{4}\\ &\le - \ln \tfrac{5}{3} + \gamma(\tfrac{1-\sqrt{1-4\gamma}}{2})^{-2}\ln \tfrac{5}{4}\\ &= - \ln \tfrac{5}{3} + \gamma^{-1}(\tfrac{1+\sqrt{1-4\gamma}}{2})^2 \ln \tfrac{5}{4}\\ &\le - \ln \tfrac{5}{3} + (\tfrac{6}{25})^{-1}(\tfrac{1+\sqrt{1-4\cdot 6/25}}{2})^2 \ln \tfrac{5}{4}\\ &< 0 \end{align} for $\frac{1-\sqrt{1-4\gamma}}{2}\le x \le \frac{1+\sqrt{1-4\gamma}}{2}$. Thus, we have $$f = f(x) \le f(\tfrac{1-\sqrt{1-4\gamma}}{2}) = \ln \tfrac{5}{2} - \tfrac{1-\sqrt{1-4\gamma}}{2}\ln \tfrac{5}{3} - \tfrac{1+\sqrt{1-4\gamma}}{2}\ln \tfrac{5}{4}$$ with equality if $x = \frac{1-\sqrt{1-4\gamma}}{2}$, $y = \frac{1+\sqrt{1-4\gamma}}{2}$, and $z = 0$.
2) $z = 0$: This case was solved by other users. Although I have my solution, I will not give it since it is not better than other users' solutions.
We are done.
$\phantom{2}$
Proof of Fact 1: Assume, for the sake of contradiction, that $x + y + z < 1$ and $z > 0$, at optimum. Let $x_1 = x + 3t, y_1 = y+t, z_1 = z - 2t$ for $0 < t < \min(\frac{z}{2}, \frac{1-x-y-z}{2})$.
We have $x_1>0$, $y_1 > 0$, $z_1 > 0$, $x_1+y_1+z_1 = x+y+z + 2t < 1$ and $x_1+y_1 = x+y + 4t > \frac{4}{5}$. Also, we have \begin{align} &\big[(y+z)^2 + 2x - x^2 - 2xy\big] - \big[(y_1+z_1)^2 + 2x_1 - x_1^2 - 2x_1y_1\big]\\ =\ & 2t(7t+4x+4y+z-3)\\ >\ & 0 \end{align} where we have used $x + y \ge \frac{4}{5} > \frac{3}{4}$. Thus, $(y_1+z_1)^2 + 2x_1 - x_1^2 - 2x_1y_1 \le 1 - 2\gamma$. Thus, $(x_1, y_1, z_1)$ is feasible.
On the other hand, we have \begin{align} &\big[\ln 2 + x_1\ln \tfrac{3}{2} + y_1\ln 2 + z_1\ln \tfrac{5}{2}\big] - \big[\ln 2 + x\ln \tfrac{3}{2} + y\ln 2 + z\ln \tfrac{5}{2}\big] \\ =\ & t \ln \tfrac{27}{25}\\ >\ & 0. \end{align} However, this contradicts the optimality of $(x, y, z)$. We are done.
I cannot prove that the maximum value is $$M:=\frac{\log(12)}{2}+\frac{\log(4/3)}{2}\sqrt{1-4\gamma}$$ but let me show one possible way to prove that the maximum value is $M$.
If $x=0$, then $\frac 45\le y\le y+z\leq \sqrt{1-2\gamma}\implies \frac 45\le \sqrt{1-2\gamma}\implies \gamma\le 0.18$ which is impossible. So, we have $x\gt 0$.
In order for $z$ satisying $$z^2+2yz+y^2+2x-x^2-2xy+2\gamma-1\leq 0\tag1$$ (seen as a quadratic inequality on $z$) to exist, it is necessary that the discriminant $\ge 0$, i.e. $$y\ge\frac{2x-x^2+2\gamma-1}{2x}\tag2$$
Under $(2)$, we see that $(1)$ is equivalent to
$$-y-\sqrt{-2x+x^2+2xy-2\gamma+1}\le z\le -y+\sqrt{-2x+x^2+2xy-2\gamma+1}\tag3$$
In order for $z$ satisfying $(3)$ and $0\le z\le 1-x-y$ to exist, it is necessary that $-y+\sqrt{-2x+x^2+2xy-2\gamma+1}\ge 0$, i.e. $$x-\sqrt{2x^2-2x-2\gamma+1}\le y\le x+\sqrt{2x^2-2x-2\gamma+1}\tag4$$
It follows from $(3)$ and $0\le z\le 1-x-y$ that $$0\le z\le\min\bigg(1-x-y,-y+\sqrt{-2x+x^2+2xy-2\gamma+1}\bigg)$$ where $$\min\bigg(1-x-y,-y+\sqrt{-2x+x^2+2xy-2\gamma+1}\bigg)$$ $$=\begin{cases}1-x-y&\text{if $\quad xy\ge \gamma$}\\\\-y+\sqrt{-2x+x^2+2xy-2\gamma+1}&\text{if $\quad xy\le \gamma$}\end{cases}$$
Case 1 : $xy\ge\gamma$
We get $0\le z\le 1-x-y$ from which we have $$\begin{align}f(x,y,z)&\le f(x,y,1-x-y)=\log(5)-\log(5/3)x-\log(5/4)y:=g(x,y)\end{align}$$ So, we want to maximize $g(x,y)$ under the condition that $$x\gt 0, y\geq 0, 0\le 1-x-y,x+y\geq \frac 45,(2),(4),xy\ge\gamma$$
i.e. $$\frac{1-\sqrt{1-4\gamma}}{2}\le x\le\frac{1+\sqrt{1-4\gamma}}{2},\frac{\gamma}{x}\le y\leq 1-x$$
So, we get $$g(x,y)\le g\bigg(x,\frac{\gamma}{x}\bigg)=\log(5)-\log(5/3)x-\log(5/4)\frac{\gamma}{x}:=h(x)$$ where $h'(x)=\frac{-\log(5/3)x^2+\log(5/4)\gamma}{x^2}$ with $h'(x)=0\iff x=\sqrt{\frac{\log(5/4)\gamma}{\log(5/3)}}$. Since $\sqrt{\frac{\log(5/4)\gamma}{\log(5/3)}}\lt \frac{1-\sqrt{1-4\gamma}}{2}$, we see that $h'(x)\lt 0$ and that $h(x)$ is decreasing, so we get $$f(x,y,z)\le h(x)\le h\bigg(\frac{1-\sqrt{1-4\gamma}}{2}\bigg)=M$$ which is attained when $(x,y,z)=\bigg(\frac{1-\sqrt{1-4\gamma}}{2},\frac{1+\sqrt{1-4\gamma}}{2},0\bigg)$.
Case 2 : $xy\le\gamma$
We get $0\le z\le -y+\sqrt{-2x+x^2+2xy-2\gamma+1}$ from which we have $$\small\begin{align}f(x,y,z)&\le f\bigg(x,y,-y+\sqrt{-2x+x^2+2xy-2\gamma+1}\bigg) \\\\&=\log(2)+\log(3/2)x-\log(5/4)y+\log(5/2)\sqrt{-2x+x^2+2xy-2\gamma+1}:=i(x,y)\end{align}$$
So, we want to maximize $i(x,y)$ under the condition that $$x\gt 0, y\geq 0, 0\le 1-x-y,x+y\geq \frac 45,(2),(4),xy\le\gamma$$
i.e. $$ \frac{3-\sqrt{27-100\gamma}}{10}\le x\le \frac{1+\sqrt{1-4\gamma}}{2},$$ $$\max\bigg(\frac 45-x,x-\sqrt{2x^2-2x-2\gamma+1}\bigg)\le y\le \min\bigg(\frac{\gamma}{x},x+\sqrt{2x^2-2x-2\gamma+1}\bigg)$$
Now, $\frac{\partial i}{\partial y}=-\log(5/4)+\frac{\log(5/2)x}{\sqrt{-2x+x^2+2xy-2\gamma+1}}$ which is decreasing with $\frac{\partial i}{\partial y}=0\iff y=\frac{(1-c)x^2+2cx+2c\gamma-c}{2cx}$ where $c=\bigg(\frac{\log(5/4)}{\log(5/2)}\bigg)^2$.
Here, we can separate it into the following six cases :
Case 2-1 : $0.24\le r\le 0.2475$
Case 2-1-1 : $\frac{3-\sqrt{27-100\gamma}}{10}\le x\le \frac{1-\sqrt{1-4\gamma}}{2}$
$$-x+\frac 45\le y\le x+\sqrt{2x^2-2x-2\gamma+1}$$
Case 2-1-2 : $\frac{1-\sqrt{1-4\gamma}}{2}\le x\le\frac{3+\sqrt{27-100\gamma}}{10}$
$$-x+\frac 45\le y\le\frac{\gamma}{x}$$
Case 2-1-3 : $\frac{3+\sqrt{27-100\gamma}}{10}\le x\le{\frac{1+\sqrt{1-4\gamma}}{2}}$
$$x-\sqrt{2x^2-2x-2\gamma+1}\le y\le\frac{\gamma}{x}$$
Case 2-2 : $0.2475\le r\le 0.25$
Case 2-2-1 : ${\frac{3-\sqrt{27-100\gamma}}{10}}\le x\le \frac{3+\sqrt{27-100\gamma}}{10}$
$$-x+\frac 45\le y\le x+\sqrt{2x^2-2x-2\gamma+1}$$
Case 2-2-2 : $\frac{3+\sqrt{27-100\gamma}}{10}\le x\le\frac{1-\sqrt{1-4\gamma}}{2}$
$$x-\sqrt{2x^2-2x-2\gamma+1}\le y\le x+\sqrt{2x^2-2x-2\gamma+1}$$
Case 2-2-3 : $\frac{1-\sqrt{1-4\gamma}}{2}\le x\le{\frac{1+\sqrt{1-4\gamma}}{2}}$
$$x-\sqrt{2x^2-2x-2\gamma+1}\le y\le \frac{\gamma}{x}$$
Now, let $$\begin{align}F_1(x)&:=i\bigg(x,\frac 45-x\bigg)\\\\ F_2(x)&:=i\bigg(x,x-\sqrt{2x^2-2x-2\gamma+1}\bigg)\\\\ F_3(x)&:=i\bigg(x,\frac{\gamma}{x}\bigg)\\\\ F_4(x)&:=i\bigg(x,x+\sqrt{2x^2-2x-2\gamma+1}\bigg)\\\\ F_5(x)&:=i\bigg(x,\frac{(1-c)x^2+2cx+2c\gamma-c}{2cx}\bigg)\end{align}$$
Also, let $x_i$ be such that $F_i'(x_i)=0$ for each $i=1,2,\cdots, 5$.
Now, if it is true that, for $0.24\le \gamma\le 0.25$, $$F_1\bigg(\frac{3-\sqrt{27-100\gamma}}{10}\bigg)\le M,F_1\bigg(\frac{3+\sqrt{27-100\gamma}}{10}\bigg)\le M,F_1(x_1)\le M,$$
$$F_2\bigg(\frac{3+\sqrt{27-100\gamma}}{10}\bigg)\le M,F_2\bigg({\frac{1+\sqrt{1-4\gamma}}{2}}\bigg)\le M,F_2(x_2)\le M,$$
$$F_3\bigg(\frac{1-\sqrt{1-4\gamma}}{2}\bigg)\le M,F_3\bigg({\frac{1+\sqrt{1-4\gamma}}{2}}\bigg)\le M,F_3(x_3)\le M,$$
$$F_4\bigg(\frac{1-\sqrt{1-4\gamma}}{2}\bigg)\le M,F_4\bigg({\frac{1+\sqrt{1-4\gamma}}{2}}\bigg)\le M,F_4(x_4)\le M,$$
$$F_5\bigg({\frac{3-\sqrt{27-100\gamma}}{10}}\bigg)\le M,F_5\bigg(\frac{1+\sqrt{1-4\gamma}}{2}\bigg)\le M,F_5(x_5)\le M$$
then we can say that the maximum value is $M$. (I have to say that the calculations are very tedious.)
(That at least one of them is larger than $M$ does not necessarily mean that $M$ is not the maximum value. If at least one of them is larger than $M$, then we have to consider the case(s) more carefully.)