How can I trickle charge NiMH batteries as part of an IOT controller?

You are not seeing an increase in your battery voltage for two reasons:

  1. The solar cell is NOT providing its full current/voltage capability, so your perception of 'full sunlight' is wrong. Test your solar panel on it's own and use a light meter to get some idea of the span of capability the panel has.
  2. Your ESP12F IS NOT asleep, it's just in a lower power mode.

See here:

enter image description here

You should be aiming to be in at least Modem-Sleep mode, and even at this, you will have limited runtime for dark periods if your IOT device is expected to be always on/ready.

For your solar battery charge schema you absolutely must have at least a voltage threshold sensor. As commented in another answer this could be as simple as a TL431 and transistor. Since you might have to dissipate all the power from the solar panel the TL431 on its own won't do.
This type of wasteful charger is ok, but means you can never fully charge the batteries since you have to make voltage compromises. However it's cheap so may be all you need.

For example if you used this Duracell AA NiMH you can see the problems with a voltage only controlled charger:

enter image description here

To charge the batteries (nominal 1.4V, so 5.6V pack) at maximum rate you need a peak voltage of about 6.4V ...above you panel capability. This assumes active charge current control and would add complexity and cost to your project.

To charge the batteries with a voltage only control, you have to set the voltage to no more than the expected terminal voltage (and this varies with temperature too). In this case a reasonable choice would be 1.35V (5.4V pack). At 5.4V pack voltage you will get a proximately 80% charge on the batteries.

At this voltage the batteries will never overcharge, and even though the charge current reduces to just a few mA for a long time this IS NOT trickle charging. Trickle charging by definition is continuing to charge the battery AFTER it is fully charged. In this case since we are not fully charging the battery there is no problem with low current charging.

With a simple charger like this you have to dissipate most of the energy from the solar cell when the voltage limit is reached. In this case if the panel current limit is 200mA then you need to dissipate about 1.2W.

I'd suggest a circuit such as this:

schematic

simulate this circuit – Schematic created using CircuitLab


You should be able to measure the sleep mode current draw and determine where the current is going.
Always active are the L293, the LM1117 and the sleeping ESP.
None of these should be close to the notional 150 mA PV input current.
The most likely "rogue" load is the ESP not being in sleep mode.

You MUST NEVER "float" modern NimH batteries. Older batteries under about 1800 mAh had the capability to absorb up to about C/10 trickle charging (about <= 180 mA). Modern cells above that capacity do not have that capability and there is no certainty that lower capacity modern ones still have trickle charge capacity. Charging fully charged NimH cells will destroy them rapidly by electrolysing the electrolyte and drying out the cell.

To "float" NimH cells long term you can absorb ALL trickle charge current by voltage clamping them at 1.45V per cell at 20 C = 5.8V for 4 cells (or lower). Your nominal 6V PV input + D1 notionally limits the voltage to under 5.8V BUT in practice it may not. Use of a zerner is suitable ONLY if it is certain to keep the voltage to <= 5.8V. Otherwise a hard clamp using a sharp cutoff clamp regulator such as a TL431 and appropriate pass transistor is recommended.