Generalization of results obtained from a paper that may be "false"
What should I do [when a published result contains a mistake]? should I try to find another proof for the theorem and write it in a paper?
You cannot use their result as-is, because it is incorrect. If their result is sufficiently interesting, then you could perhaps publish a paper that corrects their result, otherwise, you could include a correction in your paper before using the corrected result.
In the paper B, I will use theorems similar to those of Article A, but by weakening the assumptions assumed in A. Do you think there is any chance that this can be considered plagiarism?
No.
You should make it clear that the theorems introduced in Article A are incorrect, and you should clearly motivate the introduction of your similar theorems that correct the originals.
This isn't plagiarism.
The OP clarified their plagiarism concerns as follows:
my proofs will be similar to those used in paper A, except that I weakened the hypotheses.
Before stating your proof, you could explain "The following proof builds upon a proof by Original Authors [Theorem X, Paper A]," you could go further and explain "novel aspects appear in the first, third and fourth paragraphs" or "novel aspects will be highlighted in the proof" (with suitable highlighting in the proof, e.g., "this aspect is new," possibly parenthesised).
I agree with the other answer that plagiarism isn't a concern, but I don't necessarily agree with "You should make it clear that the theorems introduced in Article A are incorrect." This could actually do bad things for your career if you don't tread carefully. I'm not saying you should hide the truth because of politics, but there's likely a middle ground here.
First, it's not clear from your question how bad the mistake is. You say:
they made a passage "crucial" which is not quite correct
I find this statement hard to parse, and would appreciate clarification. As you know, pretty much every published article has mistakes, even if they're just typos. And a mistake has to be much worse than a typo before the math community considers it a big deal. In my observation, any mistake that can be fixed with standard techniques is not considered serious. (In the sense that the authors will still get credit for proving the theorem, even if the mistake regrettably gets past peer review.) Admittedly, "standard techniques" is a blurry line, but you need to think about which side of it you're on, because the right course of action depends heavily on this.
If their proof is relatively easy to fix, then the theorem will be considered theirs, not yours, and you should sell your new paper accordingly in the introduction, i.e. focus on what's really new in it. Then give your own proof of their theorem (or whatever special case you need) "for completeness" with the mistake fixed. If you can also simplify the proof or its presentation, that will make editors and referees feel good about including it in your paper. At the point of the mistake, make a small remark like "we are not able to verify this step of the proof in [citation], so we have provided our own argument."
If the mistake is somewhere in the middle, I would err on the side of not treating it as a big deal, and do more or less as in the previous paragraph.
If the mistake is really crucial, your first step should be to contact the authors via email and ask about it. Preferably, you and your advisor would send a joint email. If they agree with your objection, give them your ideas for fixing the mistake, and offer/ask to work with them. If they don't agree, then you are in a difficult situation, and it's hard to give general advice. It would depend on how important the theorem is, how much work you've done on it already, who the people are, etc.