Should I be worried that I don't have ideas of questions to ask during seminars?
A conference sessions might have 50 people in it (at least in the conferences I go to), and yet there will only be 3 or 4 questions on each talk, even if five people have questions for every 1 that has the guts to put their hand up and ask it that most people don't have questions for most talks. You might also notice that its the same people that ask questions about every talk.
I wouldn't worry about it, especially if you are just a PhD student. One thing I would say is that questions should be asked from a position of ignorance - that is you ask a question because you genuinely don't know the answer and want to. Not because you think you can trip the speaker up, or make yourself look clever, so questions that you think will make you look dumb might be the best questions.
Ian's answer is already good, as is the link to "How to ask stupid questions". But I wanted to touch on the last paragraph of your question:
Finally, a disclaimer. I know some people might say "talk to your advisor about this", but I really never had conversations about the process of doing research with him. In fact the way he works is more like "calculate this, then we talk about the calculation" and he really does not have much patience to talk about much else, so that I don't really feel comfortable bringing this up.
This is something you should seek a solution for. It doesn't have to come from your supervisor. One of the most inspiring courses I took was titled "Better Science for Computer Scientists" and it was filled with a mix of Master and PhD students, and led by a veteran professor. Much of the course was really a view at what really happens behind the scenes - how to people behave at conferences, what do they do in breaks during conferences, what really goes on at the editing desk of a journal and so forth. The key was that academics are people, there is much more of a human and social dimension to academia than you might imagine seeing it from a distance as an undergrad student.
One of the things the professor pointed out is that a lot of academics like being magicians: pulling complicated results out of a hat and making it seem like that's easy. We all like other people to think that we're amazingly smart. But magicians' tricks take a lot of practice that you don't see.
What I encourage you to do is talk more with people you're not answerable to. It's a bit easier to have an honest talk about how things really work with someone that isn't also deciding on your thesis. So have conversations with your peer PhD students, and talk with faculty members that you think are nice people. If you're finding this difficult, then that's definitely a warning sign that there may be some impostor syndrome at work; "I don't dare ask such things because I'll be found out!" indeed.
I am the opposite. I always have many questions and try to limit myself to one or at most two.
If you truly never have questions about a seminar, it can mean a number of things:
You believe the speaker is infallible and therefore you believe everything they say unquestioningly.
You find the seminar easy or obvious. You really don't need to ask anything.
You are scared of looking stupid. You can bet that half the audience are wanting to know the answer to this question and they are all unwilling to ask first. Very often I have asked a question and a number of people have turned to me and nodded approvingly or said, "Yes, I was wondering that".
You are shy. This was something that I had to overcome. Things get easier the more you do them. The first question is the hardest.
Note: I was twice offered a job at one seminar, by a scientist who had a big team and lots of funding. He did this in front of the audience! I asked what seemed to me to be a seemingly obvious question. He was impressed that what had seemed obvious to me was actually very rarely spotted by those in the field. (It wasn't even my speciality - I was attending out of interest). I didn't accept the job because I was involved in other things.
Conclusion
If you literally never have a question at any seminar then either you are a genius and you have understood everything, or the subject matter is very basic. If you understand everything then you can ask, e.g. "Why did you choose to do it that way - couldn't you have done it this way instead?" or "That was fascinating, thank you for such a clear explanation, what is your next step going to be?" If you really are fascinated by the subject, you can even drop a hint that you would be interested on researching the same subject and say what part of it - it might even pay off with a studentship or a collaboration.
Finally, don't try to 'think of questions', that is completely pointless and will come across as insincere and annoying. Ask about what you don't understand or think could have been done better or where the research will lead.
P.S. If you agree with this answer completely, then please upvote it and accept. If you disagree then please ask me a question about the part(s) you disagree with. It is much more tactful to ask than simply tell someone they are wrong. If there is a point I made that you are not sure will work, ask me a question, e.g. Has it actually worked for me in real life - can I justify my answer? Stack Exchange is all about asking questions and you have just asked a very good question, i.e. "Should I be worried that I don't have ideas of questions to ask during seminars?"! Practise asking questions on Stack Exchange.