Should I worry if my credit card payment processor's server allows only weak SSL cipher suites?
Of course you should worry. If the credit card payment processor is not able to fix well known and obvious security problems (A few days ago RC4 got explicitly prohibited for use with TLS by the IETF) which are even visible world-wide from outside, how will be the status of their internal security? Note, that it might be not that bad to offer RC4 for compatibility with older clients (see the other answer about PCI requirements), but in this case it offers only RC4 to all clients.
Not only you should think in this way but attackers will also think this way and will check the providers server and infrastructure for less obvious but maybe even more serious security problems. And they will probably find them, it would not be a first. This in effect can put your customers and also business at risk too.
A payment processor who accepts RC4 is simply satisfying PCI requirements. (WAS - see update below)
PCI does not disallow RC4. It does, however, consider the presence of BEAST to be a failure. And if they're going to mitigate BEAST and still remain widely compatible, they need RC4 - "The only reliable way to defend against BEAST is to prioritise RC4 cipher suites". The alternatives involve limiting support to TLSv1.1+, which has compatibility problems.
There's a good summary of the issue here: BEAST vs RC4 Ciphers vs PCI
That being said, @steffen-ullrich caught me in the comments to point out your provider is only supporting two RC4 suites. That's clearly not a good idea, and can't be completely explained away by the PCI issue. I'm willing to bet $2 in quarters that this was the result of someone who got told to make sure their site passed their PCI scanner, so they monkeyed around with it to get something that would pass, and didn't realize that they shouldn't go for the minimum passing ciphers.
Since Febuary 2015 when this answer was written, the PCI Security Standards Council has since prohibited RC4. See, for example, this document.