Why should browser security be prioritized?

Can you explain why browser security should be placed on the top priority ...

Because the browser is processing lots of untrusted content from the internet.

Of course, if you use any other programs which does this (like Mail client, maybe Office program, PDF reader) you should keep these updated too since vulnerabilities in these programs are a regular attack vector too.

.. Most of the websites I visit have SSL certificate,

A SSL certificate says nothing about the trust you can have in a site. HTTPS only protects against sniffing modification of the traffic during transport. A HTTPS site can serve malware as much as a plain HTTP site can do.

Apart from that "Most of the websites" is not the same as "All of the websites".

I can smell fishy websites?

Even if you might be confident in your ability to sniff websites where the URL looks fishy (which might actually be overconfidence) I'm pretty sure that you will not know up-front if the site you visit regularly got hacked and is serving malware (i.e. Watering hole attack or other kinds of hacking high-reputation sites to increase number of victims) or if it is serving malicious ads which are outside the control of the website itself (i.e. Malvertising).

EDIT:
After I've wrote my answer the OP added the following to the question:

Most of them are either big enough that I can trust that they can't be hacked, ...

Too big to be hacked? While large web sites usually employ better security than smaller ones it does not mean they are unhackable. And sites with lots of customers are especially a lucrative target for the attackers since this also means lots of potential victims. Some examples: ... malicious ads on Forbes ... or ... New York Times and BBC hit by 'ransomware' malvertising or Study: One-third of top websites vulnerable or hacked.

... or small enough that I don't think it's profitable for the hackers, ...

Too small to be hacked? That's not true either: attackers use automated tools to hack insecure CMS installations like WordPress or Django en mass, i.e. it is very cheap to take over a vulnerable site this way.

Not all the websites you visit have certificates. You can’t smell fishy websites. Certificate doesn’t mean the site isn’t trying to hack you.

The browser is the biggest attack vector against your computer. It will tend to run unvetted JavaScript code at least, and god knows what else. It constantly processes data from untrusted sources.

Each of your statements is making a false assumption here:

Most of the websites I visit have SSL certificate.

This is great, but SSL/TLS only protects you against certain types of attacks.

Pretty much, a site having a (valid) TLS certificate simply means that the owner of that website has in some way proven ownership of the domain name that is used to access the site, possibly spending a very large amount of money to do so (or possibly spending none). Usually, this means that you can trust that the site is who they say they are, barring domain spoofing (which I will cover in response to your third point). However, it may not mean that (see for example the fiasco that got everything issued by Symantec revoked/untrusted earlier this year). So, a TLS certificate really just verifies site ownership, not that the site is legitimate or that they are doing what they say they are.

TLS certificates give one more protection benefit, they let you use HTTPS. HTTPS provides exactly two types of protection if it's used correctly:

• It provides a reasonably high level of confidence that the data you are receiving is the same data that the web server sent. This is important for ensuring that nothing has been added to or removed from the site by a third party while it was in transit to you.
• It provides a reasonably high level of confidence that the data you are transferring is not visible in-transit to third-parties. This is the reason it's so important to ensure that any site you provide a password or personal data to uses HTTPS.

This still leaves you open to quite a few attacks. The most obvious is for the web server to be hacked (or the CDN if they're using one). There are others too, like XSS attacks, malicious advertisements (think of all the perfectly legitimate sites out there that have hidden auto-redirects to malicious sites because of the ads they use), and attacks on TLS itself (such attacks are why no sensible operator still uses SSLv2 or SSLv3, both are known broken). So, HTTPS/TLS/SSL by itself is not rigorous protection.

Also, as you yourself said, most of the websites you visit use TLS, not all of them. Think very hard about whether you really should trust those other sites that aren't using it, if they aren't willing to take the five minutes to set their servers up with free certificates from LetsEncrypt, what other aspects of security are they cutting corners on?

Most of them are either big enough that I can trust that they can't be hacked, or small enough that I don't think it's profitable for the hackers.

Have you not watched any news for the past decade? Just in terms of public disclosures, there are dozens of known attacks on sites with well over 100 million users (that's a big site by most people's definitions, as 100 million is more than 1% of the world population, and probably closer to 3-4% of the total number of people on the internet). I suggest taking a look at the list of public breaches on Have I Been Pwned?, there are quite a few big names on there, including ones which handle very sensitive data (Experian for example). So, no, it's not realistic to expect something to be 'too big to be hacked'. In fact, big sites are some of the most attractive targets for attackers, because they have lots of users. They also have a notoriously bad track record for publicly disclosing security breaches (they have more incentive not to, as they have more potential customers to lose).

On the other side of things, small sites are easy targets, even if not attractive. If you think of big sites as high risk investments for hackers, small ones are low-risk investments. They may not give as much in the way of returns, but they often will give much more consistent returns, so they're still attractive targets.

Also, consider that quite often attackers are targeting software that is vulnerable, not necessarily specific sites. WordPress is a good example, as it's used on sites both big and small, and it's been successfully used multiple times in the past as an attack vector.

I can smell fishy websites

First off, just because a site isn't 'fishy' doesn't mean it's not a threat. There are also quite a few legitimate sites that look 'fishy' by many people's definitions.

Second, it's not hard to copy a legitimate site but still do illegitimate things with the result. Domain spoofing, in it's various forms, is often used for this. There was a rather good blog post on a big infosec site a while back (which i unfortunately cannot find right now, otherwise I would link it here) demonstrating this with apple.com.

As an example of the type of thing you need to look out for but probably can't spot, can you tell the difference between uv and υν? No, this isn't a trick question, the first one is the lower case Latin letters 'u' and 'v', while the second one is the lowercase Greek letters upsilon and nu. In most sans-serif fonts (like those used by almost all browsers in the address bar, and the default font on most SE sub-sites), those two pairs of characters are nearly indistinguishable. Even in many serif fonts, they're hard to distinguish for most people. Similarly, АВ is actually a pair of Cyrillic characters, not Latin ones, and again is indistinguishable from the Latin 'AB' in most fonts. Both cases illustrate examples of IDN homograph attacks, a technique whereby attackers take advantage of the similarity of different characters looking similar or identical to trick people into following their links by making them look like links to legitimate websites.

Pretty much, don't assume that you will be able to recognize a site that's a threat until it's too late.