What are the risks of just clearing cookies instead of logging off?
Can I just log out by wiping cookies instead of hitting logout?
Frequently yes, for the reasons you supplied in your question: Without the session token in your cookies, a typical web application won't know who you are.
What are the issues of just wiping cookies versus clicking the logout button?
Web applications that manage authentication following the OWASP session management guidelines will invalidate the session on the server side when you log out explicitly. If you simply discard the cookie with the session token, the session may be subject to session hijacking.
Using door locks as an analogy for those who are not familiar with best practices of developing web applications (thanks to discussion in the comments):
Your account can be seen as a room in a building. When you log in, the building's owner creates a door and puts an automatic lock on it, so that only you can enter. Your session token is your key, and is typically stored in your browser's cookies, but can be stored in other places.
Discarding your token by deleting your cookies, clearing cache, etc., is simply destroying your copy of the key.
Explicitly logging off is asking the building owner to brick up the doorway. There's nothing guaranteeing that they'll secure your account, but as the user, you're explicitly making your wishes known.
There are various ways that an attacker can get a copy of your key, known as session hijacking, that are the responsibility of the site owner to mitigate, not the users.
First, the attacker can just guess. If the site generates session keys sequentially, or uses a low entropy pseudorandom generation method, this makes guessing much easier. Sites mitigate this through using high entropy tokens and periodic session recycling. Recycling sessions doesn't prevent access, but it makes it obvious when unauthorized access has been granted.
Second, the attacker can use session fixation: They give you a key before you log in, which you continue to use after you've logged in. Sites mitigate this by explicitly recycling the session when you log in.
Third, a Man-in-the-Middle attack. The attacker can see your key directly. TLS mitigates this. It is possible to decrypt TLS traffic through downgrade attacks, insecure implementations, and zero-day attacks, but these are far outside a user's domain, rare, and zero-day attacks against TLS tend to raise a LOT of noise when they're discovered (Heartbleed, et al).
As a user, your responsibilities are to log off and to hold the site accountable when they take shortcuts with security, much as your responsibility with your car in a public parking lot is to lock your doors. If the door locks are trivially bypassed, then its the manufacturer's fault, not yours.
Some websites make use of non cookie based methods of storing data on a users computer, such as HTML5 local storage and flash player cache that may contain sensitive information.
For example your email client may store your draft email in local storage so that in the event you accidentally refresh the page/close the browser you will be able to resume where you left off. This sensitive data will normally be removed when you click the logout button on a well designed website and would not be removed by merely deleting your cookies.
Can I just log out by wiping cookies instead of hitting logout?
Yes, since the web application uses cookies to uniquely identify you,deleting cookies will log you out.
What are the issues of just wiping cookies versus clicking the logout button?
The logout button serves a special purpose in that it sends a request to delete the session and returns a response to delete the cookie in your browser as well. If you don't send a request to delete the session then the session still remains alive server side. Sessions do have a maximum lifetime and will be deleted by the garbage collector eventually.