Why do we need subnet mask?

We need a subnet mask for IPv4 addresses because the address doesn't give any information on the network size. Class sizes are not the network sizes. In practical networks all IPv4 networks are broken up into subnets that are smaller than the class size.

For example you could break up the class C network 200.200.200.0/255.255.255.0 into two smaller networks (potentially at separate locations) 200.200.200.0/255.255.255.128 and 200.200.200.128/255.255.255.128 assuming neither needed more than 126 hosts. In reality most companies only get enough IPv4 addresses for the servers that need to be on the public Internet. I've personally seen set ups with 32,16, & 8 address networks (that would be masks of 255.255.255.224, 255.255.255.240, & 255.255.255.248 respectively)

Having IP networks only in class size blocks was too restrictive in limited the number of networks that could be allowed - the 127 class A networks taking half of the space. Not to mention that having a 24 billion node network is completely unmanageable Instead in 1993 Classless Inter-Domain Routing (CIDR) was introduced to allow the networks to be split up.

Also to be clear the purpose of the subnet mask is to determine which hosts are on the local network and which are outside of the network. Hosts can talk directly to hosts on the same network, but they need to communicate with a router to talk to hosts on external networks.

The 1st Octet already specify the network class (1-127: A, 128-191: B, 192-223: C etc.). A, B, or C implies the number of octets for network (respectively, 255.0.0.0, 255.255.0.0, 255.255.255.0), which automatically tells you how many hosts is allowed for each class of network.

Right, but if someone were to subnet that network, you'd need the subnet mask to know how big a subnet you were in. Yes, with classful addressing, the class tells you the size of the network and allows you to tell whether a host is in the same network as you, but if that network is subnetted, without the subnet mask, how would you know whether another node is in the same subnet as you?

Say you're on an Ethernet network. We use classful addressing with subnetting. Your IP address is 1.2.3.4 and you want to reach 1.3.1.1. Do you use ARP to reach that address? Well, it depends on whether 1.2.3.4 and 1.3.1.1 are in the same subnet. Even if they're in the same network, if there in different subnets, a router needs to be used. If they're in the same subnet, then ARP should be used.

So you need the subnet mask if subnetting is in use, even with classful networks.

I think you're confusing subnetting with CIDR, actually. Without CIDR, even with subnetting, you don't need the subnet mask between administrative regions. But you still need it inside the network!

A subnet mask is used to do a bit wise operations on an IP address, in conjunction with a network address. If my memory serves me well, you take an IP address and do a bit wise AND on it and the subnet mask for a given network. If the result equals the network address, then the IP address is on that particular network. Routers that have routing tables of network addresses and subnet masks can use simple binary maths (which is very fast, if not the fastest for computers to handle) to find out which interface to punt a packet out of.