how to set state array using react hooks

More readable and cleaner solution it would be:

Create a variable that holds a copy of the actual state:

If state is an array and you need to add an element in it

const newState = [...messages, 'Hi buddy'];
setMessages(newState);
 
or

setMessages(prevState => [...prevState, "Hi Buddy"]);

If state is an object and you need to update a property in it

const newState = Object.assign({}, message, {name: 'Michael Scott'});
setMessages(newState);

or

setMessages(prevState => {...prevState, name: "Michael Scott" });

Your state is an array so you will need to spread your previous state into a new array and add the new message using [...prevState, newMessage]

What you try to do is return an object, because {} is a code block so you need to wrap it inside () if you return an object which is what you are trying to do

setMessages(prevState => [...prevState, newMessage])

To insert new element at the end of the list

const addMessage = (newMessage) => setMessages(oldMessages => [...oldMessages, newMessage])

To insert new element at the begining of the list

const addMessage = (newMessage) => setMessages(oldMessages => [newMessage, ...oldMessages])

There is no real need to use the prevState, you could just do:

setMessages([...messages, newMessage])