The main reason you would want to do this is to prevent your machine from making reverse shell connections. A lot of attacks involve targeting a vulnerable web application that has a remote code execution vulnerability. If this attack was successful, the attacker could then download remote content to your machine or make a reverse shell connection, giving them shell access to your machine.
By blocking all outgoing traffic, you can prevent them from gaining remote access to your machine.
To do this with UFW:
You can then allow individual outgoing ports if necessary. For example, you'll probably want to allow DNS queries to your name servers.ufw default deny outgoing
ufw allow out to 126.96.36.199 port 53
Where 188.8.131.52 is the address of the name server. This allows traffic from any local IP on the machine to your nameserver on port 53.