They currently don't drop anything other than XP orbs when killed and they make fantastic pets. They don't drop any items at the moment, but future updates are planned to add at least one: goat horns.
How to get a goat horn in Minecraft Bedrock EditionWhile the future of Minecraft has goat horns for everyone, only certain Bedrock Edition players can even get access to it.
Goats will only drop goat horns when they ram into a solid block, so there's still not a lot of reason to kill them. When they do hit the block, up to two horns will drop from an adult goat.
Goats will ram every so often, roughly 30 seconds to five minutes apart. Players need to pay attention and try to get a solid block in front of them when this is happening
There's currently only one use for goat horns and it's musical. Using the goat horn will result in players blowing it, which sounds exactly like the horn in raids.
Mojang Gameplay developer Ulraf did confirm, however, that more features are planned for the goat horn. What those will be remains unclear at this point, and it's also not clear exactly when they'll arrive in all versions of the game.