In this article, I discuss Wittgenstein’s conception of music, musical understanding and the sense of comparing music to language. I argue that for Wittgenstein, musical understanding is describable as a specific kind of experience that is public and sharable. I then reject any formalist view, which asserts that musical understanding is exclusively an ability to follow a set of established rules. Second, I illustrate the scope of Wittgenstein’s comparison between languages and posit that music is useful for clarifying the concept of linguistic understanding in the case of certain specific language-games, such as making jokes or puns. I will finally show that no thesis on the nature of language parallels and follows such a comparison.