This article argues that a 1931 text by Wittgenstein discussing Beethoven, philosophy, and cultural history sheds a unique light on both Wittgenstein's later philosophy and the relationship between music and philosophical thought more broadly. The first part of the essay claims that in writing about Beethoven Wittgenstein is writing about his philosophical self as well, not only touching on what he believed to be the relationship between his project and Beethoven's, but also trying to identify the path for his later philosophy within his own cultural and historical context. The second part brings the insights of this minor text to bear on both the controversial “rule-following” paradox from Philosophical Investigations and the role of skepticism in Wittgenstein's late philosophy. I suggest two related analogies, or what Wittgenstein himself might have called “family resemblances.” First, skepticism in Wittgenstein's later work resembles dissonance in one of Beethoven's most profoundly historical works, Symphony No. 3 in E flat “Eroica.” Second, and by extension, the overall structure of argumentation in Part 1 of Philosophical Investigations resembles Beethoven's version of the sonata form. In explicating the rule-following paradox and the symphony, I assess the methodologies of prominent critical commentaries on Wittgenstein and Beethoven, in particular those of philosopher Saul Kripke and musicologist Richard Taruskin, whose approaches attempt to dramatize the process of reading a text or listening to a piece “as if” for the first time. Such an approach can refresh or enliven our understanding of a familiar work; it can also leave a difficult text or musical composition seeming stranger than before. Kripke's commentary on Wittgenstein and Taruskin's on Beethoven provide extreme cases of both the happy and more problematic possibilities of such an approach.