The museum was founded in 1874, following the models of the Victoria and Albert Museum in London. In 1877 it moved to its current premises, a building on the Steintorplatz built from 1873–75.
Between 1919 and 1933, under the direction of Max Sauerlandt, the museum acquired a large collection of Expressionist works. After 1933, the Nazi campaign against "degenerate art" resulted in the loss of numerous contemporary works, as well as the forced departure of Sauerlandt. The building was partly destroyed by bombs in 1943; the rebuilding was finished in 1959.