Joshua Reynolds, 1784

In 1789 the South and especially Virginia had been the impelling force in creating the nation. By 1815 the South and slaveholders still seemed to be in control of the national government. President Madison was a slaveholder. So too were Speaker of the House, Henry Clay, James Monroe, the secretary of state, and George W. Campbell, the secretary of the treasury. All Republican leaders of the House were slaveholders. In 1815 the United States had four missions in Europe: two of them were held by slaveholders. The chief justice of the United States was a slaveholder, as were a majority of the other members of the Court. Since 1789 three of the four presidents, two of the five vice-presidents, fourteen of the twenty-six presidents pro tempore the Senate, and five of the ten Speakers of the House had been slaveholders.

Gordon S. Wood
Empire of Liberty: A History of the Early Republic, 1789–1815 (2009)