In 1761 Rutledge became politically active. That year, on behalf of Christ Church Parish, he was elected to the provincial assembly and held his seat until the War for Independence. For 10 months in 1764 he temporarily held the post of provincial attorney general. When the troubles with Great Britain intensified about the time of the Stamp Act in 1765, Rutledge, who hoped to ensure continued self-government for the colonies, sought to avoid severance from the British and maintained a restrained stance. He did, however, chair a committee of the Stamp Act Congress that drew up a petition to the House of Lords.

In 1774 Rutledge was sent to the First Continental Congress, where he pursued a moderate course. After spending the next year in the Second Continental Congress, he returned to South Carolina and helped reorganize its government. In 1776 he served on the committee of safety and took part in the writing of the state constitution. That year, he also became president of the lower house of the legislature, a post he held until 1778. During this period, the new government met many stern tests.

In 1778 the conservative Rutledge, disapproving of democratic revisions in the state constitution, resigned his position. The next year, however, he was elected as governor. It was a difficult time. The British were invading South Carolina, and the military situation was desperate. Early in 1780, by which time the legislature had adjourned, Charleston was besieged. In May it fell, the American army was captured, and the British confiscated Rutledge's property. He ultimately escaped to North Carolina and set about attempting to rally forces to recover South Carolina. In 1781, aided by Gen. Nathanael Greene and a new Continental Army force, he reestablished the government. In January 1782 he resigned the governorship and took a seat in the lower house of the legislature. He never recouped the financial losses he suffered during the war.

In 1782-83 Rutledge was a delegate to the Continental Congress. He next sat on the state chancery court (1784) and again in the lower house of the legislature (1784-90). One of the most influential delegates at the Constitutional Convention, where he maintained a moderate nationalist stance and chaired the Committee of Detail, he attended all the sessions, spoke often and effectively, and served on five committees. Like his fellow South Carolina delegates, he vigorously advocated southern interests.

The new government under the Constitution soon lured Rutledge. He was a Presidential elector in 1789 and Washington then appointed him as Associate Justice of the U.S. Supreme Court, but for some reason he apparently served only a short time. In 1791 he became chief justice of the South Carolina supreme court. Four years later, Washington again appointed him to the U.S. Supreme Court, this time as Chief Justice to replace John Jay. But Rutledge's outspoken opposition to Jay's Treaty (1794), and the intermittent mental illness he had suffered from since the death of his wife in 1792, caused the Federalist-dominated Senate to reject his appointment and end his public career. Meantime, however, he had presided over one term of the Court.

Rutledge died in 1800 at the age of 60 and was interred at St. Michael's Episcopal Church in Charleston.