Between 1774 and 1776 Gerry attended the first and second provincial congresses. He served with Samuel Adams and John Hancock on the council of safety and, as chairman of the committee of supply (a job for which his merchant background ideally suited him) wherein he raised troops and dealt with military logistics. On the night of April 18, 1775

Gerry attended a meeting of the council of safety at an inn in Menotomy (Arlington), between Cambridge and Lexington, and barely escaped the British troops marching on Lexington and Concord.

In 1776 Gerry entered the Continental Congress, where his congressional specialities were military and financial matters. In Congress and throughout his career his actions often appeared contradictory. He earned the nickname "soldiers' friend" for his advocacy of better pay and equipment, yet he vacillated on the issue of pensions. Despite his disapproval of standing armies, he recommended long-term enlistments.

Until 1779 Gerry sat on and sometimes presided over the congressional board that regulated Continental finances. After a quarrel over the price schedule for suppliers, Gerry, himself a supplier, walked out of Congress. Although nominally a member, he did not reappear for 3 years. During the interim, he engaged in trade and privateering and served in the lower house of the Massachusetts legislature.

As a representative in Congress in the years 1783-85, Gerry numbered among those who had possessed talent as Revolutionary agitators and wartime leaders but who could not effectually cope with the painstaking task of stabilizing the national government. He was experienced and conscientious but created many enemies with his lack of humor, suspicion of the motives of others, and obsessive fear of political and military tyranny. In 1786, the year after leaving Congress, he retired from business, married Ann Thompson, and took a seat in the state legislature.

Gerry was one of the most vocal delegates at the Constitutional Convention of 1787. He presided as chairman of the committee that produced the Great Compromise but disliked the compromise itself. He antagonized nearly everyone by his inconsistency and, according to a colleague, "objected to everything he did not propose." At first an advocate of a strong central government, Gerry ultimately rejected and refused to sign the Constitution because it lacked a bill of rights and because he deemed it a threat to republicanism. He led the drive against ratification in Massachusetts and denounced the document as "full of vices." Among the vices, he listed inadequate representation of the people, dangerously ambiguous legislative powers, the blending of the executive and the legislative, and the danger of an oppressive judiciary. Gerry did see some merit in the Constitution, though, and believed that its flaws could be remedied through amendments. In 1789, after he announced his intention to support the Constitution, he was elected to the First Congress where, to the chagrin of the Antifederalists, he championed Federalist policies.

Gerry left Congress for the last time in 1793 and retired for 4 years. During this period he came to mistrust the aims of the Federalists, particularly their attempts to nurture an alliance with Britain, and sided with the pro-French Democratic-Republicans. In 1797 President John Adams appointed him as the only non-Federalist member of a three-man commission charged with negotiating a reconciliation with France, which was on the brink of war with the United States. During the ensuing XYZ affair (1797-98), Gerry tarnished his reputation. Talleyrand, the French foreign minister, led him to believe that his presence in France would prevent war, and Gerry lingered on long after the departure of John Marshall and Charles Cotesworth Pinckney, the two other commissioners. Finally, the embarrassed Adams recalled him, and Gerry met severe censure from the Federalists upon his return.

In 1800-1803 Gerry, never very popular among the Massachusetts electorate because of his aristocratic haughtiness, met defeat in four bids for the Massachusetts governorship but finally triumphed in 1810. Near the end of his two terms, scarred by partisan controversy, the Democratic-Republicans passed a redistricting measure to ensure their domination of the state senate. In response, the Federalists heaped ridicule on Gerry and coined the pun "gerrymander" to describe the salamander-like shape of one of the redistricted areas.

Despite his advanced age, frail health, and the threat of poverty brought on by neglect of personal affairs, Gerry served as James Madison's Vice President in 1813. In the fall of 1814, the 70-year old politician collapsed on his way to the Senate and died. He left his wife, who was to live until 1849, the last surviving widow of a signer of the Declaration of Independence, as well as three sons and four daughters. Gerry is buried in Congressional Cemetery at Washington, DC.