Resisting his father's wish that he become a minister, Johnson embraced law instead--largely by educating himself and without benefit of formal training. After admittance to the bar, he launched a practice in Stratford, representing clients from nearby New York State as well as Connecticut, and before long he established business connections with various mercantile houses in New York City. In 1749, adding to his already substantial wealth, he married Anne Beach, daughter of a local businessman. The couple was to have five daughters and six sons, but many of them died at an early age.

Johnson did not shirk the civic responsibilities of one of his station. In the 1750s he began his public career as a Connecticut militia officer. In 1761 and 1765 he served in the lower house of the colonial assembly. In 1766 and 1771 he was elected to the upper house. At the time of the Revolution, Johnson was disturbed by conflicting loyalties. Although he attended the Stamp Act Congress in 1765, moderately opposed the Townshend Duties of 1767, and believed that most British policies were unwise, he retained strong transatlantic ties and found it difficult to choose sides. Many of his friends resided in Britain; in 1765 and 1766 Oxford University conferred honorary master's and doctor's degrees upon him; he had a strong association with the Anglican Church; he acted as Connecticut's agent in Britain during the years 1767-71; and he was friendly with men such as Jared Ingersoll, Sr., who were affiliated with the British administration.

Johnson finally decided to work for peace between Britain and the colonies and to oppose the extremist Whig faction. On that basis, he refused to participate in the First Continental Congress, to which he was elected in 1774, following service as a judge of the Connecticut colonial supreme court (1772-74). When hostilities broke out, he confined his activities to peacemaking efforts. In April 1775 Connecticut sent him and another emissary to speak to British Gen. Thomas Gage about ending the bloodshed. But the time was not ripe for negotiations and they failed. Johnson fell out of favor with radical patriot elements who gained the ascendancy in Connecticut government and they no longer called upon his service. Although he was arrested in 1779 on charges of communicating with the enemy, he cleared himself and was released.

Once the passions of war had ebbed, Johnson resumed his political career. In the Continental Congress (1785-87), he was one of the most influential and popular delegates. Playing a major role in the Constitutional Convention, he missed no sessions after arriving on June 2; espoused the Connecticut Compromise; and chaired the Committee of Style, which shaped the final document. He also worked for ratification in Connecticut.

Johnson took part in the new government, in the U.S. Senate where he contributed to passage of the Judiciary Act of 1789. In 1791, the year after the government moved from New York to Philadelphia, he resigned mainly because he preferred to devote all his energies to the presidency of Columbia College (1787-1800), in New York City. During these years, he established the school on a firm basis and recruited a fine faculty.

Johnson retired from the college in 1800, a few years after his wife died, and in the same year wed Mary Brewster Beach, a relative of his first bride. They resided at his birthplace, Stratford. He died there in 1819 at the age of 92 and was buried at Old Episcopal Cemetery.