He met Madeline Keltie, an opera singer 24 years his junior, in 1924 at the American embassy in Rome, Italy. In May 1925 Harmon was involved in a fistfight in Monte Carlo with Arthur St. John over St. Johns unwanted advances towards Keltie. Keltie and Harmon were engaged to be married in July 1925; the marriage was postponed in November 1925 and the engagement was broken in early 1927. Later in 1927, Clifford Harmon was best man at the marriage of Pola Negri and David M'divani in Paris.

In 1930, Harmon moved to Cannes, France, taking up residence in the Hotel Martinez. In 1932, Harmon accidentally drank a quantity of disinfectant, mistaking it for medication. He quickly drank a large quantity of hot water, diluting it and preventing a fatal overdose.

Harmon suffered a severe stroke in 1939 and was effectively an invalid afterwards. He resided in Cannes during the Nazi occupation in World War II, but due to his condition German forces did not pursue internment or other confinement.

Death Clifford Harmon died on June 25, 1945, in Cannes, France. In his will he left $48,431 for the continuation of the Harmon Trophy. He was interred the Rhone American Cemetery and Memorial, Draguignan, Departement du Var, Provence-Alpes-Côte d'Azur, France.

Legacy Per stipulations in the agreements related to developing Harmon-on-Hudson, the Metro-North Railroad station Croton-Harmon is named after Harmon. When the hamlet of Harmon was annexed by the village of Croton in 1930, the "Harmon" designation was preserved in the name of the Croton-Harmon School District.