If an assessment of you is correct, then you must accept it, even if it's unflattering
Records of the expression "if the cloak fits" are found in 16th century English writing. For example, in Of the Lawes of Ecclesiasticall Polite (1593), Anglian priest Richard Hooker writes, "Which cloake sitteth no lesse fit on the backe of their cause, than of the Anabaptists."
In 1705, Daniel Defoe published a poem critical of the British parliament in which he wrote, "Gentlemen, and if the Cap fits any Body let 'em wear it."
In Britain, "if the cap fits" is still the norm; but in North America, by the late 18th century, the expression morphed into "if the shoe fits". The earliest known example being from the New York Gazette & Weekly Mercury, in 1773: "Why should Mr. Vanderbeck apply a general comparison to himself? Let those whom the shoe fits wear it."
According to phrases.org.uk, the shift from cap to shoe might have been influenced by the Cinderalla story, as versions of the story including the lost slipper were well known in the USA and Europe by 1773.