Alexander attended high school and graduated in 1977, moving on to Boston University. He never completed his tenure at the university, however, leaving in his senior year to take on work in New York City. He would later earn an honorary degree from his alma mater.

Alexander began acting in New York soon after discovering that his original passion, that of a magician, would not earn him the kind of success or money he craved as an entertainer. He proved an adept actor, however, appearing on Broadway several times in such productions as Merrily We Roll Along and Broadway Bound. He continues to appear on the stage, his most recent appearances in The Producers with Martin Short and A Christmas Carol with Kelsey Grammer.

Alexander's real success, however, came on the television. And by far his best role, perhaps one that has spoiled his other attempts at television success, was that of George Costanza on the hit sitcom Seinfeld. Originally cast as George in 1989, Alexander was a key player in the show's three-man and one woman cast for all nine seasons of the show. George proved a fan-favorite, an angry and somewhat hopeless man with a penchant for getting himself into trouble.

While starring in Seinfeld Alexander also had a number of other successful television spots, including starring in the animated show Duckman and having a recurring spot on the sitcom Dinosaurs!. He also lent his voice to the animated version of the cartoon strip Dilbert, though it didn't last very long. Alexander has also had a few film appearances, perhaps most notably when he appeared in the Julia Roberts and Richard Gere as Gere's slimy lawyer.

Unfortunately post-Seinfeld Alexander has not fared so well, and for a long time was practically a poster-boy for the Seinfeld curse, a popular belief that the stars of Seinfled - outside Jerry Seinfeld himself - would encounter little success in independent endeavors. This belief seemingly ran true, as Alexander starred in three shows that all folded quickly after their initial episodes: E/R, Bob Patterson and Listen Up!. Listen Up! was the longest lasting, though it didn't manage to run into a second season.

Alexander has since appeared in bit spots in a number of shows, including Friends, Malcolm in the Middle, Star Trek: Voyager, The Twilight Zone and Monk. He was also responsible for hosting the Comedy Central roast of veteran actor William Shatner and has appeared twice on the celebrity poker showdown, managing to win $500'000 for the United Way of New Orleans.