Samuel was born in Hamilton, Victoria, the son of Maree and Clifford Samuel. He grew up in Adelaide, South Australia and graduated from Rostrevor College in 2001. He has a younger brother, Benedict, a writer, producer and actor, as well as an older sister, Bridget, a stage manager. Despite completing his senior secondary years at Rostrevor College, Samuel undertook final year drama at Christian Brothers College under the tutelage of Amanda Portus, where he played the part of Tom Snout (the wall) in Rostrevor Colleges production ofShakespeares A Midsummer Nights Dream as well as playing "Belvile" in CBCs production of Aphra Behns The Rover (The Banished Cavaliers). Xavier Samuel later attended Flinders University Drama Centre in 2005, where he studied under renowned acting teacher Professor Julie Holledge. He played Hamlet in the graduate production.