Joakim Nilsson, Rikard Edlund (both formerly of Norrsken), Axel Sjöberg, and Truls Mörck formed Graveyard in 2006. When Norrsken disbanded in 2000, guitarist Magnus Pelander went on to form doomy folk metal act Witchcraft and Nilsson and Edlund joined Albatross, a growly blues rock band whose lineup also included Sjöberg on drums. Initially they considered Albatross a hobby project, but after five years the members began taking their music more seriously and had grown dissatisfied with the direction their sound had taken. When Albatross broke up, Nilsson and Edlund decided that for their next venture, they would head back towards their roots as musicians and songwriters. Nilsson explains, “I am a singer, but in Albatross I only played guitar. Rikard played guitar but he is a bass player. We also wanted a more straightforward [rock] sound.”
Together with Sjöberg and guitarist/singer Truls Mörck they began practicing as Graveyard. Upon their initial formation, Graveyard quickly recorded a two track demo, played a total of three shows, and began planning for a full-length album with Swedish label, Transubstans Records. In the meantime, they posted a demo of some of their material on MySpace. Songs from the band's Myspace caught the ear of Tony Presedo, founder of TeePee Records. Their self-titled debut album was recorded by Don Ahlsterberg and released in early 2008. On the recording sessions' completion, Truls Mörck was replaced by guitarist Jonatan Ramm. The debut album received generally good reviews. Graveyard performed at the 2008'sSouth by Southwest Music Festival.
Graveyard was featured on Rolling Stone Magazine's Fricke's Picks for their first ever US gig at SXSW. After SXSW, Graveyard toured with label mates Witch. In the Fall of 2008 Graveyard toured with Witchcraft and then Clutch. In 2009 they toured with the rock band CKY. Graveyard released their second album in spring of 2011.
In 2013 they toured Europe with grunge band Soundgarden.