Goldsworthy began working on the Garden of Stones in late 2002. In the winter and early spring of 2003, he traveled to forests and quarries in the northeastern United States seeking out suitable boulders, which he located in Barre, Vermont. Searching for boulders that were free of flaws, Goldsworthy selected stones that range in size and physical character. He noted that "there is an energy within a group of stones of various sizes. It becomes a family." The smallest stone is three tons, while the largest weighs more than 13 tons.
Most of the boulders he selected had been removed from nearby farmlands hundreds of years ago, something that appealed to Goldsworthy since there was a tradition of human involvement with the stone. "My working of the stones is a continuation of the journey these stones have made. They have a history of movement, struggle, and change which I hope will resonate with the garden." He chose to include eighteen boulders in part because of the number's symbolic significance: In Hebrew every letter also possesses a number value. Chai, whose number value is 18, is the Hebrew word for life, and is known to many in the traditional toast "L'chaim" - to life!
In all of Goldsworthy's works, the process of selecting and becoming familiar with the natural materials he uses is a key element of the work itself. For Garden of Stones, he researched several different methods of hollowing the stones, including coring, water jet cutting, and burning with a flame torch. "I rarely repeat a work twice, so each work is a step into the unknown," he has said. He chose the flame torch method, in part because it was the most efficient, but also because granite is a fire-formed stone: Goldsworthy saw an affinity between the way the stone came into being and the way in which it became part of Garden of Stones. This spectacular technique melted away the interior of the stones, transforming solid granite into molten liquid.
For the trees, Goldsworthy selected a species of dwarf oak, Quercus prinoides. The trees will begin as small saplings, and over the course of decades will grow to be around 12 feet tall. "Amidst the mass of stone the trees will appear as fragile, vulnerable flickers of life - an expression of hope for the future. The stones are not mere containers. The partnership between tree and stone will be stronger for having grown from the stone."