Leviticus 16
“Aaron is to offer the bull for his own sin offering to make atonement for himself and his household. Then he is to take the two goats and present them before the Lord at the entrance to the tent of meeting. He is to cast lots for the two goats—one lot for the Lord and the other for the scapegoat. Aaron shall bring the goat whose lot falls to the Lord and sacrifice it for a sin offering. But the goat chosen by lot as the scapegoat shall be presented alive before the Lord to be used for making atonement by sending it into the wilderness as a scapegoat.” Leviticus 16:6-10 NIV
This is where we get the term “scapegoat.” This ritual was repeated yearly on the Day of Atonement: One goat was chosen for the Lord and was sacrificed in order to have the sins of the Israelites forgiven. Another goat was chosen to be released into the desert in order to remove the guilt of the Israelites for the sins that they committed. Note the contrast between the imperfect and temporary sacrifice of a goat for the Israelite people, and Christ, the one perfect and final sacrifice for all mankind.