The chorus to 'By the Rivers of Babylon' was lifted, pretty much word for word, from Psalm 137 from the Old Testament.
The Psalm itself refers to the exile of of the Jews after the Babylonian conquest of Jerusalem in 586 BC & was probably written within 100 years of those events, making those lyrics around 2,500 years old.
Turn,Turn,Turn by Pete Seeger has a similar Old Testament pedigree.