Whitney Castle was once located along the banks of the River Wye in modern Herefordshire, but was for most of its years located in an area called The Marches, the border region between England and Wales. The original castle was perhaps of the Motte-and-bailey style, common to this area. The original castle was burned and destroyed in 1402 during a Welsh uprising led by Owen Glendower.
In 1404, King Henry IV granted Clifford Castle to Robert Whitney because "his property has been burnt and destroyed by our rebels of Wales, so that the same Robert has not any castle or fortress where he can tarry to resist and punish our aforesaid rebels".
Whitney Castle was rebuilt, but was destroyed again, this time washed away by the River Wye, perhaps in 1730 when it changed course. By 1754, the castle was described as 'demolished'.
The earliest mention of the location Whitney is in the Domesday Book, created in 1086. It appears that then scarcely any of the land was under cultivation.
- In Elsedune hund., Rex tenet Witenie. Aluuard tenuit tempora Regis Edwardi et poterat ire quo volebat. Ibi dimid hida geld. Wasta fuit et est.
- In Elsedune hundred, the King holds Witenie. Aluuard held it in the time of King Edward, and was able to go where he pleased. There is half a hide yielding geld. It was and is waste.