Historically from Compton Wynyates in Warwickshire, the family has owned Castle Ashby since the 1500s. Castle Ashby House is now lived in by Earl Compton, the son of the 7th Marquess, whilst the Marquess himself continues the family tradition of managing the Castle Ashby and Compton Wynyates Estates.
In 1512 William Compton purchased Castle Ashby from Sir John Hussey who, six years previously, had bought the Estate from Richard, 3rd Earl of Kent. In 1574 his grandson, Henry Lord Compton pulled down the medieval Castle, described by local historian John Leland as "'now down and is made a septum for beestes", and started the present building. The house was originally built as an ‘E’ shape to celebrate the reign of Queen Elizabeth I but in 1635 the front façade was added by Inigo Jones, thus creating the courtyard.
The Latin inscription around the parapet of the house was added in 1624 and is taken from the second verse of Psalm 127. It reads:-
Except the Lord build the house they labour in vain they that build it: except the Lord keep the house, the watchman waketh but in vain.
In the Civil War the family fought on the side of the King. In 1643 Spencer, the 2nd Earl, was killed at the battle of Hopton Heath. After his death, the east front of the house was set on fire and severely damaged. The marks of the flames can still clearly be seen on the lintels of the windows.
The House boasted a succession of Royal visitors; Elizabeth I in 1600 and King James I and his Queen Anne of Denmark in 1605. At this time the household had 83 servants and 4 chaplains! In 1695 King William III also visited the Estate and was instrumental in introducing the concept of creating impressive avenues in the gardens. In 1761 Capability Brown removed those to the North and West and replaced the one to the East with clumps of trees. This left the one to the South which stretches for three and a half miles to the old deer park.
Early in the 19th century the 9th Earl and 1st Marquess did much work in the house, Britton remarking that 'it has been wholly renovated, and adapted to the comforts of refined society, by the present noble proprietor'.
In 1870-72, John Marius Wilson's Imperial Gazetteer of England and Wales described Castle Ashby like this:
ASHBY (Castle), a parish in Hardingstone district, Northampton; near the Nen river and the Peterborough railway, 7 miles E of Northampton. It has a station on the railway at White Mill, and includes the hamlet of Chadstone; and its Post Town is Grendon under Northampton. Acres, 1,926. Real property, £3,088. Pop., 183. Houses, 38. Castle-Ashby House, the seat of the Marquis of Northampton, stands within the parish, at the north end of a wide avenue of upwards of 3 miles through Yardley Chace; and is a large quadrangular edifice, with two lofty octangular towers, built in 1625-35; and contains a good picture gallery, with valuable portraits and very old oil paintings.