Conservation experts have raised concerns about Richard Hammond's plans for his country home in Herefordshire.
Historic England says it has problems with some of the changes at Bollitree Castle in Weston Under Penyard on "heritage grounds".
The TV presenter wants to add five bedrooms to the listed building, which dates back to the 15th Century.
He also wants to build a glazed extension to link a dining room to a showroom for his motorbike collection.
Neighbours and conservation chiefs have made no objections to the plan.
The Hamster and his family moved into the estate in 2008.
Herefordshire MP remains flatly opposed to bypass project
Court blocks Ledbury Lidl plan
Wales–England bridge now closed to all access
Hereford bypass ‘will boost nature and respect landscape’
Holiday park plans refused
‘Shires’ authority plan to boost growth between Birmingham and Bristol
Shop staying open later ‘deters ASB’
Wye Valley bridge closure sparks fear residents are being ‘fobbed off’