Conservation experts have raised concerns about Richard Hammond's plans for his country home in Herefordshire.
Historic England says it has problems with some of the changes at Bollitree Castle in Weston Under Penyard on "heritage grounds".
The TV presenter wants to add five bedrooms to the listed building, which dates back to the 15th Century.
He also wants to build a glazed extension to link a dining room to a showroom for his motorbike collection.
Neighbours and conservation chiefs have made no objections to the plan.
The Hamster and his family moved into the estate in 2008.
Future of Cleobury Sports Centre secured until 2027
Glamping cabins allowed after appeal
Holiday park plans refused
Rent to rise by £400 per month at Ludlow council house
Revealed: The main roads in Herefordshire prioritised for repairs
Villagers fustrated over lack of road repairs
New Food and Drink group launched at online event