Conservation experts have raised concerns about Richard Hammond's plans for his country home in Herefordshire.
Historic England says it has problems with some of the changes at Bollitree Castle in Weston Under Penyard on "heritage grounds".
The TV presenter wants to add five bedrooms to the listed building, which dates back to the 15th Century.
He also wants to build a glazed extension to link a dining room to a showroom for his motorbike collection.
Neighbours and conservation chiefs have made no objections to the plan.
The Hamster and his family moved into the estate in 2008.
Natural causes verdict in death of newborn at Grange Hospital
Council to buy land to fix road cut off by landslip
Neighbouring towns to Battle for UK Culture crown
Wales–England bridge now closed to all access
Wye Valley bridge closure sparks fear residents are being ‘fobbed off’