The minister for sport has put a dampener on the launch of the Football Association's 150th anniversary programme by saying it is "crazy" that the chairman, David Bernstein, will be forced to leave halfway through the year.

In the buildup to the publication of a parliamentary select committee report that is expected to be heavily critical of the slow pace of reform at the FA, Hugh Robertson said: "Nobody can think it is a sensible idea to change the chairman in the middle of that celebration."

In October the FA council voted not to change a rule that forced Bernstein to stand down at 70. But Robertson, who paid tribute to the progress made by Bernstein in his two years in the post, said there was an implicit understanding when the chairman was appointed that he would be allowed to complete a full term.

"David Bernstein is recognised as being a really good chairman of the FA," said Robertson. "People find it hard to understand why you have a good chairman who is respected and you lose him in the middle of your 150th year. I've been asked by significant figures internationally how that has been allowed to happen."

Roberston, having last year called football "the worst-governed sport in Britain", said that "some progress" had been made since then. However, the select committee report expected to be delivered within the next fortnight is likely to be critical of the pace of change and increase the pressure on the game's governing bodies to deliver promised reforms.

Bernstein is seen as having made progress in building bridges with Fifa and, in particular, Uefa, which is bringing the Champions League to Wembley for the second time in three years and will also hold its congress in London in May.

"It takes you a couple of years to get up to pace, to get a relationship with people like Platini and Blatter, to get to know people on the circuit," said Robertson. "David's just got there and should be entering his most productive period and suddenly he's going."

Bernstein, who is to become chairman of the fashion label Ted Baker, said his parting with the FA in mid-July would be sudden and that he wanted a "clean break".

"I am a great believer in clean breaks to be honest. When I left Manchester City I made a very, very clean break. I don't want to be the spectre at the feast. I'm not thinking about those things – the six months ahead are very big and I want them to go as well as the last two years."

He admitted that progress on reforming the FA's governance had been "slower than it might have been" and identified "respect issues" and making further progress on implementing its "anti-discrimination" programme as key priorities, as well as qualifying for the Brazil World Cup.

Robertson said the FA would struggle to find a successor to match Bernstein: "I don't think you are going to find another David Bernstein in a hurry. He's been exceptional. It's a very difficult job. You need a mix of skills. There are some good candidates out there that will have a go but he's going to be a hard act to follow."