The sports minister has called on football authorities and the police to "throw the book" at those involved in "disappointing" and "depressing" outbreaks of football-related violence at Wembley and on the streets of Newcastle last weekend in an attempt to ensure incidents do not snowball.

As Newcastle United and Millwall promised to ban those involved for life, Hugh Robertson said the pictures of bloodied fans fighting one another and the police threatened to damage the positive image of British sport created by the London Olympics. In addition to 29 arrests in Newcastle and 14 at Wembley as Millwall fans fought each other in full view of the TV cameras British Transport police confirmed they had made 36 football related arrests last weekend. An Everton fan was also stabbed outside Goodison Park.

"It's incredibly disappointing and quite depressing. I think it's difficult to think of another time – certainly in my adult life – that British sport has been in a stronger position than it is at the moment, in the aftermath of 2012 and with so many other major events coming to this country," said Robertson. "Against that backdrop the last thing we want are the sort of scenes we saw on Saturday at Wembley or indeed up in Newcastle on Sunday. We want to absolutely make sure this doesn't snowball and become fashionable."

In common with football supporters' groups and the police Robertson was keen to stress that he did not believe the incidents represented a return to the serious disorder that gripped the game in the 1980s. Home Office figures show the overall number of football-related arrests declined 24% last season to 2,363. But he said last weekend's events should be taken as a warning against complacency.

"Back in the 70s and 80s we used to have the reputation that this happened time and time again. If anything, what the weekend does show is that the moment you take your eye off it, it pretty quickly reappears. You never entirely put the lid on all of this. But from what I've heard, it's a sort of alcohol-inspired, warm day and that's what's done it."

The Football Association chairman, David Bernstein, met the Millwall chief executive, Andy Ambler, on Monday to discuss their next steps, amid a comprehensive review that is expected to take about a week. The FA is expected to look into claims that police took too long to intervene and will also have to answer questions about ticketing and the late kick-off time.

England's match against the Republic of Ireland on 29 May, the first time the two have played since rioting away fans forced the abandonment of a friendly at Lansdowne Road in 1995, is now likely to come under the microscope. But the FA will point to the number of big matches it has been able to police effectively at Wembley. About 10,000 tickets are believed to have been sold to Irish fans.

Newcastle United issued a strong statement condemning the rioters in the city centre. Northumbria police confirmed there had been 29 arrests on Sunday, including five before the match and eight inside the stadium during it.

"We were embarrassed and appalled by the behaviour of a minority of so-called fans who last night were involved in disturbances and disorder in Newcastle city centre after the match. The club will take the strongest possible action against those involved in the disturbances and will impose immediate lifetime bans on all those found guilty," they said.

"These deplorable individuals have no place at Newcastle United and bring shame on the club and the vast majority of its proper, law-abiding, fans."

Chief superintendent Steve Neill said that at no point did police lose control but confirmed that a group of Newcastle fans turned on officers, hurling missiles and punches, after they were stopped from reaching a group of Sunderland supporters being escorted from the ground. Four officers received treatment for minor injuries but a police horse punched by one Newcastle fan was said to be "fit and well".

Amid reports that some of the fans involved had been drinking in city centre pubs since 9am before the noon kick-off, scheduled at that time on police advice, Newcastle city council confirmed that eight pubs in the city had licences to open at that time on Sunday mornings.

Stephen Savage, Newcastle City Council's director of regulatory services and public protection, said that no additional licences had been granted and premises had been visited ahead of the match to "remind licence holders of their responsibilities".

"Venues in the city centre have different licensing hours, approximately half open at 10am, others from 11am and some from 12 noon. Eight premises can serve from 9am with food," he said.

"The new Terrace Bar at St James' Park has a license to serve four hours before kick-off with the exception of derby days, this means an 11 o'clock start in normal circumstances. It opened from 9am rather than 8am but no problems were directly associated with this premises."

The British Transport Police chief constable, Andrew Trotter, said fans from Chelsea, Manchester City, Millwall, Newcastle and Watford had been arrested for racism, assault and other offences.

"This weekend saw at least 36 football-related arrests on the rail network, and the trouble-making minority were dealt with swiftly and robustly. However, the scenes at Wembley Stadium on Saturday and in Newcastle on Sunday serve as a firm and sobering reminder that there is still much more to be done in tackling football-related disorder," he said.