"It's like you are an outsider – you feel like a Gooner, you feel part of that football family. But then you hear homophobic chanting or abuse and suddenly you feel like you don't belong," says Dave Raval, a professional referee and loyal Arsenal fan, who also happens to be gay.

After years of hearing homophobic abuse and chanting dismissed as "banter" in football grounds across the country, he and hundreds like him, have decided to take matters into their own hands.

Frustrated with waiting for football's governing bodies to face up to homophobia in football grounds, Raval is a founding member of Gay Gooners, part of a football-wide movement to create lesbian, gay, bisexual, and transgender (LGBT) fan clubs and tackle prejudices. "We can now stand up and say we are proud to be Gooners and proud to be gay – that is a remarkable transformation," he says.

Gay Gooners was the first LGBT football fan group to be set up, in 2012, followed by Norwich City's Proud Canaries and Manchester City's Canal Street Blues in 2013 and, earlier this year, Tottenham Hotspur Proud Lilywhites.

Similar fan groups have been formed at half a dozen other clubs, including Liverpool, Everton and West Ham and are lobbying for official recognition.

"New groups are forming all the time," said Leviathen Hendricks, the fan group co-ordinator at the Gay Football Supporters Network. "There is huge shift going on and it's a movement that is really gaining momentum."

"Clubs are really beginning to embrace their gay fans. Recently, Spurs included a page in a matchday programme about their LGBT fan group – and a lifelong Spurs fan said he was overwhelmed to be welcomed like that. He had always thought he had his gay life and his football life – and the two could never meet."

Christina Paouros, also a member of the Proud Lilywhites ,calls homophobia "the last bastion of hate on the terraces", adding: "It's hard enough being a female football fan, never mind being a lesbian, but I think the new fan groups are going an enormous way towards changing that."

Paul Amann, a member of the Liverpool group and the LGBT rep on the Liverpool supporters' committee, believes groups have formed because of lack of commitment from the Football Association to take the matter seriously.

"As a mixed heritage fan who is also gay, I find it really offensive that the FA will, rightly, punish racism strongly but when they see homophobia, it's just a tap on the wrist," he says. "What we are seeing is LGBT fans taking control of the situation."

It is an accusation that Danny Lynch, anti-discrimination spokesman at the FA, rejects. But he admits that football has been slow off the mark. "It perhaps has taken longer for the nettle to be grasped in comparison to, say, racism," he said.

"Where we are now with homophobia is not exactly where we were with racism 20 years ago, but it's fair to say it isn't as developed as an agenda. Now I think there is an appetite to change things that hasn't been seen previously. But the FA takes every form of discrimination just as seriously."

Lynch cited a variety of FA programmes designed to combat discrimination including Opening Doors, the FA's action plan for wider inclusion of LGBT people in football, and the continuing work of Kick It Out, which uses footballers as role models speaking out against prejudice.

Abuse remains but there are signs it is being taken more seriously. German club Bayern Munich faces disciplinary action after fans unfurled a homophobic banner during a clash with Arsenal in March. And former Blackburn Rovers player Colin Kazim-Richards last week became the first footballer to be convicted of making a homophobic gesture during a match at Brighton and Hove Albion FC after receiving abuse from fans of his former club. The club has been a prime target for homophobic abuse – a recent report said its fans were abused at 72% of games last season. Lynch says the FA had taken steps to tackle the issue, including writing to visiting clubs before matches.

Dan Tickner, the founder of Tackle – a blog about homophobia in football – believes football is coming out of "the dark age" of the 1990s. He says the suicide of Justin Fashanu, the first top flight footballer to come out as gay, and overt homophobia from players such as Robbie Fowler, who notoriously taunted straight footballer Graeme Le Saux for being gay, ushered in a decade of fear. "I started the blog because, even in 2010, everyone was too scared to talk about being gay – the authorities, the fans, the clubs. I wanted to burst that bubble of fear," he says.

Fowler, who has publicly apologised to Le Saux, stated earlier this year that a Premier League star would not have a problem if they came out.In January, former Aston Villa and Germany star Thomas Hitzlsperger, became the highest-profile footballer to announce he was gay.

For Tickner, though, the real acceptance will come when there is no need to have specific fan groups for LGBT fans.