Arsenal reached the fifth round of the FA Cup with a breathless, boisterously fought and occasionally rather uneven 3-2 defeat of Brighton. In the event, Arsène Wenger will have been hugely relieved to see Theo Walcott's deflected 85th-minute winner ripple the Brighton net after a performance of both pluck and craft from the home team. Brighton twice equalised Olivier Giroud's brace of goals and at times stretched to the limits a weakened – some would say additionally weakened – Arsenal starting XI.

"Brighton is a good team and they made it very difficult for us," the Arsenal manager said. "They play a very positive and technical game and they had a great spirit. Especially on the right flank they gave us many problems."

Wenger is nothing if not goatishly unswerving in his convictions. He might have been tempted here to eschew the traditional weakening of his team against opposition from a lower division, but this was a selection that veered between brave and a little reckless. Tomas Rosicky was back in midfield, playing just ahead of his fellow long-term convalescent Abou Diaby, and André Santos, the Premier League's most notable "false three", continued at left-back against opposition that would target him mercilessly throughout another uncomfortable afternoon.

The Amex is an arched open bowl of a new-build stadium, not so much a symbol of the usual upwardly mobile aspirations as simply a welcome place for this most peripatetic of clubs to finally rest its head. With Arsenal's fans given an end-and-a-bit and the rest of the stadium packed with blue and white there was a slightly fevered atmosphere from the start. It was Brighton who had the first chance, after 14 minutes, Ashley Barnes seeing his shot saved by Wojciech Szczesny after Leonardo Ulloa's flick found him unmarked in the penalty area.

With the Amex still collecting itself, Arsenal opened the scoring. It was a beautiful finish out of nothing, Lukas Podolski cutting in from the left and then leaving the ball in Oliver Giroud's path on the edge of the area. The Frenchman placed it in the top corner with the minimum of back-lift.

Brighton were unbowed. With 30 minutes gone Will Buckley skipped past Santos with ease – as he did for most of the afternoon – and crossed low for Ulloa, newly arrived from Almería of the Spanish second division, to put the ball in the net from an offside position. Naturally enough it was from that side that the equaliser came on 32 minutes.

Buckley, who by this stage had Santos pretty much dangling from his back pocket each time he gambolled down the right touchline, forced a corner that was taken short. With Arsenal dithering Barnes met the cross unopposed and power his header past Szczesny. The Amex erupted. For Arsenal it was another moment of alarmingly flaccid central defending. With Brighton's captain, Gordon Greer, forced off injured Brighton lost a little of their momentum, but it was still an energetic first 45 minutes for the Championship team with Ramsey, Rosicky and Diaby out-ferreted in midfield by Dean Hammond, David López and the outstanding Liam Bridcutt.

After half-time Arsenal pressed towards their own fans with renewed vigour, Podolski smacking the bar on 54 minutes with a flighted free-kick. Giroud's second goal came two minutes later, bringing with it shades of the now defunct Alex Song-Robin van Persie axis of last season. Diaby dinked a lovely pass forward into space. Giroud caught the ball on his right foot and finished smartly.

Still Brighton fought back. With 62 minutes gone Barnes found some pace on the right to swing in a wonderful cross and Ulloa stooped to head home and make it 2-2. On the touchline Gus Poyet, the Brighton manager, could barely contain himself. The FA Cup does tend to do this. Journeyman debutant signed just 10 days ago? Of course he's going to score.

It was cavalry time for Arsenal. With 22 minutes left Wenger brought on Theo Walcott and Jack Wilshere, left out here to protect his niggly ankle. Wilshere again looked by far Arsenal's brightest midfielder and inevitably it was from his corner that the winner came. Casper Ankergren punched out to where Walcott had taken up his "Sweden" position on the edge of the penalty area and his measured shot was deflected in off Adam El-Abd.

Brighton will feel unlucky to lose here, but in the end the quality of Arsenal's substitutes – and the belated appearance of the first team – made the difference.

Wenger was quick to praise Giroud, while also suggesting striking reinforcements are far from imminent. "He's scored four goals this week and he had several other chances today. He's powerful, he's dangerous, his physical presence is very important to us. Quality wise we don't need a new striker. Number wise we could be a bit short. But you do not find them in the street, strikers who can play for us and strengthen our team."