But Arsenal's pressure in the final half hour turned the game back their way and Aaron Ramsey equalised with seven minutes remaining before Giroud headed the 85th minute victor.
Giroud's compatriot Alexandre Lacazette marked his league debut with a goal in just 94 seconds, only for Shinji Okazaki to level for the Foxes less than three minutes later. Substitutes: Olivier Giroud (for Holding, 67) 7, Aaron Ramsey (for Elneny, 67) 7, Theo Walcott (for Welbeck, 75) 6.
"Going forward and character wise we have been great", said a relieved Arsenal manager Arsene Wenger, who will be seeking a first league title since 2004 after being handed another two years in charge.
Wenger lauds Giroud as Arsenal beat Leicester 4-3
Jamie Vardy then struck twice either side of Welbeck's equaliser as the Foxes looked set for a first league win over Arsenal since 1983.
"Giroud? He's fantastic", Wenger said of the France worldwide, who usually started on the bench last season and had been linked with a move away from The Emirates. "He made a decision to stay and it was one of the great days of recent weeks for me".
"I don't know whether it was a lack of concentration or just a lack of game management in the last ten minutes, but you then concede again from a set play in terms of being overpowered". Giroud had been linked with an exit during the summer, and Wenger admitted he considered allowing his most authentic target man to leave but the 30-year-old resisted the idea. "But he's really loved here, we love him, and in the end he's chose to stay".
"If you look at the third goal, I'm a bit disappointed with the handball leading up to it but that aside, once you get the corner, you have to defend it better".