Watch highlights, interviews and more video from Paris right here on wtatennis.com!
With her trademark combination of aggression and consistency, Azarenka gave Williams all kinds of trouble for the first half of the match, taking the first set, 6-3, then breaking early en route to leads of 3-1 and 4-2 in the second. It looked like Williams' 16-match Grand Slam winning streak was about to end.
The No.1-seeded Williams rifled one last forehand to beat the No.27-seeded Azarenka, 3-6, 6-4, 6-2.
"I was really disappointed with my last match, and I knew I couldn't play like that against someone like Vika. It would be impossible to win," Williams said. "Even though I lost the first set, I told myself, 'Well at least you're playing better than in your second round match.' I would have lost so easily if I played that same way. It was definitely better today - it's definitely a step in the right direction.
"And I just really want to be here."
"The level of the match was pretty high quality from both players, and she really stepped it up," Azarenka said. "She went for a lot of shots that landed right on the line, and she really just stayed aggressive. Looking back at it, I've got to learn from that and maybe stay a little bit more aggressive and go a little bit more for my shots too. But there wasn't much wrong that I've done, I don't think.
"She just played a good match and I have to give her credit for that."
With the win, Williams becomes the first woman in the Open Era to win 50 matches at all four Grand Slams - not even Chris Evert, Martina Navratilova or Steffi Graf achieved that milestone (more here).
The World No.1 was told the stat in her post-match press conference and seemed to be in disbelief.
"Yeah, I don't understand that, because it's just weird to me," she said. "I was thinking, 'Are you sure Martina didn't do it, or Chrissie Evert?' Yeah, that's just strange. But I feel really good about it."
Awaiting Williams in the fourth round will be WTA Rising Star Sloane Stephens, who won eight of nine games from 4-all in the first set to get past Tsvetana Pironkova in an hour and 15 minutes, 6-4, 6-1.
Williams leads Stephens in their head-to-head, 4-1, and won their only clay court meeting, too - in fact that came just a few weeks ago at the Mutua Madrid Open (watch highlights from the match here).