Strachan's position come under heavy scrutiny after they made a poor start to qualifying for the 2018 World Cup, but they rallied and finished the campaign unbeaten in their last six matches.
Scotland last participated in a major tournament at the 1998 World Cup.
The SFA announced Strachan's departure after a board meeting at Glasgow's Hampden Park.
"On behalf of the board I would like to thank Gordon for his excellent service to the national team," said SFA chief executive Stewart Regan.
"I know how disappointed he was that we couldn't secure a play-off place, but to come so close was a tribute to the belief he instilled in his squad.
"After almost five years the board felt it was time for a new direction to prepare for the UEFA Euro 2020 qualifying campaign and also the forthcoming Uefa Nations League.
"We are grateful to both Gordon and Mark for their efforts and wish them the best of luck in the future."
Strachan paid tribute to Scotland's fans' "unwavering support" and thanked his back-room staff and "everyone at the Scottish FA".
He said the experience of managing the national team had been a "real privilege".
Former Manchester United manager David Moyes is the early favourite to succeed Strachan with British bookmakers.