The Hawks, one of only three teams in the Eastern Conference with winning records, improved to 18-14.
They are 2-1 since Al Horford was lost for the season with a torn right pectoral. Since Horford’s departure, Millsap has stepped up with an average of 28 points and 11.7 rebounds a game.
Jeff Teague added 16 points for the Hawks and Kyle Korver contributed 14, including a three-pointer in the fourth quarter to stretch his NBA record streak of games with a three-pointer to 101.
Boston reserve Kelly Olynyk scored a career-best 21 points on 8-of-11 shooting while Kris Humphries finished with 18 points and 10 rebounds, also coming off the bench.
The Celtics scored the first 13 points of the second quarter and stretched the lead as large as 18 points before Atlanta closed the first half on a 15-4 run to pull within 44-39 at half-time.
Boston’s Jordan Crawford missed two shots in the final seconds as the Celtics fell to 13-18 with their fourth loss in five games.