Irving is the stand-out star at the Celtics having moved from Cleveland Cavaliers one year after winning the NBA Championship in 2016.
But Jennings feels the point guard is doing more damage than good in the current squad.
Irving is out of contract this summer and despite verbally committing his future to the franchise, there are strong reports he is pushing for a move to Los Angeles Lakers.
The 26-year-old also repaired his relationship with LeBron James last month following a fall-out while team-mates at the Cavs.
The Celtics are fifth in the Eastern Conference and Jennings feels they should be higher.
“The biggest problem with Boston Celtics is Kyrie Irving,” Jennings told the Undisputed.
“This is not a shot at Kyrie Irving because statistically he is having an unbelievable year. Individually.
“But when you look at Boston Celtics you look at how they’re made up. Their DNA.
“Brad Stevens is the biggest asset that they have.
“Although Kyrie Irving is as electrifying, as exciting, as stellar as he is on court, he takes away from what everybody else does.
“He doesn’t compliment everybody else.”
The Celtics took the Conference Finals down to the seventh game last season, subsequently losing to the Cavs while Irving was out injured.
And Jennings believes to progress so far in the campaign without their key man speaks volumes.
“When you look at their success last year with Kyrie Irving off the court,” he added.
“The ball popped, they moved around, threw to one another. There was no animosity.
“There wasn’t any individual play or iso-ball.”