NBA commissioner Adam Silver said Donald Sterling would be forced to sell his interest in the team.
And the league will fine him $2.5m (£1.5m), the maximum allowed.
Mr Sterling was recorded asking a woman not to associate in public with black people nor bring them to games.
He later said the recording “does not reflect his views”.
In a news conference on Tuesday, Mr Silver told reporters that Mr Sterling’s “hateful opinions… simply have no place in the NBA”.
“That they came from an NBA owner only heightens the damage,” he said. “I am personally distraught that the views expressed by Mr Sterling came from within an institution that has historically taken a leadership role in matters of race relations.”
Mr Silver said Mr Sterling would be banned from participating in all team business, as well as attending any NBA practices or games.
The fine will be donated to organisations focused on combating racism, he said.
And Mr Silver said he would urge the NBA’s board of governors to force Mr Sterling to sell his stake in the team, which he purchased in 1981.
It is considered the harshest punishment ever meted out by the NBA.
“Commissioner Silver showed great leadership in banning” Sterling, basketball great Earvin Magic Johnson tweeted following the announcement.
Mr Johnson reportedly posed for a photo with Mr Sterling’s female companion, leading to his comments on the controversial recording.
Following the announcement of Mr Sterling’s NBA expulsion, the Clippers’ website posted the message: “We are one.”
“We wholeheartedly support and embrace the decision by the NBA,” the statement added. “Now the healing process begins.”
The row erupted on Friday when celebrity news website TMZ published a 10-minute audio recording in which a man can be heard criticising a woman, believed to be Mr Sterling’s girlfriend V Stiviano, for posting online photographs of herself with black friends at Clippers games.
“It bothers me a lot that you want to broadcast that you are associating with black people. Do you have to?” the man says.
“The little I ask is not to promote it on that… and not to bring them to my games.”
On Tuesday, Mr Silver said Mr Sterling had acknowledged it was his voice on the recording. It was not clear how or when the conversation was recorded.
The remarks caused an immediate uproar in Los Angeles and among basketball fans across the country.
The Clippers are competing in the first round of the NBA playoffs, drawing extra scrutiny to Mr Sterling’s remarks.
On Sunday before a game against the Golden State Warriors, the players staged a silent protest, going through a pre-match warm-up with shirts on inside-out to hide the team’s logo.
The players also wore black wristbands or armbands and all wore black socks with their normal jerseys.
Mr Sterling did not attend the game.