A woman was killed on Saturday when a auto rammed into a crowd protesting against a white supremacist rally.
Trump responded to Frazier's resignation in two tweets.
In a news conference after the event at the weekend, Trump in a statement denounced the "egregious display of hatred and bigotry" displayed by antagonists "on many sides".
Even before President Donald Trump began speaking at Trump Tower on Tuesday afternoon, America's most prominent CEOs knew they had a problem.
One of the US's most high-profile African American executives quit Donald Trump's business advisory panel on Monday, citing "a responsibility to take a stand against violence and extremism" and triggering an nearly immediate attack from the president.
"America's leaders must honor our fundamental values by clearly rejecting expressions of hatred, bigotry and group supremacy, which run counter to the American ideal that all people are created equal", Frazier wrote. Head to IBD Stock Checkup for a look at other drug stock leaders.
"For every CEO that drops out of the Manufacturing Council, I have many to take their place", he tweeted.
The heads of drugmaker Merck, sports apparel giant Under Armour, tech multinational Intel, the Alliance for American Manufacturing and the largest group of unions in the USA, the AFL-CIO, have all exited Mr Trump's council.
Far-right clashes: Merck CEO quits Trump panel
The outrage eventually forced Trump to issue a new statement condemning racism and hate groups by name on Monday. Michael Dell is the only tech CEO left on Trump's council, although he's still in good company.
Trump attacked Frazier shortly after the announcement.
Mr Trump slammed the executives who quit the council, telling reporters "they're leaving out of embarrassment because they make their products outside" the US.
"Under Armour engages in innovation and sports, not politics", Plank said.
Other business leaders on the committee include Andrew Liveris, CEO of Dow Chemical Co.
The latest to leave is the chief executive of AFL-CIO. But despite his insistence on "knowing the facts", many industry-watchers argue Trump's got the wrong target in Merck.
Disney's Bob Iger, another member of the economic advisory council, also quit after the Paris decision.
At Detroit-based General Motors, Barra elected to stay on the White House's Strategic and Policy Forum because she wants a voice in policy decisions and realizes she would face Trump's wrath if she left, which could shred any chance of ever working with him, according to people familiar with the matter.
"President Trump's remarks today repudiate his forced remarks yesterday about the KKK and neo-Nazis", Trumka said in a statement.