As part of a $650 million settlement with Bank of America in which American International Group agreed to dismiss all outstanding residential mortgage-backed securities claims, AIG Wednesday filed a notice in Manhattan state court formally withdrawing its objection to the bank’s $8.5 billion settlement with institutional investors.

The dropped appeal was part of the settlement deal, reached late Tuesday, one day before Bank of America reported its second-quarter earnings and announced in a press release its litigation expense in the quarter was $4 billion.

On Thursday, AIG was scheduled to file its appellate brief objecting to Justice Barbara Kapnick’s approval of the bank’s settlement with institutional mortgage investors such as BlackRock and PIMCO. AIG had said the $8.5 billion fell short of covering the extent of losses it suffered from the purchase of defective mortgage loans from Countrywide Financial, which Bank of America acquired in 2008.

AIG mounted a challenge to the June 2011 deal, culminating in an Article 77 proceeding held before Kapnick last year. The insurance giant argued that Bank of New York Mellon, as trustee, failed to take reasonable steps to protect the investors’ interests in reaching a resolution with the bank.

Led by Mark Zauderer, a partner at Flemming Zulack Williamson Zauderer, AIG also had filed a motion for leave to reargue Kapnick’s approval of the deal, saying it offered little analysis or reasoning as to the judge’s findings.

In her Jan. 31 decision, Kapnick said the deal was made in good faith and that BNY Mellon did not abuse its discretion in entering into the settlement. Her ruling did not apply to loan modification claims.

Oral arguments on the reargument motion were scheduled to take place this fall before Commercial Division Justice Saliann Scarpulla, who assumed Kapnick’s docket when Kapnick was elevated to the Appellate Division, First Department in February.

Zauderer declined to comment on the $650 million settlement and the withdrawal of the appeal. Matthew Inger, a partner with Mayer Brown who represents BNY Mellon in the Article 77 matter, also declined to comment.

Tuesday’s settlement disposes of AIG’s mortgage securities fraud claims that were pending in federal court against Bank of America in the Central District of California and Southern District of New York. In the California action, AIG’s opposition brief to Bank of America’s motion for summary judgment was due this past Monday, although AIG received a two-week extension from U.S. District Judge Mariana Pfaelzer as settlement negotiations drew near.

Representing AIG in the federal litigation was Quinn Emanuel Urquhart & Sullivan, led by partner Michael Carlinsky. Counsel to Bank of America was Munger, Tolles & Olson, led by partner Daniel Fry of its San Francisco office and Goodwin Procter.

In a press release announcing the deal, AIG said it would receive $650 million in cash plus its “pro rata share of whatever amount is ultimately paid out to investors in connection with the Countrywide repurchase settlement.” In a press release, Bank of America said the settlement with AIG resolves about “95 percent of the unpaid principal balance of all RMBS as to which RMBS securities litigation has been filed or threatened for all Bank of America-related entities.”