Senior White House aides are exasperated with Environmental Protection Agency Administrator Scott Pruitt, the already embattled Cabinet secretary who now finds himself at the center of an expanding controversy over his decision to rent a room in Washington, DC, from the family of an energy lobbyist.

The White House has yet to officially comment on Pruitt’s standing, but a government official tells CNN that the lack of responsiveness shows they are not ready to step in and defend Pruitt, who they believe has created a series of headaches for an administration looking to move past controversies involving Cabinet secretaries.

A White House official said the source of the frustration stems from the fact that White House officials began learning about this issue from media reports, not Pruitt himself, an issue that officials have complained about in the past over controversies involving Housing and Urban Development Secretary Ben Carson and recently-fired Veterans Affairs Secretary David Shulkin.

Pruitt rented a room in a condo from Vicki and Steven Hart, lobbyists whose firm has lobbied the EPA on behalf of an Oklahoma Energy company, ABC Newsand Bloomberg reported. According to Bloomberg, Pruitt leased a room for around $50 a night, only paying when he stayed there. In total, Pruitt paid $6,100 for the room over six months, a rate significantly lower than market value.

CNN has confirmed Pruitt’s landlords, the Harts, were political donors of his when he was an Oklahoma state official. According to campaign finance records, Steven Hart hosted a “fundraising reception” for Pruitt’s 2014 campaign for Oklahoma attorney general. The contribution report filed with the Oklahoma Ethics Commission states that Hart spent $1,616.43 on the expense.

In addition, Hart and his wife Vicki made a $500 contribution to Pruitt in October 2010, a $1,000 contribution to Pruitt in September 2011 and a $250 contribution to Pruitt in April 2012.

“It’s not good,” one of the sources said bluntly when asked about Pruitt.

An EPA source was even more dour on Pruitt’s standing.

“People are freaking out” over the volume of negative stories about Pruitt, the source said about the feeling inside the agency. The source wasn’t even sure if the former attorney general of Oklahoma could survive the controversy.

Pruitt’s “goose is cooked,” the source said.

The White House did not respond for comment for this report.

One possible reason for the doubt over his future is the fact that Pruitt has found himself in trouble for months because of the 24-hour security that the EPA administrator has required and his use of first-class travel. On Friday, CNN reported that Democratic Sen. Sheldon Whitehouse obtained information from a confidential, unnamed source that he says indicates Pruitt’s extensive EPA-funded security included non-official business, like trips home to Tulsa, Oklahoma, a family vacation to Disneyland and the Rose Bowl game.

Pruitt was one of four Cabinet secretaries to be called to the White House last month to be scolded for a series of embarrassing stories about questionable ethical behavior. During the meetings, according to a source, chief of staff John Kelly made clear that “optics matter” and White House officials urged agencies to flag any possible problems before they appear in the media.

Pruitt, according to a source familiar with the arrangement, leased the room in the Capitol Hill condo at 223 C Street NE from the lobbyist couple. The condo is owned by an LLC that shares a business address with Vicki Hart’s lobbying firm and property records show that Hart purchased the condo.

Hart is a health care lobbyist who runs Hart Health Strategies Inc. Lobbying disclosures do not show that she did any lobbying at the EPA, but Hart’s husband, J. Steven Hart, has lobbied the EPA on behalf of glass bottle manufacturer, Owens-Illinois, Inc. He has worked with the bottling company since 2009.

At some point, the EPA source told CNN, Pruitt’s relationship with the Harts soured and one of the landlords called the administrator’s chief of staff to complain. “He’s destroying my hardwood floors,” Pruitt’s chief of staff was told, the source said.

An EPA official with direct knowledge of the situation told CNN the ethics counsel had reviewed Pruitt’s living arrangement. It was not considered an ethics issue because Pruitt paid rent. “It was not a gift because he paid value for it,” the official said. In addition, the landlord was a friend of Pruitt’s, and the law “does not ban federal employees from receiving a gift from a friend.”

“As EPA career ethics officials stated in a memo, Administrator Pruitt’s housing arrangement for both himself and family was not a gift and the lease was consistent with federal ethics regulations,” agency spokesman Jahan Wilcox said Friday evening.

Pruitt’s agreement with the Harts is also drawing attention from Capitol Hill, with at least one Democrat already calling for the EPA administrator to resign.

“Pruitt’s secret rent deal with a lobbyist for companies regulated by the EPA is much like his many other scandals,” Rep. Don Beyer, a Virginia Democrat, tweeted on Friday. “Pruitt must resign. If he refuses to do so he should be fired immediately.”

CNN’s Gregory Wallace and Jeremy Diamond contributed to this report.