Barnaby Joyce says he's not going anywhere but has lost the support of the West Australian Nationals who say his leadership is "no longer tenable".
The deputy prime minister, who's taken personal leave after his affair with former staffer Vikki Campion was made public, played down the significance of a nationwide phone hook-up between Nationals officials on Monday afternoon.
But on Tuesday the WA Nationals became the first state to officially withdrew support for the federal leader.
WA Nationals state leader Mia Davies issued a statement saying she told Mr Joyce he is now a distraction.
"Mr Joyce's actions have caused pain for his family but it is the ongoing damage Mr Joyce is causing the Nationals organisation that is of greatest concern to me as WA leader," Ms Davies said.
The WA branch of the party does not have any federal MPs and will play no role in any potential vote on the leadership, but it was second only to the NSW branch in political donations last financial year, receiving $1.785 million.
Mr Joyce shot back in a statement to Sky News, pointing out WA didn't have any federal MPs and the eastern states, which had more "skin in the game", supported him.
His federal Nationals colleague Matt Canavan said there was a "level of disappointment" about the situation, but party supporters wanted Mr Joyce to keep fighting for regional Australia.
Asked about Ms Davies' statement, Senator Canavan said: "It's a sentiment I've heard from lots of people ... (but) it's obviously not 100 per cent in one direction."
He said the vast majority of Nationals MPs backed Mr Joyce as leader, but a spill was a matter for the party room.
Mr Joyce said the nationwide phone hook-up was not an official meeting, reiterating the leader of the Nationals is decided by party MPs.
"People are starting to see this as a witch hunt. I'm not going anywhere, I never would," Mr Joyce told Fairfax Media on Tuesday.
Finance Minister Mathias Cormann, who will be acting prime minister when Malcolm Turnbull heads to the US on Wednesday, said Mr Joyce's future is up to his colleagues.
But he was very confident Mr Joyce and Mr Turnbull will continue to work together, despite their sometimes "robust relationship".
"Barnaby's had a difficult week, there's no two ways about it," Senator Cormann told reporters in Canberra.
Senator Cormann left a voicemail message with the deputy prime minister in the last few days but said the pair haven't had a chance to talk.
Mr Joyce's NSW Nationals colleague Michael McCormack, who's been touted as a potential replacement, refused six times to explicitly back Mr Joyce's leadership.
"There is no challenge at the moment ... he has the party's support," the minister told Sky News on Monday.
A Newspoll published by The Australian on Monday found 65 per cent of Australian voters believe Mr Joyce should quit as Nationals leader and either go to the backbench or quit politics.
News Corp reports friends of Mr Joyce's wife Natalie say she doesn't want him to lose his job.
Australian Associated Press
