As support for Malcolm Turnbull grows, leadership spill rumours are rife.

Pressure is building in the Liberal Party for another tilt at removing Prime Minister Tony Abbott.

Both ministers and backbenchers have told the ABC they believe Malcolm Turnbull now has the numbers to win a challenge and should use them, though the Communications Minister is not agitating for a spill.

But other MPs are playing down the tension, saying the Prime Minister has “strong” support.

Mr Turnbull deflected questions of another potential spill on Thursday, saying only: “I’ll leave you guys to speculate about all that stuff.”

Mr Abbott, who was in central Queensland inspecting cyclone damage, was asked whether he had any concerns that Mr Turnbull was being urged to run against him.

“We’re here to look at the cyclone damage today, OK,” he replied.

Mr Abbott survived a move on his leadership earlier this month, when a motion to spill the position failed 39 votes to 61.

After that party room meeting, the Prime Minister declared that “good government starts today” and promised to consult his colleagues more.

Read more: Liberal MPs: “Malcolm Turnbull has numbers to win new challenge on PM’s leadership”.

However, there is now open despair in Coalition ranks, even among those who supported the Prime Minister in the leadership spill.

Liberals talk of little else and worry that there will be permanent brand damage if Mr Abbott stays.

Backbenchers do not want to lead the charge this time, and instead want ministers to push for a change at the top.

Some argue that it should be resolved quickly to give a new leader the opportunity to re-frame this year’s budget, which is just 12 weeks away.

There is also the view that NSW Premier Mike Baird should be given clean air to fight the state election next month.

The leadership talk has also been fuelled by Mr Abbott’s criticism of Human Rights Commission president Gillian Triggs, with some MPs uncomfortable with the way she was treated.

This week in the party room Mr Abbott was asked about his tactics and whether he could have, for example, highlighted the positives of what the Government had done to get children out of detention.

Some MPs described the Prime Minister’s response to their queries as a slap down and way too harsh, feeding anxiety about his leadership.

Mr Turnbull distanced himself from Mr Abbott’s remarks, describing Ms Triggs as “distinguished” and saying the criticism of her “misses the point”.

‘Our party room is as placid as a lake by moonlight’

Read more: The Prime Ministerial promise for change: Can it work?

But not all are pushing for a new leader. One Liberal MP told the ABC the opinion poll figures this week were better and people should give Mr Abbott more time.

Another Liberal, Attorney-General George Brandis, was on Thursday questioned in Senate estimates about the general issue of “friendly fire” within his party room.

“Our party room is as placid as a lake by moonlight,” he said in response to a question from the Greens.

Luke Simpkins, who filed the motion for the failed leadership spill, said he thought the Prime Minister was getting the Government in order.

“I think we just give the Prime Minister time, lets get back on track and everything will be fine,” he said.

Dennis Jensen, one of Mr Abbott’s most outspoken critics prior to the leadership spill, also praised the Prime Minister for his actions over the past few weeks.

Read more: Malcolm Turnbull auditioned to be Prime Minister last night. (And he nailed it).

“It’s evident that the Prime Minister’s heard the message and adjustments are being made,” he said.

“It’s very important that those changes are made and obviously it’s an ongoing process.”

Abbott ‘continues on with support of party room’

Coalition frontbencher Steve Ciobo said he had not been hearing the leadership discussions within the party.

“Look I’m loathe to go into these types of things but in summary, no I’m not,” he said.

“We had a vote of the party room in relation to a spill motion that has been decided, the Prime Minister continues on with the support of party room.”

Frontbencher Jamie Briggs joined Mr Ciobo in his dismissal of another possible spill motion, saying the issue was resolved and the Government was “getting on, doing its job”.

As he left Parliament on Thursday Liberal MP Rowan Ramsey was questioned about whether Mr Abbott would lead the Government to the next election.

“Who knows, I mean we are falling in behind Tony, we’ve had that day in the party room,” he said.

“Not everybody’s happy all the time, but that’s life.”

Another backbencher, Russell Broadbent, reiterated that Mr Abbott had qualified support.

“He’s been given time, the time that he’s been given is – he’s on a test at all times – so every Prime Minister is,” he said, adding there was “no deadline”.