Labour has demanded that David Cameron make clear whether he talked to former HSBC boss Lord Green about tax-dodging allegations linked to the bank before appointing him to his Government or during his three years as trade minister, after the Prime Minister four times ducked the question in the House of Commons.

As leader Ed Miliband and three Labour backbenchers posed the question at Prime Minister's Questions in the Commons, Mr Cameron responded by saying that "every proper process" was followed in the appointment.

Mr Miliband said that a list of clients at HSBC's Swiss subsidiary included a string of Tory supporters who had between them given the Conservative Party more than £5 million, and branded Mr Cameron a "dodgy Prime Minister surrounded by dodgy donors" who was "up to his neck" in the HSBC tax avoidance scandal,

But the Prime Minister hit back, pointing out Labour donor Lord Paul - who now sits as a crossbencher - had also been named as holding an account at the Swiss bank branch at the heart of the controversy.

And he insisted: "When I appointed Stephen Green, every proper process was followed. I consulted the Cabinet Secretary, I consulted the director for propriety and ethics, and the House of Lords Appointments Commission now looks at someone's individual tax affairs before giving them a peerage."

The clash came as the head of HM Revenue and Customs, Lin Homer, prepared to answer questions from MPs on the House of Commons Public Accounts Committee on the growing controversy surrounding files of data about clients at HSBC Suisse's Geneva branch, which were leaked to the French authorities by an IT worker in 2007.

Labour said Mr Cameron had "refused to answer" the key question of whether he personally challenged Lord Green at the time of his appointment in January 2011 over how much he knew about activities in HSBC's Geneva office, which is facing allegations that staff encouraged customers to dodge tax.

A senior Labour source pointed out that details of thousands of UK-based clients were handed to HMRC in May 2010 and press reports that HSBC was involved appeared in September. In October 2011, HMRC issued a press release naming HSBC and carrying a comment from Treasury minister David Gauke.

"It beggars belief that David Cameron didn't know about these allegations," said the source, who said Labour was calling on both Lord Green and the Prime Minister to explain what discussions they had.

Labour also called on Mr Cameron to say whether he believed donations from people caught up in the tax-dodging allegations were acceptable.

A senior Conservative source said: "All of our donations are fully declared and above board."

The Guardian published a list of nine Conservative donors who it said were listed in the files relating to clients of HSBC's Swiss subsidiary The newspaper stated that the accounts were held legally for a wide variety of reasons, and made no allegation of wrongdoing against those listed.

Of more than 6,000 names understood to have been passed to HMRC, UK tax authorities say they have pursued around 1,100, recovering £135 million in unpaid tax, fines and interest.

Mr Miliband told the Commons that on the list was Stanley Fink, who gave £3 million to the Conservatives and was appointed party treasurer and given a peerage by Mr Cameron.

By contrast, Mr Miliband said that none of those named had paid "a penny" to Labour on his watch. Referring to Mr Cameron, he told MPs: "There is something rotten at the heart of the Tory Party and it's him."

Mr Cameron responded: "When people donate to the Conservative Party they don't pick the candidates, they don't choose the policies, they don't elect the leader.

"When the trade unions fund the Labour Party they pay for the candidates, they pay for the policies, and the only reason you are sitting there is because a bunch of trade union leaders decided you were more left wing than your brother."

The coalition Government had been tougher on tax avoidance than the previous Labour administration, said Mr Cameron. And he told MPs that Labour had welcomed the appointment of Lord Green, who had served as head of Gordon Brown's business advisory council and been invited to an event by shadow business secretary Chuka Umunna as recently as 2013.

Meanwhile, Conservative MP Zac Goldsmith, denied The Guardian's report that he was the holder of an account at the HSBC Suisse private bank.

Mr Goldsmith said in a statement: "It has never been a secret that I am a beneficiary of a Trust set up by my late father and administered by a family office in Geneva ... I have never had a Swiss bank account, and do not control any Swiss bank accounts. I have never sought or been given tax minimisation advice by HSBC, directly or indirectly."

During a grilling by the House of Commons Public Accounts Committee, HMRC permanent secretary Lin Homer was asked why she had not warned Mr Cameron about the HSBC allegations at the time when Stephen Green was being considered for appointment to the House of Lords, and later as a minister.

Committee chair Margaret Hodge asked: "Wouldn't it have been sensible to draw his attention to the information? ... You knew about the allegations of evasion and you didn't think to tell the Prime Minister about it."

But Ms Homer told the committee that HMRC had no responsibility to report direct to the Prime Minister on individuals' tax affairs.

"Our general role in appointments is to give information about the tax affairs of people," she said. "We undertake a role of putting information into a number of procedures, and I don't think that what you are asking HMRC to do would be possible in all circumstances.

"We provide a certain limited role to a range of appointment committees. We have a very limited responsibility for giving limited information about individuals."

Ms Homer added: "It was in the public domain that there was data available to many organisations, including us, about HSBC Suisse."

Lord Fink said Mr Miliband's suggestion at Prime Minister's Questions that he had been involved in "tax avoidance activities" was "untrue and defamatory".

He challenged the Labour leader to repeat the claim outside the Commons, where he would not be protected from legal action by parliamentary privilege.

Lord Fink said he had a Swiss bank account because he was working for the Man Group in the country for four years from 1996 to 2000.

"During this time I had need of a local bank account to do simple things like receive my Swiss franc salary and pay grocery bills," he said.

"As I already banked with HSBC in London, I set up an account with HSBC. I subsequently set up an account with Credit Suisse as they had a branch much closer to my home and office.

"I submitted tax returns in both Switzerland and Britain showing my revised tax status, which was accepted by the Inland Revenue.

"The only way I have ever sought to depress my income tax liability is by giving a lot of my income to charity."

He told Mr Miliband: "I challenge you to repeat your allegation outside the House of Commons - or to withdraw it publicly."

Speaking to journalists later, Lord Fink repeated that he viewed the allegations as "defamatory and untrue".

"It's called playing the man not the ball, which I am afraid is something that has become a habit for him," he added.

Lord Paul said that he had a Swiss bank account because he had a house in the country. "I use it when I go to Switzerland," he told Channel 4 News.