The custodians of the statue of Seán Russell say the Taoiseach needs to take a history lesson after calling him a "Nazi sympathiser".
Leo Varadkar said yesterday it was a memorial "we need to talk about" in light of calls to remove statues with racist links.
Russell died in 1940 on his way to Nazi Germany to get guns for the IRA during the Second World War.
But Seán Whelan, chairman of the National Graves Association, says Russell was no fascist.
"When it was erected, the dedication of the statue got huge support," said Mr Whelan.
"There was a huge crowd turned out for it. The band was led by the Irish Transport and General Workers' Union.
"The Communist Party of Ireland was officially represented at it. There were TDs from the Labour Party.
"The first time the statue was vandalised was in 1951 and it was vandalised by a right-wing group who said that Seán was a communist."
Mr Whelan said Fine Gael needs to examine its own past.
"Three-times Taoiseach John A. Costello stated: So as the brownshirts have been victorious in Germany and the blackshirts have been victorious in Italy so too shall the blueshirts be victorious in Ireland.
"That was Fine Gael of the 1930s. Now, that is not a reflection on Fine Gael today but we are talking about history here.
Seán Russell was a person opposed to the spread of fascism, Fine Gael were the people supporting it.
Following on from the Taoiseach's comments yesterday, a Dublin councillor has said there needs to be a conversation about who the city's street names and statues honour, in light of the Black Lives Matter movement.
Among those mentioned was Westmoreland Street which is named after John Fane, the 10th Earl of Westmoreland.
Critics have pointed to a speech he made in the House of Lords in 1799 when he argued against ending the slave trade in the British Empire.
The Green Party's Janet Horner says it is time to have a debate over Ireland's involvement in the slave trade.
"I've learned a lot this week. I have learned Westmoreland Street has connections to somebody who profited from slavery.
"Elgin Road is apparently named after the Governor-General of India.
"These are all names that we don't give thought to and maybe we need to give a little bit more thought to them and give thought to whether we can do better."
