"I am resigning the whip for two principal reasons", Field wrote.
The rebel MP said he will now serve as an independent as he does not want to be seen as part of a "racist" party.
Labour has been dealing with a row about the extent of anti-Semitism within the party for more than two years.
Sky News's political correspondent Tom Rainer said Friday said that people close to Corbyn's office have pointed to Mr Field's conflict with his constituency party and the reason for his resignation was to avoid deselection - a mechanism being threatened against fellow Labour Brexiteer Vauxhall MP Kate Hoey.
The first centres on the latest example of Labour's leadership becoming a force for anti-Semitism in British politics.
The one-time minister for welfare reform, who was charged by Tony Blair to reform the welfare state by "thinking the unthinkable", sensationally resigned saying that the party was now governed by "a culture of intolerance, nastiness and intimidation".
Jeremy Corbyn has apologised for hurt caused by anti-Semitism in the party and pledged to stamp it out.
A senior Labour source told Business Insider that "Frank has been looking for an excuse to resign for some time".
He described a failure to act on what he called the "thuggish conduct" of some members demonstrates a "wilful denial" adding that, at worst, "it serves to legitimise appalling levels of bullying and intimidation of lifelong Labour supporters".
He said he disagreed with the leader's "blind support" for social governments elsewhere in the world, but believed his domestic policies could be "shaped" to give a voice to those affected by "social and economic upheaval" in the UK. I will argue on that basis that I have a right to remain a member of the party.
Microsoft Announces Xbox All Access
Say you go out and get a credit card, at 19.99% interest (the terms of the Dell Preferred Advantage program after two years). This offer is only available to new Xbox Live Gold subscribers or previous members who do not have an active subscription.
He said: "This is a serious loss to the party and I deeply regret Frank's decision".
On Thursday, he denied he was "jumping before being pushed" and used the article to further condemn some local members. This issue alone compels me to resign the whip.
Field also complained of "bullying and intimidation" in his local branch of the party.
A number of Mr Field's Labour colleagues have praised him and expressed their own concerns about their party.
In his resignation letter, the 76-year-old representative of Birkenhead called Corbyn "a force for antisemitism in British politics".
Cllr Kelly, who was Mr Field's Liberal Democrat opponent in the General elections of 2005 and 2010, continued: "Mr Field has never been what could be described as left wing and the current Labour party must feel like an very alien place even worse perhaps than the militant takeover of the 1980s".
He was also applauded by some Conservative MPs, with Home Secretary Sajid Javid tweeting that he was "a man of integrity and principle".
However, Mr Field added that he felt Mr Corbyn's thanks for his service was "as though I was resigning from a whist club" and pointed out that he had been a member of Labour longer than his leader.
The veteran MP explained to Nick Brown that a "specific bullying issue" also contributed to his decision as the party had failed to take "decisive action" following "numerous complaints".
"It's so sad to see someone like Frank Field trashing Labour's anti-racist record".