Mr Gray, 54, has twice survived deselection attempts after the revelation that he had a 19-month affair with Philippa Mayo, 46, while his wife Sarah, 53, was undergoing chemotherapy for breast cancer.
Having left the marital home of 26 years he upset local Tories when he made Mrs Mayo his Commons diary secretary - a role previously performed by his wife. At the weekend he married her in a low key register office ceremony.
The couple's three children, John, 24, Olivia, 21, and William, 16, who have remained loyal to their mother, were conspicuous by their absence.
The Register Office marriage took many local Tories by surprise and as they debated whether to try to force another deselection battle a friend of Mrs Gray said: "It's another blow to her health. Of course she knows about it, and she was upset, but she has had to be strong after what she has been through. But she is moving on from him."
Mr Gray has also compounded his difficulties in the constituency by claiming £2,000 on his parliamentary expenses for the future redecoration of his "second home" on the day that he moved out of it.
Toby Sturgiss, a local Tory county councillor of 14 years standing, said: "I am afraid many people, who are good Conservatives, will be unable to support him. Even thought this is a strong Tory constituency some people just hate what he has done."
The Gray scandal was exactly the sort that David Cameron had hoped to avoid in his attempts to decontaminate the Tory Party brand. Now Wiltshire North, a text book Conservative constituency in the heart of Middle England, has returned again to haunt him.
It was not just that Mr Gray left his wife but his attempts publicly to play down the extent of her cancer which caused such distaste. On the ConservativeHome website he claimed that this wife was not stricken by cancer but had undergone treatment for "mainly precancerous cells''. In trying to belittle her struggle with the disease he outraged even some of his own supporters.
One friend of the first Mrs Gray said: "It is just so awful. Of course he should go. But I just don't know if we have got the stomach for another fight so close to the general election."
Mrs Gray, in her own only public comments in the Daily Telegraph two years ago, said: "I do not want a divorce at all. Without wanting to go into personal details, my understanding of this [her husband's behaviour] is deep. But it doesn't make it any less hurtful. My children, who have been the most amazing support to me in their different ways, say I have not been angry enough. But I have got to live with myself".
The couple married on Friday in front of just a few close friends and held a garden party at their home near Corsham when they told 150 guests they were man and wife. Mr Gray, the MP since 1997, said: "She and I have been through a lot of things in the past several years, but it's very nice that we have now got married."
Mrs Mayo met Mr Gray in 2005 when they were campaigning against the hunting ban. She left her barrister husband Rupert after he discovered incriminating text messages on her mobile. Mr Mayo has claimed that Mr Gray is not "fit for office".
A number of members of the local Conservative association agree and will write to David Cameron asking him to intervene in a constituency where Mr Gray is defending a 7,000 majority. "In this climate, post expenses, it's seat we could now lose," said a stalwart of the association. "We must act now."