This defeat is the latest loss taken by Labour Party in what has been a stunning collapse of the party over the past decade.
"We agree with Corbyn on a lot of priorities: a growth orientated economy built on public investment; a social and youth agenda; fiscal justice and policies to tackle tax evasion; a common and shared EU migration policy; and closer European cooperation on defence".
Theresa May has said Labour's "devastating" defeat to the Conservatives' in the Copeland by-election shows Jeremy Corbyn's party is "out of touch with the concerns of ordinary working people".
Holding on to Stoke was some consolation for Corbyn, with political commentators predicting Labour would lose both of the seats it was contesting in Thursday's two by-elections, which would have threatened Corbyn's leadership of Labour where he remains estranged from many of his own members of parliament (MPs). Corbyn has in the past expressed eurosceptic views, but officially backed Remain during the referendum campaign.
The Conservatives have scored a historic by-election victory in Copeland, winning a constituency that had been held by Labour since 1935 - on the same night that UKIP failed to take Stoke-on-Trent Central following a series of embarrassing mistakes by their candidate and party leader Paul Nuttall.
Many first-time Conservative voters in Copeland cited the Conservative Party's commitment to supporting nuclear power as the motivation behind their vote.
The only other seats gained by an incumbent government since the war were the 1960 Brighouse and Spenborough by-election, won by the Conservative and National Liberal candidate Michael Shaw, and the 1953 Sunderland South by-election, won by the Conservatives during Sir Winston Churchill's final term as Prime Minister.
Shadow Cabinet minister Baroness Smith told HuffPost UK that the Labour leader will have to "think long and hard" about the way the Tories toppled the party in its northern heartland.
He wrote: "As a party we have forgotten our roots, and have arrogantly assumed that our core support would stay loyal because it has nowhere else to go". I wish her well in that.
She added: "We've got to reflect on this". We mustn't bury our head in the sand, this is a very bad election result for us. If either man had any sense of shame, UKIP's Paul Nuttall and Labour's Jeremy Corbyn would now resign.
The Ukip leader said the "whole Hillsborough thing" had not been an issue on the doorstep.
Outside of Team Corbyn, Labour backbencher David Winnick said Corbyn was an "obstacle" to victory and should consider his position.
Instead Labour's Gareth Snell, a 31-year-old local councillor, held on to the seat vacated by Tristram Hunt in January, with 7,853 votes.
Labour has traditionally been the Tory party's main opposition but has withered under Corbyn, a veteran leftist whose directionless and sometimes chaotic leadership style has fractured his party.
He went on: "While it was pleasing to see Ukip put in its place, Stoke should never have been in doubt and the result in Copeland was disastrous".
"This isn't a decline that's happened under Jeremy; it's been happening for 20 years".
Irrespective of the results from the Copeland and Stoke byelections (Report, 24 February), it was clear that Jeremy Corbyn's detractors were preparing to impose their chosen narrative, whether on victory or defeat.
Mr Nuttall said Ukip was "not going anywhere" and insisted the party's "time would come".
A spokesman from Mr McCluskey's campaign for re-election told The Independent that Unite members would be "baffled" by any suggestion the general secretary was not putting members first.