By-election

By-elections can be crucial when the ruling party has only a small majority. In parliamentary systems, party discipline is often so strong that the governing party can only lose a vote of no confidence after losing enough by-elections for it to become a minority government. Examples are the Labour government of James Callaghan 1976–1979 and Conservative government of John Major 1992–1997. In the United States Senate, Scott Brown's election in 2010 ended the filibuster-proof supermajority formerly enjoyed by Democrats.

By-elections can also be important if a minority party needs to gain one or more seats in order to gain official party status or the balance of power in a minority or coalition situation. For example, Andrea Horwath's win in an Ontario provincial by-election in 2004 allowed the Ontario New Democratic Party to regain official party status with important results in terms of parliamentary privileges and funding.

Political scientists generally caution against overinterpreting by-election results, which non-experts often take as a bellwether or early indicator of the results of the next general election. The evidence suggests that while the margin of victory relative to the district's normal performance may be relevant, other indicators generally provide stronger evidence with a larger sample size.[10]

A 2016 study of special elections to the United States House of Representatives found "that while candidate characteristics affect special election outcomes, presidential approval is predictive of special election outcomes as well. Furthermore, we find that the effect of presidential approval on special election outcomes has increased in magnitude from 1995 to 2014, with the 2002 midterm representing an important juncture in the nationalization of special elections."[11]

Seats which have unexpectedly changed hands in by-elections often revert to the former party in the next general election. One reason for this is that voter turnout at by-elections tends to be lower and skewed toward highly motivated supporters of the opposition party.

By-election upsets can have a psychological impact by creating a sense of momentum for one party or a sense of impending defeat for a government. Deborah Grey's 1989 by-election victory in Beaver River was seen as evidence that the newly formed Reform Party of Canada would be a serious political contender and that it posed a serious political threat for the ruling Progressive Conservatives. It also gave important momentum to the new party. Similarly, the upset 1960 by-election victory of Walter Pitman in Peterborough as a "New Party" candidate was a significant boost for the movement to replace the Co-operative Commonwealth Federation with an unnamed "New Party" which would be integrated with the labour movement. Pitman's candidacy in a riding in which the CCF was traditionally weak was seen as a test of this concept and his upset victory convinced the CCF and the labour movement to launch the New Democratic Party (NDP). Gilles Duceppe's 1990 upset landslide by-election victory in Laurier—Sainte-Marie with 66% of the vote on behalf of the newly formed Bloc Québécois was the first electoral test for what was initially a loose parliamentary formation created two months earlier after several Quebec MPs defected from the Progressive Conservative and Liberal parties to protest the failure of the Meech Lake Accord and provided the first indication that the party could be a serious force in the province of Quebec. On the strength of the by-election victory, the BQ went on to be officially formed as a party in 1991 and to win 54 seats in the 1993 federal election, enough to form the Official Opposition.

By-elections may occur singly, or in small bunches, especially if the authority responsible for calling them has discretion over the timing and can procrastinate. They are sometimes bunched to save money, as holding multiple by-elections is likely to cost more than holding a by-election to fill the vacancies all at once. In Canada, in 1978, 15 by-elections were held on a single date, restoring the House of Commons to 264 members. The media called it a "mini-election", a test of the Liberal government's popularity with a general election due in less than a year. The 15 districts stretched from Newfoundland to British Columbia, and produced some unexpected results, for example, an NDP candidate winning in Newfoundland for the first time. In September 1984 the Leader of the Greater London Council Ken Livingstone and 3 other Labour councillors resigned and stood in simultaneous by-elections in an attempt to stage a mini-referendum on the Thatcher government's proposal to abolish the GLC. The effect of the manoeuvre was blunted when the Conservative Party refused to stand candidates against them, and the following year the GLC was abolished.[12]

The 1918 Swan by-election was held following the death of John Forrest. The seat was traditionally a safe seat for the Nationalist Party against the Labor Party, but the emergence of the Country Party lead to a "three-cornered contest". As Australia used a first-past-the-post system at the time, the conservative vote was split between the Country and Nationalists, allowing Labor candidate Edwin Corboy to come in first place and win the seat. The Swan by-election is cited as the reason for the introduction of preferential voting, to prevent Labor from benefiting from a divided opposition in the future.[13]

The 2018 Wentworth by-election was held after the resignation of former prime minister Malcolm Turnbull, who had served as the member for Wentworth since 2004. Wentworth was considered an exceptionally safe seat for the Liberal Party, as it had only ever been held by the Liberal Party and its predecessor parties since its creation in 1901. Former Ambassador to Israel Dave Sharma was preselected as the Liberal Party's candidate for the by-election. The major challenger in the by-election was independent candidate Kerryn Phelps. A huge 17.7% two-party-preferred swing was required for the Liberal Party to lose the seat. Ultimately, the Liberals suffered a 19.0% swing to Phelps, the largest by-election swing in Australian history, which won her the seat. This loss deprived the Liberal Party of its majority in federal Parliament, forcing them into a minority government.

On November 1, 1944, General Andrew McNaughton was appointed to Cabinet as Minister of Defence without having a seat in parliament, after his predecessor resigned during the Conscription Crisis of 1944. A by-election was arranged in Grey North which the opposition Progressive Conservative party contested. The major campaign issue became the government's policy of "limited conscription" during World War II, which McNaughton supported, and which the Conservatives rejected. They called, instead, for "full conscription". McNaughton was defeated in the February 5, 1945 by-election. As a result, with confidence in his government undermined, Prime Minister William Lyon Mackenzie King called the 1945 federal election several weeks later; originally he had intended to postpone the election until the war was clearly won. McNaughton sought a seat in the 1945 contest but was again defeated, and resigned shortly after.

In 1942, the Conservatives' Arthur Meighen (who had already served as Prime Minister during the 1920s) sought to re-enter the House of Commons of Canada through a by-election in York South. His surprise defeat at the hand of Joseph Noseworthy of the Co-operative Commonwealth Federation ended his political career, and may also have been a factor in the Conservative Party's decision to move to the left and rebrand itself the Progressive Conservative Party under Meighen's replacement. Noseworthy's victory was also a significant breakthrough for the CCF giving it credibility as a national party where it has previously been seen as a Western Canadian regional protest party.

The most recent example of a cabinet minister appointed from outside parliament having to resign after losing a by-election was in 1975, when Minister of Communications Pierre Juneau was appointed to Pierre Trudeau's Liberal cabinet directly from the private sector, and tried to enter parliament through a by-election in Hochelaga. Juneau unexpectedly lost to the Progressive Conservative candidate and resigned from cabinet 10 days after his by-election defeat.

In Ontario, John Tory, leader of the Progressive Conservative Party of Ontario ran in a 2009 by-election in Haliburton—Kawartha Lakes—Brock, after he convinced one of his caucus members to step down, in hopes of re-entering the Ontario legislature. His by-election defeat resulted in his resignation as party leader.

In the March 2018 Hong Kong by-elections, the pro-democracy camp lost their majority status for the first time in the Geographical constituency part of the Legislative Council of Hong Kong By-elections were held due to six pro-democracy lawmakers were disqualified by the High Court of Hong Kong during Oath-taking controversy. The pro-democracy camp was considered safe in de facto first past the post by-election because both pro-democracy camp and pro-Beijing camp would only nominate one candidate to fill in the by-election. However, pro-democracy camp lost twice in Kowloon West, which was considered as a safe seat for the pro-democracy camp.

A by-election held in Dublin South-West during 2014 provided a very surprising upset. The Sinn Féin candidate, Cathal King, was the favourite to take the seat. Sinn Féin had done extremely well in the area during that year's local elections. Sinn Féin captured high percentages of the first preference vote across the constituency. However, the Anti-Austerity Alliance candidate, Paul Murphy, was elected on the eighth count. Despite Murphy having received a lower first preference total than Cathal King, he outperformed the Sinn Féin candidate in attracting transfers. Murphy then took his seat in the 31st Dáil. As a direct result of this defeat in the by-election, Sinn Féin hardened their stance against Irish Water and called for the complete abolition of water charges in Ireland.

In 1965 the British Foreign Secretary Patrick Gordon Walker stood in the Leyton by-election for election to the UK Parliament, having been defeated in controversial circumstances in Smethwick at the previous year's general election. His appointment as a senior minister while not a member of either house of Parliament was against convention, and he therefore sought to regularise the position by standing in the first available by-election, which was at Leyton in January 1965. However a strong swing against Labour resulted in Gordon Walker's defeat: as a result, he resigned as Foreign Secretary.