Video technology in sports adjudication: Part 2 – use on the sports field
Published 09 April 2015 By: Craig Dickson
Part 1 of this article looked at the growing use of video technology in the resolution of sporting disputes in the courts. Part 2 below moves on to investigate the evolving role that video has played on the field of play itself, and raises important questions about its fairness and accuracy.
The first uses of film in sports
The gradual uptake but uncertain use of technological developments in the legal environment has influenced and been replicated in the sporting context. Moreover, the conceptual change from illustrative confirmation to evidential verification has also been duplicated.
The development of photography saw sporting contest being reported in the new visual medium even though the speed of sporting endeavour was initially beyond the capability of the technology to reproduce. Radio commentary of sporting events had begun in the early 1920s but as far as moving pictures were concerned, the first live television broadcast (of a football match) occurred on 16 September 1937 in the UK – the BBC showing a specially arranged fixture between Arsenal and Arsenal Reserves. The world’s first live televised sporting event had however, been the 1936 Summer Olympics from Berlin. Three years later, on 17 May 1939, the first televised sporting event in the United States was of a college baseball game between the Columbia Lions and Princeton Tigers. It was broadcast by National Broadcasting Corporation using just a single camera positioned down the third base line.
TV in Australia was officially launched in September 1956, although the 1956 Melbourne Olympic Games had been broadcast as a test transmission by all three television stations operating in Melbourne at the time. Footage was also sent overnight to Sydney where it was broadcast the next day. On April 20, 1957 the Victorian Football League (VFL) was televised for the first time, with 3 local Melbourne channels broadcasting live coverage of only the final quarter of (as it turned out) the same league match, Collingwood v Essendon.
It was not until 1972 that the first Ashes test series was broadcast live by satellite from England and then on 1 March, 1975 television stations converted to full-time colour transmission. In New Zealand, it took until 1969 for a national network to be operational (although channels had operated regionally since 1 June 1960). Colour TV appeared in 1973 in time for 1974 Commonwealth Games (although due to limited facilities only athletics, swimming, and boxing could be broadcast in colour).
The modern day role of film
From such humble beginnings, the broadcasting of sporting events has grown into a multi-billion dollar, worldwide industry and as the technology has become more sophisticated so too has the substance, number and complexity of the images broadcast. The US and Canadian TV networks experimented with ways to replay key moments in a sports broadcast in the early 1960s but the debut of the instant slow motion replay was on CBS’s broadcast of the Army-Navy (American) football game late in 1963. The “Chalkboard” (which allowed commentators to draw freehand over television pictures) followed in 1982 and there have been a number of advancements since: super Slo-Mo, Stump Cam, Spider Cam and the computer-graphics enhanced Hawk-Eye ball tracking system etc.
The latest of these advanced technologies debuted during NBC’s broadcast of a NY Giants v Dallas Cowboys American Football game in September 2013 and is now also used in a number of sports, particularly baseball. This is the 360-degree instant replay that uses what is termed freeD (freeDimension) technology developed by Replay Technologies.1 Using a linked system of 24 cameras, freeD allows three-dimensional images that can be spun 360 degrees in any direction creating so-called "bullet time" footage — nicknamed for Keanu Reeves' bullet-dodging scenes in the movie "The Matrix". It is envisaged that eventually such replays will be able to be streamed in close to real time and that viewers will have their own joystick that will permit TV spectators to choose their own replay perspective.
360 degree cameras are already being used for game adjudication, particularly in Major League Baseball (MLB), as both the NFL and MLB allow a limited number of challenges by the coach in respect of umpiring decisions.
From illistrative to adjudicative use
In the first instance however, these broadcasts were purely illustrative – providing images enabling the public to follow the play: originally through for example, black and white news reels of past events followed by “live” transmissions that became increasingly sophisticated.
Increasingly however, film and video recording began to be utilised for adjudication. Perhaps, surprisingly, the photo finish has been used in the Olympics since as early as1912, when theStockholmOlympics used a camera system for the athletics events. However, a rudimentary photo-finish electronic-timing system was only used as a back-up to the hand-held judges' stopwatch. That is, the timing was not fully automatic and only in the 1,500 metres final was it necessary to separate the athletes finishing 2nd and 3rd by means of a photo finish.
As with TV replays, the technology has come a long way since then – horse racing in Britain has used photo finishes since 1947 and the whole race has been filmed and then videoed since the early 1960s. The London Olympics in 1948 marked the advent of modern photo-finish time-keeping at the Games but the 1952 Helsinki Olympics were the first Games to employ photo-finish cameras that could record athletes' times and it wasn’t until in 1968, at the Mexico City Olympics, that the photo-recording technology would officially become the determining factor for all track and field events. Currently, the latest photo-finish cameras (the OMEGA Scan'O'Vision Star camera) can take up to 2000 images each second and are capable of measuring time to one half of a millisecond.
From objective to subjective decisions
While the use of developing technologies to help determine entirely objective decisions (i.e. who finished first) was uncontroversial, the use of video replays during play to assist officials took longer to take hold. The National Football League (NFL) in the United States introduced a limited instant replay system in 1986 but the system was abandoned in 1992 because of the time it took for disputes to be reconciled and a view that it overly questioned the referee’s integrity.
Anecdotally, it seems that similar problems befell the original introduction of video adjudication into rugby league. It was introduced into Australia by the breakaway Super League in 1997 – perhaps not all that surprising since that league was the creation of a TV broadcast network. National Hockey League has used a video system to confirm goal scoring since 1991 and cricket began using replays to adjudicate on catches and run outs in 1992.
Most other major sports have followed suit in the previous 10 years with many (including, tennis, cricket, the AFL and the GAA) adopting the Hawk Eye2 RTD (reconstructed track device) technology. The NFL re-introduced a video review system in 1999 bringing with it the opportunity for coaches to “challenge” on-field calls in order to send them for video adjudication. Rugby Union began its video review system in 2001 which initially limited the TMO only to adjudicating issues that arose during the act of scoring expanding those powers in the 2013 season.
Replicating its increasing admission as evidence in the courts of law, video images have also been employed by sports disciplinary bodies to aid post-match deliberations evaluating the accuracy of officials during the game as well as support or defend charges of foul play arising from games. Those videos are routinely considered demonstrative evidence and perform a role well beyond being purely illustrative. Moreover, the admissibility of video evidence as supporting evidence in sporting (disciplinary) tribunals never appears to be questioned.
Pursuing fairness and concerns over accuracy
Customary acceptance of video evidence can also be seen as conforming to the rules outlined by the Canadian courts, particularly the requirements that the evidence be accurate in truly representing the facts and overall, be ostensibly fair. Indeed, the pursuit of “fairness” is further substantiated by the receipt of all available video evidence in order to try and provide the maximum number of perspectives possible in the search for the “truth” of what occurred. This perhaps goes further than the “best evidence” rules in the law courts and also illustrates that sports tribunals are alive to some of the problems that can result from particular camera positioning.
For example, in Korean Olympic Committee (KOC) v International Skating Union (ISU)3 the Court of Arbitration for Sport was minded to explain that a “different camera showing the same incident from a different position may well give an entirely different perspective of the same incident.”4 Often the law courts do not have such access to multiple perspectives thereby compromising the objective “truth” and fairness of parts of the video evidence proffered, not least because of the implied narrative discourse that a restricted field of view can provide.
However, even if the amount of video evidence tendered may contribute to determinations with regards to “fairness”, the same issues that compromise the admission of evidence in the legal context also affect evidence available to sporting tribunals. Problems remain regarding the accuracy and representation of the “facts” in both the technological and narrative contexts. The great majority of evidence produced technologically (that is, without human “interference”) is uncritically accepted as accurate. This is also reflected in statute: to the effect that where the evidence under consideration has been produced by a machine, device or technical process then the relevant device is considered to have working properly at the time the evidence was created - see New Zealand’s Evidence Act 2006, s 137; and Australia’s Evidence Act 1995 (Cth), ss 146 & 147.5
The Hawk-Eye system
It is not clear that the technological production of data for the adjudication of sporting events, particularly fast moving balls, should be accorded the same level of acceptance of faithful reliability. For example, during the 2007 Wimbledon men’s singles final between Rafael Nadal and Roger Federer, Nadal challenged a shot which had been called out. He was able to do so because after the implementation of the ball tracking system pioneered by Hawk-Eye Innovations, the entities governing international tennis had been operating a system of rules permitting a maximum of three unsuccessful challenges to officials’ line calls per set, with an additional challenge available in the event of a tie-break.
When a player challenges a line call it generates a video review by the Hawk-Eye system. That is, the Hawk Eye image that appears on the screen is a reconstructed representation created from the visual images and timing data provided by a number of high-speed video cameras located at different locations and angles around the area of play. Tennis for example, uses 10 cameras. The cameras track the path of the ball and employ a procedure to filter the pixels in each frame. Certain pixels are taken to represent the position of ball and others to indicate the position of the line or other features of the playing arena. The space and time coordinates of these pixels are represented numerically and a statistical algorithm reconstructs the flight and impact point of the ball and crucial features of the playing area by combining information about the pixels in the different frames with information about the size of the ball, the physics of its distortion, the width of the line, and so forth.6
What viewers see of this effort is the virtual reality graphic. This is either an image or a short video clip showing the path and impact point of the simulated ball. In the case of tennis, information about the likely distortion of the ball upon bouncing is used to estimate the size and shape of the contact footprint and the visually reconstructed bounce point is elongated to represent the way the ball compresses and skids as it hits the ground. These details, together with life-like colours and other details, give the simulation an appearance approximating the real setting, albeit with sharpened edges and idealised precision. Because of its statistical nature, any RTD technology is subject to statistical uncertainty. However sharp the reconstructed images appear, they represent an `estimate’ which has errors that will affect the accuracy of the estimated bounce point, the shape of the reconstructed footprint and the reconstruction of the line.7
Margin of error
Nadal’s challenge resulted in the “Out” call being reversed as Hawk-Eye “proved” that the ball had clipped the line by a millimetre. However, the Hawk-Eye Innovations website indicates that the system has an average error of 3.6mm and there is no indication of the maximum error. The standard size of a tennis ball is 6.54~6.86 cms which means that there is about a 5% error relative to the diameter of the ball. For the sake of comparison, approximately 5% of the diameter is the fluff on the ball’s surface. This realisation could, of course throw into doubt the accuracy adjudications, such as the one made in the 2007 Wimbledon final.
In cricket at least, some level of the statistical uncertainty of the reconstructed images is recognised in the way that the ICC has formulated the parameters for the use of the Hawk Eye system in adjudicating LBW decisions. Even for the on-ground umpire the LBW “in or out” decision requires an extrapolation from what did happen to what would have happened if the ball’s flight had continued uninterrupted.
In the review system, where Hawk-Eye is used to check the umpire’s decision, the crucial element is what has been called `the zone of uncertainty’8 – which it is noted, does not exist as far as the system is used in tennis. The zone of uncertainty is an area about 55 millimetres (half the width of the ball plus half the width of one stump) around the edge of the stumps. In that area, Hawk-Eye cannot over-rule the on-field umpire. Accordingly, in the case where an umpire gives a ‘not out’ decision (i.e. he decides that the ball would not have hit the wicket) and there is an appeal, the umpire is not over-ruled unless the inside edge of the ball is shown to be striking a point at least 55mms inside the outer edge of the wicket. Similarly, if the umpire gives an ‘out’ decision, this is only overturned if the system suggests that no part of the ball was inside the zone of uncertainty.
This systematic allowance for a margin for error is reflected in the guidelines for the use of the Hawk Eye system and there are also zones of uncertainty in respect of the other two judgments that have to be made to apply the rule correctly. These concern the impact point of the ball on the ground as it bounces before striking the batsman and the impact point of the ball on the pad. Where the point of impact on the pad is 3 metres or more from the stumps and/or the distance between the ball bouncing and then contacting the pads is short, the margin for error is magnified.9 The distance or lack of ball flight data can contribute to “tracking error” and the displayed results are commensurately less precise. These factors are also relevant to the freeD 3600 technology which also reconstructs an image from video footage.
It also has to be realised that the Hawk Eye system still requires the consistent intervention of human technicians to fine tune the recordings being made by the cameras – in cricket for example, manually to isolate the point on the trajectory of the ball where interception with the batsman was made. Automatically the system is only able to determine the interception point to the nearest frame of Hawk-Eye video running at 106 frames per second. This can be improved manually and is the only way to ensure that the interception point is accurate to 5mm.
In fact, although the number of recorded instances are very small, it was conceded that “human error” was responsible for the inaccuracies displayed in images of a score in a hurling semi-final game between Limerick and Galway in august 2013. Apparently all but one of the camera settings had been calibrated for the smaller ball10 used in hurling, while one camera was still calibrated for the diameter of a football, which resulted in a skewed result.11
Use as an aid and further limitations
None of this of course, suggests that such technologies cannot aid human adjudication. It remains clear that, notwithstanding identified margins of error, accurate ball tracking systems like Hawk-Eye outperform human observation. Moreover, for purely objective decisions (such as “did the player go into touch?”) or timing problems (for example, “which player touched the ball first?”), slow motion video replays often provide definitive results that are impossible to achieve with the naked eye.
Even then however, the technology is a long way from infallible. In Neykova v International Rowing Federation (FISA) and International Olympic Committee (IOC),12 a Bulgarian rower unsuccessfully challenged her second placing in the women’s single sculls at the 2000 Olympic Games in Sydney. Neykova had been beaten into second by a margin of 0.012 seconds, as determined by the official Swatch photo finish cameras which had been permanently fixed to the structural steel frame of the finish tower.
The Bulgarian Olympic Committee challenged the result based on video evidence provided by broadcast television cameras. The challenge was therefore, essentially based on questioning the reliability of the official photographic equipment in comparison with images produced by TV cameras primarily positioned to produce pictures for a viewing audience.13 In dismissing Neykova’s challenge, the CAS tribunal outlined that:14
“…the television camera was located 10 centimetres ahead of the finish line. When this 10 centimetre discrepancy is projected across the 200 metre width of the course, it is not surprising that the television camera’s perspective is different to that of the … photo finish. … Considering these differences of the television camera, the [Tribunal] is not surprised that the relative position of the boats and the rowing position of the athletes in the boats is different when the television view is compared to the official photo finish.”
That distortion is manifest through the lens of a television camera is graphically illustrated by the use of on-field advertising. Sponsors’ advertising is often painted directly onto the grass of a sports field so that it can be picked up by the broadcast cameras during the game. Even casual observation at such a prepared stadium shows that the on-field painting is often quite misshapen to allow for and moderate both foreshortening and perspective distortion. When captured by the relevant camera however, (that is, the camera for which the necessary angles have been measured and apportioned for in the painted advertisement), the broadcast image is viewable without any angle or design distortion. This process has been further developed by a process that makes a two‑dimensional image appear three-dimensional when viewed on television.15
Angle of perspective
Again, when the image is picked up by the relevant camera, a flat advertising logo appears to be a 3-D image on the screen. All of which says nothing about contemporary digital broadcasts which permit the insertion of “virtual advertising”, enabling the superimposition of advertisements on designated areas and which automatically adjust as the camera follows the action.16 To the television viewer the advertising is recognisable and clear, but the signage does not actually appear at or on the ground at all.
However, it not just the angle of perspective that results in distortion or “through-the-lens” manipulations. The “point of view” shortcomings in relation to film and filmic evidence outlined earlier are also apparent in sporting video. Any broadcast footage remains framed and temporally specific so that what is outside the frame cannot be viewed nor it is always obvious what came before and after the available image.
Moreover, the use of long shots, relative close ups and wide shots of the sporting event can operate to provide a similar narrative perspective as is seen in commercial films. For example, a close-up shot of a robust tackle can make the collision of bodies seem more immediate and brutal than a wide-angle shot of the same tackle. This tendency can be further exaggerated by a slow-motion replay, as was acknowledged in Maloney.17 Such difficulties echo the trade-off between probative value and prejudicial effect recognised by the courts that is often not always appreciated by sporting bodies. It must also be recognised that the perspective provided by the narrative viewpoint of the camera can be (and often is) reinforced by the associated vocal commentary.
The substance of and the dangers of video adjudication are particularly relevant to the unique characteristic of rugby league and rugby union – the requirement to only propel the ball backwards. A forward pass is proscribed by the laws of both games and as defined, a pass is thrown forward when it is propelled in the direction of the opponent’s dead ball line. However that formulation must still allow for the laws of motion. Newton’s first law stipulates that an object that is in motion will not change its velocity unless an external force acts upon it. Most observers usually refer to this principle by speaking about momentum – thus, when a player is running forwards carrying the ball, the ball must be travelling forwards at the same speed as the player and when passed even directly sideways will continue to move forwards at that speed (until acted upon by another force). A video produced by the Australian Rugby Union (ARU)18 outlines this process and further explains that if player is running at half‑pace, a pass made over a distance of 4 to 5 metres “travels about 2 metres forward from the point of release”. Indeed, at full pace, a pass made over a distance of 9 to 10 metres “travels about 5 metres forward from the point of release”. The important realisation is that in both instances, the pass still remains behind the passer as he moves forward and in neither case results in a “forward pass.” Accordingly, adjudication of a forward pass remains highly subjective and involves a significant “sniff test” – that is, everyone knows what a forward pass looks like when they see one.
It is also important to realise that the law defining a forward pass has not changed, it is only the evidence deemed available to adjudicate on one that has been modified. In fact, due to apparent problems when the video referee was first introduced to rugby league, that sport does not allow any ex post facto video adjudication of forward passes. Moreover, even the officials on the ground are instructed as to how they may adjudicate such passes. The notes to Section 10 of the NRL Rule Book outline that “the direction of a pass is relative to the player making it and not to the actual path relative to the ground.”19
The new TMO protocol for Rugby Union is also alive to the problems associated with forward motion and mandates that “the TMO must not adjudicate on the flight of the ball but on the action of the player who passed the ball, i.e. were the player’s hands passing the ball back to that player’s own goal line”.20 Practically speaking then, the adjudication is made on the basis of whether or not the ball was passed backwards out of the passer’s hands. It is also noteworthy that in order to advise the on-field referee of his opinion, the TMO must verify that any offence is “clear and obvious”. This is arguably a higher threshold than the criminal “beyond reasonable doubt” standard of proof and is presumably designed to overcome any problems associated with momentum or camera distortion – but without adequate allowance for the distortion of images and the other technical shortcomings of camera lenses, may still not provide appropriately accurate adjudication.
What must be always firmly kept in mind is that what we see projected in film on television is not real – it is a mere depiction of reality. As the law courts have grappled with the introduction into evidence of “documents” produced by ever advancing technology, so too have sports and sporting tribunals had to deal with similar issues.
The courts have responded by developing ground rules for the admission of technologically-produced evidence but appear to have yet to come to grips with the full impact that images (particularly, moving images) can have on the adjudicative process. Partly that is to do with the way humans process images but is also manifest in the assertive narrative discourse that the language of film authorises. In particular, the demarcation between visual evidence as illustrative as opposed to being demonstrative (and determinative) of facts at issue has been blurred, perilously changing the landscape of decision-making. The remaining technical deficiencies of the technology have also allowed (optical) distortions to cloud the judgment process.
For many reasons, both conceptual and technical, the processing of visual imagery is beset by distortion and illusion. In order to appropriately adjudicate both in the court room and on the sports field adequate allowance must be made for this actuality. In coming to this conclusion it is impossible to ignore the Magritte’s famous painting of a pipe.21 Although it is a realistic image of a pipe, it remains merely an image of a pipe. The painting remains a representation of reality, an idea. The representation cannot be equal to reality because it is an interpretation of it. We are never totally objective and can never show reality in its totality. Magritte himself reputedly explained that if he had included a legend on the painting to the effect that it was a pipe, “I’d have been lying” – and he presciently titled the work, The Treachery of Images.
Similarly, without adequate contingency for the potential perfidy of televised images, we must also be careful to question the veracity of whether what is depicted is actually a forward pass or whether the ball (or the batsman) is actually “out”. Spectators may to have to get used to the realisation that it is unlikely that for example, more forward passes are being thrown in the modern game than were previously, although modern technology allows more of them to be observed. That is, the throwing of passes or the demonstrated position of the ball can be more closely watched and better reviewed which will allow greater accuracy in adjudication – as long as the image per se is not to be considered determinative in every respect. Whether it is in the adjudication of the law generally or of sport specifically, technology can positively assist adjudication, but it cannot by itself reveal truth. The one reality is that the process of adjudication within sport, as within the law, must be based on the construction of contingencies in the making of decisions.22
This work was written for and first published on LawInSport.com (unless otherwise stated) and the copyright is owned by LawInSport Ltd. Permission is granted to make digital or hard copies of this work (or part, or abstracts, of it) for personal use provided copies are not made or distributed for profit or commercial advantage, and provided that all copies bear this notice and full citation on the first page (which should include the URL, company name (LawInSport), article title, author name, date of the publication and date of use) of any copies made. Copyright for components of this work owned by parties other than LawInSport must be honoured.
- Tags: Adjudication | Australia | Australian Rugby Union | British Broadcasting Corporation (BBC) | Canada | Canadian Supreme Court | Contract Law | Court of Arbitration for Sport (CAS) | Dispute Resolution | England | Football | Governance | Hawk-eye | Intellectual Property | International Rugby Board (IRB) | Korea | National Football League (NFL) | National Hockey League (NHL) | New Zealand | Olympic | Regulation | Rugby | Rugby League | Rugby Union | Super League | Tennis | The FA | United States of America (USA) | Video Replay
- Video technology in sports adjudication: Part 1 – use in the courts
- Goal-line technology or instant replay (video evidence)?
- Digital and social media rights: Can the fans be forced to take down online goal videos and should they be?
- Key sports law cases and developments to watch in 2015
Craig Dickson is a senior lecturer in the Law School at AUT University in Auckland where he teaches Sports Law as part of the regular undergraduate LLB electives and he has occasionally appeared in front of sporting disciplinary tribunals. After obtaining an LL.M. at the Centre for Innovation Law and Policy, University of Toronto, he spent some years in private practice before taking up his current position. Craig’s other teaching and research interests include insurance law and intellectual property law and he was recently appointed as inaugural treasurer of the newly formed Asia-Pacific Copyright Association.