Some Thoughts based on 6 Nations penalty data

                                                             At its best                         (c) Getty Images

At the onset of the 6 Nations tournament when looking at some of the talking points for each team I discussed the penalty counts incurred last year and the impact that had for the respective sides. At the time I thought it would be interesting to look back once the Championship had concluded to examine the penalty count as a reflection of Referee performance. You often hear from the respective media of each nation that so-and-so don't like their country and always referee them harshly, and thought it would be interesting to see if there was any evidence of that during these 15 games.

Additionally, the last couple weeks of Super Rugby action has seen some terrible officiating recently that led to a discussion of relieving referees from their duties (which frankly should have happened but alas, another battle). There has been a lot of different debate about where to place the fault of these disputed contests: should it be on the players, who are committing infractions or is the referee and his interpretation wrong? Both sides have merit, poor officiating can have a disturbing effect on the direct outcome of a game, while players can also make poor choices.

The 6 Nations is a great spectacle because each match has the potential to be won by either side (even if it's a small chance) and because each point is valuable. Referees play a role in the outcome of matches, as their interpretation of each action . The point of this article though isn't to get bogged down in semantics about the intent of a referee, so let's look at some statistics!

I compiled to spreadsheets from this year's data here and I will be referencing it so please take a look for your own interest's sake. It is a list of the penalty numbers each team conceded under each official (sorted chronologically by appearance) and the second sheet compares the number of penalty conversions completed by the teams against each other. Simple. Turns out making charts in Excel requires more time then I have patience, so you'll have to enjoy the more static presentation.

I was surprised to see such a wide group of officials used, I had not expected that. I didn't look at nationality of the official because it wasn't an aspect I was concerned about, though maybe some individuals would want to see how many appointments each nation got. The numbers are there for you. 11 Referees for 15 matches seemed surprising to me, especially considering there are only so many available, and I anticipated that the IRB would use a smaller pool. This made it harder to see what sort of overlap there was and if there was anything worth detailing. In hindsight, I can understand why there would be a more diverse use of officials, given there are only so many tier-one Test matches each year, and they a decent pool of capable officials.

One of the things that strikes me is that 3 of the 4 officials that received two appointments (Nigel Owens, Steve Walsh, and Craig Joubert) all had per-game penalty totals lower than the average (20), which means they collectively a) lowered that number and b) had games that featured less penalties and allowed more flowing rugby. These respective officials either had both teams playing within the rule (HA!) or were more tolerant of certain actions/let the advantage go to see what would develop when others might get all whistle song on us. 

Looking at the highs and lows for each nations' penalty count and comparing that with the average for their 5 games was an interesting way to look at this small sample, mostly because it re-enforces the above point with respect to the three highlighted officials, who accounted for 3 of the 6 low points (and one of the high totals, though Ireland were penalised 9 times in 3 different games so that point isn't really something to think about) while Jerome Garces (the 4th double-appointment official) awarded Scotland both its highest and lowest totals, with the former occurring before the latter (follow?). Joubert was the only other official to see a team twice, and he called Ireland on 7 and 9 infractions respectively, so his view of the team did not alter in an appreciable way. 5 games is probably not enough to derive value from a team's performance and discipline and as this chart demonstrates there can be wide variance game-to-game that is dependent on personnel, performance and interpretation. We can say that even though Ireland was the most disciplined team a penalty count of 2 (Round 4) seems ridiculously low and an outlier. France's total against Italy (16) was nearly double their average and not typical of their play, but within the context of a game that had 29 penalties called by Jaco Peyper (or once every 2:46 min, if you prefer) it does not seem as shocking.    

I thought that we might see if there was any sort of inconsistency with the officiating, but after looking at this data it's unfortunately too small of a sample size to draw any conclusions as far as I can tell. I didn't look at data from previous years because penalties and interpretations seem to be changing so much year-over-year that just comparing context-less numbers would be inaccurate, and would also not account for these officials changing their priorities and interpretations of what infractions should be targeted. If we could delve further into the data to see what sort of penalties are being called we could have a better idea about the different aspects the each officials are focusing on. But that would be a lot of manual work from the resources I'm using so we'll leave it for another day.

For the second Excel sheet on the link above I was looking at penalty points conceded by each team, and have recorded the separation between the two totals from best to worst. There aren't too many surprises, if we're being collectively honest:

Wales: +12
France: +9
Ireland: +9
England: +0
Italy: -12
Scotland: -18

Some interesting notes from this:
- I didn't look at all penalty conversion attempts, just those that counted, but after thinking about it I've complied that here!: This showed the amount of attempted penalties conceded by each team, thus we could say the amount of kickable penalties given away. This chart is equally useful, but was a bit of an afterthought by yours truly. 
- Wales were the second most penalized team in the tournament and conceded the most converted penalties while scoring the most points directly from penalties, which lead to their very strong positive differential
- England seem like a team that goes for points quite often, so I was surprised that they ended up with no difference, though on both fronts they were slightly above the average (10.8 for each coincidentally).
- Scotland gave away the most penalties in the tournament and as a result tied (with England) for the second-most converted kicks against. Additionally they left the most points on the board by missing 7 kicks. Their poor offensive showing is reflected in the difference.
- These numbers do not directly reflect the kicking locations and difficulty level. A kick from in front of the posts has a greater chance at being slotted then one from the sidelines, but here they're all the same. Not sure yet where to access that information

These results are really an outcome of territory, so the teams with worse territory percentages tend to give away more kickable penalties because they're in their end more often. Leigh Halfpenny is probably the consensus best place-kicker on the 6 squads, but even his range is limited. So to be in an optimal range for most kickers a team would have to within a 35-40 metre radius of the goalposts for an attempt to make sense. Certainly outliers to this statement exist, but on the whole, penalties between half and the 10-metre line would result in a kick to touch unless there was another factor. 

Which leads us to in-game strategy, and the conscious choice teams make to forsake 3 points for a chance at more. I think this can best be demonstrated by considering the state of Italy and the amount of penalties charged to the Azzurri but the below average amount of kicks at goal that their opponents took. Both Ireland and England were awarded 24 penalties against Italy for numerous infractions in the Round 4 and 5 encounters but each only took one kick at the posts. The play was focused in the Italian end 79% of the time in Round 4 and 68% in Round 5, so both Ireland and England would have had ample chances to pick up 3 points here and there, but both saw a weak opponent and knew that winning the Championship could come down to point differential, and went for field possession to give their teams better chances to win.

Lastly, I made a chart (here) of the percentage of total points gained and conceded from penalty kicks by the teams over this tournament. It highlights how beneficial kicking was for France and Wales, accounting for over 40% of their teams point, while also pointing out how effective the England defense was as 58.1% of the points taken by opponents was via penalty kicks, which is significantly above the average. The inverse of this is Italy's penalty indiscretions only directly took 17.4% of the total points they conceded, but this is somewhat misleading and relates to the above paragraph: They hemorrhaged tries in Rounds 4 and 5, with opponents crossing 14 times. which also explains the deflate the offensive percentages of Ireland and England.

In summary, penalties and penalty kicks played a significant role in this year's tournament, as they accounted for a little under a third (32.5%) of the total points scored, and the 3 least penalized teams were the ones vying for the title on the final weekend. The best offensive (by tries) and defense (by tries and points against) team won the title this year, and their discipline played a key role in Brian O'Driscoll lifting a final international trophy.