Monday, March 13, 2017

CRA website goes dark - Canadians should worry

It was an interesting weekend in Canadian government, and as often happens in my world of data, bits and nerdiness, it meant some news coverage. The story led the agenda during my Clicked In segment on CTV News Channel last night, and continued into the morning with live hits on Newstalk 1010 Toronto with John Moore, and Newstalk 1290 CJBK London with Lisa Brandt and Ken Eastwood. Whenever a major tech story breaks, I like to draft some high-level talking points to give the producers and hosts who I work with a sense of the story, and what I think of it. Here's what I've been using so far for this one...

Here's what we know: The Canada Revenue Agency (aka CRA, aka the folks to whom we pay taxes, aka The Tax Man) took down a large chunk of its online services on Friday afternoon, and they remained offline until Sunday night. The CRA's main website at featured the following message during the outage:
Upon becoming aware of an internet vulnerability that affects some computer servers used by websites worldwide, we took down our online services, including electronic filing, and are taking steps to ensure that all information and systems remain safe.
At this time, we are not aware that any personal information has been affected; however, we continue to assess and remedy the situation.
You can still complete your tax forms, but will have to wait before filing.
What we DON'T know: The CRA isn't saying precisely WHAT that vulnerability that prompted the outage was. Their handling of this matter has echoes of the Heartbleed bug, which was a vulnerability in open-source OpenSSL security code used on millions of websites around the world. In 2014, soon after Heartbleed was discovered in the wild, the CRA took its websites and online filing services down in response to the vulnerability.

We also don't know if an outage at the Statistics Canada website this weekend was related. No one there is saying much of anything, either.

Could it have been Cloudbleed? Without official confirmation from CRA officials, it's impossible to conclude at this time. However, the timing is unfortunate, as Cloudbleed has been a major tech story for the better part of the past 3 weeks.

What is Cloudbleed? It is a bug found in code operated by the Cloudflare web infrastructure company. Cloudflare provides security and hosting services for thousands of major internet sites - including  Uber, Yelp, Fitbit, OkCupid, the Pirate Bay,, Feedly, 4chan, and many more. Thanks to a tiny but significant error in some of Cloudflare's code, sensitive user information from some of these sites was being randomly inserted into web pages when visited by other people.

My $0.02: Whether or not it was Cloudbleed is almost irrelevant. What's crystal-clear is that almost 3 years after the Heartbleed episode, the CRA still hasn't learned its lesson. One would expect government IT personnel to be horizon-scanning constantly, to anticipate weaknesses in systems architecture BEFORE they go critical. Internet vulnerabilities are being discovered All The Time, and IT/security specialists are charged with proactively addressing them. Everyone else manages to deal with these issues appropriately, by installing security fixes, patches and updates on the fly, without having to close up shop for two days.

Yet three years post-Heartbleed (where the CRA was the ONLY major global website operator to actually SHUT DOWN its services) the agency continued to chase its tail, reactively downing the website and related services while it frantically tried to figure out what was going on. The only reason our tax filing services went dark for days is because the people running them are clueless, don't learn from previous mistakes, and no one's holding them accountable.

The Statistics Canada website was also partially down over the weekend, and again no one's saying much of anything, so we don't know if the two incidents are related, what's behind them, or what's being done to address the root cause and prevent them from recurring in future. Neither agency is telling Canadians the whole story, and given this is peak filing season for taxpayers, they deserve answers. And better government IT coverage than they've been getting.

No one's going to complain that the feds made it harder for them to pay taxes for a couple of days. But given the growing signs of IT ineptitude within at least two major departments in  recent days, it's entirely reasonable for Canadians to be concerned.

1 comment:

Anonymous said...

Any other time of the year, and I would have stated that we, Canadians, would not necessarily have placed much heed to CRA's outage. However, with this outage occurring during their supposedly most critical period of the year (aka Tax Time), I would have fully expected that CRA would have had a better crisis communications plan in place: messaging utterly useless!
We are being asked to put a significant amount of trust in CRA with our personal and financial information. The least that CRA can do is ensure that this trust is front and centre throughout such that we continue to hand over that information via their preferred means, their website.
Did CRA's crisis comms folks decide to take the weekend off, and address this incident after the morning coffee break on Monday? Harsh, but not entirely without merit.
CRA seemed to have let the media run with the narrative, instead of owning it: basic crisis comms fail!
Will be interesting to read their follow-ups / excuses on this outage. Cynically, they will probably just issue a quick news release with the hope that it just goes away.
For something so visible and sensitive to Canadians during tax-time, I am hoping that they step-up to better inform Canadians on this outage.