A new case challenging the design of a public transport ticketing system on privacy grounds has broad implications for any organisation which collects personal information, especially in this age of Big Data.

In 2016, an unrepresented NSW resident, Nigel Waters,* launched a legal challenge to the collection of data about his physical movements by Transport for NSW (TfNSW).  After two years of legal arguments, Mr Waters has won his case, with the NSW Civil and Administrative Tribunal finding in February 2018 that the design of the Opal Card system breached the privacy obligations of NSW law, by over-collecting passengers’ personal information.  The implications of this case will be profound.

First, some background.  The Opal Card, like the Oyster card in London and the Octopus card in Hong Kong, is a contactless smartcard used across different modes of public transport.  Launched on a limited scale in 2012, it now covers greater Sydney and surrounding regions, and offers a single ticket with integrated fares across ferries, buses, trains and light rail.  Passengers store value on the card, from which fares are then deducted.

The Opal Card comes in four types:

• Adult (aka Opal Card, coloured black) for adults who pay full-fare
• Child/Youth (aka Green Opal Card) for children who pay half-fare
• Concession (aka Silver Opal Card) for people entitled to a half-fare concession rate by virtue of being tertiary students, unemployed, or one of a number of other concession categories, and
• Senior/Pensioner (aka Gold Opal Card) for seniors, aged and disability pensioners and some other categories of individuals, who pay a flat $2.50 per day of use.

However only Adult and Green Opal Cards can be purchased and used anonymously.  Registration of the passenger’s card is optional for children and full-fare-paying adults, but compulsory for Silver and Gold passengers.

Registration of a card means that the card itself is linked to an identifiable individual.  For passengers, the benefits of registration include being able to top-up the stored value automatically, or to cancel the card and retrieve its stored value if the physical card is damaged, lost or stolen.  For TfNSW, which operates the system, the benefits of registration include being able to ensure that people entitled to concessions can only have one valid card issued at a time, and that the cards of passengers whose entitlement to a concession has expired can be remotely cancelled or suspended.  (On-going entitlement is routinely checked by TfNSW in a data-matching process, for example verifying with Centrelink whether people are still on unemployment benefits, and with universities to check whether students are still enrolled full-time.)

Mr Waters did not object to the collection of information about his identity.  Nor did he object to the processes by which he had to initially demonstrate his eligibility to claim the seniors’ entitlement to the Gold concession rate.  Importantly, he also did not object to any requirement to demonstrate his entitlement to the seniors’ discount whenever he was travelling, such as if challenged by a ticket inspector; or to the process by which his on-going entitlement is periodically verified by data-matching with the issuer of Seniors cards.

But what Mr Waters did object to was the by-product of registration – the creation of a record of his physical movements when using public transport, linked to his identity via the Card number.  (The physical movements of all cards are tracked and the data can be easily interrogated; but only if the card is registered can TfNSW link that card’s use back to the identifiable individual assumed to be using that card.)

In other words, the complainant objected to the fact that some passengers could choose to use public transport anonymously – i.e. without their physical movements as identifiable individuals being tracked by an arm of the State Government – but others could not.

He brought his challenge by alleging that TfNSW was in breach of Information Protection Principle 1 in the Privacy and Personal Information Protection Act 1998 (NSW), which requires:

(1) A public sector agency must not collect personal information unless:

(a) the information is collected for a lawful purpose that is directly related to a function or activity of the agency, and

(b) the collection of the information is reasonably necessary for that purpose.

(2) A public sector agency must not collect personal information by any unlawful means.

The critical phrase tested in this case was “reasonably necessary for that purpose”.

TfNSW had characterised the reason for compulsory registration of Silver and Gold cards as to manage cases of fraudulent claiming of concession fares.  The question was therefore whether the collection of data about passengers’ physical movements, in an identifiable form, was “reasonably necessary for that purpose”.

And after 50 pages, the Tribunal concluded no: the collection of the travel history data is not reasonably necessary for a fraud-prevention purpose:

“there seems little basis for the collection of the travel information for the stated purpose of enforcement/eligibility for the entitlement to the concession card”.

The Tribunal found that TfNSW had therefore breached IPP 1 in its design of the Opal Card system, and ordered TfNSW to stop collecting Mr Waters’ travel history data.

The Tribunal also found that while it could not order the same outcome directly for all other Gold Opal Card customers (because Mr Waters did not have standing to make a claim in relation to the personal information of anyone other than himself), it also noted that its findings as to the breach of IPP 1 “would be applicable to persons who wished to avail themselves of an unregistered card”.  In other words, the Tribunal is saying that because every other Gold Opal Card customer could make the same privacy complaint as Mr Waters and receive the same outcome, TfNSW may as well assume that the Tribunal’s orders apply to all Gold Opal Card customers.  (Although not mentioned, one would expect the same outcome would also apply for Silver Opal Card customers.)

Importantly, the Tribunal found that “some action must be taken … to make the system compliant”.  While the Tribunal didn’t say exactly how the Opal Card system should be re-designed, it recommended the agency take legal, privacy and IT design advice, and look at equivalent systems in Queensland, Victoria and Hong Kong.

There were some interesting twists and turns along the way as this case developed over the past two years, which illustrate a number of important points about the maturation of privacy law and practice.

Whether the data was ‘personal information’

Although TfNSW began its case accepting that the data in question was ‘personal information’, and indeed the published Opal Card Privacy Policy asserted as much, as Mr Waters’ matter was progressing the convoluted Grubb v Telstra / Privacy Commissioner v Telstra decisions came along.  Seizing this opportunity, TfNSW argued the view taken by the AAT in the Grubb case that geolocation data collected from Mr Grubb’s mobile phone was ‘about’ his phone, rather than ‘about’ Mr Grubb.  Applying it to the Opal Card scenario, TfNSW suggested that travel history data was ‘about’ the Opal Card, not about Mr Waters, and thus did not meet the definition of ‘personal information’.

The Tribunal made short shrift of this argument.  The Tribunal noted the full Federal Court’s view, which was contrary to that expressed in the earlier AAT case, that information can be ‘about’ more than one thing.  In any case the Tribunal found that “the travel information was more about CNS* than about the card. There was no purpose attached to the card information … that was not about CNS”.

TfNSW also argued that because the card registration data is held separately from the travel data, with the former being held by TfNSW in the ‘PAS database’ and the latter in the ‘Opal database’ which is managed by a contracted third party, and because the two datasets were not routinely linked and did not offer a ‘live tracking’ function, Mr Waters’ identity was not apparent or reasonably ascertainable from the travel data ‘by itself’, and thus did not meet the definition of ‘personal information’.

Again the Tribunal rejected this argument.  The Tribunal noted that regardless of whether or not TfNSW routinely links the two datasets, the fact remains that they are linkable, by virtue of the card number being present in both, and data is linked between the two by TfNSW in order to respond to requests from either law enforcement agencies, or from customers themselves, including by customer online queries.

The Tribunal found that information about “the tapping on and off at various locations was information about CNS, as his identity could be ascertained”.  Thus the Tribunal concluded that the travel history was ‘personal information’ covered by the IPPs.

Yet another nail in the ‘notice and consent’ coffin

TfNSW also argued that Mr Waters had ‘consented’ to the collection of his travel history data, because in order to obtain a Gold Opal Card he would have had to first tick that he ‘agreed’ for his personal information “to be used as outlined in the Opal privacy policy”.

The Tribunal rejected this argument, on two grounds.  First, because ‘consent’ does not provide an exemption to IPP 1.  In other words, whether or not an individual consents to, agrees with, is lukewarm about or violently objects to, the collection of their personal information is entirely irrelevant to IPP 1, which is about whether or not the collection is both lawful and ‘reasonably necessary’.

Second, the Tribunal noted that in any case, the notion of ‘consent’ in privacy law, such as might be relied upon as the basis for later secondary uses or disclosures of personal information, is predicated on “whether an individual acts in a purely voluntary manner”, compared with “something more akin to a lack of choice”.

While these later observations were only obiter comments by the Tribunal, they provide yet another example of the futility of the US-driven approach to privacy protection, which is to pretend that everything will be tickety-boo so long as you bury the detail about what you are planning to do with people’s data in privacy policies and notices, and make your customers ‘agree’ to them as part of your standard terms and conditions.

That’s not how privacy law works in Australia, and it’s not how privacy law works in most of the rest of the world.  The GDPR, the new European privacy law commencing in May, will claim the credit for the coming revolution in how ‘consent’ is managed, but it’s the same position as we have had here for decades.  Australian privacy case law, and guidance on the meaning of ‘consent’ from Privacy Commissioners both state and federal, has been consistent on this point, but Australian privacy law just doesn’t get the kind of airplay that the GDPR does.

So website designers and lawyers everywhere take note:  Notice is not consent.  Terms and conditions are not consent.  Opt-out is not consent.  Making people ‘agree’ to your privacy policy is pointless.  It’s time to separate out your collection notices from your consent forms and your privacy policy, and know when each one is needed, but understand that none of them offer an exemption from the requirement to only collect personal information that is ‘reasonably necessary’ in the first place.

Implications for Big Data

Much of the value of Big Data is built on our digital breadcrumbs – the digital traces we leave behind as we go about our day-to-day activities like travelling to work, buying goods, using social media or searching the web.

But if an organisation does not have a sound reason for collecting those breadcrumbs – in other words, if collecting our data is not reasonably necessary for the primary purpose for which we were transacting in the first place (getting on a bus, buying a pair of shoes, chatting to our friends on Facebook) – then it might not be able to lawfully collect it at all.

It’s right up there as privacy principle number 1 here in NSW: don’t collect personal information unless you really, truly need it for your primary purpose.  And yet this most fundamental of privacy principles is so often ignored.

Ultimately, the impact of this case is to place organisations everywhere on fair warning: ignore the Collection Limitation rule (known in other jurisdictions as Data Minimisation) at your peril.

The importance of Privacy by Design

For years, advocates of the idea of Privacy by Design have asserted that it is better to design privacy in from the start, and to have pro-privacy settings as the default, than to try and retro-fit a system later.

The Opal Card system design is the perfect illustration of the wisdom of that theory.  As the Tribunal said in this case, the collection of the travel history data in an identifiable form was just a by-product of the system design:

“The respondent has never strongly submitted that it desires the travel history of the Gold Opal card holders, merely it would appear that this is a necessary attendant function intertwined with the technology … In some ways the movement history function … is an unintended or unnecessary functionality that the respondent has no view about. Whilst the respondent clearly argued that travel history is useful to law enforcement situations (where authorised), significantly much of that travel history involves unregistered cards which cannot (via the Opal technology alone) be matched to any individuals.”

The Opal Card system could – and should – have been designed differently from the start.  (Indeed, as part of his case Mr Waters sought to bring forward evidence not only about how public transport ticketing systems work in other jurisdictions in more privacy-protective ways, but also about the political promises made years ago, early on in the Opal Card’s design that the system would allow anonymity for all passengers; and how the former NSW Privacy Commissioner’s criticisms of the later design had been ignored.)

For organisations still not convinced of the need to properly consider privacy in the design of their systems, right from the start, this case might be the wake-up call they need.  Otherwise watch out: when the GDPR commences in May, organisations which offer their goods or services to (or monitor the behaviour of) people in the European Union will be subject to an updated privacy law, which requires organisations to practice ‘data protection by design’.  The penalties for non-compliance are significant.

Where to next for the Opal Card system?

As mentioned above, the Tribunal found that TfNSW has to do something to “make the system compliant”.  This doesn’t mean free public transport for everyone, or undermining the means by which fares are accurately deducted from Opal cards.  It doesn’t mean dismantling the entire concession card system, or changing the rules about the compulsory registration of Silver and Gold cards, or giving up on enforcing rules around verifying on-going entitlements to concessions.  But it does mean that TfNSW will need to re-design its processes well enough, to stop collecting the travel history data, in an identifiable form, of those customers who do not wish that data to be collected.

The key will be to irretrievably break the nexus between a card’s trip data and the identity of the card holder.  If the trip data exists in complete isolation from any reasonable means of identifying who was likely using that particular card, it will no longer be ‘personal information’, and thus IPP 1 will not apply.

Without knowing exactly how the databases are constructed, I imagine that one potential solution would be to assign a new, randomly generated identifier for each card number in the ‘Opal database’, and then strip out the original card number.  Such a mechanism would need to ensure that the new identifier could not be reverse-engineered back to the original card number, and that the new identifier could not be guessed or re-generated by entering the original card number (which would still held with identity details as part of the customer profile in the separate ‘PAS database’), which would rule out a simple hashing algorithm.  This still would not guarantee perfect anonymity, because geolocation data is highly identifying on its own.  That is, the patterns illustrated by an individual’s movements could of themselves allow disambiguation of an individual, and thus further steps such as applying differential privacy might also be needed to achieve true anonymity; but this is an existing problem even for those customers with unregistered cards.

Who this mechanism should be applied to would be the next question.  If you applied it to all card holders, this would disadvantage those customers who want to be able to cancel their card and redeem its stored value if their card is damaged, lost or stolen.  (Note that for those customers who wish to save and interrogate their travel history – for example, to query whether they have been charged correctly, or to claim the cost of trips that are work expenses – this can be done now even on unregistered cards simply by querying the card number.)  Removing the nexus to identity for all customers would also make the data far less valuable to law enforcement; a tempting prospect for a privacy advocate, but in NSW politics unlikely to pass muster.

If you applied this mechanism only to customers who want anonymity, then the question is which setting should be the default.  Should having your travel history recorded in an identifiable form be opt-in or opt-out?  If the collection was to be based on consent, I would say it would have to be opt-in.  But given that ‘consent’ is not a way of avoiding IPP 1’s requirements that the collection be ‘reasonably necessary’ for a lawful purpose, what is instead needed is a framing of the purpose of collecting travel history data, to give it a purpose, as distinct from collecting the identity-related and concession-entitlement data.

If you frame the rationale for collecting trip data in an identifiable form as ‘to provide an added-value service to customers at their own choice’, then the collection could be argued as ‘reasonably necessary’, but only for those customers who want their movements tracked.  (If you frame it as ‘because we want to create a surveillance state in which the Government can tell where everybody is’ … well, then you might have less of a problem with the ‘reasonably necessary’ part of IPP 1, but suddenly run into problems with the ‘lawful purpose that is directly related to a function or activity of the agency’ part of IPP 1.  Not to mention some potential public push-back.)

My view is that passengers entitled to a concession fare should be entitled to the same choice as full-fare-paying customers already enjoy, which is being able choose anonymity.  The most privacy-protective stance would be to design the system to be opt-in, with no data collection about travel history (in an identifiable form) as the default setting.

Offering all passengers, no matter how much they pay for their public transport, the option to travel about this fair city without an arm of Government knowing where they are is what the Government originally said it would do.  It’s what the Privacy Commissioner told them to do.  And now, thanks to one unrepresented but determined senior citizen, it’s what the Tribunal has said they must do, in order to comply with the law.

I look forward to Transport for NSW now just getting on with it.

* A few days after I first wrote this blog, the published Tribunal case was de-anonymised.  Where the complainant had previously been known simply as ‘CNS’, which was the name I used in the original version of this blog, on 12 March the Tribunal altered the judgment, at the request of the complainant, to show his real name.  On 13 March I edited this blog to reflect that change.  Nigel Waters is a long-time privacy advocate and privacy professional.

(April 2018 update: If you would like some privacy tools to help you assess the risks posed by a new project, or if you are wondering how the GDPR’s requirement to incorporate Data Protection by Design should be implemented in practice, our Compliance Kits are now available.  The privacy tools included in our Kits include a template Privacy Risk Assessment Procedure & Questionnaire you can download and easily customise for your organisation for use when conducting Privacy Impact Assessments or privacy audits, as well as a template Privacy Policy.  Both templates cover both the GDPR and the Australian Privacy Act.  Our Comprehensive Compliance Kit also includes a GDPR Compliance Checklist, and loads more templates, checklists, eBooks including our guide to Demystifying De-identification, and eLearning modules, including on identifying and mitigating privacy risks in projects.  Check out our range of Compliance Kits to see what suits your needs.)

Photograph (c) Anna Johnston

On January 28, when many Australians were enjoying the last hurrah of summer holidays and getting the kids ready to go back to school, our northern hemisphere colleagues were celebrating International Data Privacy Day.  A day to reflect on privacy challenges, and to applaud the efforts of Privacy Officers, and all who support them.  One of the photos doing the rounds on Twitter was a t-shirt bearing the slogan ‘Keep Calm and Trust the Privacy Officer’.

The Privacy Officer’s role is a delicate balancing act – helping your organisation answer the critical question: How can we best realise the value of the data we hold, while still protecting our customers’ privacy?

But for someone whose job involves a complex mix of fighting fires, ticking boxes, holding hands, telling truths, predicting the future and sometimes saying ‘no’, how does the Privacy Officer demonstrate her or his worth?  How can organisations be persuaded that there will be a positive return if they invest in a robust privacy management program?

We hear a lot about how customer and citizen trust is essential to maintaining an organisation’s reputation, but … can you quantify it?

I have long been interested in this question.  For the privacy officers out there, you need to know that doing good work – championing Privacy by Design for example – will lead to good outcomes for your business or government agency.  But where is the evidence?

Some years ago, the UK Information Commissioner’s Office commissioned The Privacy Dividend, which concluded that customer trust, gained through proactive privacy protection, delivers business value.  More recently the IAPP published a white paper on the ROI of Privacy, which noted that privacy protection can be used to “secure brand trust, contribute to the bottom line and gain competitive advantage”.  And a study released by Cisco found that privacy-mature organisations experience shorter sales delays, fewer data breaches, and smaller losses from cyberattacks.  But sometimes I feel like these efforts to quantify the value of privacy protection go unheeded.

Privacy regulators have long sprouted the line that ‘privacy is good for business’, but as my colleague Steve Wilson loves to point out, it ain’t necessarily so.  For all we laud Apple and Microsoft’s legal stands against the US Government to protect the privacy of their customers, in fact many of today’s tech giants have thrived on hoovering up personal information and re-purposing it for their own ends.

It is difficult to confidently articulate a positive ROI on protecting personal information, when others seem so hell-bent on finding the ROI in exploiting it instead.  Consider this example, from a story explaining how Bluegogo, one of the dockless bike share companies, went spectacularly bust in November 2017, having burnt through almost $120M in venture capital churning out bikes:

“none of the share bike companies have a working business model and are instead burning through venture capital to place their bikes on the street in a race to become the market leader. They hope that the data the GPS-enabled bikes generate could lead to revenue opportunities in the future.”

That’s right, some mugs stumped up $120M on a scheme that might have been able to exploit the personal data of its customers, in some unknown way, at some indeterminate point on the future, to possibly earn some income, which may or may not have been enough to pay them back.  (Pssst, hey rich and reckless venture capitalists, let me show you this bridge I’ve got for sale.  It’s on the blockchain, so you know it’s gonna be amaaazing.)

Even the recent revelations about Facebook and Cambridge Analytica have simply exposed the brutal truth at the heart of Facebook’s business model.  When you consider that the loss of around US$70 billion in share-market value in the first 10 days after the Cambridge Analytica story first broke still only represents around a 13% decline in Facebook’s value, there is clearly oodles of money left in a business model which is predicated on collecting, generating and inferring personal information about individuals, in order to use and sell the data for unrelated purposes.

So as much as I would like to tell you that governments and businesses have all jumped on board the ‘privacy is good for us’ bandwagon, it’s not entirely true.

While smart CEOs will realise that the Privacy Officer can be a source of strategic value for the organisation, others will still only see privacy as a compliance cost, rather than an investment.  I know this is a situation that many Privacy Officers are in.  For those of you in that boat, focusing on the bad instead of the good may be the only way to get your message through to the C-suite.  Case studies about stuff-ups might work better for you than promises about improving customer trust.

A couple of recent examples highlight the importance of ensuring privacy risks are properly considered before projects are rolled out.

In the first example, insurers AAMI and Suncorp launched an online insurance quote tool, which then had to be rapidly shut down when customers began pointing out that the tool could be used to look up whether or not any address had particular safety features like deadlocks or an alarm.  The predictable fear was that of burglars, but my first thought was for victims of family violence or other people facing physical safety risks.  (By the way I believe that the insurer’s claim that there was no privacy issue because the information related to buildings not people is well off the mark, both in terms of the scope of what is ‘personal information’, and customer expectations about how data will be treated.)

And the second example is that of a change to Vodafone’s IT systems, which “allowed customers to self-select online that their identity had been verified in store, without any further check that this had actually occurred”.  More than 6,000 pre-paid mobile phones were sold over a 12 month period, without the identity checks required by law for anti-terrorism and other law enforcement reasons.

If conducting Privacy Impact Assessments (PIAs) is routine in your organisation for all projects (whether big ticket items or minor changes to existing systems or processes), and if your PIA methodology includes mapping out the data flows and testing customer expectations, these are the kinds of errors which should be picked up at the design stage.  Preventing costly disasters is a critical way the Privacy Officer can add value to their organisation.

Of course, it can’t be the Privacy Officer’s job alone.  Everyone needs to be on board if you are going to tackle privacy risks properly.

We conducted a survey of Privacy Officers in 2017, and asked them an open-ended question: ‘What do you see as your organisation’s biggest challenge?’  We were expecting to hear about hot topics like data analytics, de-identification and artificial intelligence, but instead the majority of responses boiled down to one thing: the need for better staff awareness.  Not sexy, but so important.

Employees need to be actively engaged in good privacy practices for a privacy management program to be effective.  Staff need training to be able to understand their obligations, know how to implement those obligations in practice, recognise and report privacy near-misses and breaches, know how to handle complaints, and remember where to go for advice.  Plus anyone involved in projects, IT or otherwise, needs extra skills-based training to be able to identify and mitigate privacy risks when designing projects or implementing changes.

A Privacy Officer’s job is complex enough.  It shouldn’t have to also involve constantly quantifying or justifying the role’s very existence.  The value for organisations in having a skilled and dedicated person in charge of a broader privacy and data management program should be self-evident.

But since that message has apparently failed to get through to some organisations, from July 2018 the new Privacy Code for Australian government agencies will make it a legal requirement for those agencies to have a dedicated Privacy Officer, as well as a ‘Privacy Champion’ drawn from their senior executives.  (The Code will also make PIAs and staff training about privacy responsibilities mandatory for Australian government agencies.)

So in much the same way as the GDPR is driving demand for more privacy officers in private sector businesses around the world, expect to see a jump in government job adverts coming your way soon.  Might be time to polish your CV, or ask for a pay rise to stay put.  But meanwhile, keep calm, dear Privacy Officers, and carry on.

PS – If you need a hand walking the tightrope of privacy protection, give us a call, or check out our new Compliance Kits to help you build your privacy management program.  They include a swag of template policies and procedures you can quickly customise for your organisation (including a template Privacy Risk Assessment Procedure & Questionnaire to help with PIAs), as well as a staff manual, checklists, online training modules, and eBooks. We’ve updated the Kits recently to include extra resources like a GDPR Compliance Checklist too.

Photograph (c) Anna Johnston

How do you solve a problem like Facebook?
How do you catch a cloud and pin it down?*

By now we all know the story: Facebook allowed apps on its social media platform which enabled a shady outfit called Cambridge Analytica to scrape the profiles of 87 million users, in order to serve up targeted ads to benefit the Trump election campaign in 2016.  More than 300,000 Australian users of Facebook were caught up in that particular example of data harvesting, despite only 53 Australians using the app.

Sitting here in Australia, you might be thinking: So what? I never saw a Trump ad, or if I had I would have ignored it because I’m not in America. Or even if the same thing happened here, it’s just ads anyway, I can make up my own mind.

But that’s not the whole story.  The Facebook scandal is about so much more than serving up ads in a foreign election campaign.  Facebook, and other companies involved in data mining and analytics, are invading our privacy and harming us economically and socially, in ways that are only just starting to become clear.

It’s not just the data you choose to share

Up until recently, Facebook has been successful in steering most discussions about privacy towards either information security, or a focus on the controls that users have over what they post, and who they allow to see those posts.  CEO Mark Zuckerberg likes to say that users have choices, and that users stay in control of what they choose to share.

In one sense that’s true: you get to choose which photos you post on Facebook, and which inspirational quotes or worthy news stories you share with your friends.

But in another sense it’s not true at all.  Because the information you post is not the whole story.  It’s only the tip of the iceberg of data Facebook has collected about you.

Every time you go online, you leave a trail of digital breadcrumbs.  Facebook has been busily sweeping up those breadcrumbs, and using them to categorise and profile you.  Facebook obviously knows when you click on a Facebook ‘like’ button; but also, unless a web developer has gone out of their way to find tools to block them, Facebook knows every time you simply look at a website that has a Facebook ‘like’ button somewhere on it.

So if you only post or ‘like’ stories about inspirational mountain climbers and funny cat videos, but also do things online that you don’t share with your family, friends or work colleagues (like looking at stories about abortion or dealing with infidelity, googling how to manage anxiety or erectile dysfunction, whingeing about your employer in a chatroom, or spending hours reviewing dating profiles, gambling, playing Candy Crush or shopping obsessively for shoes) – Facebook has you pegged anyway.

Plus, Facebook obtains data from other sources which know about your offline purchases, to build an even richer picture of who you really are.  And of course, Facebook may have access to your address book, your location history, the contents of your private messages, and depending on your brand of phone, possibly even a history of your phone calls and text messages.

(And even if, like me, you have never had a Facebook account, they still monitor and create ‘shadow profiles’ on non-users, based on information scraped from other people, over which we non-users have zero transparency or control.  Facebook has been conducting what my colleague Steve Wilson and I have described as unlawful indirect collection of personal information, including photographs for facial recognition purposes, for many years now.  Regulators are only just starting to have some success in pushing back, with a Belgian court finding in favour of the Belgian DPA that Facebook’s collection of data on non-users is illegal.)

All that information is used to draw inferences and assumptions about your preferences, and predict your likely behaviour.  The results are then used to categorise and profile you, and ultimately target you, in a process usually described as ‘online behavioural advertising’.

It’s not ‘just ads’

The objective of online behavioural advertising is to predict your purchasing interests, and drive a purchase decision.  So far, the same as any other advertising.  But online, the implications for us as individuals are much greater.

In the hard copy world, advertisers will choose what ads to place in which newspaper or magazine, based on the target audience for that publication, and what they know about the demographics – in aggregate – of the readership.  You might place an ad for a luxury sedan in the Australian Financial Review, an ad for a family SUV in the Australian Women’s Weekly, and an ad for a ute in Fishing World.  Anyone can walk into a newsagent or library, and buy or flick through a newspaper or magazine.  Everyone looking at that newspaper or magazine will see exactly the same ads as everyone else.

But in the digital world, advertisers might want to find busy middle class mums – if that’s their target market for a family SUV – no matter what they read online.  Ad space is sold according to precisely who they want to target.  Enter micro-targeting.  Facebook’s promise to advertisers is that it can find exactly who you want, and show them your ad – and exclude everybody else.  So two people reading the same newspaper story, or looking at the same website at the same time, will see two different ads.

However by allowing exclusion, the platform also allows discrimination.  Facebook has been caught allowing advertisers to target – and exclude – people on the basis of their ‘racial affinity’, amongst other social, demographic, racial and religious characteristics.  So a landlord with an ad for rental housing could prevent people profiled as ‘single mothers’ from ever seeing their ad.  An employer could prevent people identifying as Jewish from seeing a job ad.  A bank could prevent people categorised as ‘liking African American content’ from seeing an ad for a home loan.

The opaque nature of online behavioural advertising also allows fake ads and fake news to proliferate.  Further, the content we see is so filtered that we each live in an individually tailored echo chamber which serves only to reinforce stereotypes, or push people towards extremism.  (Consider for example the Facebook ads used for a biopic about the hip-hop band NWA: the ad for ‘white’ audiences highlighted gang culture, guns and police chases, while the ad for ‘black’ audiences suggested their music was art and a form of non-violent protest.)

Existing patterns of social exclusion, economic inequality, prejudice and discrimination are further entrenched by micro-targeted advertising, which is hidden from public view and regulatory scrutiny.

Predictive analytics can narrow or alter your life choices

Once we move beyond straight-up advertising and into predictive analytics, the impact on individual autonomy becomes more acute.  Big Data feeds machine learning, which finds patterns in the data, from which new rules (algorithms) are designed.  Algorithms predict how a person will behave, and suggest how they should be treated.

Algorithms can lead to price discrimination, like surge pricing based on Uber knowing how much phone battery life you have left.  Or market exclusion, like Woolworths only offering car insurance to customers it has decided are low risk, based on an assessment of the groceries they buy.

(So when you read about companies ‘tailoring their offers’ for you, it’s not just discounts they could be offering you.  It can mean the price you see is higher than another customer; or you might not see the product or service exists at all.)

Banks have been predicting the risk of a borrower defaulting on a loan for decades, but now algorithms are also used to determine who to hire, predict when a customer is pregnant, and deliver targeted search results to influence how you vote.

Algorithms are also being used to predict the students at risk of failure, the prisoners at risk of re-offending – and then launching interventions accordingly.  Based on some deeply unethical psychological experiments it conducted on 700,000 unsuspecting users some years ago, when it played with altering news feeds to manipulate users’ emotional states, Facebook now believes it can predict people at risk of suicide, and offers intervention strategies to help.  Even leaving aside the accuracy of that claim, interventions are not all well-intentioned.  It was revealed last year that Australian Facebook executives were touting to advertisers their ability to target psychologically vulnerable teenagers.

Instead of asking or assessing us directly, business and government decisions about us are increasingly being made according to algorithms, designed on the basis of correlations found through Big Data processing.

Automated decision-making diminishes our autonomy, by narrowing or altering our market and life choices, in ways that are not clear to us.  People already in a position of economic or social disadvantage face the additional challenge of trying to disprove or beat an invisible algorithm.

In a predictive and pre-emptive world, empathy, forgiveness, rehabilitation, redemption, individual dignity, autonomy and free will are programmed out of our society.

Privacy is a collective right

Waleed Aly has written about how privacy (or more precisely the lack of it) is no longer an individual’s problem – it has become society’s problem:  “In the networked world of Facebook, your lack of privacy is everyone else’s problem. You could dump Facebook altogether and you’d still be living in a country whose democracy is vulnerable to corruption in new ways”.

Fiddling with users’ privacy settings on Facebook won’t fix a thing.  Aly warns us against being ‘duped’ by promises to improve controls set at the individual level.  Instead, we need collective, political action.  Scott Ludlam has similarly argued that this latest Facebook scandal should be the catalyst we need to “draw a line under surveillance capitalism itself, and start taking back a measure of control”.  We need to remember that above all we are citizens first, consumers and users second.

If we want our lives to be ruled by human values and individual dignity, instead of by machines fed on questionable data, we need robust, enforced and globally effective privacy laws.  Specifically, what we need is for the American legislature to pass effective privacy laws which rein in Facebook and the data brokerage industry, imposing limits on what personal information they are allowed to collect, and the purposes for which it can be used.  The self-regulatory model of privacy protection favoured in America (but rejected by most of the rest of the developed world) has failed us all.

The GDPR commences this week.  The obligations include that businesses and governments must offer understandable explanations of how their algorithms work, and allow people to seek human review of automated decision-making.  This is a step in the right direction, which Australia, the US and the rest of the world should follow.

* with apologies to Rodgers And Hammerstein

Photograph (c) Shutterstock

If 21 is the age at which a person is considered to have matured, what are we to make of a law when it turns 21?

2019 marks the 21^st birthday of PPIPA (aka the Privacy and Personal Information Protection Act 1998), the key privacy statute in my home state of NSW.  After such a long infancy, it seems an appropriate time to reflect on the law’s effectiveness.

Does PPIPA deliver the goods?  After 21 years, I tend to wonder: do the good people of NSW actually enjoy a greater degree of privacy protection than, say, their cousins in WA – who by contrast have no equivalent privacy law covering their state government agencies, local councils or public universities?

I have reason to doubt it.  Several reasons, in fact.  There are the problems with disturbing loopholes in the legislation, which I have written about before; details like the fact that the maximum compensation payable to a person who has suffered significant harm is set too low and hasn’t been increased in 21 years; and the continued under-resourcing of the NSW Privacy Commissioner’s office.

But today I am more concerned about a really fundamental question.  Given PPIPA is recognised has having both a beneficial and a normative purpose – in other words, the legislation as drafted was intended to set new standards across the public sector, to the benefit of individuals’ privacy – is it working?

Agencies not embracing Privacy by Design

For years, advocates of the idea of Privacy by Design have asserted that it is better to design privacy in from the start, and to have pro-privacy settings as the default, than to try and retro-fit a system later.

The on-going Opal Card system design is the perfect illustration of the wisdom of that theory.  Over some years, privacy advocate Nigel Waters argued that the collection of data about his travel history – his physical movements – was in breach of the collection limitation principle (known in other jurisdictions as data minimisation), IPP 1, because knowing information about his movements as a passenger was not reasonably necessary for the agency to pursue its lawful purpose of enabling or verifying his entitlement to a concession fare.

And in a ground-breaking case last year, the Tribunal agreed, finding that the collection of travel history data in an identifiable form was just an unnecessary by-product of the system design.

As I have written before, the Opal Card system could – and should – have been designed differently from the start.  (Indeed, as part of his case Mr Waters sought to bring forward evidence not only about how public transport ticketing systems work in other jurisdictions in more privacy-protective ways, but also about the political promises made years ago, early on in the Opal Card’s design that the system would allow anonymity for all passengers; and how the former NSW Privacy Commissioner’s criticisms of the later design had been ignored.)

So in order to comply with the Tribunal’s decision and IPP 1, Transport for NSW should now be busy exploring more privacy-protective design options, such as allowing individual passengers to choose to have their travel history data de-linked from their identity data, or otherwise anonymised.

However instead Transport for NSW decided to appeal.  In a ruling last August, the Appeal Panel determined that the Tribunal made an error in the way it cast the purpose of the travel history data.  Thus far the Appeal Panel has set aside the earlier decision, and determined to conduct a new hearing.

One thing to note however, is that the Appeal Panel hinted that it was taking a dim view of the agency making post-hoc justifications for collecting travel history data that were not mentioned in the original case, and/or not included in the collection notice provided to customers.

If anything, coming up at this late stage with new arguments about why the travel history data might be necessary in an identifiable form only serves to cement the conclusion that from the outset, TfNSW did not consider the right to anonymous transport when making key design decisions, despite knowing both their obligations under privacy law and also that successive NSW Privacy Commissioners had expressed the need for anonymous travel options for all passengers.

We will find out soon enough if the Appeal Panel comes to a different conclusion about whether the design of the Opal Card system is in breach of IPP 1.

If the Appeal Panel allows Transport for NSW to succeed in casting a wide net in its interpretation of what data is ‘reasonably necessary’, the agency could continue collecting personal information without forethought, further encroaching on the privacy of public transport users.  Other agencies will gratefully receive and act on that message, allowing the open slather collection of data, or implementation of monitoring or surveillance systems, without having to weigh up the impact on privacy, or consider less-intrusive options.

However if the Appeal Panel upholds the principle of collection limitation, then all public sector agencies would be sent a strong message about ‘privacy by design’, and the need to think carefully when designing new systems: The need to pause and think about why they are proposing to collect personal information, and to only proceed if they can justify the data collection by reference to a legitimate purpose, and with evidence that the data collection will actually achieve that purpose.

An absence of representative complaints

So if the existence of privacy principles alone has not driven change over the past 21 years, we need robust enforcement to ensure the law has the standard-raising effect it was intended to have.  Enforcement can be regulator-driven, or caselaw-driven.

One of PPIPA’s success stories has been its mechanism enabling access to justice.  By allowing complainants to seek an external review of conduct in an independent tribunal, without requiring legal representation, NSW has seen several hundred privacy cases decided in NCAT, and its predecessor the ADT, since 2001.  (By my count, there have been over 400 reported judgments made under PPIPA and her sister law HRIPA, including interlocutory and appeal decisions.  We annotate them all in our quarterly guide, PPIPA in Practice.)

Compare that with the handful of cases brought under the federal Privacy Act, which has been around since 1988 but which requires complainants to lodge their cases in the much more expensive Federal Court, and you see the genuine difference legislation can make to people’s lives, if they can quickly and cheaply access a tribunal in which to seek an enforceable remedy to a harm they have suffered.

And yet, I feel like most of those PPIPA and HRIPA cases have just been tinkering around the edges.  So far almost every case has been brought by only one or two people, seeking remedies to mitigate the damage done to themselves: an apology here, a small amount of compensation there.  And fair enough!  It should not be the job of individual citizens to drive systemic change.

But there is also a role for advocacy groups to agitate on behalf of the citizenry at large.  In fact, PPIPA appears to allow for this.

First, the law allows any “person who is aggrieved” to seek review of conduct under PPIPA or HRIPA.  There is no threshold requirement in the statute that the complainant’s own personal information needs to have been involved in the conduct; they simply need to be aggrieved because they believe the conduct breached a privacy principle.  The Tribunal has noted the beneficial purposes of the legislation, and has promoted this broad reading of the phrase “person aggrieved”.

Indeed, the Tribunal noted that it is possible that a “person aggrieved” by conduct could be a person other than the person who was the subject of the personal information at issue, and thus a third party may be able to seek a review and a remedy for any breach.  (And the fact that the definition of ‘personal information’ includes information about people dead for less than 30 years indicates an intention to protect the interests of survivors who may be affected by the handling of the deceased’s personal information, rather than their own.)  Similarly, successive NSW Privacy Commissioners have stated that a ‘person aggrieved’ is a wider concept than simply a person whose personal information is in issue.

Second, the orders available to the Tribunal are not limited to providing remedies to the complainant, but can be directed to requiring systemic change by a public sector agency, such as requiring certain conduct to be stopped, or proactive actions to be taken in order to comply with the privacy principles.

So, why don’t we have a history of class actions brought by representative groups, aiming for systemic change?  I believe part of the reason is that Australian privacy advocacy groups are stretched too thin, having to put their (entirely volunteer-based) resources into making submissions on countless policy and legislative proposals, and running social media campaigns to draw attention to travesties like Robodebt.  They don’t have the time, money or energy to run class actions too.

But it doesn’t help that the Tribunal has been slow to embrace the idea, and indeed has offered some conflicting interpretations.  A complaint brought by an individual who claimed to be a member of a class of people potentially aggrieved by the disclosure of case studies about workers compensation claimants was dismissed, with the Tribunal stating that the complainant “is only permitted to agitate matters before the Tribunal in proceedings that relate to conduct or alleged contraventions concerning him personally and where he has suffered some tangible and measurable impact”.

Similarly, in the Opal Card case, the Tribunal rejected the complainant’s argument that he could represent all members of a certain category of people (all passengers using Gold Opal cards), and thus his standing was limited to how he was personally aggrieved.  (Nonetheless, the Tribunal did note that its findings about non-compliance, and recommendations to the respondent about how to deal with the complainant’s personal information, would equally apply to any other member of that category of people who came forward with the same complaint, and thus it would be ‘prudent’ for the respondent to make their system complaint for all passengers using Gold Opal cards.)

Also, the reactionary responses by governments on the losing end of privacy cases with the potential for systemic change is hardly encouraging.  It’s not just on the Opal Card case that agencies fight back after losing a privacy case on its merits.  In 2013, farmer Adam Bonner brought a case under PPIPA, and successfully argued that the CCTV system installed by his local council was not fit for purpose, could not achieve its crime prevention objectives, and was thus beyond the council’s power to run in the first place – and had poor data security practices where the video feed ended at the local police station.  One man, using his democratic right to object, and his legislated right to demand legal compliance by his local council with the State’s privacy laws, won his case and held the local council to account.  Hurrah!  It’s the stuff of Hollywood movies, David vs Goliath, right?

Only NSW politics doesn’t work that way.  Did the local council or the State government take a step back and re-evaluate the efficacy of their CCTV crime prevention program?  Did they promise to only use CCTV when it is actually fit for purpose, such as to justify the intrusions on privacy?  No, politicians confected outrage, characterised the complainant as a trouble-maker, simply asserted that all CCTV works fine, and swiftly drafted blanket exemptions for local councils operating CCTV.  So, no accountability, no scrutiny; taxpayers waste their money and citizens lose their privacy.

21, but not yet an adult

Much of the value of Big Data is built on our digital breadcrumbs – the digital traces we leave behind as we go about our day-to-day activities like travelling to work, buying goods, using social media or searching the web.

But if an organisation does not have a sound reason for collecting those breadcrumbs – in other words, if collecting our data is not reasonably necessary for the primary purpose for which we were transacting in the first place (getting on a bus, buying a pair of shoes, chatting to our friends on Facebook) – then it should not be collected at all.

It’s not rocket science.  It’s not impossible, or unrealistic, or crazy-advocates-wish-list thinking.  It’s right up there as privacy principle number 1, and has been the law for 21 years now: don’t collect personal information unless you really, truly need it for your primary purpose.  And yet this most fundamental of privacy principles is so often ignored.

In my view, all the other privacy principles are subordinate to this one.  All the access and correction rights, all the data security, all the transparency requirements, are pointless if there are no meaningful limits on what governments can collect about us in the first place.

If agencies still don’t care about getting privacy right, and if individual citizens, the Tribunal or the Privacy Commissioner cannot make them care, then PPIPA is not doing its job.  NSW residents are no better off than we were in 1998.

So happy birthday, PPIPA.  You may be 21, but I am not yet convinced that you are a functioning adult.

Photograph (c) Shutterstock

February marks 12 months since the start of the notifiable data breach scheme here in Australia, and nine months since the European notification scheme started under the GDPR.  American notification laws have been running for years now. All of which means overworked regulators, executives losing their jobs, and plenty of media attention on individual cases of data breaches.

But are any lessons being learned by organisations about how to prevent data breaches in the first place?

With the latest stats from the OAIC showing that yet again the private health sector topped the list of sectors reporting data breaches, and the latest news suggesting that 15,000 cardiology patients’ records were rendered inaccessible and held to ransom in a cyberattack, it seems that there is plenty of work yet to be done, by health service providers in particular, to get data protection right.

(But by the way if that’s you, there is some good news!  We recently partnered with the AMA to produce a free eLearning module specifically for health service providers.  Just released, Privacy Compliance for Medical Practices is even RACGP-accredited for continuing professional development, so Doc – you’ve got no excuse anymore.)

So in the interests of trying to prevent the next data disaster from happening, I thought a review of some of the causes of data breaches might be order.  Not that this is in any way scientific, more like my personal musings, but here goes … the Salinger Privacy List of the Top 10 Things Not To Do.

# 1 – Not redacting documents properly

Remember back in the good old days, when redaction was easy? When my latest crush turned sour I would simply slather Liquid Paper on my tartan pencil case to paint over the love heart featuring said boy’s initials.  Evidence covered up, I could move on to the next unwitting subject of my fickle affection.

OK, I’ll admit, it wasn’t a perfect method.  Anyone could have scratched the Liquid Paper off to reveal the original writing, thick texta having soaked through the fabric.

These days, the digital equivalent of scratching off the paint seems to happen with disturbing regularity, particularly in relation to documents released under FOI or published in court filings.  Redaction software exists, so why do people keep getting this wrong?

Examples of personal information released by accident because redaction either didn’t happen at all, or was done so badly that it was trivial to reverse, have included the accidental publication of the private mobile phone numbers of hundreds of federal politicians, former prime ministers and senior political staffers; the publication by Comcare of the personal details of an injured worker; the publication of information contained in hundreds of confidential submissions from families of children who have self-harmed and been the victims of bullying; calculations of actor Geoffrey Rush’s historic income and predicted future earnings submitted in court documents; data released under FOI revealing both prison security details and personal information about hundreds of prisoners; and data mistakenly embedded in a Word document published online by the Department of Immigration revealing sensitive personal information about more than 9,000 asylum seekers.

# 2 – Leaving databases and backups on publicly facing servers

This was the cause of the Red Cross data breach affecting more than 1M people in Australia, the Capgemini leak of Michael Page recruitment data, as well as the leak of more than 43,000 pathology reports in India, and the personal information about more than 198 million American voters from the Republican National Committee.  IT managers should know better.

# 3 – Leaving unsecured AWS ‘buckets’ of data in the cloud

This has happened to the ABC, as well as Accenture, Viacom and a recruitment company holding data on military veterans and others holding security clearances.  Plus to a contractor holding staff records from AMP, the Department of Finance, the Australian Electoral Commission and others.

And then, in a data breach affecting 123 million American households, to credit reporting bureau Experian and its partner analytics firm Alteryx.  And then FedEx. Really, why does this keep happening?

# 4 – Storing passwords in plain text

Not hashing or encrypting user passwords was the cause of an app maker being fined for breaching the GDPR.  Although it’s not clear, this might also be how a bank employee managed to disclose online banking passwords of customers to a third party.

# 5 – Allowing sensitive data to be stored on unencrypted mobile devices

A paediatric hospital in Texas, contrary to prior security advice, failed to deploy encryption or other measures on all of its mobile computing devices.  So no surprise the heightened risk when a staff member left behind at an international airport an unencrypted non-password-protected BlackBerry, containing the electronic health records of 3,800 patients.  Yet still not learning the importance of information security, a few years later the same hospital suffered the theft of an unencrypted laptop from an unsecured work area; the laptop contained the electronic health records of 2,462 individuals.  The hospital was fined US$3.2M for the two instances providing evidence of their failure to comply with data security rules.

This also happened to a company providing mobile monitoring of patients with cardiovascular disease.  When the employee’s laptop, containing health information about 1,391 patients, was stolen from their parked car, the company was fined US$2.5M.

# 6 – Mishandling the mail or other transmission of records

There have been examples from Victoria of posting confidential children’s court records to a violent family member; or in NSW where 2,693 photo ID cards, including driver licences and gun licences, were sent to the wrong people.

# 7 – Poor disposal of paper records

Examples include the medical letters about more than 1,400 public and private patients found in a public bin in Sydney after being dumped by a contracted transcription service provider; and private hospital medical records found lying in the street in Victoria.

And while last year’s data leak involving Cabinet documents may have revealed more about government affairs than personal information per se, news that decades’ worth of Cabinet documents were found inside locked filing cabinets sold off by the Government suggests that the Australian Government is also quite good at screwing up data disposal.

# 8 – Poor handling by a third party supplier or contractor

A study of data breaches by the Ponemon Institute and IBM found that third-party involvement was the top ranking factor that led to an increase in the cost of a data breach.  Examples include customer data leaked from a supplier to Domino’s Pizza, and a data breach involving 8,500 current and former staff of the Department of Social Services which was blamed on a third party contractor.

# 9 – Failing to use audit logs to identify rogue behaviour

What is the point of all those audit logs, if no-one is using them to look for evidence of unusual activity by staff such as to suggest misuse of their access to data?  An investigation by the UK privacy regulator the ICO found that an employee of a health fund was able to deliberately extract (and illegally sell) personal information about more than half a million customers from its CRM system, because the audit log was not being monitored.

# 10 – Not configuring tech to protect emails leaking data

A shipping company discovered that data was being ever-so-slowly exfiltrated from its finance and payroll departments over an 11 month period, with around 50,000 emails being auto-forwarded from three employee email accounts to two email addresses outside the company.  Security commentators suggested the mail settings should typically be configured to prevent auto-forwarding of protected emails outside a company.

So what are the lessons to be learned from our Top 10 Things Not To Do?  In other words, what should you do?

Make sure your information security settings are tight, and that controls like audit logs and email gateways are actually being used and monitored.  Check your contractors are doing the same.  And train all staff to get privacy and security right in absolutely everything they do, from sending out the mail to taking out the trash.  Train, train, and train again.

Want to know more?  Check out our 5 March webinar on Notifiable Data Breaches; our up-coming Privacy Management in Practice workshops, which include plenty of focus on identifying and mitigating privacy risks; and our template Data Breach Response Procedure.

Photograph © Shutterstock

If a year ago I had attached a piece of string to the personal information I provided in order to enter an online competition, would I be surprised how many organisations had my string threading through them by now?

Almost certainly my string would by now lead in multiple directions – and I am willing to bet that my privacy was infringed somewhere along the way.

In between all the organisational filters such as privacy policies, collection notices and consent mechanisms that my personal information should not have been able to pass through, it would only take one organisation to have set up a non-compliant privacy framework (or to have no framework at all) for my personal information to have spread like wildfire.

I am typically an optimist.  I like to think that insufficient privacy frameworks are due to a lack of expertise rather than questionable practices.  However, when it comes to my own privacy string, my optimism breaks down.

So, I think to myself. What are three simple things that all companies (and government agencies alike…I haven’t forgotten about you) can do, so that if my piece of string led me to your organisation, I would not be unpleasantly surprised?

First, I think privacy needs to always be considered in an organisation’s initial project plan, with controls ultimately being incorporated into the final design prior to implementation.

The term for this is ‘Privacy by Design’. And yes, it’s actually a thing!

Privacy shouldn’t be something to tick off at the last minute before launching your next lucrative venture. It should be one of the first things considered, and incorporated into designs or project plans.

This can be as light-touch as checking the collection notice drafted for a new form is appropriate, or as comprehensive as performing a privacy impact assessment on a complex project being worked on by multiple departments.  Failing to do so will ultimately lead to uncontrolled privacy risks post-implementation.  Many organisations can attest to this, trust me, but let’s look at one briefly.

Suncorp’s insurance arm built a new feature for their online quoting platform. In order to speed up the process for customers, the feature pre-filled the physical security details of residential properties where these were already known to Suncorp (which also owns AAMI, GIO and other insurance brands), such as whether there were deadlocks or alarms installed.  However because online users were not required to verify whether they owned or lived at the residence in order to view those details, this amounted to publicly exposing physical security features (or the lack thereof) to the public at large.

So, to give an example, if I wanted to see what kind of security measures I’d need to overcome in order to shake our Prime Minister’s hand in his own home, I’d simply have to type in the address for The Lodge and hope a quote had been generated or a policy held for that address with one of Suncorp’s insurance brands in the past.  Of course, there’s also the tricky subject of bodyguards but you get my drift.

Following complaints from the public, Suncorp immediately removed the feature.  So although built as a time saving feature for customers, we can only speculate that the grave privacy implications of disclosing that level of information in the absence of identity verification had not been comprehensively considered by Suncorp.  Privacy by Design could have saved the day.

Moving on, the second thing organisations can do is strengthen their de-identification methods. It’s well known that de-identification of personal information when performed properly allows an organisation to draw powerful insights from datasets while simultaneously protecting the privacy of individuals.

But what is de-identification really?

I’d take a guess that for most organisations, simply stripping out direct identifiers such as name, address and date of birth constitutes their preferred de-identification technique.  However, the process of de-identification can be complex, and simply stripping away personal identifiers from a dataset may not be sufficient to prevent re-identification or constructive identification.

I’m of the opinion that organisations need to consider a few things here.

First, can the data be linked with other datasets via data points that would not – at face value – constitute personal information?

Second, in addition to the de-identification techniques used, what other controls will be placed on access to or storage of the data?

And third, who is the data being disclosed to, and can they re-identify it using their expertise? In the case of public disclosures, the worst needs to be assumed here.  I know, I know, I thought I was an optimist too.

Don’t take my word for it though, let’s look at an example. In 2016 the Department of Health (DoH) published Medicare Benefits Schedule and Pharmaceutical Benefits Scheme data on approximately 2.5 million Australians.  The data was treated with several different de-identification techniques before being published online.

However following the publication, researchers at the University of Melbourne were able to re-identify data belonging to several high-profile individuals, primarily through a cross matching exercise with other publicly available datasets.  Well, you know what they say, where there is a will, there is a way.

The DoH was subsequently found by the Office of the Australian Information Commissioner (OAIC) to be in breach of three Australian Privacy Principles.  Being a government agency, the effect of such breaches can lead to the corrosion of public trust and confidence in the way government agencies in general handle personal information.

I guess I would like to think that if my string led me to a government agency, my personal information would have been subject to extra care, right?  (Yes, I’m still optimistic!)  But the lesson here is the importance of talking re-identification risk seriously.

The final thing I’d want organisations to do would be to exercise more transparency in the way they handle personal information. Of all my recommendations, this would be the easiest to implement.

More transparency doesn’t just show individuals how their personal information will be handled, it also provides other organisations with the tools to help them decide whether or not personal information can or should be exchanged with your organisation.

An organisation’s failure to be transparent in its handling of personal information is one of the most fundamental privacy risks it faces, because transparency is an enabler of other privacy rights for the individual consumer or citizen.

Having a clear, concise and easy to read privacy policy, collection notice and consent capturing process will go a long way in combating the risk of non-transparency.

Now let’s look at my third point in action.  HealthEngine, a health service booking platform, was recently found to be routinely disclosing to a law firm information about individuals who had booked appointments with medical professionals through their site. That law firm then direct marketed legal services pertaining to occupational injuries back to those individuals.

While HealthEngine argued that individuals consented to the disclosure via its Collection Notice, that Notice seemed contradictory to its Privacy Policy which was radio silent on the nature of that disclosure.  Additionally, acceptance of HealthEngine’s Terms, Privacy Policy and Collection Notice was ‘bundled’ and the ability for an individual to make a booking was contingent on that acceptance.  Probably not what their customers expected, and certainly not within the spirit of voluntary and informed consent.

Given the public outcry, it’s fair to say that individuals did not expect their sensitive health information to be used and disclosed in such a way. In other words, if a string had been attached to their information, they certainly would not have expected to follow it back to a law firm.

This case outlined the importance of dealing with information, particularly that sensitive in nature such as health information, in accordance with customer expectations.

So my three pieces of advice for your organisation, to help reduce your privacy compliance risks and keep your customers happy, is to engage meaningfully with Privacy by Design; tread carefully when it comes to de-identification; and be clear with your customers about what exactly you are planning to do with their data.

I have no doubt that one day, if I follow the millions of strings attached to the personal information I have ever provided to anyone else, I will be content with where it leads me. Unfortunately, though, that is not the case today. Being part of a privacy consulting firm myself though, I will definitely do my part to correctly guide the strings of others.

Join one of our Privacy Management in Practice workshops to learn more about managing privacy risk in your organisation.

Photograph © Shutterstock

Imagine reading an ethical framework for organising birthday parties, which says that it will be important to meet legal requirements in terms of not making too much noise, that matching napkins and paper plates are fundamental to planning your party, but that the success of your party could potentially be affected by a tsunami hitting your house.

The framework fails to mention some key things to plan for such as the number of guests, food, drink, music, decorations, lighting, dress code, the birthday cake, speeches, or wet weather backup plans, let alone any actual ethical questions such as deciding whether you have to invite your boorish brother-in-law, or whether you should cater specially for guests with a gluten intolerance.

You would be a bit worried about the utility of such a framework, right?  Over-stating the importance of some factors, over-stating the risk of others, but also missing some really key things to consider in your planning.  Plus, not actually addressing any ethical questions at all.

That’s how I felt when reading Artificial Intelligence: Australia’s Ethics Framework, a Discussion Paper by CSIRO’s Data61, released by the Department of Industry, Innovation and Science on 5 April, the objective of which is “to encourage conversations about AI ethics in Australia”.

Its definition of what is covered by the Privacy Act is wrong, it fails to mention other critical privacy laws which could apply to organisations developing or applying artificial intelligence (AI) or machine learning (ML), its assumptions about what matters in the application of privacy law in practice is wrong, it misses the bulk of what is important, and it throws into the mix a random consideration which is unrelated to the discussion at hand and the risk of which is overstated.

Surely, a discussion of ethics must begin with foundational concepts and accurate explanations of the law, and then move on to ethical dimensions which challenge how to apply the law in practice, or which ask difficult questions about topics which go beyond the requirements of the law.  A framework which does not achieve this – and which could lead its audience into misunderstanding their legal obligations – could be worse than no framework at all.

I was worried enough to gather a couple of other privacy pros, with whom I prepared a joint submission to the Department of Industry.  That submission is reproduced below, along with the names of additional colleagues working in the privacy field who agree with its sentiments.

You can make your own submission until 31 May 2019 at the Department’s website.

UPDATE, NOVEMBER 2019: In November the Department replaced its lengthy Discussion Paper with a short set of 8 AI Ethics Principles, without further explanatory or supporting material, which comprises motherhood statements (such as this gem, the principle dealing with privacy: “Throughout their lifecycle, AI systems should respect and uphold privacy rights and data protection, and ensure the security of data”) which despite being valueless statements of the bleeding obvious will somehow now be tested by industry.  The sole suggestion for “maintaining privacy” is to use “appropriate data anonymisation”.  This v2 approach to resolving privacy risks (‘We can fix everything with anonymisation!’) is as simplistic and wrong as the v1 approach (‘We can fix everything with consent!’)

The Department states that as a result of the submissions received on their Discussion Paper they “analysed the submissions and engaged with AI experts from business, academia and community groups to help analyse the feedback. This enabled us to develop the revised set of AI ethics principles”.  I note however that none of the authors of our joint submission (below) were contacted.

The Principles don’t even start to touch on whether personal data should be being used to train AI in the first place, let alone how the application of the results will impact on human dignity or autonomy.  The end result looks like the Department dealt with our critique of the Discussion Paper (along with 129 other submissions, including multiple others also critical of their approach to privacy) by simply backing away from all discussion of either the law, ethical complexities or moral nuance of using ML/AI, and producing some airy-fairy gumpf in place of pragmatic guidance.  That’s a big chunk of $29.9M of your tax dollars at work, folks.  – Anna Johnston.

THE SUBMISSION AS LODGED

Introduction

The world of machine learning (ML) and artificial intelligence (AI) is set to dominate the technology of the future, and reframe human interactions.  Ensuring that there is a strong legal and ethical framework to underpin the development and implementation of ML and AI is critical to ensuring that technology serves humans in a manner that is fair, non-discriminatory, and beneficial.

For these reasons, we are pleased to see the Australian Government’s commitment to starting the discussion on this important topic.  While the principles espoused in the Artificial Intelligence: Australia’s Ethics Framework Discussion Paper are a good start, we submit it lacks a firm or accurate basis.

We are only just beginning to understand what ML and AI could do.  But we must thoroughly understand what we are lawfully allowed to do, before we can truly understand what we should do.

Any discussion of ethics must therefore begin with foundational concepts and accurate explanations of the law, and then move on to ethical dimensions which challenge how to apply the law in practice, or which ask difficult questions about topics which go beyond the requirements of the law.

It is our submission that unfortunately, the Artificial Intelligence: Australia’s Ethics Framework Discussion Paper does neither.

This submission focuses on our area of expertise, which is privacy law, privacy management in practice, and the ethics of collecting, using or disclosing personal information.  While mention of privacy is only a small part of the Discussion Paper, an understanding of privacy law, both in theory and in practice, is essential to developing an ethical framework which is accurate and beneficial for its audience, as well as protective of the humans who will be affected by the development of AI.

Our concern is that the Discussion Paper, as it stands, requires substantial re-writing to accurately reflect Australia’s legal privacy landscape.

The Discussion Paper’s definition of what is covered by the Privacy Act 1988 (Cth) (Privacy Act) is wrong, it fails to mention other critical privacy laws which could apply to organisations developing or applying ML or AI, its assumptions about what matters in the application of privacy law in practice is wrong, it misses the bulk of what is important, and it throws into the mix a random consideration which is unrelated to the discussion at hand and the risk of which is overstated.

This submission relates primarily to Chapter 3 of the Discussion Paper, titled ‘Data governance’, as this is our particular area of expertise.  It also comments on the proposed Principles and Risk Assessment Framework in Chapter 7, in relation to privacy compliance and privacy risks.

This submission seeks to provide context for our answers to the following questions posed in the Discussion Paper:

1. Are the principles put forward in the Discussion Paper the right ones? Is anything missing?

Our submission:  There is so much missing in the description of privacy law as to be misleading.  A sound ethical framework cannot be developed in the absence of a robust understanding of privacy law.

3. As an organisation, if you designed or implemented an AI system based on these principles, would this meet the needs of your customers and/or suppliers? What other principles might be required to meet the needs of your customers and/or suppliers?

Our submission:  No.  The description of privacy law is so inaccurate as to be misleading.

4. Would the proposed tools enable you or your organisation to implement the core principles for ethical AI?

Our submission:  No.  The fourth Principle at Chapter 7 introduces a concept not reflective of privacy law: “private data”.  Believing privacy law is only about ‘private’ data is a common misunderstanding.  Its repetition here will not assist your audience.  This misconception could leave those entities engaging in AI and ML activities vulnerable to data breaches and sanctions.

Phrases such as “protected and kept confidential” relate only to a sliver of what is covered by privacy law, and are too vague to be edifying.  It is not only data breaches or unauthorised disclosures which could cause privacy harm to a person.  It is the very nature of data collection for ML development, or the re-purposing of existing datasets for ML development, or the application of algorithms derived from such data in AI or automated decision-making, which could generate privacy harms.  Such activities underpin important decisions made by businesses and government that directly and significantly impact people’s lives.  This has been either misunderstood or downplayed by the authors of these principles.

Further, the Risk Assessment Framework at chapter 7.2 contains an over-reliance on ‘consent’ as if it were the sole mechanism by which the collection, use or disclosure of personal information may be lawfully authorised.  Consent is not a magic bullet.  It’s not even the rule when it comes to collecting, using or disclosing personal information.  It is the exception to the rule.  Additionally, any consent must be freely given, informed, and specific – elements of validity very difficult to attain in the contexts described.

As such, the Risk Assessment Framework is seriously misleading for its users, and not fit for purpose as a risk assessment tool.

Getting the law right

The content of the section on privacy in Chapter 3 is so under-developed and inaccurate as to be misleading.

A discussion of ethics must begin with foundational concepts and accurate explanations of the law, and then move on to ethical dimensions which challenge how to apply the law in practice, or which ask difficult questions about topics which go beyond the requirements of the law.

This Discussion Paper does neither.

Its definition of what is covered by the Privacy Act is wrong, it fails to mention other critical privacy laws which could apply to organisations developing or applying AI or ML, its assumptions about what matters in the application of privacy law in practice is wrong, it misses the bulk of what is important, and it throws into the mix a random consideration which is unrelated to the discussion at hand and the risk of which is overstated.

In any final guidance to be offered on this topic, we suggest that the following errors must be corrected, at the very least:

1. Privacy, while not defined in legislation, is defined in Office of the Australian Information Commissioner (OAIC) resource materials as ensuring that individuals have transparency and control regarding the way in which organisations and government handle their personal information, and can where practicable choose to interact with those entities without identifying themselves.
2. Privacy laws in Australia cover ‘personal information’, not ‘personal data’.
3. The Privacy Act is not the only privacy law which will apply to organisations working with AI. For instance, State and Territory-based privacy laws will apply to those working in the public university sector (other than ANU), and State and Territory-based health privacy laws will apply to organisations managing health information, no matter whether they are also covered by the federal Act or more local privacy laws.  The European General Data Protection Regulation (GDPR) will apply to organisations in Australia which have an establishment in the EU, or which offer their goods or services to, or monitor the behaviour of, people in the EU.
4. Although noted in the Bibliography at 58., we submit that a thorough review of the Office of the Victorian Information Commissioner (OVIC)’s Artificial intelligence and privacy: Issues paper (June 2018) in the formulation of this Discussion Paper would have assisted with terminology nuances and an understanding of the breadth of the privacy law landscape in Australia.  The OVIC issues paper, for example, reflects considerations in relation to the State of Victoria’s privacy law and references the fair information handling principles underpinning that law (and others worldwide) first set out by the Office of Economic Cooperation and Development (OECD) in 1980.
5. At the outset, the Discussion Paper makes no distinction between personal information and sensitive information, nor the differing legal requirements relating to the two. Personal information, in most privacy laws in Australia, has a sub-set known as sensitive personal information, to which higher standards apply.  Using the language of ‘sensitive’ to describe all personal information confuses readers about the actual legal definitions, and the differing obligations that arise from those definitions.
6. The Discussion Paper comments that there may be a need to explore what privacy means in the digital world. The authors of this paper may have benefitted from exploring the meaning of privacy and its iteration in the Australian privacy law landscape by reviewing the extensive research on this topic compiled by the Australian Law Reform Commission in its Report 108.
7. In terms of exploring key concepts, we additionally submit the authors of the Discussion Paper should further explore what consent means; particularly in terms of its proper application in existing privacy law (both in Australia, and internationally) and the hallmarks of a true consent. The matters discussed in 3.1 (Consent and the Privacy Act) and the related sub-sections confuse the topic entirely.
8. Consent is not the sole privacy issue, nor is it the sole solution to all privacy problems. “Protecting the consent process” is not “fundamental to protecting privacy”.
9. If it is truly intended to collect, use and disclose personal information in accordance with the law (e.g., the Privacy Act), the authors of the Discussion Paper must first understand that consent is not ‘the rule’; it is an exception to the rule – the ‘rule’ here being the restrictions or limitations on collection, use and disclosure of personal information as set out in the law.
10. Consent is just one of many exceptions that may be applied, as appropriate, during decision making processes.
11. Privacy law creates privacy obligations covering the entire life cycle of handling personal information, and in many cases consent is utterly irrelevant. For a plain language explanation of the role of consent in privacy law, we refer you to a paper by one of the authors of this submission, ‘Why you’ve been drafting your Privacy Policy all wrong’.
12. Consent is only explicitly required under the APPs at the time of collection, where the information being collected is ‘sensitive’ personal information.  Consent may or may not be sought at the time of collecting personal information. It can also be sought later, before an unrelated secondary use or disclosure is to occur.  Nor is consent everlasting, irrespective of when it was sought.  Suggesting that consent must be gained at the time of collecting personal information will conflate and confuse consent with the requirement to offer a ‘collection notice’, which is a separate legal obligation (unrelated to whether or not you need consent for your proposed data use), which is indeed required at the time of collection.
13. The paper confuses the acts of providing a collection notice or transparency obligations in privacy law (which are about “making people aware”) with consent, which is a separate act of seeking permission or agreement to stray from the rules set out in privacy law.
14. In any case, consent is not a ‘get out of jail free’ card and is a significant problem where there is misuse of personal information in a way that compromises individuals’ trust, even if otherwise lawful. The HealthEngine incident is a good example of this.  Further, consent must be reviewed periodically in a way commensurate with the sensitivity and risk, so as to ensure that it remains current.
15. Conceptually, consent is an even less appropriate means to authorise data flows in the context of AI than in other contexts. Consent would likely be ineffective where AI is concerned; most people would be unaware of the impacts of AI or its possible consequences and as a result ‘informed’ and ‘specific’ consent would be near impossible to achieve.
16. The introduction of the topic of consent in the Discussion Paper without any context for that discussions begs the question: this discussion is about consent to what? Any discussion about privacy laws must first explain what they actually do, which is to regulate both data flows and data governance.
    1. Data flows are regulated to the extent that privacy principles say when personal information can be collected, used or disclosed. For each of these, the privacy principles offer various grounds on which personal information may be lawfully collected, used or disclosed, and ‘with the consent of the individual’ is but one of those grounds.  In the context of AI and ML, it is likely the least useful ground.
    2. Much more challenging for organisations developing or applying AI or ML are compliance with the rules around the use or disclosure of personal information for secondary purposes. The datasets on which ML is trained will almost certainly have been created in the first place for a primary operational purpose related to the individual (e.g. to treat a patient, to transport a passenger from A to B, to connect a customer’s phone call).  Re-use of that dataset for training ML is a secondary use, unrelated to the primary operational purpose.  The starting point in privacy law is that secondary uses are not allowed, unless an exception to that rule applies.  ‘With the consent of the individual’ is one such exception, but is generally not pragmatic in the case of large datasets.  There are other exceptions such as ‘for law enforcement purposes’ which will generally not apply, which leaves research exceptions as the most likely path for the development of ML in particular.  However research exceptions (which differ in scope between the federal Privacy Act and State and Territory-based privacy laws) typically define ‘research’ narrowly; require elaborate processes to balance and test the ethical implications of allowing the secondary use or disclosure of personal information without consent; and raise additional questions about whether the proposed research is in the public interest, such as beneficence and impact on vulnerable populations.  An ethical framework which fails to mention the process by which Human Research Ethics Committees must wrestle with the ethical implications of an AI or ML project, before allowing it to proceed lawfully, seriously underplays the legal and ethical requirements of organisations working in the AI or ML fields.
    3. Data governance includes the need for transparency, amongst other matters such as enabling rights of access and correction. This includes notice to individuals about how their personal information will be collected, used or disclosed.  Notice is not the same as consent.  The delivery of a meaningful notice poses considerable challenges in the context of the application of technologies quite removed from the individual whose personal information is at issue, such as AI and ML.  This should be an important focus of any discussions around privacy and AI.
17. The paper conflates the currency of consent with the absence of a ‘right to be forgotten’, as though consent can always be considered current unless someone has asked to be erased.  This is just nonsense. The ‘right to be forgotten’, which is a unique feature of the EU General Data Protection Regulation (GDPR), is unrelated to the issue of consent.  It is related to the rights of access and correction.
18. The right to be forgotten is overstated in this Discussion Paper in terms of topics for organisations to worry about. Even in the GDPR, the right to be forgotten is not an absolute right, and should not impact on the business practices of companies which are only collecting or using personal information lawfully and fairly, and still have a current need for it.
19. The right to be forgotten may not explicitly exist in the Australian Privacy Act, but APP 12 requires that information once no longer needed for the purposes for which it was collected be deleted, destroyed or de-identified.  Not doing so, irrespective of any specific request from an individual, would be a breach of the Act.
20. The Facebook/Cambridge Analytica case study is less about consent, and more an illustration of the failure of either of those companies to adhere to legal limitations on the secondary use or disclosure of personal information, beyond the expectations of the individual.
21. In describing the Facebook/Cambridge Analytica case study with language such as “this incident … demonstrates that it may not be sufficient to merely follow the letter of the law” implies that Facebook complied with privacy law when it allowed members to ‘consent’ on behalf of their friends to let Cambridge Analytica scrape and re-use their data, but that these practices somehow fell foul of ethical considerations beyond the law. This is seriously misleading, as Facebook has already been found to have not complied with privacy law by the UK privacy regulator, and more recently by the Canadian privacy regulator, with investigations in other jurisdictions still open as at the time of writing, such as in the United States where Facebook is reported to be expecting a fine of between $3 billion and $5 billion.
22. The Discussion Paper contains a very poor description of the notifiable data breach (NDB) scheme and requirements, and in any case provides only a retrospective action for breaches whether or not they involve AI.  AI also proceeds at such a speed that significant damage is likely to be done in the time that any breach takes to be detected, and the AI activity ceased.
23. The NDB scheme covers ‘personal information’, not personal data. The notifiable data breach scheme is not limited to ‘personal information’, but also two other types of data: Tax File Numbers and credit-related information.  The NDB scheme is not limited to unauthorised access or disclosure, but also covers loss of relevant data.
24. Organisations may also be subject to additional data breach notification schemes under other privacy laws, most notably the GDPR, which has a broader definition of what constitutes a data breach, and stricter timeframes for reporting.
25. Hypothesising that human error data breaches are indicative of ‘security gaps’ in the same / similar way as malicious or criminal attacks is wildly misleading.
26. The impending Consumer Data Right (CDR) would not significantly change the privacy landscape for the consumer nor provide any effective safeguards or transparency in the specific use of AI.  Consumers already have a right to access their information under the APPs and the CDR would provide another mechanism by which businesses could share customer information and use it for a range of purposes, included those that utilise AI.
27. The ‘Key Points’ at 3.5, by focusing only on ‘consent’ as a mechanism for resolving any and all privacy risks, fail to deliver an even baseline level of explanation about what organisations must do in order to meet their legal requirements, in terms of either authorising data flows, or enabling data subject rights as part of routine data governance.
28. Critically, the proposed Principles outlined in Chapter 7 do not reflect the scope of privacy law, let alone grapple with ethical considerations beyond the law.
29. Chapter 7 proposes the following Principle: “Privacy protection. Any system, including AI systems, must ensure people’s private data is protected and kept confidential plus prevent data breaches which could cause reputational, psychological, financial, professional or other types of harm to a person.”
30. This introduces a concept not reflective of privacy law: “private data”. Believing privacy law is only about ‘private’ data is a common misunderstanding.  Its repetition here will not assist your audience.
31. Further, phrases such as “protected and kept confidential” relate only to a narrow understanding of what is covered by privacy law, and are too vague to be edifying. It is not only data breaches or unauthorised disclosures which could harm a person; it is the very nature of a data collection, its use for ML development, or its application in AI or automated decision-making, which could lead to privacy harms.  We discuss this further below.  This has been either misunderstood or downplayed by the authors of these principles.

Implementation issues

Further, in our view:

32. The discussion of de-identification and re-identification is overly simplistic. Further, de-identification is not the panacea for compliance with privacy law, particularly if the de-identification is not permanent and irreversible.  Where that is the case, de-identification is merely a protective measure, but does not remove the information from the obligations of privacy law, nor the community’s expectations about how their information should be used.
33. The discussion of the risks from location data is overly simplistic. A more nuanced discussion could be developed from considering the fallout from the public release of Strava fitness data, as one example.  For more on the privacy risks posed by location data, see the following papers by one of the authors of this submission:
    1. ‘Too much cyber, not enough privacy 101’,
    2. ‘Where’s Wally? Geolocation and the challenge of privacy protection’, and
    3. ‘Bradley Cooper’s taxi ride: a lesson in privacy risk’.
34. The Discussion Paper suggests a number of impractical solutions to legal and ethical problems. For example:
    1. It poses a strange and impractical distinction between ‘training data’ as though it can always be a discreet dataset, and AI would not continue to use and learn from live, ongoing data.
    2. The suggestion that a Code could be applied to data scientists, whilst helpful, does not resolve the issue.  AI is becoming so accessible that every employee with access to data is a ‘data scientist’ although that is not their primary function.
    3. Similarly the suggestion that AI systems can be isolated and regulated is impracticable.  AI is becoming so embedded in regular business practices that ‘system’ regulation is very difficult.
35. The Risk Assessment Framework at chapter 7.2 repeats the over-reliance on ‘consent’ as if it were the sole mechanism by which the collection, use or disclosure of personal information may be lawfully authorised.
36. In terms of outcomes, the Discussion Paper does not express in any real terms how consent can be translated into a reliable governance approach; rather, it seems to presume a generic level of appropriateness associated with using a consent model based on a view (however formed) that having consent will address both privacy compliance risks and allay privacy-related fears in the community. In this way, consent is treated as both the primary mechanism to (get the community to) allow a thing and the benchmark for success in terms of the proposed Risk Assessment Framework for AI Systems. On the former, it presupposes a level of engagement and sophistication within the populace from whom consent will be sought.  On the latter, it fails to address the intricacies and risks in decision making where personal information is concerned.  While it is clear that consent is intended to be a decisive, rigorous and universal proposition, in the context of this Discussion Paper it appears to be an ill-considered broad brush approach to a complex area of public policy.
37. Reliance on consent also pre-supposes that data scientists know what data they will be using and what insights or results it will generate. Most often, they don’t.  They start with an unfettered dataset and let the AI create and apply algorithms from and to the data.  You can’t get valid pre-consent from individuals for the future use of their personal information when the data scientists don’t even know what they’re looking for.
38. Consent is not a magic bullet. As such, the Risk Assessment Framework is seriously misleading for its users, and not fit for purpose as a risk assessment tool.

Ethical considerations in relation to privacy

Once you have guidance which at least starts with an accurate description of the law, then you could move on to ethical considerations.

The ethical issues considered in the Discussion Paper needs greater breadth and depth.

An example of an ethical issue involving privacy in the context of ML and AI is how the data used to train ML in the first place was obtained.  For example, the questionable ethics of scraping personal information from the web was recently highlighted in an NBC News investigation of IBM’s development of facial recognition algorithms from photos used, without either the subject or the copyright owner’s consent, from photo-sharing site Flickr.

Another example of ethical issues in the collection of the data used to train ML is the recent revelation that humans listen to and transcribe conversations heard by digital assistants.

Any discussion of these types of examples must however start with the recognition that what may be lawful (if unethical) behaviour by technology companies in the United States would not necessarily be lawful in other parts of the world with more comprehensive privacy laws, such as Australia.

An example of an ethical issue not covered by privacy law is the practice of individuating individuals without identifying them.  The rights protected by privacy laws currently stop at the point where an individual can be claimed to be no longer identifiable.  Leaving aside the vexed question of whether data can ever be truly described as unidentifiable, privacy harms can still be done to humans, such as through targeting an individual for communication, intervention, or denial of goods or services, even when the human is not identifiable.  Organisations involved in the development or application of AI or ML must grapple with the ethical implications of activities which can cause privacy harms, even if legal.

For more on the topic of individuation, see the paper by one of the authors of this submission: ‘Individuation – Re-thinking the scope of privacy laws’.

The extent to which predictive analytics can narrow or alter people’s life choices, in ways that are not necessarily transparent to the affected individual, must also be more comprehensively considered in any serious discussion of ethical issues in ML and AI.  For more on the topic of predictive analytics, see the paper by one of the authors of this submission: ‘How do you solve a problem like Facebook?’.

We also refer you to the work of international legal scholar Daniel Solove, who has written extensively on the taxonomy of privacy harms.

Discussion of this topic must also grapple with issues of community expectations around privacy, and the importance of gaining a social licence for the use of people’s personal information or impacts on their privacy.  Thus the objective of any research or development activity is relevant, as will be the likely applications of that development in the real world.  The use of ML to train AI applications to detect treatable cancers more effectively than humans or current technologies can would likely sit high on a measure of social licence, while the use of AI or ML in other scenarios, from how to prioritise child protection interventions, to decisions around policing, bail or sentencing, to which potholes should be fixed first, will be more problematic.

We note that the new Ethics Guidelines for Trustworthy AI from the European Commission have a more definitive position, which embraces a fuller understanding of existing privacy law, and the need to adopt a protective position.  The Commission’s Privacy summary says: “Privacy and data governance: Citizens should have full control over their own data, while data concerning them will not be used to harm or discriminate against them.”

Further, where the ethical principles are concerned, the principle of Do No Harm is poorly named and described, as well as impracticable – Do no harm is not the same as minimise harm, or design without any intention of harm as the definition in the paper suggests.  AI and ML activities often have no indicators of harm, until applied in the real world.

Conclusion

Downplaying or inaccurately describing legal requirements does not assist those working in the AI/ML fields to understand where their legal requirements end, and their ethical requirements start.

It is our submission that the Risk Assessment Framework is seriously misleading for its users, and not fit for purpose as a risk assessment tool.

By presenting ‘consent’ as a mechanism for resolving any and all privacy risks, this Discussion Paper fails to deliver an even baseline level of explanation about what organisations must do in order to meet their legal requirements, in terms of either authorising data flows, or enabling data subject rights as part of routine data governance.  We suggest that for the most part, consent will be irrelevant to the development of ML or AI technologies, and other privacy compliance considerations come into play.  These privacy compliance requirements require nuanced solutions, not a misplaced faith that ‘getting consent will fix everything’.  Consent is not a magic bullet.

Once you have guidance which starts with an accurate description of the law, then you could move on to ethical considerations which help flesh out the application of the law in practice, or which grapple with ethical considerations beyond legal requirements.

We suggest that the CSIRO and Department of Industry should engage with privacy regulators, and practitioners with specialist expertise in privacy law and practice, to assist in a redrafting of the Principles and Framework, as well as the contextual discussion underpinning them.

This submission was authored by:

• Anna Johnston, Director of Salinger Privacy, and former Deputy Privacy Commissioner of NSW, CIPM, CIPP/E, FIP;
• Nicole Stephensen, Principal of Ground Up Consulting, and Executive Director (Privacy and Data Protection) at the Internet of Things Security Institute; and
• Nicole Hunt, Privacy and Ethics Specialist, former Director of Privacy for the Australian Digital Health Agency, Senior Privacy Advisor for the NBN, and Deputy Director at the Office of the Privacy Commissioner.

Each author is an experienced privacy specialist.

In addition the following individuals, also experts in the privacy field, lend their name to this submission, in order that CSIRO and the Department of Industry appreciate the importance of accurate and nuanced discussion of privacy law and privacy-related ethical dimensions in any guidance, principles or risk assessment frameworks being developed for industry and academia working on the fields of machine learning and artificial intelligence.

• Malcolm Crompton, AM, FAICD, CIPP. Privacy Commissioner of Australia 1999-2004, Founder and Lead Privacy Advisor to Information Integrity Solutions Pty Ltd
• Melanie Marks, Principal, elevenM Consulting
• Sophie Bradshaw, Principal, Elgin Legal
• Dr Monique Mann,Vice-Chancellor’s Research Fellow, Technology and Regulation, Faculty of Law,Queensland University of Technology
• Kara Kelly LLB, CIPM
• Dr Roger Clarke, Visiting Professor, UNSW Faculty of Law, Visiting Professor, ANU Computer Science, and Principal, Xamax Consultancy Pty Ltd
• Stephen Wilson, Principal of Lockstep Consulting
• Nathan Mark, BA LLB. LLM Research Student focussing on inter-jurisdictional data and digital evidence
• Andrea Calleia, Privacy Learning Manager, Salinger Privacy, CIPM
• Nathan Kinch, Co-founder and CEO of Greater Than X and inventor of Data Trust by Design
• Mathew Mytka, Chief Platform Officer of Greater Than X, former Head of Platform Product at Meeco

Photograph (c) Adobe Stock

Hey get ready people, it’s almost Privacy Awareness Week!

OAIC’s theme for 2019 is ‘Don’t be in the dark about privacy’, while OPC NZ and OVIC’s theme is ‘Protecting privacy is everyone’s responsibility’.  No matter which slogan you prefer, the point is to spread awareness of the privacy message.

Much like an infectious toddler let loose in a childcare centre, we here at Salinger Privacy are doing our bit by scattering our privacy expertise all over the populace in little droplets of knowledge.  We’ve got a webinar explaining privacy for IT professionals, a free webinar with the IAPP on Privacy by Design in Privacy Law, an article about liability for privacy breaches by rogue employees in the May edition of the Law Society Journal, and here, a foundational explainer for anyone new to privacy.

In this Privacy 101 post we’re going to cover what is privacy, what is personal information, what privacy laws actually do, and when consent is needed.

Feel free to share this Privacy 101 for the edification of your workmates, your family, or even the person sitting next to you on the bus.  Go on, raise awareness!

Here goes…

What is privacy?

The first thing to know about privacy law is that mostly when we talk about privacy law we are talking about only one aspect of privacy.  There is no neat definition of what privacy means.  Some people call it the right to be left alone.  Some people equate privacy with secrecy, solitude, or anonymity.  Others think of it as control over who sees our information.  All of these factors do come into play, but privacy is a very broad, and ill-defined concept.

When we deliver privacy training, we tend to break down the concept of privacy into four categories.  These four categories are not set out in law and they are not black and white rules; there is a lot of overlap between them.  But when people talk about something having a privacy element to it, they are probably referring to at least one of the following four things:

• informational privacy: the appropriate handling of information about you, aka ‘personal information’;
• communications privacy: the confidentiality of your communications, which can be invaded if someone you did not authorise was to read your private mail, or intercept your phone calls;
• behavioural privacy: the autonomy of your behaviour, which can be impacted by surveillance or monitoring; and/or
• physical privacy: the autonomy of your body and the solitude of your territory, which can be impacted if someone touches you without your permission, or intrudes on your personal space or personal time.

Anything that impacts on any of these four aspects of our lives, we might think of as a privacy issue.

However when we start to think about privacy law in Australia, we are really only concentrating on the first category of privacy, which is the privacy of personal information.

What is personal information?

I have often found that people who work in information security, or people who have a lot to do with American companies or American businesses, use the phrase PII.  PII stands for Personally Identifying Information.  In the UK and Europe they use the phrase’ personal data’.  In Australia we use the phrase ‘personal information’, as do privacy laws in New Zealand and Canada.

Basically they all mean roughly the same thing: information about a human being, where that human being is clearly identified or might be identifiable.

(Exactly what makes someone identifiable is contested, with differing legal tests and interpretations of the related concepts of de-identified or anonymous data, but let’s leave that aside for another day.)

But in a nutshell, privacy laws regulate how certain types of data, described as something along the lines of ‘personal information’, is handled.

What does personal information include?

One of the common questions we get asked about the scope of privacy law is whether things like IP addresses, MAC addresses, or information about devices, is personal information.

This has been a contested area of the law for many years, but is increasingly so in this world of the Internet of Things, because information about things can be linked back to a human being who controls that thing.  There have been cases which turned on this issue, such as the Grubb v Telstra case which sought to answer the question of whether or not information about calls made to or from a mobile phone, or messages sent to or from a mobile phone, is ‘personal information’ – i.e. whether it is information solely about the phone, or also information about the person who is using that phone.

Similarly, the Waters v Transport for NSW case turned on whether data about the movements of an Opal Card is personal information about the person using the card.  (Answer: Yes it is.)

So with the Internet of Things, data that can be collected from different devices may start to show a pattern of a person’s behaviour.  To the extent that the data is about the behaviour of an identifiable person, it should be considered ‘personal information’ just as clearly as the laws regulate information about someone’s name, home address, or bank details.

Is it only ‘private’ information that’s regulated?

A common fallacy I hear about the scope of personal information is that personal information is only information that is ‘private’.  In fact the words ‘private’ and ‘public’ rarely come into privacy law.  Information is either about a person or it’s not.  Whether or not that information is already publicly known is a separate question.

The scope of what’s covered by the definition of ‘personal information’ (and thus, what is regulated by privacy law) is very broad.  It is not only what you might consider to be ‘private’, sensitive or embarrassing, but also information that might be publicly known, such as a person’s name or job title, or publicly observable, such as gender or eye colour.

Within most Australian privacy laws (yes, there are multiple privacy laws, because most of the States and Territories have one or two laws each, in addition to the federal Privacy Act 1988), the law starts with a definition of ‘personal information’, but then may also have a sub-set within that of what’s known as ‘sensitive information’ or ‘sensitive personal information’.  This sub-set is often then subject to some slightly tougher rules about how you can collect, use or disclose that information.  The kind of information that you typically (though not always) find in the definition of ‘sensitive’ information is personal information that is about a person’s health or disability, ethnicity, religion, sexuality, criminal history, political affiliations or trade union membership.  The law applies tougher standards to these categories of data because typically these are the kinds of information that might be used to discriminate against an individual.  Unfortunately, this notion of ‘sensitive information’ is easily confused with data classification schemes which also use the word ‘sensitive’, but to mean something else.

What do the privacy laws require us to do?

Privacy laws regulate both data flows and data governance.  They do so by laying out a number of ‘privacy principles’.

Data flows are regulated to the extent that privacy principles set the conditions under which personal information can be collected, used or disclosed: the what, how, and why.

The rules around collection of personal information first regulate the what.  They typically require that personal information can only be collected if having that data is reasonably necessary in order for the organisation to pursue a lawful purpose, related to the organisation’s legitimate activities.

There will usually also be some rules about the how of collection, for example requiring that personal information be collected fairly and transparently, directly from the person who is the subject of the information.

The main reason for which the organisation is collecting the personal information – the why – should be thought of as the ‘primary purpose’ for which that information can also be used.  Privacy principles typically allow the information to be used for the ‘primary purpose’, but also for a directly related secondary purpose, within the reasonable expectations of the individual.

An example would be a patient who has come into a hospital with a broken leg.  The primary purpose for which personal information about the patient is going to be collected will be to diagnose and treat their broken leg, and manage their stay in hospital.  A directly related secondary purpose for which their personal information might also be used internally could be to issue the patient with an invoice, or could be to commission a quality assurance review of the orthopaedic unit.  A directly related secondary purpose for which their personal information might be disclosed (given to a third party) would be sending a referral to the patient’s physiotherapist for further treatment after they are discharged.

However the default position in privacy law is that using or disclosing personal information for any other secondary purpose is generally not allowed, unless an exemption applies.  ‘With the individual’s consent’ is one such exemption, but consent is not always a pragmatic solution, for the reasons explained below.  You will usually find exemptions on grounds to do with law enforcement or national security, to prevent serious harm, to comply with another law, or to enable research in the public interest.

The other area of focus for privacy principles is data governance.

Data governance includes the need for transparency, amongst other matters such as enabling people to access the personal information held about them, and to seek correction of that information where appropriate.  It also refers to having a privacy compliance program in place, with appropriate pathways for people seeking to make a privacy complaint or report a data breach.

Proper transparency includes having a Privacy Policy, and giving notice to individuals about how their personal information will be collected, used or disclosed.  Giving notice is not the same as seeking consent.

So when do we need consent?

When thinking about data flows, my starting point is always to think about whether or not an organisation has the legal authority to collect, use or disclose an individual’s personal information.  The precise answer will depend on which privacy law/s apply to that organisation, but as noted above, typically privacy principles will offer multiple options for legally collecting, using or disclosing personal information.  Only one of those grounds will be ‘with consent’.

So you don’t need consent to do most things.  Consent should only be necessary if you are planning to do something not directly related to the primary purpose for which the personal information was collected from the individual in the first place, or sometimes (depending on the exact privacy law you are subject to) if you are planning to collect ‘sensitive’ personal information – and no other exemption applies.

But if you do need to rely on consent as the basis on which to authorise your collection, use or disclosure of personal information, make sure you know what consent means, and how to get it in practice.

Under Australian privacy law, for consent to be valid, as the legal basis on which an organisation can collect, use or disclose personal information, it must have five elements:  it must be voluntary, informed, specific, current, and given by a person with capacity.

Of these five elements, the most commonly misunderstood is the voluntary aspect.  To be considered voluntary, a consent must be a proactive choice exercised by the individual.  A valid consent must be an unequivocal ‘yes’ from a person who was given a genuine choice to say ‘no’ (without suffering any detriment), and where the default position is ‘no’.

What does that mean in practice?  Consent can’t be ‘opt out’.  It can’t be a condition of doing business with you.  And consent must be revocable; it must be as easy for someone to later withdraw their consent as it was for them to give it.

So consent is only useful for authorising data flows if your business process can cope with a whole bunch of people saying no, or saying nothing at all, when you ask them the question: “Hey, can we please also do X with your information?”

A collection notice is not consent.  Your Privacy Policy is not consent. (A Privacy Policy is not magic. It cannot authorise you to do anything that the privacy principles don’t already allow.  Your Privacy Policy is solely there to inform people, in general terms, how you handle personal information.  So don’t ask your customers to acknowledge, agree or consent to your Privacy Policy. It’s pointless.)

Clicking on mandatory T&Cs is not consent.  Offering an opt-out is not consent.  Pre-ticked opt-in boxes are not consent.  You cannot gain, infer or imply your customer’s consent to something simply because you mention it in T&Cs, a collection notice or your Privacy Policy.

So make sure you have separated out your collection notices from your consent forms and your Privacy Policy, and know when each one is needed and what they should include. They are three different things, serving three different purposes.  (Check out our Compliance Kits for templates of all three if you need assistance.)

Want to know more?

We hope you have enjoyed this Privacy 101.  Want more privacy knowledge?

Salinger Privacy has privacy training options from bite-sized webinars to professional certification programs, as well as online privacy awareness training modules.  Plus stacks of useful resources in our Compliance Kits.  Or contact us to see how our privacy specialists can assist your organisation.

Happy Privacy Awareness Week.

Photograph (c) Shutterstock

We’ve written before about the common causes of data breaches, but what about all the other types of privacy risks your organisation might face?

This month we have helpfully compiled for you a list of Ten Things To Do or Not to Do or Privacy Risks to Avoid and Other Things to Worry About Generally.  Which is too long for a blog title, sadly, so let’s just call them Things to Lose Sleep Over.

#1: Not understanding the value of your data

The public release of fitness app Strava’s data was a classic demonstration of an organisation not even realising the value to be found in the richness of its own records – and therefore not protecting them appropriately.  Geolocation data not only can make individual customers targets for harm, but can also create risks for groups of people – or even nation states – somehow related to your customers.

#2: Not understanding the identifiability of your data

The devil is in the detail: sometimes, despite purportedly being de-identified, data can reveal the identity of an individual, or at least lead them to be disambiguated from the crowd.  This might be because of poor de-identification techniques, like the MBS/PBS dataset. Other times it is the richness of the data which creates new privacy risks, such as the taxi trip data which revealed details about celebrity passengers, or could have allowed individuals to be targeted on the basis that they had visited a particular site – for example, a mosque or an abortion clinic.

#3: Thinking ‘notice and consent’ authorises data flows

Despite how the American model of privacy law works, in the rest of the world you can’t just legalese your way out of privacy obligations, burying expansive or permissive powers in mandatory T&Cs and then claiming your customers ‘consented’ to your practices.  (I mean, sure, some companies try, but the law is not on their side.)

There are several problems with relying on consent to authorise the collection, use or disclosure of personal information.  The first is that to be valid, the consent must be genuinely free, without a penalty attached to saying ‘no’.  So threatening an employee with dismissal if they refuse the collection of their biometric data does not allow the employer to claim that any such collection was conducted on the basis of consent. Further, to be voluntary, consent must be indicated with a proactive ‘yes’ from the individual.  A failure to opt out is not consent.

The second problem is that to be valid, the consent must also be informed and specific, which means that the organisation seeking consent must be precise about all the potential uses and disclosures that might occur, and the potential harms that might arise, if the person says ‘yes’.  But in the world of open data, predictive analytics, machine learning, algorithms and artificial intelligence, that’s not always possible to predict.  When it comes to AI in particular, consent is almost certainly useless as a mechanism to authorise your collection or use of personal information.

And finally, even if you manage to obtain a consent that is voluntary, informed and specific (plus current and given by a person with capacity), consent does not absolve you of compliance with all privacy principles.  Privacy law creates obligations covering the entire life cycle of handling personal information, and at many of those points in the life cycle consent is utterly irrelevant.  You still have obligations to only collect personal information that is reasonably necessary for a lawful purpose, to ensure that the data is fit for purpose, that you take all reasonable steps to protect data security, and so on.

#4: Thinking authorising data flows is all you need to worry about

When we conduct Privacy Impact Assessments, we are often asked whether or not a proposal to “let X access our data for Y purpose” will be compliant with the privacy principles.  But this is not only a question of whether, but also how.  Sure, first you need to determine whether the purpose for which you are proposing to disclose data to a third party is lawful – for example, whether there is another law authorising it, or if there is a public interest exception which applies, such as medical research or law enforcement.

But you still need to think about howthe data will be disclosed or accessed, because there is a spectrum of design options to choose from, and neither the lawyers nor the solution architects are going to come up with the most privacy-protective option without some prompting.

An example is the case of identity verification services run by marketing/data aggregation firms which are reported to now have ‘access’ to the electoral roll, following some law changes.  What was not explained in the media reporting is whether the firms are simply given copies of the entire electoral roll, whether they can access or extract bulk records, whether they can freely search on any name, how much data they can see about each person, or if they can only ‘ping’ the roll on a case-by-case basis by presenting a suite of already-known data about the individual to return a limited yes/no verification response.  Not all data ‘access’ is equal or carries the same privacy risks.  Notwithstanding your legal position, your social licence may depend on you choosing the most privacy-protective of all possible design options.

#5: Doing stuff your customers don’t expect

Other than your friends throwing you a lovely surprise birthday party, you probably don’t much like surprises.  Your customers don’t either.  Some examples of what not to do: collect mobile numbers you told your customers were to enable two-factor authentication, but then use them for spam; offer soccer fans an app which sneakily accesses their location data and microphone to listen out for illegally streamed matches; or ask people to make a submission to a parliamentary inquiry about tax matters but then use their details for political fundraising and disclose their details to an asset management firm for their own marketing purposes.

#6: Not doing stuff your customers do expect

This should be a no brainer, but if you are going to promise your customers a particular privacy control, make sure you follow through. Unlike in the UK, where the NHS asked people to opt out of having their patient records used for secondary purposes beyond their direct care, but then failed to properly record and respect the wishes of 150,000 patients, whose records were shared despite them opting out.

#7: Not implementing ‘need to know’

It’s not just external bad actors you need to worry about, there are also significant risks posed by trusted insiders.

A NSW auditor-general’s report in 2017 found that a third of NSW government agencies were failing to properly safeguard their data, by not limiting access to personal information to only those staff with a ‘need to know’.  The first fine issued under the GDPR in Portugal was €400,000 for a hospital which failed to follow the ‘need to know’ principle, when it allowed indiscriminate access to clinical data about all patients to both clinical and non-clinical staff.

And just the other week, the Commonwealth Bank had to enter into an enforceable undertaking with the OAIC, after it was revealed that there had not been appropriate user access controls to stop staff in the banking arm from seeing customer data related to the separate insurance arm.

#8: The rogue employee

Speaking of trusted insiders, watch out: a survey of healthcare workers in the USA and Canada suggests that one in five employees would sell confidential data, for as little as between $500 and $1,000.  Then there are the employees who access or disclose customer records for their own benefit or to assist a mate, examples of which have affected several banks, NSW Police and Queensland police; or to sabotage a company’s reputation, which is the claimed reason for a deliberate leak of data from valuation firm Landmark White.

#9: The helpful employee

It’s not just the bad apples you need to watch out for; it’s also the keen beans.

A staff member of a Bunnings store took it upon himself to create a database of customer records (with the aim of notifying customers about activities and events at their local store), as well as an employee performance monitoring database.  Unfortunately he did so contrary to organisational procedure and on his insecure home computer, causing more than 1,000 customer records and staff performance reviews to be publicly exposed on the internet.  And a NSW government agency was found in breach of the Disclosure principle when an employee responded to a solicitor’s request for information with more details then he was asked to provide, because he was “trying to be helpful”.

#10: The bored employee

Beware boredom.  A police officer who looked up the records of 92 women he saw on dating sites claimed he carried out his crimes “partially due to curiosity but also boredom with his job and during slow periods at work”.  And a review of 1,368 data breach incidents in the healthcare sector across 27 countries found that the majority of data breaches are caused by trusted insiders, with 31% of them involving staff looking up the records of celebrities or family members “for fun or curiosity”, such as the dozens of hospital staff who were fired after accessing the medical record of an actor in the news after an alleged attack.

The list goes on

This top ten list doesn’t even scratch the surface of the complexities involved in implementing privacy by design, but it does touch on issues that almost every type of organisation needs to deal with.

Managing privacy risks is not just about getting on top of data security.  You need to appreciate the value of your data, map how that data is being used throughout the organisation, ensure all those data flows are authorised, limit access on the basis of ‘need to know’, understand where and why staff might be tempted to use or disclose personal information for novel purposes, and follow through on the privacy promises made to customers.  And then train, re-train and train again all staff, so they know what they can and cannot do.

If you need pragmatic tools to help with your privacy risk management, check out Salinger Privacy’s Compliance Kits which include resources such as a Privacy Impact Assessment Framework, Privacy Risk Assessment Questionnaire, and Data Governance Protocol.  We also have online Privacy Awareness Training in multiple modes, from ready-to-roll to customised options, as well as professional development training for privacy officers, in either a one-day Privacy Management in Practice workshop, or a two-day IAPP privacy professional certification program.

Now that should help you sleep a little easier.

Photograph (c) Shutterstock

You say data breach, I say cybersecurity incident

You say privacy breach, I say an individual sending out emails

Potato, potahto, tomato, tomahto

Let’s call the whole thing off

(With apologies to Ella Fitzgerald)

What is a data breach?  Is it the same as a privacy breach?  Or a cybersecurity incident?  And how should we describe their causes?

A flurry of debate over where to lay the blame for data breaches was whisked up with the release earlier this year of the OAIC’s report into the first 12 months of the notifiable data breach (NDB) scheme in Australia.

964 eligible data breaches were notified to the OAIC.  Of these, 35% were categorised by the OAIC as “attributed to human error”, such as unauthorised disclosure of personal information or loss of a portable storage device.  The remainder were either “malicious or criminal attacks” (60%) or “system faults” (5%).

Speaking at an official Privacy Awareness Week event to launch the report, panel member Richard Buckland, who is Professor of lots of things starting with ‘Cyber’ at UNSW, suggested that really, when it comes down to it, all data breaches should be considered caused by human error.

His point, as I understood it, was that humans program computers, and humans design systems, and humans make decisions about how much energy, budget and time to put into training and supporting other humans who operate or use those systems (we poorly named ‘users’), and how best to protect systems and users from malicious attacks from other humans.  He was speaking about the need to improve the data literacy and cyber awareness of all humans, but especially the humans in his student body who will be programming the computers and designing the systems the rest of us humans will use.

Of course, that subtlety was lost when I tweeted simply that the professor had suggested that “all data breaches should be considered caused by human error”.

To which a fellow privacy expert Tim de Sousa responded with horror:

“Really, really, no. To err is human. Humans err. We know this. It’s risk management 101 (and a legal obligation under APP 11) to design your systems to mitigate known risks. Breaches ’caused’ by human error are *systems design flaws*.”

And then various other people who were most likely not in the audience and had not read the OAIC report jumped into the debate about who causes data breaches and how to categorise them, because that’s Twitter sometimes.

So who is right, Richard or Tim?  They’re both right of course.  The debate only arises because of the need for the OAIC to make crude distinctions for reporting purposes when categorising how breaches occurred, when the reality of root cause analysis can be far more complex than simply attributing blame along the lines of ‘system vs human’.

As the OAIC’s own report notes, “most data breaches—including those resulting from a cyber incident—involved a human element, such as an employee sending information to the wrong person or clicking on a link that resulted in the compromise of user credentials.”

Then there’s the confusion about what is a ‘data breach’ in the first place.  There is the legal definition under the NDB scheme, the short version of which is: any loss of, unauthorised access to, or unauthorised disclosure of, certain types of data including personal information.  Such a data breach becomes notifiable if it is assessed as ‘likely to result in serious harm’ to one or more individuals.

(I draw a distinction between a ‘data breach’ as per the NDB scheme, and a ‘privacy breach’ which I would define as any conduct in breach of one or more privacy principles.  A failure to take reasonable steps to protect data security, which leads to an unauthorised disclosure of personal information, would be both a data breach and a privacy breach.)

Note that the legal definition of data breach is deliberately tech-neutral.  Not every data breach is a cyber security incident; leaving a manila folder of paper client files on the bus is a data breach.  And conversely not every cyber security incident is a data breach; a Denial of Service attack may significantly impact your business operations, but without risking the data you hold.  What matters under the NDB scheme is whether personal information was put at risk of misuse.

So that’s the legal definition of a data breach.  Which judging by what we see or hear in the media, is apparently a very different thing to the politician’s definition.  Or the PR playbook.

When arguments were raging out the shift to an opt-out model for MyHealthRecord, the government was keen to spruik the benefits and downplay the risks.  But along with others I called out the then PM and Health Minister for misleading the public about the extent of data breaches that had already happened.  The agency responsible, ADHA, had already reported publicly on 11 data breaches it notified to the OAIC over the previous year, as required under the MyHealthRecord legislation (which contains its own reporting scheme that pre-dates the NDB scheme and has a slightly more expansive definition).

Yet the then PM Malcolm Turnbull stated “There are six million records — six million My Health records. There have been no privacy breaches.”  And the then Health Minister Greg Hunt said “When you look at six million people, six years, though, on the latest advice today, no data breaches.”

How did they get away with saying this?  Part of the problem is that people tend to talk at cross-purposes.  If you think ‘data breach’ or ‘privacy breach’ only refers to external bad actors getting through your cybersecurity defences, then maybe the politicians’ semantics start to make sense.  But that’s certainly not what the law states, and it is irresponsible at best, or actively misleading at worst, for politicians to say there have been no data breaches when the law says there have been, the agency responsible says there have been, and the OAIC says there have been.  11 of them, to be precise.  That ‘human error’ was to blame rather than human hackers does not necessarily lessen the privacy risks posed to the individuals affected.

The “nothing to see here, no cyber problems, just human error” line seems to be a favourite of organisations responding to a data breach, including NAB.  LandMark White, a valuation firm, managed to devalue itself to the tune of $7M after suffering a data breach which it described as a “cybersecurity incident”.  Perhaps learning from the effects on its business model by admitting to information security vulnerabilities, the next time it suffered a data breach, the chairman was quoted as saying “it was not a data breach but an individual who was sending out individual emails with separate attachments”.  Right, because knowing you have staff who are deliberately leaking data is somehow less of a problem?

There have been other lame semantics, blame-shifting and weasel words recently, such as when a contractor to the Department of Home Affairs accidentally emailed the medical details about hundreds of visa applicants to the wrong person.  The official line was: “The document contained bio-data details of visa applicants. No actual personal client medical records were disclosed as part of this incident.”

I can picture now the media management guru advising “instead of calling it health information or medical records let’s call it ‘bio-data’ so no-one knows what we’re talking about and it sounds less serious”.

Then there’s Neoclinical, which accidentally exposed 37,000 Australians’ particularly intimate health information by placing the records on an insecure cloud server.  It quickly attempted to shift the blame to the security firm which identified the insecure records and eventually went public about it, as a ‘marketing’ exercise.

With so much debate – and some deliberate obfuscation – about data breaches, it’s no wonder people are confused about where to look, who to blame, or how to prioritise their limited data loss prevention and privacy risk management resources.

But consider this.  The single most common cause of a data breach in the first year of the NDB scheme was personal information sent to the wrong recipient by email (28%).  Failure to use BCC when sending an email accounted for another 8% of human error breaches.  Thus more than a third of all data breaches caused by human error involve the simplest of tasks: sending emails.

Data breaches can be low-tech or high-tech, deliberate or accidental, featuring trusted insiders or external bad actors.  Staff training and awareness, tech tools to help with data loss prevention (like email and document classification, and encryption) and accountability (like collecting and monitoring audit logs), vendor and supplier contract management, data breach response plans and phishing simulations – the privacy practitioner needs to cover them all.

Because data protection is not just about your cyber defences, but requires active management of your entire data ecosystem.

If you need assistance with data protection, consider our September webinar on outsourcing and managing contractors, our October CIPM certification training, or our template Data Breach Response Plan which comes included in most of our Compliance Kits.

Photograph © Anna Johnston

A recent case illustrates the need to think about privacy in both system design and human decision-making.  Plus, how keeping user experience (UX) front of mind when designing systems or processes should result in better privacy outcomes too – and maybe help preserve some human dignity along the way.

In DQJ v Secretary, Department of Family and Community Services [2019] NSWCATAD 138, a mix of poor system design and human error caused a distressing disclosure.

DQJ was a homeless woman applying for housing through the Department of Family and Community Service’s online application form.  The online form made it mandatory to enter a contact residential address.  Thus even though she had no residential address, in order to lodge her application she had to nominate an address.  DQJ therefore listed a previous address.  However she also made clear in the application form that she had no fixed address, and preferred to be contacted by email.

The Terms and Conditions for using the system, which DQJ had to ‘accept’ in order to make her application, said that once an outcome on the housing application had been made, she would receive communication (via email, SMS and/or letter) either shortly after receiving the last required supporting documentation or within two months of lodgement.

The Respondent argued that this constituted DQJ’s ‘consent’ to be contacted at the old address.  The Tribunal certainly thought this lent some weight to the department’s argument.

Personally, I would disagree.  Mandatory Terms and Conditions cannot indicate consent, because DQJ had no alternative.  She was a homeless woman in need of emergency housing, so if she refused to accept the Ts&Cs she would remain homeless, which is hardly a position from which to offer ‘voluntary’ consent.

Further, the system made a field mandatory which, in the context of applications from potentially homeless people, seems illogical.  The system also asked for her preferred contact mechanism, but then did not respect her answer.  And importantly the Ts&Cs said she would be contacted by “email, SMS and/or letter”, which in our view can also be read as meaning that hard copy letter was not the only mechanism, especially for someone who had clearly nominated email as her preferred mechanism.

But what actually happened was this.  The online system automatically generated a hard copy letter which was sent to the old address.  The contents of the letter included that DQJ was homeless and in need of accommodation, and mentioned a health condition relevant to her accommodation needs.

DQJ complained about the letter being sent to the old address.  There was no evidence to suggest the letter had been opened by the new occupants, but nor was the letter returned to the Department, so there was a potential disclosure to whoever received that letter, if they opened and read it.

Although found to be out of scope for this litigation, from a design and privacy risk management perspective it is useful to understand what happened next.

After DQJ made a complaint to the Department about them mailing hard copy letters instead of emails as she had requested, the Department conducted an internal review, in which it admitted this conduct was an unauthorised disclosure.  The Department apologised and updated the system to show that the old address was no longer in use.

Nonetheless, even after that update, three further letters were sent to the old address.  This was described as human error, because unlike the first letter they were not system-generated letters.  The description of the problem was that:

“the Respondent … end-dated DQJ’s contact address to avoid system generated correspondence being sent and a client specific notification (was) updated within the system advising that DQJ’s is only to receive correspondence by email”.

However, “the notation … had not been read”.  The officer responsible for managing the privacy complaint then had to speak to the team leader about staff training, and the agency had to add an additional pop-up warning on the system.

So we’ve got multiple points of failure here.  Poor design of the application form, especially in the context of the Department’s client base.  Poor translation of the answers given in the application form about preferred contact mechanism into system-generated outcomes.  And then even when the problem was supposed to be remedied, poor staff practices meant that notes went either unread or ignored.

In this case, only the disclosure in the first letter was in scope in the Tribunal, along with a separate matter.  The complainant had brought her complaint to the Tribunal because she was seeking compensation.  Despite the Department admitting this was an unauthorised disclosure in the internal review, the Tribunal actually found it was authorised, on the basis that DQJ had only expressed her ‘preference’ to be contacted by email, and had not requested that all communication be by email.  (There is no mention in the case as to whether this was expressed as an option on the form.)  The Tribunal also gave weight to the wording of the Ts&Cs.

Personally, I think that decision puts too much onus on individuals to have to proactively object to something, going beyond already saying “here is how I prefer to be contacted”, when in fact a better system design would have prevented this problem in the first place.

Imagine instead an online housing application form which allows people to say: “I don’t have a fixed address, I am homeless, so please only contact me via this email or this phone number.”

So some takeaway lessons here:

1. Don’t make data fields mandatory if doing so forces people to give you incorrect data, just to get through a web page.
2. Think about UX, or in other words consider your client base when you design points of data collection.  For example, if you are in the business of offering emergency housing assistance, do not be surprised that some of your clients are homeless, and therefore will not have a current residential address.
3. Offer multiple ways of receiving communications.
4. If you are going to let people nominate their preferred contact mechanism, respect their wishes, and only use that mechanism to contact them.
5. Make sure that both system-generated and human-generated communications follow the same business rules.
6. Ensure that staff are trained in those business rules.

This case illustrates the need to think about upskilling staff throughout your organisation to think in terms of ‘privacy by design’ – and, indeed, user experience.  From the precise way online forms are designed, to the way systems act on the data collected, and the staff who need to stop and think before they use or disclose personal information, at least basic privacy skills, and the ability to stand in the shoes of your customer, are needed at every decision-making point.

Photograph (c) Shutterstock

A recent case illustrates the importance of robust, mandatory privacy training for staff, to avoid privacy breaches – or, if a breach does happen, in order to avoid liability for when a rogue employee goes off on a privacy-invading frolic of their own.

A rogues’ gallery of privacy violations

Up until the recent case of CJU, public sector agencies responding to privacy complaints in the NSW Civil and Administrative Tribunal (the Tribunal) have successfully pursued the argument that if a disclosure was not authorised within the terms of the NSW Disclosure principles (IPP 11 or HPP 11), the agency can simply claim that as the disclosure was not authorised, it must, by definition, have been the act of a rogue employee – for which the agency is not liable, because the rogue employee’s actions should not be attributable to the agency.

In the words of the Tribunal, the ‘rogue employee’ defence is “that not only are the actions unsanctioned by the agency, but the individual is acting in effect contrary to direction and in a rogue and aberrant manner”.

This idea of agencies being able to escape liability for the unauthorised actions of a ‘rogue’ employee date back to 2006, when the NSW Court of Appeal found that not every action by an employee can be attributed to their employer, under the NSW privacy statutes:  “Where … the “use” or “disclosure” of information was for a purpose extraneous to any purpose of the Department, it should not be characterised as “use” or “disclosure” by the Department or conduct of the Department. … it was not, in my opinion, Parliament’s intention to expose every such agency to a form of absolute liability for the unauthorised private conduct of its employees or agents”.

The Court of Appeal thus found that the Department of Education was not responsible for the actions of their employee when he disclosed information in his “private capacity” as a soccer coach, rather than in his employed capacity as a teacher at the school – even though it was in his capacity as a school teacher that he discovered the information in the first place.

As a result of that decision, in multiple cases since 2006 the Tribunal has applied the ‘rogue employee’ defence to the benefit of government agencies seeking to avoid liability for privacy breaches caused by the conduct of their staff.  In each case the agency escaped any liability for a breach of the Disclosure principles, and as a result the victims received no remedy.

However in our view there is considerable space between conduct that is authorised by the law (being conduct in compliance with IPP 11/HPP 11, or allowed under an exemption to the relevant principle), and conduct that is not only not authorised, but is also so outside the normal standard of conduct for that agency, and motivated by malice or corruption, that the responsible employee should be prosecuted under the criminal offence provisions of the legislation.  It is in this middle ground that an agency should – and as a result of a recent case, can – be held liable for an unauthorised disclosure.

Rogues no more: the effect of the CJU case

The CJU case provides an example of a disclosure that was not authorised by the agency, but nor was it malicious or corrupt.  And critically, the adequacy of staff training was a pivotal element in determining that the agency was liable for the conduct.

In CJU v SafeWork NSW, the respondent admitted that an unauthorised disclosure had occurred.  In the words of the Tribunal: “The respondent’s case was that the Disclosure was a discrete breach of information privacy made in good faith by Mr Covi who was trying to be helpful in responding to an enquiry from a government agency’s solicitor”.

The Tribunal sought to test whether or not the disclosure was indeed made in good faith, questioning why the employee disclosed more information than was requested by the solicitor, without first seeking the consent of the subject.  The Tribunal did so by enquiring into the employee’s understanding of his privacy obligations, and what privacy training he had received.  Evidence was provided which showed Mr Covi had completed the agency’s online induction training course.

He should have known better: the link between inadequate training and liability

The Tribunal reviewed the agency’s training materials, which offered an overview of the IPPs and informed staff of their legal responsibilities.  The Tribunal described the assessment as consisting of “only 10 relatively simple questions”, and noted that Mr Covi said the online course had taken him no more than half an hour, and possibly materially less.

The Tribunal found that “the training steps that have so far been carried out by the respondent are inadequate to convey to staff their responsibilities concerning the disclosure of personal information obtained during the exercise of the powers and functions of the respondent”.

The Tribunal concluded that the evidence “suggests a step taken in ignorance of the applicant’s rights rather than acting in bad faith or maliciously to harm or undermine the applicant’s interests”.  The result was therefore a disclosure that was not authorised under the law (and thus was a breach of IPP 11), but because the employee had not been adequately trained in his privacy responsibilities, his actions were attributed to ignorance, rather than bad faith.

In these circumstances, the agency could not claim the ‘rogue employee’ defence, and it was found liable for the breach.

The need for comprehensive privacy training

The effect of CJU is clear: train your staff properly, to reduce both the likelihood of, and liability for, privacy breaches.

The Tribunal characterised the “inadequate training concerning information privacy protection” as “giving rise to a sufficient risk of a future breach”.

To be effective, privacy training must set out clear examples of what is and is not allowed under the privacy law applicable to that organisation, in a way that speaks to their employees’ experiences.  Training content must be thorough, and not simply recite the law.

The style must be interactive, to keep staff paying attention.  The concepts must set a high enough bar that staff cannot simply cruise through; staff should be challenged, and stretched, by training.  The final assessment must likewise be challenging, and must generate proof of understanding and completion for each employee.  Hosting your online training in a Learning Management System can help, by enabling you to report to management on the rollout of your training program, keep a track of who has completed training, and retrieve evidence of completion for particular personnel in the event of a privacy breach.

And make sure staff training is repeated, or periodic updates provided.  After conducting a ‘sweep’ of Victorian public sector agencies’ compliance, the Office of the Victorian Information Commissioner OVIC recently advised that organisations “should provide refresher training for privacy, not just during employee induction.”

Only once you have a robust, comprehensive and high quality staff training program in place, can your organisation then argue, in the event of an unauthorised disclosure, that the responsible employee was ‘rogue’, such that your organisation should not be liable for their actions.

Salinger Privacy offers quality comprehensive, interactive online privacy awareness training modules, which can be purchased off-the-shelf, or further customised to suit your organisation.  See more at www.salingerprivacy.com.au/training/online-training/

Photograph (c) Shutterstock

An earlier version of this article was first published in the Law Society of NSW Journal, May 2019 edition.

As you might expect, the recent investigation by the Office of the Victorian Information Commissioner (OVIC) into the public release of data about myki card users includes important insights into de-identification and re-identification, which were picked up and commented on in the media.  But a secondary impact of the report, not much commented on to date, is that OVIC has offered important critiques of Privacy Impact Assessment (PIA) as a methodology.

The report highlights the importance of approaching PIAs in a fulsome, defensible and iterative way; to be careful about making assumptions concerning safe use of data; and to ensure that all parties involved in a project understand who is responsible for what.

I have drawn these critiques into eight lessons we can all learn about PIAs.

The background

In mid-2018, Public Transport Victoria (PTV), the agency with responsibility for public transport administration across Victoria, released a dataset of 1.8 billion records of transport users’ activity to Data Science Melbourne for use in the Melbourne Datathon.  The Datathon is an annual event in which entrants (typically data scientists, academics and students) compete to find innovative uses of a dataset.  The dataset contained the records of ‘touch on’ and ‘touch off’ activity of 15.1 million myki cards over a three year period to June 2018.

PTV maintained that the dataset was disclosed in response to a request from the Department of Premier and Cabinet (DPC), which oversights the government’s open data platform, through the DataVic Access Policy and Guidelines.  DPC had been represented on the Datathon judging panel and provided sponsorship to Data Science Melbourne for the Datathon.

Based on advice that certain de-identification techniques would be applied to the data prior to release, PTV completed a threshold PIA checklist and gave the ‘all clear’ for the release.  However, on their receipt, a number of Datathon competitors reported concerns that the dataset was still readily identifying.  Whilst names were excised and card numbers randomised, in a number of cases, taking what might be known from as little as one shared trip with an acquaintance was enough to deduce all trips they had made in the three year period.

A re-identification exercise conducted by a team from the University of Melbourne also found that combining information from other sources with information in the dataset about the relatively small number of some categories of card holder (police and politician card holders), rendered the dataset ‘personally identifying’.  As a consequence, it revealed a significant amount of location data about individuals and their likely travel patterns.

Due to a number of factors, including the heightened risk posed by the sheer size of the dataset, and the potential impact of the breach on public trust, OVIC determined to investigate the circumstances, including the steps taken to assess and approve the data for release.

There’s a degree of irony in the concept of assessing re-identification risk for a massive dataset intended for use by data enthusiasts.  Unlike most other instances of proactive public release of data – for general human interest, or to foster public sector accountability, this was a group itching to investigate what powerful insights could be drawn from the data when manipulated, drilled, mixed and matched.  In those circumstances, one would expect that the most robust of de-identification techniques might be employed, in addition to other data security and assurance measures.

There are eight poignant lessons to be learned from the investigation, on the purpose of, and the approach to, PIAs.

Plan for the unknown

PIAs that are fulsome, measured, defensible and iterative are powerful tools.  They can anticipate the foreseeable (and the not so foreseeable) privacy risks.  They highlight current controls around general data governance frameworks and established processes to deal with ‘known knowns’.  They suggest new treatments or risk mitigations to deal with ‘known unknowns’, and data breach management plans to deal with ‘unknown unknowns’.  They suggest strategies for dealing with risks such as the risk of re-identification.  PIAs foreshadow and plan for the consequences in the event that things do not actually go to plan.

Use existing frameworks

In relying exclusively on ‘de-identification’ of the data to manage the risk of ‘re-identification’, PTV overlooked other possible means of protecting the information, such as the Five Safes Framework for managing statistical disclosure risk, which would have suggested:

• Limiting disclosure to a known and fixed list of Datathon participants
• Ensuring participants were subject to contractual or legal obligations to not attempt to re-identify the data, or on-disclose the data, and to destroy the data at the end of the Datathon
• Ensuring the data was held on a secure system, to limit extraction and retention by participants, and
• Testing whether the data was ‘safe’ for the planned release.

The importance of data literacy

Deficiencies in governance and risk management in relation to data can undermine the protection of privacy, even for well-intentioned projects.   As OVIC saw it, this matter demonstrated the significant challenges in identifying privacy risks in large, complex datasets, and the need for the Victorian public sector, which possesses many large and sensitive data holdings, to have a high level of data literacy.

In particular, over the course of many months during which PTV considered the proposed data release, consulted, undertook its PIA, and prepared the data for the release, a raft of guidance, from both OVIC and the OAIC, was published on managing re-identification risks.

The OAIC’s March 2018 report into the problems associated with the public release of MBS and PBS data highlighted the risks of taking a simplistic approach to de-identification before an open data release.  OAIC’s guidance on de-identification was also updated in March 2018.  Even before that, in late 2017, the OAIC and CSIRO had released a detailed De-identification Decision-Making Framework.

Meanwhile in May 2018, OVIC’s guidance on de-identification suggested that analysis of unit level data “is most appropriately performed in a controlled environment by data scientists” rather than being released publicly, which OVIC described as “a risky enterprise”.  Most pertinently, in that report OVIC had called out PTV’s counterpart agency in NSW, Transport for NSW, as offering a benchmark for how to safely share and analyse public transport smartcard users’ data, without unnecessarily raising re-identification risks.

Yet this critical regulator guidance on de-identification had not filtered through to impact on key decisions made by PTV and DPC, before the PTV data was released in July 2018.

Be clear about responsibilities

At its broadest, the OVIC investigation highlighted the consequences of a manifest disconnect between PTV as data custodian, what it thought its role was, and its obligations and accountabilities, in relation to Datathon competitors’ use of the data, and DPC as a more distant sponsor, supporter and public sector lead for the initiative.

Throughout the process of developing and disclosing the myki dataset to the Datathon, and OVIC’s investigation, both PTV and DPC displayed a lack of clarity about which agency was responsible for protecting the dataset and identifying and managing privacy risks.

Use experts and test assumptions

The PIA was premised on the assumption that the dataset had been successfully ‘anonymised’ by one area of PTV, and so concluded that the dataset could therefore be safely released for use in the Datathon.

For example as to whether the program was going to collect, use or disclose re-identifiable information, PTV’s PIA stated:

“No. There is no way to link the public transport travel patterns of individual mykis to specific people via the encrypted internal card ID – this is not publicly available and will be encrypted in any case. The only remaining risk is that someone may attempt to identify a specific myki card based on the travel patterns but this would require a detailed knowledge of when and where a person had used public transport – basically a travel diary – and it would be very difficult to distinguish from other cards with similar travel patterns. In the unlikely event that this succeeded it would only reveal which Public Transport modes and stops the card had appeared at”.

This view, that the dataset had been de-identified, formed the basis for the subsequent governance of the released data.  However as OVIC and the University of Melbourne team demonstrated, re-identification from the dataset was considerably easier than PTV had imagined.

OVIC noted that:

• PTV did not seek any external expertise to assist with de-identifying the dataset, and
• PTV’s decision-making processes were not clear or well documented, and appeared to lack both the support of an effective enterprise risk management framework and suitable rigour in the application of a risk management process.

OVIC cautioned that appropriate processes and expertise should sit behind any decision to release de-identified personal information.

Templates are useful, but only to a point

PTV had used a PIA template report issued by the predecessor agency to OVIC.  But by design, templates are generic in nature, and will not be a neat fit for every project.  Answering template questions without critical analysis will not magically produce the answers needed to properly document, assess and manage risk.

The PIA did not describe in detail what data would be released, other than to say it would be anonymised myki data.  The PIA concluded that “no personal information capable of identifying an individual” would be used, but without sufficient analysis or reasoning for that conclusion.

The template did ask users to consider the risk of re-identifiable information and why. But given PTV had firmly concluded that there was no ‘personal information’ involved, the remaining sections of the template, which were designed to manage risks including that of re-identification, were left incomplete.

Know the project’s scope (and keep the PIA current)

Unfortunately, the PIA did not appear to envisage the dataset being released as ‘open data’, despite contemporaneous documents providing mixed accounts of how the data would be released or used as part of the Datathon.  Further, the scale of release was significantly broadened following the PIA, but the PIA was never revisited to align with the change in project scope.

OVIC cautioned on the importance of re-visiting a PIA and its assessment of risk if, over time, the scope and quantity of data to be managed is to be expanded or clarified.

A PIA is not a project approval

The PIA was approved by the PTV ‘owner’ of the myki dataset, and the PTV chief information officer. It was the only authorising decision or documentation for PTV’s decision to release the data. There was no other written agreement with respect to use by Data Science Melbourne of the dataset for the Datathon.

Conclusions

• Be fulsome and factually correct. Properly account for the relationships between parties involved in the project, the data flows, and describe the data governance arrangements to put in place.
• Ask, “Where has something similar been done elsewhere?”, “What approach was taken?” and “Was it successful?”
• Approach the consultations with stakeholders that will inform the PIA’s content, exploratively and collectively – not as shuttle discussions. Both PIAs and the consultations that support them are exercises in translation, ensuring everyone is on the same page about a project’s objectives, risks and their management.
• Plan alternative strategies to deal with incorrect or misunderstood conclusions about how identifying your data might be – in both planned and alternative contexts.
• Keep the PIA scope broad, considering whether links between purpose of collection and disclosure are logical, and try to gauge community expectations about use (aka ‘the pub test’).
• Remember that PIAs conducted at a single point in time must be revisited as projects evolve in scope.
• Don’t rely on a template to achieve more than it is designed to do, or use it as the sole source of approval for release of a significant and complex dataset. This gives a false sense of security for all stakeholders involved.

In its investigation report, OVIC acknowledged that while data-driven insights can bring great benefit, they can also put individuals at risk, and some old assumptions about data de-identification require revisiting.  For datasets containing unit level data about individuals, and particularly longitudinal data about behaviour, OVIC noted that some research now indicates that such material may not be safe for open release, even where extensive attempts have been made to de-identify it.

Not everyone can read a crystal ball, but a well-considered PIA that foreshadows and plans for all risks – even if considered unlikely – will at least go some way towards identifying and mitigating privacy risks for your projects.

Photograph (c) iStock

Privacy dies yet again

In another masterful piece of privacy reporting, Kashmir Hill in the New York Times has exposed the nefarious use of facial recognition technology by start-up Clearview. The business offers face searching and identification services, ostensibly to police forces, on the back of a strikingly large database of reference images extracted from the Internet. Clearview claims to have amassed three billion images ― far more than the typical mugshot library ― by scraping social media and other public sources.

It’s creepy, it offends our intuitive sense of ownership of our images, and the potential for abuse and unintended consequences is enormous.  But how should we respond objectively and effectively to this development? Does facial recognition, as the NYT headline says, “end privacy as we know it”?

First let’s get one distraction out of the way. I would agree that anonymity is dead. But this is not the end of privacy; instead I feel it might be a new beginning.

If there’s nowhere to hide, then don’t

Why would I say anonymity is a distraction? Because it’s not the same thing as privacy.  Anonymity is important at times, and essential in some lines of work, but it’s no universal answer for the general public.  The simple reason is few of us could live much of our lives in hiding.  We actually want to be known; we want others to have information about us, so long as that information is respected, kept in check, and not abused.

Privacy rules apply to the category of Personal Data (aka Personal Information or sometimes Personally Identifiable Information) which is essentially any record that can reasonably be associated with a natural person.  Privacy rules in general restrain the collection, use, disclosure, storage and ultimate disposal of Personal Data. The plain fact is that privacy is to protect information when it is not anonymous.

Broad-based privacy or “data protection” laws have been spreading steadily worldwide ever since 1980 when the OECD developed its foundational privacy principles (incidentally with the mission of facilitating cross border trade, not throttling it).  Australia in 1988 was one of the world’s first countries to enact privacy law, and today is one among more than 130.  The E.U.’s General Data Protection Regulation (GDPR) currently gets a lot of press but it’s basically an update to privacy laws which Europe has had for decades.  Now the U.S. too is coming to embrace broad-based data privacy, with the California Consumer Privacy Act (CCPA) going live this month.

Daylight robbery

Long before the Clearview revelations, there have been calls for a moratorium on face recognition, and local government moves to ban the technology, for example in San Francisco.  Prohibition is always controversial because it casts aspersions on a whole class of things and tends to blur the difference between a technology and the effects of how it’s used.

Instead of making a categorical judgement-call on face recognition, there is a way to focus on its effects through a tried and tested legal lens, namely existing international privacy law.  Not only can we moderate the excesses of commercial facial recognition without negotiating new regulations, we can re-invigorate privacy principles during this crucial period of American law reform.

I wonder why privacy breaches for some people signal the end of privacy?  Officially, privacy is a universal human right, as is the right to own property. Does the existence of robbery mean the end of property rights? Hardly; in fact it’s quite the opposite! We all know there’s no such thing as perfect security, and that our legal rights transcend crime.  We should appreciate that privacy too is never going to be perfect, and not become dispirited or cynical by digital crime waves.

What is the real problem here?

Under most international data protection law, the way that Clearview AI has scraped its reference material from social media sites breaches the privacy of the people in those three billion images.  We post pictures online for fun, not for the benefit of unknown technology companies and surveillance apparatus. To re-purpose personal images as raw material for a biometric search business is the first and foremost privacy problem in the Clearview case.

There has inevitably been commentary that images posted on the Internet have entered the “public domain”, or that the social media terms & conditions allow for this type of use.  These are red herrings.  It might be counter-intuitive, but conventional privacy laws by and large do not care if the source of Personal Data is public, so the Collection Limitation Principle remains.  The words “public” and “private” don’t even feature in most information privacy law (which is why legislated informational privacy is often called “data protection”).

Many privacy laws also require the collection of Personal Data to be conducted fairly, transparently, and directly from individuals themselves unless a particular exception applies.  And commonly privacy laws also place heightened restrictions on the collection of sensitive categories of information such as biometrics.

Scraping images from social media sites for the purpose of developing facial recognition systems offends all of these privacy principles.

Data untouched by human hands

The second problem with Clearview’s activity is more subtle, but is a model for how conventional data protection can impact many more contemporary technologies than just facial recognition. The crucial point is that technology-neutral privacy laws don’t care how Personal Data is collected.

If an item of Personal Data ends up in a database somewhere, then the law doesn’t care how it got there; it is considered to be collected.  Data collection can be direct and human-mediated, as with questionnaires or web forms, it can be passive as with computer audit logging, or it can be indirect yet deliberate through the action of algorithms. If data results from an algorithm and populates a database, untouched by human hands, then according to privacy law it is still subject to the same Collection Limitation, Use & Disclosure Limitation and Openness principles as if it had been collected by another person.

The Australian Privacy Commissioner has developed specific advice about what they call Collection by Creation:

The concept of ‘collects’ applies broadly, and includes gathering, acquiring or obtaining personal information from any source and by any means. This includes collection by ‘creation’ which may occur when information is created with reference to, or generated from, other information the entity holds.

Data analytics can lead to the creation of personal information. For example, this can occur when an entity analyses a large variety of non-identifying information, and in the process of analysing the information it becomes identified or reasonably identifiable. Similarly, insights about an identified individual from data analytics may lead to the collection of new categories of personal information. 

The outputs of a face search service are new records (or labels attached to existing records) which assert the identity of a person in an image. The assertions are new pieces of Personal Data, and their creation constitutes a collection. Therefore existing data protection laws apply to it.

The use and disclosure of face matching is required by regular privacy law to be relevant, reasonable, proportionate and transparent. And thus the effect of facial recognition technology can be moderated by the sorts of laws most places already have, and which are now coming to the U.S. too.

It’s never too late for privacy

Technology incidentally does not outpace the law; rather it seems to me technologists have not yet caught up with what long-standing privacy laws actually say. Biometrics certainly create new ways to break the law but by no means do they supersede it.

This analysis can be generalised to other often troubling features of today’s digital landscape, to better protect consumers, and give privacy advocates some cause for regulatory optimism. For instance, when data-mining algorithms guess our retail preferences or, worse, estimate the state of our health without asking us any questions then we rightly feel violated.  Consumers should expect the law to protect them here, by putting limits on this type of powerful high tech wizardry, especially when it occurs behind their backs.

The good news is that privacy law does just that.

This blog was first published by Constellation Research, and is reproduced here with permission.

Photograph © Shutterstock

If you enjoyed this blog, subscribe to our newsletter to receive more fab privacy news every month – sign up below.

I love Islamic architecture, for the arched doorways offering little glimpses into private worlds.

‘Smart cities’ are all fine and good for life’s necessary conveniences, with their state-of-the art parking tech lighting your way to vacancies, conveyer belts that kick in when they see you coming, always-on wifi, and other technological perks that keep us moving, switched on, and wired for daily life.

If at times smart cities represent ‘digital overload’, then surely passing through an ancient archway to a tranquil space must be its antithesis – that civic digital de-tox we don’t usually attain unless resolutely electing to break the circuit, leave the city and ‘go bush’.

Privacy in architectural Design 

Privacy, articulated by Louis Brandeis and Samuel Warren as our ‘right to be let alone’ has multiple dimensions – bodily privacy, communications privacy, information privacy and territorial privacy.

To protect information privacy, our regulators advocate Privacy by Design.  However, in this article, I’d like to explore the concept of ‘Privacy in Design’.

• How are privacy interests reflected in the physical design of homes, workplaces and recreational or learning spaces?
• How do they differ across cultures?
• When did this concept of privacy as an architectural feature first come into being?
• How has a culture of privacy influenced architecture over time?

A historical perspective

The word ‘privacy’ derives from the Latin privatus, meaning separation between the communal (household) and public spheres.  It later came to signify the separation of personal (individual) and public.

The development of privacy as a human interest or psychological construct mirrors the significant cultural shifts societies experienced moving from the Renaissance period, to the Age of Enlightenment.  Enlightenment witnessed further exploration of science and nature, new think tanks, and the rise of the novel – with increasing numbers of those who could read and write moving away from traditional ecclesiastical content, to exploration of individual discoveries, plights, travel, and other human experience.  These were stories of individual identity, reflection and self-determination.  And thus, the concept of privacy as a form of individual identity arose.

Sociologists observe that on a macro level, in the western world, we also moved from a collectivist ‘shame culture’ in which our identity and reason for being was to be found in our group or tribe, to an individualist, ‘guilt culture’, in which we became members of a nuclear family unit – identifying for the first time as truly separate from extended kin.  The concept of ‘self’ differs dramatically in each.  In a ‘shame culture’, individual actions either bring honour to the family or ‘shame it’.  In a ‘guilt culture’, you ought to be either be proud of your personal achievement, or ashamed of yourself.  Identity (and life choices) are also more pluralistic – some have said, making us more prone to identity crises, career flips and other sea changes, at different points in life.

With rising standards of living, and an ability to alienate other nuclear families from property, we sought to keep up with the Joneses, hiding dirty laundry from those outside our nucleus.  This significant cultural shift bore out in architectural design.

House design changed from single-room (i.e. open plan living) feudal thatched roofs, to rooms separated by corridors and quarters – building more impressive drawing rooms to the front, and shifting sculleries to the back.  In Europe and England, we first saw hallways built for passage to private rooms and chambers, and proper stairways to those hallways emerge around the late 17^th century, and then become widely used from the 19^th century.

The pendulum of time still swings back and forth in domestic spaces – bringing solitude and reprieve, or ‘family time’.  In the post-WW2 ‘west’, we saw a return to those open-plan living spaces (thankfully without thatched roofs and dirt floors), making the kitchen, for the first time, a central hub of collaboration and control. Our kitchens were encircled with a bar so friends and family could engage in natter whilst meals were being prepared.  How else could women have been encouraged to step back into full time domestic duties when men folk returned from war and needed their jobs back?

And then there are those funny ‘throwbacks’ in architectural design.  You tend to notice them when travelling as physical travel alters (sometimes strengthens) your sense of self and ‘other’.  I used to wonder, for example, why so many Amsterdam homes have big, curtainless bay windows to display thoughtfully positioned knickknacks?  My Dutch friends tell me this is a cultural hang-over sourced in Amsterdam’s 15th century mercantile history.  If ever there was a society that could successfully mix work and pleasure, it would be the Dutch.  Most Amsterdam households were trading households and maximising opportunity for trade within the local community was the foundation for prosperous family and social life.  Any wares to be offered for trade would be left in the bare window of each home, for purchase at any time of day or night.

A cultural perspective

The above is of course a very rough and blunt dichotomy between individualism and collectivism.  And just as globalisation and immigration have significantly blended these two cultural concepts, so too do we see this blending in architectural design – the Spanish Mission home, the French Façade, the quiet Japanese courtyard.

We see key differences within cities, between quarters, or from one building to the next.  In Marrakech, for example, the outward-facing balconies and large windows in the Jewish quarter are markedly different to the surrounding Islamic architecture, where homes feature lavishly decorated, peaceful and plant abundant internal courtyards, but offer little decoration or display of wealth (or even windows) on the outside.  And there are still nuances between cultures in how we conceive personal space, as evidenced by Vic Groskop in ‘Personal Distance – why Russian life has no room for privacy’.

Other aspects of design that enhance personal privacy include the use of fountains in parks to provide enough noise for ‘acoustic privacy’, or that special breed of Lilly Pilly, the ‘neighbours be gone’ tree or ‘spite hedge’.

However design can also be used to deliberately erode privacy.  A privacy advocate once observed that in her home country, Soviet-era Czechoslovakia, the walls of houses were built deliberately thin so that neighbours could spy on each other, as a form of social control.

In her article, “Nothing to hide – What happened to privacy?” novelist Linda Jaivin observes that the idea behind the Separate Prison at Port Arthur in Tasmania, one of the first supervised prisons, or panopticons, was that if inmates thought they were being watched at all times, they would reform themselves and become better people.  In reality, at Port Arthur as elsewhere, it more often drove them to madness.

Margaret Attwood, in her review of Dave Eggers’ novel, The Circle, noted: “What happens to us if we must be ‘on’ all the time? Then we’re in the 24-hour glare of the supervised prison. To live entirely in public is a form of solitary confinement.”

Privacy in design of workplaces and learning spaces

Sometimes change hits you like a tonne of bricks. That was the experience I recall when I stepped into a classroom as a parent for the first time some years ago, having not been in one since the 80s.  Gone are the days of front facing rows or desks.  They are collaboration spaces, with tables in groups reflecting more modern teaching principles.  Gone are the days of being scolded for not keeping your eyes on your own work.  Group work is now so much more valued.  ‘Show and tell’, reconceived as ‘Show and share’.  Students are expected to be sociable.

Classrooms open and unfold like pop-up books creating new, flexible learning spaces.  As with open plan workplaces and hot desks, this makes for a much noisier, unsettling environment.  A well-designed classroom will have a quiet corner, equipped with noise-cancelling headphones.

Libraries too are designed with private study areas and small conference rooms, but there is a counter argument to this – that libraries are no longer a source of silence but an escape from it.  As writer Fleur Morrison reflects,

“in this age of voiceless communication via Facebook, Twitter and Snapchat, I have a suspicion that it is not silence most people are missing, but chatter. We have silence when we gaze at our iPhones on public transport, as we order groceries online and as we call customer service and receive an automated response. Libraries no longer need to be an escape from conversation. Now they are sources of that very noise that they used to forbid”.

What does the future hold for privacy in public spaces? 

History repeats itself in odd ways.  When the ‘west’ moved from a shame culture to a guilt culture, we privatised punishment, in the sense that we ceased flogging people in the public square and putting them in the stocks in front of an audience.  Now, it’s posited that Google and Facebook have re-conceived and re-opened the public square.  Our soap boxes and traditional billboards are now online posts and collaborative forums.  Demonstrations of the power of snake oil need only be advertised through targeted, algorithmic-based marketing.

Amnesty International argues that Google and Facebook have built the world’s new digital “public square”, but on their own terms, meaning that surveillance of us in that public square is ubiquitous, and incompatible with our human rights and core values of dignity, autonomy, and privacy.

Clearly increased housing density in areas of population growth, new technology, and environmental changes will be the largest of influences on architectural design.  We may see smaller living quarters and changing dimensions to public space.  There will likely be further commodification of public spaces, necessitating another balancing of vested interests over both public and privatised space.

In “Why the Opera House backlash was so fierce”, Waleed Aly gives a well-summarised account of public reaction to these issues:

“these symbols are vaguely sacred because they are in some respect civic. They are special because when they bring us together, they bind us as citizens rather than consumers. This is what makes them public spaces in the fullest sense. They immediately connect us to those who surround us either physically, or by some social or national mythology.  And so they are places we visit and experience for their own sake, on their own terms, and to be with our society rather than ourselves… Without public spaces – without public forums – eventually there can be no public.”

It’s ironic to think that by coming together in truly ‘public’ spaces, unmediated by the technology giants, we can enjoy more freedom and privacy than we do at home in front of our screens.

Photograph (c) Anna Johnston

If you enjoyed this blog, subscribe to our newsletter to receive more fab privacy news every month – sign up below.

Welcome to the new normal.

This month we offer an overview of the privacy issues to think about as you navigate the new normal, with pointers to the best guidance we have found.

Privacy is not just about data

Is there a better illustration of the importance of freedom of association to our sense of autonomy than the whiplash we’re all feeling now, being told how many people you can have at your wedding, or how many metres apart mourners at a funeral must be?

I’m not arguing against the importance of these physical distancing rules, just pointing out that their sudden imposition highlights a truth about the role of physical privacy, which is that, like other types of privacy, what we value is self-determination: the ability to make our own decisions.

So it may seem counter-intuitive, but privacy is not only about the right to be left alone.  Privacy includes the freedom to be in a crowd.  Or not.  When we feel like it.

When it is about data, the law is your guide

Your organisation might be on the receiving end of calls for increased data sharing.  Keep in mind that privacy law already anticipates data sharing without consent in a number of ways.

Most privacy laws will have a number of exemptions, which allow personal information to be used or disclosed for secondary purposes, without the consent of the subject individual.  Those exemptions will typically include:

• ‘any other law’ which authorises or requires the use or disclosure

For example, we already have public health legislation which deals with reporting of notifiable diseases and biosecurity hazards.

• law enforcement purposes

Law enforcement exemptions are not open slather, so when police come calling, ask questions about what criminal offence they are investigating, and ensure you can meet the threshold test in the privacy law that applies to your organisation.  For example, APP 6.2(e) requires that you (not the police asking for the information) reasonably believe that the use or disclosure of the information is reasonably necessary for one or more enforcement related activities conducted by, or on behalf of, an enforcement body.

• serious and imminent threat

Most privacy laws have some kind of ‘emergency’ exception, but be careful: the differences in wording are subtle, but significant.

For example, the NSW test for the disclosure of non-health, non-sensitive personal information (IPP 11) is narrow: the agency holding the personal information must “(believe) on reasonable grounds that the disclosure is necessary to prevent or lessen a serious and imminent threat to the life or health of the individual concerned or another person”.  Case law suggests this is for extraordinary circumstances such as an active threat of homicide (as assessed by someone such as a patient’s long-term psychiatrist), and even then only if the precise disclosure is necessary to contain that threat.

By contrast the NSW rule for health information (HPP 11) also includes “to lessen or prevent … a serious threat to public health or public safety”.  A test involving “a serious threat to public health” is much easier to satisfy than a test which requires “a serious and imminent threat to the life or health” of an individual.

So taking the two NSW privacy laws together, even in the midst of a pandemic which arguably poses “a serious threat to public health”, only health information could be disclosed under this test.  Non-health information such as location data could not be disclosed, even if it might be useful for contact tracing.  But see again the first point: there may be other public health laws which override the privacy laws in certain circumstances.

(If you need help navigating quickly through all the potential exemptions under which personal information, sensitive information and/or health information can be used or disclosed under the privacy principles, see our Untangle guides: Untangling the APPs, or Untangling Privacy: the NSW rules.)

See specific guidance from your privacy regulator

Check your privacy regulator’s website for their guidance.  Here are some quick links:

• OAIC – guidance for employers around staff privacy issues
• OVIC – tips for working remotely, safely
• NSW IPC – preserving privacy rights
• OPC NZ – FAQs on managing COVID-19 and privacy, including for specific groups such as employers, landlords, the hospitality sector and healthcare workers
• Europe – the EDPB has guidance on compliance with the GDPR and ePrivacy Directive from the European regulators
• Everywhere else – see the IAPP’s global roundup of regulatory guidance

The foundational principles hold firm

In the midst of a crisis, it is tempting to run around like headless chooks (keeping at least 2m away from all the other headless chooks, of course), blindly grasping for a solution.  But there’s no need to make decisions in the dark, because privacy laws already shine a light on what to do.

Privacy laws around the world are built on core principles such as legitimacy, necessity, proportionality, fairness and transparency.

This means that at all times – whether you are building an app, or considering a request to disclose personal information about your customers, or wondering if you can collect health information from your staff – you must ask yourself:

• are our objectives legitimate?
• what is reasonable and necessary to achieve our objectives?
• will our actions actually achieve our objectives?
• is achieving our objectives worth the resulting impact on privacy?
• can we design a more privacy-protective (or less privacy-invasive) way of achieving our objectives?

Two examples of headless-chook thinking spring to mind.

The first is those calling for more information to be published about people with confirmed cases of COVID-19, such as down to the suburb level.  This is usually posed as ‘so that we can protect ourselves’.

But we should not forget the lessons learned in the early days of AIDS, which is that both public health and privacy interests are best served by a policy of universal infection control, instead of stigmatising individual patients.

We have surely now reached the point of universal infection control (i.e. everyone should take the same precautions, and assume anyone else is infectious; and anyone showing symptoms should stop work and isolate themselves even further), that knowing how many people in your suburb are infectious is kind of irrelevant.

And in fact knowing how many people in your suburb are infectious is inviting citizens to make their own risk judgments about whether or not to follow universal infection control, which undermines the public health objectives of physical distancing and universal infection control.  And will lead to discrimination.  And let’s face it, the data will be so quickly out of date as to be useless anyway.

The second example is using mobile phone metadata for contact tracing.  Which leads me to my next point:

Don’t design in a panic

Contact tracing refers to trying to find all the people who were in recent physical proximity to a person subsequently diagnosed with COVID-19.

There are a few different ways to go about contact tracing.  As a government you could broadcast the details of infected persons’ movements, as South Korea did.  Or you could, as a government – as some are calling for – suck up all location data (both past and on-going) from everyone’s mobile phone metadata, stick it all in a centralised, searchable database, and use it to meticulously trace who goes near who else, in case it becomes useful later.  Yikes!  Not even national security agencies have dared ask for such a dystopian degree of surveillance infrastructure.

Or you could aim for some more privacy-protective approaches.

Indeed a national committee of all the Australian privacy regulators has been formed to respond to COVID-related proposals with national implications (such as development of a contact tracing tool), and they have reiterated the value of conducting short-form Privacy Impact Assessments on proposed solutions to public health and economic problems, to make sure privacy is considered in the design process.

One option is an app to download in which users voluntarily hand over their location data to a centralised government agency, which then notifies users if they happened to be in the vicinity of another app user who subsequently tested positive.  This is the model apparently being considered by the UK’s NHS and Ireland’s health service (though by the time I finish writing this paragraph my claim may already be out of date).  This is better than the ‘make the telcos hand over everyone’s location data’ model, but not by much.

Genuinely better options start from a position of Privacy by Design.

One is also a voluntary app, but in the case of Israel’s Hamagen app, the location data never leaves the phone.  The user’s location data is matched, on their own device, against published details of the “tracked locations of Covid19 patients”.  While the details of diagnosed patients are purportedly de-identified before publication, we also know that location data alone is highly individuating, if not necessarily identifying.

One of the best examples I have seen comes from Jaap-Henk Hoepman, whose work on privacy engineering I follow closely.  His recent blog outlines how, like Hansel and Gretel, we could allow our mobile phones to leave their trail of breadcrumbs, but in a privacy-protective way.  Location data never needs to leave the individual’s device; when one device comes into physical proximity with another, both devices record that fact (an identifier for the other device, a location and a timestamp) within their own device; and that record is purged a couple of weeks later.  The records are all encrypted, so that the humans passing by remain blissfully anonymous to each other (or not, as the case may be), and their phones cannot be interrogated to give up meaningful data.  But if, within that two week period, a user is diagnosed with COVID-19, the app could be activated (but only using the public key of the proper health authority), which would then trigger notifications from the diagnosed person’s phone to the other devices for which a record was created.

And finally, Australian cryptographer and re-identification guru Vanessa Teague has proposed two tweaks to the Singaporean TraceTogether app, to make it even more privacy-protective.  Like Hoepman, Teague focuses on encrypted device-to-device links (the digital equivalent of elbow-bumping, if you like), and using distributed processing instead of a centralised government database to manage the post-diagnosis notification step.

So there are some elegant technical solutions possible which do not result in a panopticon nightmare.  Could we really have our public health and our privacy too?  Let’s hope so.

But meanwhile, for those of us not in the business of designing national pandemic response apps…

Don’t drop your guard on data security

There are four key risk areas to be aware of, when thinking about data security right now.

• Data security risks when employees shift to home

OVIC has some succinct tips for your staff when they need to work remotely; and Danish ThinkDo Tank Data Ethics has a round-up of tech solutions to use or avoid.

(And if you need to roll out online privacy awareness training to help your workforce remember the privacy and data security basics even when they are working from home, check out our online privacy training options.)

• Data security risks when switching rapidly to new modes of service delivery

While the US government moved swiftly to change the rules so as to allow medical professionals to conduct telehealth appointments, this meant throwing the data security obligations on the health sector out the window.  Imagine the implications for a psychiatric patient once Facebook knows about their condition because their doctor chose the Messenger platform for their video consult.  Even if you have to make decisions about deploying new tech quickly, take a few minutes to research the data security pro and cons of each option.

• Ensure your messaging doesn’t accidentally encourage risky user practices

It’s a bit embarrassing that the text message sent unexpectedly to all mobile phones on 25 March on behalf of the Australian Government didn’t say much beyond ‘wash your hands’ except to offer a link to a website … thus undermining the standard government cybersecurity advice to people about not clicking on links in messages from unknown numbers, and more disturbingly specific Australian Government advice published the week before about coronavirus-related scams containing malicious links purporting to come from government agencies or telcos.  Left hand, right hand anyone?

• Customer authentication processes when F2F is not available

Just because your customers can’t visit you in person is no reason to throw caution to the wind.  Data security and integrity should remain front of mind, even as you try to adapt to the new normal.

Centrelink has created a new set-up for the large volumes of people suddenly seeking unemployment benefits, who cannot get through on the clogged phone lines, and understandably don’t fancy standing in long queues to access Centrelink offices during a highly contagious health crisis.  The new set-up is that people are supposed to lodge an ‘intent to claim’ online but then wait for a call back … eventually … from an unknown number, which the person is supposed to answer with their identity documents ready.  You can hear Virginia Trioli simultaneously gasp and sigh when the Centrelink spokesperson explains this on her ABC Radio program (about 10 minutes into the segment, if you’re keen).

Telling people to take calls at an unknown time from an unknown number and then hand out their identity details undermines sensible and official government security advice about how to avoid scammers, and also poses risks for victims of family violence or stalkers trying to avoid particular callers.  As Trioli called it on the spot, this design is a “massive scammers problem waiting to happen”.

If the caller is supposed to authenticate themselves to the customer by demonstrating they must be from Centrelink because they already know X and Y details about the customer, how does Centrelink first know the (alleged) customer is legit, and didn’t enter their ex-partner’s name etc online but their own phone number, in order to find out new details about their ex?

Or if the customer is supposed to authenticate themselves to a caller from an unknown number, how can the customer first know the caller is legit, before they hand over their details?  The spokesperson said in this radio interview that if the customer is concerned, they can ask for a number to call back the (alleged) Centrelink staffer on.  Now either this number goes direct to the (alleged) Centrelink staffer, in which case the customer still has no idea if they are talking to a legitimate Centrelink staffer or a scam artist, or it goes via the main switch number, in which case they are stuck back on the clogged phone lines, and … oh dear, they have fallen into the seven circles of Centrelink hell.

When it all ends

I know that right know we all feel like kids in the backseat of the car, whining Are we there yet?  But this too shall pass.

However if we accept widespread surveillance of our movements in the name of public health, how hard will it be to wind that back again once the pandemic is over?

September 11 was used to create a false dichotomy between security and privacy, and that thinking ultimately led us to surveillance capitalism.  It has taken almost 20 years for critiques of surveillance capitalism to pierce public consciousness.  Where will this pandemic take us?

I recommend two insightful pieces of writing which focus on ‘what next’, not only in relation to privacy:

• a global view from historian, philosopher and author Yuval Noah Harari, and
• predictions for Australian politics, the economy and society – a thoughtful piece from Walkley award winning The Age journalist Michael Bachelard.

Stay safe dear readers.  And stay vigilant to protect privacy.

Photograph (c) Shutterstock

If you enjoyed this blog, subscribe to our newsletter to receive more fab privacy news every month – sign up below.

Last Sunday evening, as I was cosily snuggled up on my sofa watching a murder mystery, my phone started beeping like mad.  I had multiple text messages from friends and family, all asking my advice on the same thing: ‘What’s with the covid app? Should I download it? Is it a privacy risk?’

There is no ‘one size fits all’ answer to this question.  So let’s run through the privacy pros, cons, and the ‘yet TBD’ features of COVIDSafe, so that you can decide for yourself.

Privacy positives

• Only people who download and register for the app have any information collected at all.
• The app does not track geolocation data.
• Registration data (the name, age range, postcode and telephone number you supply at the time you download the app and register to use it) is held in the National COVIDSafe Data Store, which although operated by the federal government is (we have been told) not accessible by the federal government, only by State and Territory health officials. (But then again, we have also been told that the federal government will use the system to send out SMS messages; that “data about generation of encrypted user IDs to create de-identified reports about uptake of COVIDSafe will be prepared by the Digital Transformation Agency” which is a federal government agency; and that if you want to delete your data you have to go through the Administrator which appears to be either the federal Department of Health or the DTA or a contractor.  So maybe I should put this one in the ‘cons’ column.)
• You don’t need to provide your real name. (But if you call yourself Mickey Mouse on the app, remember to not hang up if you receive a call for Mickey saying you have maybe been infected… )
• There are multiple points at which you must consent; and you can delete the app any time you like.
• Using a Bluetooth ‘handshake’, the app collects encrypted device IDs from other devices your device was near.  People who were physically near you will not find out your name or phone number from the app.  (But, the app does collect the make and model of other devices you were near and stores this unencrypted on your device; so you or someone with access to your phone could start to figure out who you were near.  And it is data about all devices with the app, not just those you were near for more than 15 minutes.)
• The app only stores the data from other devices for a rolling 21-day period.
• According to the Department of Health, State and territory health officials can only access app information if someone tests positive and agrees to the information in their phone being uploaded. The health officials can only use the app information to help alert those who may need to quarantine or get tested.
• Legal protections specific to this app and the related National COVIDSafe Data Store have been introduced. Importantly, the Determination issued by the Minister for Health under s.477 of the Biosecurity Act recognises and prohibits behaviour which could otherwise create pseudo-compulsory scenarios, such as employers only allowing staff to work if they are using the app, or service providers only allowing you into their shop or onto their train if you have the app.  (The criminal penalties come via s.479 of the Biosecurity Act 2015.)  Also, it is a crime for an unauthorised person to decrypt the ‘handshake’ data exchanged between users’ phones.

Privacy downsides

• My friends and family could not figure out their ‘is this a privacy risk for me?’ answer for themselves. That suggests the comms around the tech are not clear enough.  This in itself is a privacy fail.  Transparency is critical to facilitate informed consent.
• Contrary to earlier government promises, the source code has not been released for independent cybersecurity experts (or armchair amateurs for that matter) to review and test. Promising that the source code will come in a couple of weeks leaves room for concern that there might be vulnerabilities in the system which have not yet been found; for example that devices’ locations could be tracked by third parties (shopping centres, burglars) from the unencrypted data shed via Bluetooth from devices using the app.  Going live without fulfilling that promise is not a good look for a government which has struggled with public trust in its data handling practices and technological competence.
• On Android phones, users must give permission for location to be recorded. Not that the app’s current design actually does track location (we are told), but a later update could change that for unsuspecting users.
• While the Privacy Impact Assessment (PIA) Report was publicly released as promised, it was done so at the same time as the app. No time for journos to source expert review and commentary before the app went live with uncritical exhortations to the public to download the app ‘because it got the privacy tick’.  In fact the PIA on COVIDSafe did not examine compliance, or risks posed, by the State and Territory health departments which will actually be accessing and using the identifiable data, which are covered by a different patchwork of privacy laws.  (And in the case of SA and WA, no privacy laws.)  The scope of the PIA was limited to the federal Department of Health’s compliance with the federal Privacy Act.  The PIA Report’s authors called out this limitation in their report, along with the lack of time available to consult with either State and Territory privacy regulators, civil society representatives or other experts.  The PIA Report is not quite the all-encompassing ‘privacy tick’ the government would like us to believe.
• The legal protections in the Determination are temporary, and currently at the whim of a Minister, so they could be scrapped tomorrow without Parliamentary oversight. Parliament is expected to create a more permanent legal framework when it sits in May.
• Contrary to promises by government ministers, the legal framework does not yet prohibit law enforcement access to app metadata, which is why the Attorney General vowed to amend the telecommunications laws. Promising to introduce a more robust legal framework to deliver on those political commitments sometime afterthe app has gone live is reckless and a breach of faith with the Australian public.
• There is also an argument that the legal framework does not prohibit access to the data in the National COVIDSafe Data Store by agencies armed with a warrant, court order or their own ‘notice to produce’ powers.(This is because while s.477(5) of the Biosecurity Act says that the Minister’s Determination applies despite any other law, it also says at s.477(1) that the scope of the Determination is about what is necessary to prevent or control the spread of the disease; so to the extent that police conducting a murder investigation want access to data and their access to that data would not prevent the app or contact tracing from operating as per the Minister’s Determination, does the Determination really stop them?)  This issue is not only in relation to police agencies, but national security agencies, anti-corruption bodies, Centrelink and the ATO, all of which have their own powers to compel other organisations to hand over data to them.
• Also, there is an open question as to whether the legal framework does (or even constitutionally can) regulate what happens ‘downstream’, once data has been copied by State and Territory health officials from the National COVIDSafe Data Store into their local systems. Minister Hunt’s Determination implies that it does cover State and Territory officials (because at cl.7(4) it exempts them from the requirement to keep all data in Australia); but the PIA Report states that the federal Department of Health loses “effective control” once the data passes to the States and Territories, and the Department of Health’s acceptance of the PIA Report’s Recommendation 12 implies that at best, the Commonwealth can only seek to have the States and Territories ‘agree’ to a data use protocol.  Once held by State/Territory governments, we must rely on our existing (incomplete) patchwork of State and Territory privacy laws to regulate how officials in State and Territory Health Departments can or can’t use the data, how long they store it, how securely they store it, how they authenticate authorised users, what prohibitions and penalties are available to deter misuse, and preventing police (etc) to access the data via their local health department.  Binding State and Territory departments to an agreement with the Commonwealth Department of Health not to use the data for any purpose beyond contact tracing does not remotely cover the privacy risks to individuals posed by data breaches, deliberate misuse or police access via State and Territory Health Departments.  Instead we need a legal framework which includes all downstream users and uses, as law firm Gilbert+Tobin has suggested.
• The legal framework does not make provision for independent audit, assurance or oversight of the operations of the app, the National COVIDSafe Data Store, or the downstream use of the data by State and Territory health agency users. There is no single complaints mechanism or opportunity for redress for victims of a privacy breach.  (Criminal penalties don’t help victims.)  So we have to rely on our patchwork of federal, State and Territory privacy laws, which often don’t allow redress for the victim of a privacy breach if a government agency can describe the breach as the action of a rogue employee.
• The legal framework does allow the federal government to use app data for “producing statistical information that is de-identified”. There have been some spectacular de-identification fails by governments here in Australia, so this makes me nervous.  And of course, someone in the federal government needs to access the identifiable data in order to first de-identify it.  Who will that be?  Who checks they are doing the right thing?
• Deleting the app from your phone will not trigger deletion of your data in the National COVIDSafe Data Store. For that, you have to ask the “COVIDSafe Administrator” (which is who? the government website does not clarify) and the form includes a broadly-drafted ‘consent’ (which is not a valid consent because it is not optional), and which talks about using the data to respond to the disease, not, as you would hope, ‘to complete my deletion request’: the tick box says “I … consent to the information provided being used and disclosed by the Australian Government to enable the Commonwealth, state and territory governments to respond to COVID-19”.
• The period for which identifiable information (the name, phone number etc you supplied at registration, plus if you were infected, and presumably also if you were contacted about possible infection) will be held in the National COVIDSafe Data Store, and accessible by State and Territory health officials, until the Commonwealth decides to delete it “after the COVID-19 pandemic has concluded”. Which might be… whenever.
• We don’t yet know if the app will work. (And there is no clear metric for what success looks like.)  Operational issues are also privacy issues, because if you are trying to weigh up privacy risk versus public health benefit, you need to be able to quantify whether those health benefits are going to be realised.  Will enough people download the app?  Will it work properly on iPhones?  Is the ‘15 minutes at 1.5m’ an accurate proxy for virus-catching risk?  (If an infectious person sneezes in my face as they walk past me, I am at risk, but the app won’t know.)  Will there be so many false positives that the manual contact tracers give up using the app as a contact tracing tool? (If I was on the other side of a sealed glass wall from the infectious person, was I really at risk of catching the virus from them?)

Could try harder

• There were other design decisions which could have been taken, to make the app way closer to achieving privacy and security via anonymity for users. Instead of creating a data store of people’s names and phone numbers, data could have been processed and push notifications issued almost entirely on and between people’s phones, with only randomised strings of gibberish stored in a public register.  (Check out this comic for a quick and easy explanation of the privacy-preserving alternative model known as the DP-3T protocol being developed in other countries.)  If the Australian Government had taken a proper Privacy by Design approach, as promoted by over 300 academics across 25 countries, we could have had an app with almost none of the privacy concerns, which wouldn’t have then triggered the need for urgent bespoke legal protections, because no identifiable information would ever have been stored by anyone.  The failure to implement a DP-3T decentralised anonymised trace-and-notify model (without at least first considering it and then justifying on public health grounds why it was rejected as a model) is a significant privacy fail.  (The PIA Report suggests the Department had chosen the current model, and it was not within scope for the PIA to consider other, less privacy-intrusive models.)  It might also be a public health fail, because a more privacy-preserving model might have engendered greater public trust which might have led to higher download rates.
• Some people face higher privacy and safety risks in their everyday lives than many other people. Systems and products should be designed to protect those who are most vulnerable to privacy harm.  These include victims of family violence, celebrity stalking and other physical threats; serving members of the judiciary, law enforcement and defence forces; political activists, journalists and whistle-blowers; and people who could be blackmailed or sacked if they were known to be frequenting a brothel or having an affair or talking to a competitor.  If a system is not designed to protect the most vulnerable, their particular risks should at least be highlighted in a transparent way by the government, so that those individuals can make their own informed decision.  Neither happened in this case.
• Politicians responding to genuine concerns from the public about privacy and security should not resort to jingoistic ‘Team Australia’ pseudo-patriotic rubbish, or ‘download this app or we can’t lift the restrictions’ bargaining or ‘maybe I’ll make it mandatory after all’ threats in response. Be truthful about the limitations, and seek to understand why some people have entirely legitimate fears about their privacy and safety.  Be humble in acknowledging that concerns and criticisms come from a place of deep distrust and disquiet which has been entirely caused by a series of own goals by the Australian Government, given its appalling recent record on privacy across many fronts including Robodebt, doxing a Centrelink recipient for criticising the government, CensusFail, anti-encryption laws, pursuit of whistle-blowers and journalists, failures to comply with its own metadata laws, attempts to silence researchers who identify re-identification risks in government-published datasets, and more.
• Journalists and commentators responding to genuine concerns from the public about privacy and security should not rely on lazy ‘Facebook/Google already knows everything about me anyway’ non-analysis in response. Wake up and be critical in your thinking!  (Um, maybe you shouldn’t let Facebook/Google know everything about you?)  Government can tax you, fine you, cancel your driver’s licence, outlaw your profession, restrict your movements, seize your goods and throw you in gaol.  Facebook cannot.  The citizen/government relationship significantly affects the privacy risk profile of our interactions with government compared with private companies.
• Know that middle aged white men in white collar professions (yes I am looking at many of you, above-mentioned politicians, journalists and commentators) are the least likely category of people to face discrimination, violence, harassment or economic uncertainty, and thus have the least to lose from any privacy violations. So please understand that not everyone shares your risk profile.  Be more inclusive in your thinking, and calibrated in your calculations of risk.
• The same goes for those who declare ‘I don’t trust the government therefore I won’t use the app’. Instead of holding a default ‘don’t trust’ position, I would argue for a more nuanced balancing of the pros and cons, by each individual, reflecting the privacy and safety risks they face, as well as their willingness to contribute in a small way to (maybe) helping to slow the spread of a terrible disease.
• The legal protections need clarification. Privacy advocate and legal academic Graham Greenleaf described the Determination as “flawed (despite good points): ‘proximity’ is undefined; uploading is at risk if someone else has possession/control of your phone; the deletion date is ill-defined; and the anti-coercion (clause) needs tightening”.

Conclusion

The app is not ‘one size fits all’, but maybe it is ‘one size fits enough’.  From the point of view of a privacy advocate, it could certainly be better, but I also give the government credit for understanding that implementing privacy protections would be essential.  And as they say, perfection is the enemy of good.  And maybe, for this project, at this particular time in history, the privacy protection is good enough, for enough people.  But – this experience also illustrates the importance of considering privacy earlier in the design cycle, and with an open mind about alternative designs.

So, what to do about the app?  To download or not to download, that is the question.

No, the app is not as privacy-invasive as Facebook.  (But if that is the standard by which I measured privacy risks in projects, I would have given up years ago.)  Yes, it could have been designed better.

But, ever the optimist, my advice is this: If you don’t face a particular threat to your privacy or safety in your everyday life (such that if your name, postcode and phone number, and possibly inferences about who else you have been near and where you have been, were accessed by your violent partner or police or your boss or by a third party in a man-in-the-middle attack or leaked in an almost-inevitable data breach you would not have a particular reason to be worried), and if you need to commute on public transport or serve customers or otherwise be close to strangers or large groups of people for decent chunks of time, then the health benefits you can offer to the people in your physical proximity by downloading the app likely outweigh any privacy risk to you.  So if you feel comfortable with the app, go ahead and do something great for your fellow humans.

If you’re not yet comfortable (but don’t face particular privacy or safety risks in your everyday life), wait until there is a proper legal framework in place, and the source code has been pulled apart by independent experts and found to be secure.

Just remember that you will only receive health benefits for yourself if those around you also download and can use the app and are in fact using the app when they are near you.  Those of us with all the best intentions, and no specific reason to be concerned for our own privacy or safety, but who have an iPhone (or who have a flat battery, or who left their phone at home, or who don’t own a phone at all) … well, given all those ‘ifs’, whether the app will penetrate the populace deeply enough to enable the benefits of the app to be realised is an open question.

And be vigilant in your scrutiny and demands for accountability.  The federal government cannot be allowed to backtrack on any of its privacy promises about the app, and yet more can be done to the legal framework to improve the privacy protections further, without impacting on the public health benefits.

Perhaps most importantly, from a privacy advocate’s point of view, as with multiple governments’ many fiscal, legislative and policy responses to the pandemic (everything from stimulus payments to changes to liquor licensing rules to procuring stockpiles of hand sanitiser for schools), we have seen how quickly good things can be done by governments, when they care.  Don’t ever forget that lesson.  When we demand legislative protections, policy solutions and better technology design to protect our privacy, governments are actually capable of delivering, fast.  Don’t ever take ‘too hard’ for an answer on privacy protections again.

Photograph (c) Shutterstock

If you enjoyed this blog, subscribe to our newsletter to receive more privacy insights and news every month – sign up below.

The concept of consent may not yet be broken, but it is certainly under strain.

Nowhere was this more evident than at the IAPP ANZ Summit in late 2019, where speaker after speaker hammered another nail in the coffin of the ‘notice and consent’ model of privacy regulation.

Keynote speaker Professor Woody Hartzog spoke about how notice and consent, choice and control are attractive options, both for legislators needing to ‘do something’ about privacy and for companies tackling product design decisions, because at face value they seem empowering for consumers.  But, he said, such an approach asks too much from a concept which works best in limited doses.  In reality, people don’t have enough time, energy or understanding to navigate every interaction they have with technology: “Our consent cannot scale; it loses its meaning”.  Illustrating his point with visuals of endless privacy control buttons and convoluted click-throughs, he concluded: “If we get our wish for more control, we get so much privacy we end up choking on it”.

Next up was Privacy Commissioner of New Zealand, John Edwards, who in a powerful call to arms for both governments and regulators to address the power asymmetry of Big Tech, warned that “the days of click to consent are numbered because it is not meaningful consent”.

And then Victorian Information Commissioner Sven Bluemmel asked whether consent in an online and hyper-connected world can ever be fully informed, or whether anyone can ever voluntarily consent when dealing with government.  He posed the question: “Is consent still fit for purpose, as a tenet of privacy regulation?”

Of course you might think that for we antipodeans (and indeed for most of the rest of the non-American world), whose privacy laws have never relied wholly on a notice and consent model, criticising the business practices of Silicon Valley is a well-trodden path, leading nowhere.  I’ve seen the ‘my model of privacy regulation is better than yours’ argument at countless global privacy conferences.

Except that this time it feels different.  This time, the Cambridge Analytica scandal has thrown a star-spangled spanner in the works.  You know the fix is in for notice and consent when even conservative American think-tank the Brookings Institute is arguing that this particularly American model of privacy protection should be killed off.

Is news of the Notice and Consent model’s demise premature?

But like the Monty Python character who protests “I’m not dead yet”, the regulatory model of notice and consent just won’t die.

Indeed the California Consumer Privacy Act, which commenced in January 2020, directly ties collection limitation and use limitation back to transparency: section 1798.100(b) says “A business shall not collect additional categories of personal information or use personal information collected for additional purposes without providing the consumer with notice consistent with this section”.

It’s not only California; other American legislators are similarly still focussed on the myth that transparency, and consumer controls like opting-out of the sale of personal information, can deliver privacy protection, instead of setting meaningful limits on when personal information can be collected or used in the first place.

Even the Australian consumer protection regulator, the ACCC, has proposed strengthening notice and consent provisions in the Privacy Act as a solution to the twin problems of information asymmetry and unequal bargaining power between consumers and the Big Tech digital platforms.  But will more transparency really help?

Privacy academic Daniel Solove describes the idea of ‘privacy self-management’ – notice, consent, user controls, “more buttons, switches, tick boxes and toggles” as just more homework.

What the Notice and Consent model means

OK, I hear you saying, wait up a bit.  What is this notice and consent model anyway, why do we suffer through it, and what else is on offer?

I’m going to let privacy academic Dr Gabriela Zanfir-Fortuna set the scene for you:

“A ‘notice and consent’ framework puts all the burden of protecting privacy and obtaining fair use of personal data on the person concerned, who is asked to ‘agree’ to an endless text of ‘terms and conditions’ written in exemplary legalese, without actually having any sort of choice other than ‘all or nothing’ (agree to all personal data collection and use or don’t obtain access to this service or webpage).”

The alternative model of privacy regulation is to start from a point of restricting data flows (i.e. the collection, storage, use and disclosure of personal information) unless they can be justified within a framework of overarching principles like necessity, proportionality and fairness; create some public interest exemptions like law enforcement and medical research; create a ‘with consent’ exemption; build in some actionable rights (access and correction, and sometimes more); and then layer transparency over the top (ideas like privacy policies and collection notices).

Much of the developed world has omnibus privacy laws which cover wide swathes of the economy, including public sector agencies and much of the private sector.  They incorporate most if not all of the above features in those privacy laws.

But in the US, instead they have what is known as a sectoral law approach.  They have one piece of legislation that just talks about the privacy of financial records in banks, and another one just for federal government agencies. They have a separate law about the privacy of health insurance records; another law that talks about students’ privacy; and yet another law about the privacy of video rental records.  And there is an Act which protects the privacy of children online.  So the US has a few privacy laws, each designed for a different sector.

But what they don’t have is one set of rules that applies to all sorts of different businesses.  So as a result Big Tech – the data-gobbling tech companies like Facebook, Amazon, Alphabet (Google), Apple, Microsoft, Netflix, Uber and Air BnB – are for the most part not regulated by privacy legislation.  (OK yes there is the new Californian law CCPA, which applies to all sorts of industries including tech companies, but for the most part that should have been called the “Don’t Sell My Data” Act, because that’s about all it covers; it doesn’t come close to being a principles-based privacy law like most other countries have.)

Because of this gap in privacy regulation, the default form of privacy protection for most industries in the US, including those industries which matter most in the online world, is consumer protection and trade practices law.  This where the ‘notice and consent’ model comes from.  When you come at the issue of authorising data flows purely from a trade practices angle (instead of a human rights angle), the chief requirement is to ensure that contracts are not misleading or deceptive.

In other words: under the ‘notice and consent’ model, you just have to tell people up front what you are going to do with their personal information, and then you can go ahead and do it.  So long as you bury somewhere in some fine print some kind of explanation of what your company is going to do with people’s data, then if people choose to buy your product or use your service anyway, well then, they must have ‘consented’ to whatever it was you said you were going to do with their data.

So what’s wrong with that?

The first problem with the ‘notice and consent’ model is that companies can bury whatever they like in those terms and conditions – because, let’s face it, almost nobody ever reads them.  Rather like the Londoners who ‘consented’ to give up their first born child when signing up for free wifi, most of us don’t read T&Cs, because they are longer than Shakespearean plays.  And deliberately so: privacy notices under the US model are not about delivering transparency; they are legal tools for arse-covering.

Notice and consent just doesn’t scale.  This art installation illustrates the problem, as does this video, in which the advocacy group Norwegian Consumer Council asks their Consumer Affairs Minister to go for a jog while they read her the privacy policy from her fitness tracker.  The Minister manages to run 11km in the time it takes for the policy to be read out to her.  When the same group tallied up the T&Cs for the apps found on an ‘average’ mobile phone, reading them took 37 hours.  And that’s before you get to the quagmire posed by the Internet of Things: where on your smart toothbrush are you going to adjust your privacy settings?

Second, if a customer does read the fine print, they probably don’t understand that phrases like ‘share data with our partners to personalise your experience’ means the kind of privacy-invasive profiling practices on which data brokers and AdTech thrive.  Speaking at a webinar to mark Privacy Awareness Week in May 2020, Australian Privacy Commissioner Angelene Falk noted that the OAIC’s 2020 Community Attitudes to Privacy Survey found that only 20% of people felt confident that they understood privacy policies.

And sometimes that is deliberately so.  The Notice of Filing in the OAIC’s lawsuit against Facebook for disclosing the personal information of 311,127 Australian Facebook users in the Cambridge Analytica scandal states: “The opacity of Facebook’s settings and policies hampered (Australian Facebook users) in understanding that their data was disclosed to the app.  The design of the Facebook website was such that Users were unable to exercise consent or control over how their personal information was disclosed”.

Third, consumers don’t have enough power, knowledge or time to genuinely exercise what little choice or control they might be offered.  There is a power imbalance between consumers and corporations; and between citizens and governments.  The OAIC’s submission to the ACCC’s Digital Platforms Inquiry calls this out clearly:

“consumers may be informed and understand the inherent privacy risks of providing their personal information, but may feel resigned to consenting to the use of their information in order to access online services, as they do not consider there is any alternative. Further, while ‘consent’ is only a meaningful and effective privacy self-management tool where the individual actually has a choice and can exercise control over their personal information, studies also show that consumers rarely understand and negotiate terms of use in an online environment”.

Describing consent as a “legal fiction”, the editorial board of the New York Times nailed the pointlessness of even reading privacy policies: “Why would anyone read the terms of service when they don’t feel as though they have a choice in the first place? It’s not as though a user can call up Mark Zuckerberg and negotiate his or her own privacy policy. The ‘I agree’ button should have long ago been renamed ‘Meh, whatever’.”

As Digital Rights campaigner Sam Floreani remarked at NetThing last year, there is an element of elitism and privilege behind the very notion of notice and consent: suggesting to consumers that if they don’t like what’s happening with their privacy, they should just opt out of using Google / Facebook / Uber / etc ignores the reality that much of our civil and political life depends on or is mediated through a small number of dominant technology platforms and service providers.

Fourth, it is a fantasy to think that consumers can calculate the privacy risks arising from every single transaction they enter into, let alone whether the benefits to be obtained now will outweigh the risks to be faced later.  Rachel Dixon, the Privacy and Data Protection Deputy Commissioner in Victoria, has said about the role of consent that because most data is collected during transactions where we as consumers or citizens want something, the ‘consent’ obtained is almost never fair: “There is always an inherent lack of attention paid to the downstream consequences”.

Privacy risks are usually time-shifted, and obscure.  And in the context of artificial intelligence in particular, ‘consent’ can almost never be informed.  Rachel Dixon again: “No matter how much you think you can explain how the AI works to a regular person … people don’t understand what they’re giving up”.

And if you don’t understand the risks, your consent will not meet the test for ‘informed’, let alone any of the other elements needed to gain a valid consent under privacy law. (To be valid under privacy law, consent must be voluntary, informed and specific, current and given by a person with capacity.)

Why can’t we be informed about the risks?

So why can’t companies and governments do a better job of explaining the risks to us?  Well, because sometimes they don’t even know.

Lawyer Andrew Burt has written about how the nature of privacy risks has shifted.  Where once organisations and individuals alike worried about personal information being misused or disclosed without authority, now, in this world of Big Data and machine learning, he suggests the biggest threat comes from the unintended inferences drawn from our personal information: “Once described by Supreme Court Justice Louis Brandeis as ‘the right to be let alone’, privacy is now best described as the ability to control data we cannot stop generating, giving rise to inferences we can’t predict.”

Daniel Solove says it is “nearly impossible for people to understand the full implications of providing certain pieces of data to certain entities. … Even privacy experts will not be able to predict everything that could be revealed… because data analytics often reveal insights from data that are surprising to everyone”.  The benefits of ‘consenting’ are usually obvious and immediate, while the possible privacy risks are unpredictable, obscure and time-delayed.

By way of example, the public release of Strava fitness data, although de-identified, gave rise to privacy and security risks that the company themselves had failed to predict.  Strava is a social network of people who use wearable devices to track their movements, heart-rate, calories burned etc, and then share and compare that data with fellow fitness fanatics.  After releasing a data visualisation ‘heat map’ of one billion ‘activities’ by people using its app, an Australian university student pointed out on Twitter that the heat maps could be used to locate sensitive military sites.

So if service providers cannot imagine the risks posed by the data they hold, how is a consumer expected to figure it out?

When data is combined from different sources, or taken out of context, or when information is inferred about individuals from their digital exhaust, the privacy issues move well beyond whether or not this particular app, or device, or type of data, poses a risk to the individual.  We have to assess the cumulative impact.  The herculean task of assessing the likely risks posed to an individual’s privacy means that notice about likely risks is impossible to deliver, and therefore informed consent is impossible to obtain.

It’s just not fair

Even if you could magically solve the problems of digital literacy, power imbalances, and the difficulties of calculating privacy risks, and deliver your consent solution at scale, the notice and consent model still suffers a terrible weakness: it’s just not fair.

At the IAPP ANZ Summit in 2019, Professor Hartzog described how notice and consent, as well as the related idea of solving privacy problems by offering more user controls, are both ways of shifting risks onto individual consumers and citizens.  He has also written about the “fallacy” that it is up to us as individuals “to police Facebook and the rest of the industry”.

Even in the context of discussing privacy laws such as ours which do not rely entirely on consent – i.e. the consent of the subject is what you need when you can’t rely on any other ground to lawfully collect, use or disclose personal information – Australasian privacy regulators are calling time on the over-reliance of consent as a mechanism, on the grounds of fairness.

NZ Privacy Commissioner John Edwards has said of consent that it asks too much of a consumer, and described it at the same 2019 conference as an “abdication of responsibility”.  He has published guidance telling companies to lift their game when it comes to designing consent mechanisms, saying the practice of ‘click to consent’ is simply not good enough anymore.

Likewise, the Australian Privacy Commissioner, in the OAIC’s submission to the ACCC in response to its Consumer Loyalty review, has said that “Overreliance on consent shifts the burden to individuals to critically analyse and decide whether they should disclose their personal information in return for a service or benefit.”  In a similar vein, the OAIC has also said that that burden “should not fall only on individuals, but must be supported by appropriate accountability obligations for entities, as well as other regulatory checks and balances”.

Consent should be the last resort, not the first or only choice from a menu of regulatory or design responses to privacy problems.  The responsibility for protecting our privacy should fall on privacy regulators, government legislators, and organisations themselves – not on us as individual consumers or citizens.

So what are the alternatives?

Let’s turn now to some positive steps being taken to improve matters: enforcing the law we have, making transparency meaningful, and regulating for fairness.

Enforcing the law we have

This is easier said than done.  Privacy regulators around the globe are under-resourced, compared with the budgets of Big Tech.

The European law GDPR has already been in place for two years.  It explicitly requires that personal information can only be processed (collected, used, disclosed) on one of six legal grounds, one of which is ‘with consent’.  It also says that to be valid, consent must be voluntary, informed and specific: a proactive ‘yes’, with the freedom to say no (or say nothing), not bundled up with any other choices, or built into T&Cs.  Yet complaints lodged by privacy advocacy body NOYB against Facebook’s ‘forced consent’ model on the first day of GDPR’s operations are still yet to be ruled upon by the Irish Data Protection Commissioner.  Fingers crossed.

Making transparency meaningful

I am always on the lookout for new ways to do privacy comms better.  So I got all excited to see what the Brookings Institute were suggesting, when they proposed making transparency “targeted and actionable”.  And then I deflated again when I realised that in effect, what the American think-tank came up with is what Australian privacy law already requires: a comprehensive privacy policy available to the public at large, and specific notices provided to the consumer at the precise point of collection.  Ho hum.

Sure, there are some fun ways to deliver your privacy policy, like using graphics or animated videos.  But even with pictures of cute fish, I can’t be bothered reading to the end.  Because actually, as a consumer, if I am looking at this stuff at all, I just want to know (quickly, at the point in time that suits me, in language I understand, in a format that works for me) what I would not already expect, what my choices are, and what I don’t have choice about.  In other words: Just tell me now if it is safe for me to proceed.  Or: Just tell me if this app is safer than this other one.  But because how one person might be harmed is different to the next person, and what I value is different to what the next customer values, and because children are different to adults, that information should be contextual for me.

European think-tank DataEthics suggests taking a layered approach, and using icons to categorise types of information.  They were part of a consultation group reviewing IKEA’s new app, which promises to put customers in control of their data.  Watching a video on how the app works for a fictional customer, it is encouraging to see how explanations, toggle controls, prompts and links to more information are integrated within the user experience.  IKEA’s customer data promise and the app’s design were seen as innovative enough to be the subject of a presentation at the World Economic Forum in Davos earlier this year.

The IKEA app is a great example of building in privacy thinking as part of a customer-centric experience, instead of a legal compliance bolt-on after the fact.  But it is not perfect.  I noticed that the app still has a button which makes the customer “accept the Privacy Policy” (argh, WHY?), and DataEthics has also pointed out that IKEA uses third party tracking cookies, and stores data in the US.

But taking the IKEA app design as a step forward, what next?  First, we need universal terms and icons, as well as ways to present them better, to quickly communicate privacy messages and choices to individuals.  (And I mean all individuals, not just highly educated, literate adults with time to spare.)

To help consumers better understand data practices and exercise choice, the OAIC has expressed the need for “economy-wide enforceable rules, standards, binding guidance or a code”, to create “a common language in relation to privacy and personal information, which could include the use of standardised icons or phrases”.  The new Consumer Data Right scheme provides a positive example of what can be achieved.  The CSIRO’s Data61 has legislated power to set standards under the CDR scheme; their guidance on translating the legal elements of consent into design specifications is extremely valuable.

Other ideas are modelled on successful designs used in safety messaging (traffic light indicators), and product labelling (star ratings, nutrition labels).

Privacy advocate Alexander Hanff has developed a proof of concept matrix of data types, internal uses and disclosure types, each one coloured red, amber or green, allowing users to click through for more information.

Researchers in privacy and trust from Carnegie Mellon University and Microsoft have presented an idea for using nutrition labelling as a model for communicating privacy messages.  Having tested and rejected matrix designs as too complex, their simplified label broke messaging into What, How and Who: what categories of personal information are being collected; the different purposes for which the data will be used, including whether the user can restrict those purposes by opting in or out; and to whom it will be disclosed, including any choice over such disclosures.  They struck difficulties: even University students didn’t understand the symbols used to illustrate opting in versus opting out.

The idea of nutrition labels received a more recent boost from Ghostery President Jeremy Tillman, who argued that the US government should develop uniform labelling:  “What consumers need is a privacy nutrition label – something quick and scannable they can look at to see what the privacy impact of a digital service is before they use it, the same way they would look at nutrition info before eating a candy bar.”

But do you see the problem here?  In this scenario, the consumer checks the nutrition label but then still eats the candy bar.  If consumers made only rational decisions, they would put down the candy bar and bite into an apple instead.  But we all know, that’s not how humans necessarily behave.  Sometimes we crave instant gratification, whether that is a sugar rush or to download a game of Candy Crush.

And nutrition labels still depend on we humans to stop and read them, over and over and over again, and use them to compare one product to another.  Ugh.

The most innovative idea I have seen in this space comes from Data61, who propose a machine-readable solution. Senior experimental scientists Alexander Krumpholz and Raj Gaire wrote: “Wouldn’t it be nice if we could specify our general privacy preferences in our devices, have them check privacy policies when we sign up for apps, and warn us if the agreements overstep?”

Basing their idea on Creative Commons icons which are universally agreed, legally binding, clear and machine-readable, their proposal would also need legally binding standards and universal icons, but it would work more efficiently than star ratings, traffic light matrices or nutrition labels alone.  They propose ‘Privacy Commons’ classifications to cover what they call Collection, Protection and Spread: the categories of personal information collected, the data security techniques applied, and who the personal information will be disclosed to.  In my view they need to add also include the purposes for which the personal information will be used within the organisation, but the idea is a great start.

It would be a brilliant time-saver, by getting consumers to think deeply once about what they are trying to achieve in terms of their personal privacy goals, and then automating the legwork of reading and comparing privacy policies against those goals.  I would love to see a legal and technology framework which allowed an individual to set their own privacy risk profile (e.g. add me to a mailing list is OK, never share my home address, don’t collect my date of birth, don’t collect location data without checking with me, etc), and then facilitated an automated ‘reading’ of a company’s data practices against that profile (and even better, automating the toggling on or off of settings to matchthe profile), to come up with tailored gatekeeping advice about whether it is safe for them to proceed.

Now that might actually deliver on the promise of transparency.

Fair use limitations

And finally, we need to better define what Solove describes as “the architecture of the personal data economy” – the rules under which personal information can be collected, stored, used and disclosed.

Because even with innovative approaches like machine-readable privacy policies, it will still be hard to code for whether any given collection or use of data is necessary, proportionate, reasonable or fair.  As privacy and technology lawyer Peter Leonard argues, consumers shouldn’t even be put in the position of having to figure out for themselves whether a company’s data practices are reasonable: “Regulators don’t require consumers to take responsibility for determining whether a consumer product is fit for purpose and safe… Why should data-driven services be any different?”

Thus we need a more wholistic and protective approach to privacy regulation, in which an organisation can only collect, use or disclose personal information when it is fair to do so.  There are some practices so privacy invasive or socially damaging that even ‘consent’ should not be allowed to authorise them.  The late Giovanni Buttarelli, European Data Protection Supervisor, argued that “The right to human dignity demands limits to the degree to which an individual can be scanned, monitored and monetised — irrespective of any claims to putative ‘consent’.”

Because we care about human dignity and autonomy, in Australia we do not allow trade in human organs or tissue.  ‘Consent’ doesn’t even come into it.  It’s time we outlawed some types of data exploitation too.

Canadian privacy law includes a gatekeeper provision.  Section 5(3) of PIPEDA says: “An organization may collect, use or disclose personal information only for purposes that a reasonable person would consider are appropriate in the circumstances.”  As a result the Canadian Privacy Commissioner publishes guidance on ‘no-go zones’, based on court interpretations of s.5(3) as well as consultations with stakeholders and focus groups.

Of course what is ‘reasonable’, ‘appropriate’ or ‘fair’ are subjective assessments, but the Canadian model at least creates a space for reflecting community expectations in the application of legal tests.

The OAIC has proposed introducing a similar “general fairness requirement for the use and disclosure of personal information” as a way of addressing “the overarching issue of power imbalances between entities and consumers” and “protecting the privacy of vulnerable Australians including children”.  The Australian Government has since committed to reviewing the Privacy Act “to ensure it empowers consumers, protects their data and best serves the Australian economy”.

Reforming the Australian Privacy Act to create no-go zones, in which even ‘consent’ would not be sufficient to authorise data practices which would otherwise be unfair, have discriminatory impacts or diminish human dignity, would be a fantastic result.

No more privacy theatre

The US-influenced model of ‘notice and consent’ has failed.  User controls, notice and consent are too often just privacy theatre: smoke and mirrors offering the illusion of control and choice, but within confines that are invisible.

Successful privacy protection should not depend on the actions of the individual citizen or consumer.  Placing the burden of privacy protection onto the individual is unfair and absurd.  It is the organisations which hold our data – governments and corporations – which must bear responsibility for doing us no harm.

Designing, implementing and enforcing privacy protection is the task of legislators, the organisations which want to use our data, and privacy regulators.  Not consumers, and not citizens.  Under a truly effective model of privacy regulation, the hard choices about limiting the use of personal information, protecting autonomy and dignity, and avoiding privacy harms, must be made well before the individual user, consumer or citizen ever becomes involved.

Photograph (c) AdobeStock

If you enjoyed this blog, subscribe to our newsletter to receive more privacy insights and news every month – sign up below.

We don’t live in a bubble, so why do privacy laws act like we do?

It may seem counter-intuitive, but privacy is very much a public good.

If we think of managing privacy as a private exercise, governed by our own actions and choices, then the regulatory model of notice and consent, choice and control makes sense.  But that’s not actually the world we live in.  As individuals, we have no more control over the data ecosystem we find ourselves in, than we have over the quality of the air we breathe or the water we drink.

Further, privacy is a shared experience, where the actions of one can negatively affect the whole.  That is why privacy protection cannot be left to the choices made or controls exercised by individual consumers or individual citizens.  It must be treated and managed and regulated as a public good, because privacy harms are increasingly collectiveharms.

This notion of privacy as a public good, or a public commons, has been raised by a number of privacy regulators and privacy thinkers recently.  Many offer analogies with our physical environment, and parallels are drawn with other global challenges such as tackling climate change.

At the IAPP ANZ Summit in late 2019, Australian Privacy Commissioner Angelene Falk compared managing privacy protection to dealing with oil spills, while Victorian Information Commissioner Sven Bluemmel ran with a comparison to the harms of passive smoking.

Privacy and open government advocate Martin Tisné describes the nature of the collective harms which arise from privacy intrusions at scale as “invisible to the average person”, which is why forcing regulatory action and co-operation from governments is so difficult – much like action on CO² pollution.

Shoshana Zuboff, author of The Age of Surveillance Capitalism, calls the belief that privacy is private “the most treacherous hallucination of them all”.  In reality, writes Tisné, we are “prisoners of other people’s consent”.

Because none of us live in a bubble affected only by our own choices.  The consequences are shared.  One person’s privacy can be negatively affected by a different person making choices about their own personal information.

The most obvious example of privacy as a shared concern is genetic privacy.  If one family member shares their DNA with a site like 23andMe, it impacts on all their genetic relatives, no matter whether those relatives agreed, vehemently objected, or were entirely ignorant of the situation.

A second example is the impact of one person’s decisions on the people around them.  Military personnel stationed in remote or secret locations should not have their privacy and safety compromised by the choice made by a colleague to use a fitness app when they jog around the base.  (And this is not to criticise the joggers themselves.  Even Strava users who carefully calibrated their privacy settings to not share their Strava data with others were nonetheless exposed by Strava’s release of ‘aggregated’ data.)

Similarly, even if you have never used Facebook yourself, you will be subject to online behavioural advertising based on information collected about you, and inferred about you, from Facebook users who do know you, because Facebook generates ‘shadow profiles’ on non-users.  Academic Zeynep Tufekci has noted that the power of Big Data means that companies such as Facebook can now infer, even about privacy-protective individuals who have deliberately tried to protect their privacy online, “a wide range of things about you that you may have never disclosed, including your moods, your political beliefs, your sexual orientation and your health”.  As individuals we have no choice about this.  There is no way to opt out of being swept up in this vast dragnet of data.

A third example of one person’s privacy choices affecting other people’s lives is for classes of people with shared characteristics.  Let’s say an algorithm has been built on data, that was collected from people who consented to share their data in a research project.  That algorithm then makes predictions about people who share certain characteristics.  For example: that indigenous students are more likely to fail first year Uni than non-indigenous students.  Or that people who buy lots of pasta are at higher risk of car accidents.

When that algorithm is operationalised, it is going to result in decision-making affecting everyone with those characteristics, never mind that they were not part of the original group who ‘consented’ to the use of their data for the research project.  The result is, as philosopher and mathematician Rainer Mühlhoff puts it, that “data protection is no longer a private matter at everyone’s own discretion… Rather, data protection in the age of predictive analytics is a collective concern”.

The cumulative impact of years of situation-specific choices, made by millions of individuals, about what they agree to share and with whom, is terrifying.  Algorithms are based on predictive analytics, built by machines learning from datasets collected from unwitting individuals who just wanted to connect with their friends on social media, or stream movies, or earn points on their grocery shopping.  Those algorithms are increasingly used to determine who gets access to housing, finance or employment opportunities, and who gets targeted for intrusive surveillance, government interventions or policing.

Digital rights activist Lizzie O’Shea says the result is that privacy is a class issue: “Our digital experiences are crafted for us on the basis of our membership in a class—an intersecting set of collective traits, rather than an individual with agency or dignity. To suggest a person alone can challenge these practices by taking individual responsibility for their privacy fundamentally misunderstands our social and political environment.”

Fiddling with your privacy settings on Facebook, Spotify or Strava won’t fix a thing.  Waleed Aly warns us against being ‘duped’ by promises to improve controls set at the individual level.  Former Australian Senator Scott Ludlam has argued that the Facebook / Cambridge Analytica scandal should instead be the catalyst we need to “draw a line under surveillance capitalism itself, and start taking back a measure of control”.

We would do well to heed the “unequivocal call for more regulation” made by New Zealand Privacy Commissioner John Edwards in his blistering speech addressing the harms caused by Big Tech, at the IAPP ANZ Summit last year. Because privacy laws, as they exist today, are not enough.  Privacy laws focus only on the effect of conduct on an individual, who must be identifiable in order to trigger the law’s protection.

Instead, we need recognition of the social and collective impacts of the data economy, and political action to protect us from the invisible threats posed not only to our individual autonomy, but also to our democracy, civil rights and social cohesion.

Photograph © Shutterstock

If you enjoyed this blog, subscribe to our newsletter to receive more privacy insights and news every month – sign up below.

We’ve all heard the saying: “if you’re not paying for the product, you are the product.”  The underlying assumption of this phrase is that if we were paying for the product or service, then we wouldn’t need to ‘pay’ for it with our data.  

But what if you could pay for privacy instead: Would you?  Should you have to?

Pay-for-privacy models are characterised by users having the option to either pay a monetary fee to opt-out of data collection, or use the service at a discounted (or free) rate by ‘agreeing’ to data collection terms. This raises some pretty significant questions around how we perceive privacy and the value we place on it. It also forces us to grapple with the relationship between privacy, privilege and power.

The idea of putting a price on privacy is not new, but is once again rearing its head as technology companies and organisations face growing pressure to take privacy seriously. People are increasingly aware of, and dissatisfied with, data-hungry and privacy-invasive platforms. Alongside this, the looming threat of increased regulation, and other steps to reign in AdTech which could diminish the ability to fuel targeted advertising revenue models, mean that many companies are being forced to consider alternative revenue streams. For example, in July of this year, Twitter announced it was considering a subscription model to mitigate their advertising losses. In 2018, Facebook COO Sheryl Sandberg said that any ability for their users to completely opt out of tracking and data profiling would be a paid offering.

It’s also currently a topic of heated debate in California as controversy surrounds Prop. 24, a ballot initiative which, if passed, would amend California’s Consumer Privacy Act. One of the issues that has raised complaint from privacy advocates is that it would entrench the ability for businesses to charge users more if they opt-out of sharing their information, effectively allowing for pay-for-privacy models.

As an alternative to the more traditional ‘pay-for-privacy’ models, California’s Prop. 24 also introduces an alarming concept of consumers being reimbursed for the use of their data. This means that rather than individuals ‘paying’ for privacy, they would be paid for their privacy invasion.  This concept is problematic in several ways.  Not only does it further cement the commodification of data, it also does nothing to reduce the harm caused by excessive and intrusive data collection, and the myriad ways that data can be used, manipulated and shared. Rather than minimising privacy harm, it justifies it – as long as some miniscule part of the profit is shared with consumers.

Privacy and privilege

Before we even think about putting a dollar sign on privacy, we should consider the extent to which privacy already has a privilege issue.

For example, those experiencing financial or social disadvantage are often required to share much more of their personal information than others in order to receive services and support. The devices and software you can afford also dictate the standard of privacy and security you can enjoy from day to day. It is well known that an iPhone has extra privacy and security features, but they also are 2-3 times the price of an android device. Not everyone is able to pay for an email service rather than defaulting to a free gmail account. Lack of access to your own stable internet connection means you’re more likely to forfeit your browsing anonymity and online security by joining an open wifi network.

Making the decision not to use well known platforms that collect, use and share your information often comes at a price. But the price is not always monetary— your choice to disconnect from Facebook out of concern for your privacy may also come at a ‘cost’ that is much less than that of someone else who depends on that platform to connect with community they may otherwise lose access to, such as fellow LGBTQ+ people, or family members from their home country. 

Even the very nature of being tech and privacy literate comes from a position of privilege: to have the tools, support, knowledge, and ability to protect yourself online is an immense advantage in and of itself, before we even start to talk about putting a monetary value on privacy.

Indeed, the experience of privacy is different for Black and Indigenous communities and People of Colour, who have historically (and continue to be) subject to disproportionate surveillance. This is often overlooked when wealthy white people lament the ‘modern’ erosion of privacy. As highlighted by activist and abolitionist, Mariame Kaba: “these violations span centuries for Black people, and are one reason for a racial disconnect in discussions about privacy and civil liberties. Black people have always been under the gaze of the state.”

It is clear that privacy already has to face up to its privilege. It’s not much of a leap to see how putting a price on privacy would serve to exacerbate inequality and create a classist privacy divide. Requiring people to pay for privacy would turn it into a luxury only available to those who are able (and willing) to pay for it.

So how much is privacy “worth”?

The value we place on privacy depends on how we think about it. As a notoriously difficult-to-define concept, privacy means different things to different people. It also evolves over time, between cultures, and across age groups.

From a company’s point of view, the value of any one individual’s data is fairly minimal. The actual value lies in the inferences that are able to be drawn about people’s interactions, and the meaning that is able to be extracted from access to millions of data points. It’s very much a case of the whole being greater than the sum of its parts (thanks Aristotle!). While it’s unlikely that Facebook and Google care too strongly about your data in isolation, they are very interested in the value of your data once it is combined with data about everyone else.

A 2013 study placed the average amount a smartphone owner is willing to pay for privacy at approximately $5. Conversely, in 2018, Facebook generated $10 per year per active daily user. But does this mean your data is worth $5? $10? No.

The reality is, humans are worth more than their data, and privacy is worth more than the arbitrary monetary value we assign it.

Valuing privacy

In Western societies, we overwhelmingly treat privacy as an individualistic issue. The idea of privacy is so wrapped up notions of my data or information about me, that we equate the value of privacy to what it means to us as individuals.

It’s this individualistic conceptualisation of privacy that spawned the adage “nothing to hide, nothing to fear”.  Because if the value of privacy is purely personal, then if you don’t care about it, what does it matter if yours is invaded?  But the problem is, it’s not actually only about you. 

Instead, we need to consider the value of privacy as a collective.  And once we look at it in this light, the idea of putting a price on privacy is revealed to be at best, overly simplistic, and at worst, actively harmful.

Shoshana Zuboff, author of The Age of Surveillance Capitalism, calls out the fundamental misconception that privacy is only an individual concern or value: “We have imagined that we can choose our degree of privacy with an individual calculation in which a bit of personal information is traded for valued services — a reasonable quid pro quo.” Similarly, Daniel Solove argues that “the value of privacy cannot be determined empirically by examining individual valuations of privacy.”

Unlike a product, privacy has social value beyond the price tag associated with your data under a system of surveillance capitalism. Beyond its inherent value as a human right, privacy also holds immense worth in its role in upholding other rights and freedoms. It’s difficult to enjoy freedom of speech or association if you don’t have privacy as a prerequisite. Privacy Commissioner of New Zealand, John Edwards, made the point in 2019 that “increasingly we are seeing privacy as a public good, a necessary precondition to the survival of some of the fundamental institutions of liberal democracy.” This concept has even reached those who aren’t self-confessed privacy nerds, with Waleed Aly writing on how privacy (or the lack thereof) is a problem for a society more so than the individual: “you could dump Facebook altogether and you’d still be living in a country whose democracy is vulnerable to corruption in new ways.”

This raises another important point: even if some people did opt to pay to protect their own privacy, it would do very little to protect the collective notion of privacy. The impact of mass data collection and pervasive surveillance would remain unchanged, even if the dataset excluded the data of the select few who would fork out to safeguard their own data.

What can be done?

First, we need to challenge the way we think about privacy and how we value it. This means reframing privacy from something purely individualistic, to a collective public good. We also need to resist the urge to think of data as capital (or, ‘the new oil’). Treating data like capital, or personal information like a tradable good, strengthens the problematic relationship between data and power.  Each of us is more than merely a series of data points.

Second, we need to re-think the scope of privacy law. Too much of how we protect privacy is through legislation that is narrowly focused on the definition of “personal information,” which often acts as a gatekeeper to privacy protections. Further, rather than focusing on solely protecting data, we ought to be reframing it to protect people. With an eye toward harm-reduction, privacy law could be, as Marietje Schaake and Martin Tisne propose, “reflective of the harms incurred through collective data analysis, as opposed to individual privacy violations.”

Much of the privacy discourse has not fully accounted for the growing power asymmetries between organisations and companies that collect data, and those who create it. While there are some changes on the horizon, our privacy framework in Australia still does not reflect the reality in which people are forced to make privacy-related decisions when dealing with systems they don’t always understand, while the system has learnt, by way of ingesting their data, how to manipulate their preferences. Adding the option to pay for privacy within this dynamic will only exacerbate the power discrepancy and deepen inequalities.

Third, we need to disrupt surveillance capitalism. This means denying the narrative that pervasive (and invasive) surveillance is a technological necessity. Privacy professionals, civil society groups and governments must push back against the idea that if we want technology and services that they must come with privacy invasion as byproduct. As Shoshana Zuboff notes in The Age of Surveillance Capitalism, mass collection of data is done not because it is necessary, but because it is profitable. Surveillance capitalism is an ideology, not a technological necessity. Adding a price tag to privacy would further strengthen surveillance capitalism’s grasp.

Finally, we need to absolutely deny the prospect of people paying for privacy. Putting a price on privacy stands to transform it into a luxury that only those with money can enjoy. The onus of respecting privacy should be on companies and organisations, not hand-balled over to individuals who may or may not be able to afford it. Beyond this, allowing a handful of people to pay for their privacy, or worse, ‘reimbursing’ people for the use of their data while doing nothing to address the underlying issues that cause harm, only creates an illusion of change. It is a red herring that tricks us into thinking that privacy is being respected or valued — all the while the data-hungry machines continue to work quietly in the background.

Photograph © Shutterstock

If you enjoyed this blog, subscribe to our newsletter to receive more privacy insights and news every month – sign up below.