Tuesday, December 25, 2012

Impact of Convergence Review on Australian Television Industry


The final report of the Convergence Review was published recently with many points of contention. While many pundits believe that the television is a dying media and will be over taken by the IPTV and the internet, there has been overwhelming number of research which proves otherwise. The television industry is in fact the medium of choice for Australians and therefore, this paper saw it necessary to analyse the impact that the proposals contained within the review will have on the industry. While it does seem that the television industry will have to meet higher standards, thus favouring its consumers, the paper found it difficult to measure the financial impact on the industry. This is because the review did not specify important information, especially in terms of the exact costs and percentage expenditures required by the industry. Therefore, while the review posts a win for consumers it remains to be seen what will happen once the proposals get implemented in 2013.


By 2020, digital television (DTV) will replace analogue broadcasts in Australia. Up till now television (TV) signals have been analogue waves, broadcast from towers and received by home antennas to be displayed on analogue TVs. This form of broadcasting limited the number of channels viewers could watch while taking up a lot of bandwidth. However, with digital technology, TV signals and other information are broadcast more efficiently and with better quality picture and sound. Television stations can also broadcast over more channels using the same bandwidth. However, with the advent of convergence and especially with the internet’s prevalence in Australians’ lives, the Department of Broadband, Communications and Digital Economy saw the need to review out-dated media regulation and thus embarked on the mission of the Convergence Review. The review which took about a year to compile, published its final proposals in the middle of 2012. There are many points of contention with the review and this paper decided to focus especially on the television industry in Australia. This is because television, despite contrary belief, is still the medium of choice among Australians.
In a research done by Think TV, a marketing representative of Free TV Australia, it found that TV pays back more than any other medium, in terms of returns on investments (ROI) and that the sales effect of a TV advertisement continues to be strong for up to about two years, while the effects of other media on sales fade quickly. It also found that when advertisers pull back on advertising on TV, brands may take up to six years to recover their market position (Clift, 2011). According to research firm Ovum, digital terrestrial TV would reach about 5.894 million households in Australia, with an additional 823,000 accessing mainstream TV via satellite, by 2015. Pay TV will also reach 2.175 million households, while IPTV services are expected to grow impressively; from just 90,000 now to 342,000 in 2015 and that figure is still only a fraction of the reach which free-to-air TV will have (Kidman, 2012). Therefore, it is important to understand how the proposals in the review will impact this dominant medium.  Hence, this paper will now pick out proposed regulatory changes in the Convergence Review to critically analyse its impact on the dominant television industry.

Convergence Review: Proposed Changes Affecting Television in Australia

Content Service Enterprises

Due to the increasingly convergent nature of media, the review coined the term “Content Service Enterprises” (CSE) to describe media organisations, including print, broadcast and online entities. However, only those CSEs which earn annual revenue of $A50m, generated from professionally produced content, and with a monthly viewer numbers of 500,000, will face the proposed regulations. This means that organisations such as Telstra, Apple, Google and its YouTube web video off-shoot will may be classified as a CSE (Australasian Business Intelligence, 2012). This new classification will not affect the delivery of digital television in any way as television is an industry that has always been regulated and will continue to be while it exists. The new content rules may also make Google re-think the introduction of Google TV into Australia as it will be subject to the CSE regulations as soon as it meets the set threshold (Turner, 2012). Hence, this could prevent Australians from enjoying a brand new technology, which goes against the spirit of introducing the National Broadband Network (NBN) and to decrease the digital divide.
As IPTV will soon become prevalent in Australia following the NBN rollout, this will also present a threat for free-to-air and pay TV broadcasters. This is due to the fact that IPTV services can escape classification as a “broadcasting service” if it is distributed via the internet or provided on-demand on a point-to-point basis. Thus, it does not need a licence under the Broadcasting Services Act (BSA), unless is provided over a proprietary network and not on-demand. But that is unlikely to be the case. In fact, IPTV is just like watching traditional TV, except it is delivered over broadband rather than through an antenna or cable. So, it is unclear why it bears less regulation than broadcasters who have to carry the burden of regulations such as content quotas and advertising standards. This would most certainly create an uneven playing field for traditional broadcasters (Bradshaw, 2012).


Content Standards

Under the new content standards, the review proposed that once a particular content has been classified, it would apply uniformly regardless of the platform it is delivered on. However, the review made an exception that:
 “Content providers that are not of sufficient size and scope to qualify as content service enterprises should also be able to opt in to content standards or develop their own codes”
It is unclear why the review made such an exception. Does this mean that say a horror movie classified as “PG13” – Parental guidance advised for viewers under the age of 13, could be classified as something completely different just because it is being streamed on YouTube?  Another scenario may hold true as well where a young viewer who watches an episode of Simpsons for instance, may not register the variations in content standards when he watches it on free-to-air channels, on a PG (Parental Guidance advised) rated episode on Pay TV or downloaded through Apple TV (Kidman, 2012).

The Push for more Australian Content

One of the major highlights of the convergence review is to increase locally produced television content by regulating the amount of Australian made programs aired on their networks (Department of Broadband, Communications and the Digital Economy, 2012). The review proposes that television broadcasters would keep their current 55 per cent quota for Australian content while increasing the quota for drama, documentary and children’s content by 50 per cent. At the moment, these quotas will not include content shown on digital multi-channels. One of the main reasons why the government has to step in to regulate Australian made content is because, producing locally made content is more expensive than buying content from overseas. Therefore, without regulation, there would no incentive for broadcasters to air locally made programmes. Indeed, in the final report of the review, Price Waterhouse and Coopers (PwC)  concluded in its analysis that without existing Australian content requirements, documentaries would plunge by 50 per cent, drama by a staggering 90 per cent and children's content would simply not exist.
This is especially true for children’s programmes, since most of its children’s programming is currently only from the Australian Broadcasting Corporation (ABC) and hence lacks variety.  A research into the health of children’s television in Australia found these quotas to be the utmost important. 
“The research confirmed the centrality of the CTS (Children’s Television Standards) to the production of children’s television in Australia. In an environment in which Australian adult drama production has been declining, and financing children’s television has been becoming more difficult, the CTS quotas mean the production of children’s television plays a significant role in the overall health of Australia’s production industry.” (Blumenau, 2011)
Therefore, the review was justified in raising the quotas for children’s programmes, although, one may argue that Australian children's content simply does not pay, particularly when there is no junk food advertising (McCreadie, 2012).
In the current state of almost non-existent content regulation, the free-to-air networks' digital multi-channels, have carried next to no Australian content, with the exception of Neighbours (McCreadie, 2012). Not surprisingly, with the exception of Channel 31, which airs only locally made programmes, both free-to-air and subscription television were opposed to the raise in quotas. Julie Flynn, chief executive of Free TV, the industry’s umbrella body, said: local content quotas
“Should not be increased and they should be more flexible than they are”, in light of the convergence of television and digital platforms. (Holgate, 2012)
James Warburton, chief executive of the Ten Network (Ten), also criticised local content rules for not allowing local programs broadcast on the FTA digital channels to count for the quotas. He wrote in an opinion piece for The Australian Financial Review,
“Unfortunately the Australian Content Standard, which sets out the local content rules, was framed well before digital multi-channels arrived.” (Warburton, 2012)
He was particularly referring to Ten’s longest running and successful drama series, “Neighbours,” which airs on its free-to-air channel which would have met the quota requirements if it was counted towards it. Despite these criticisms, a news article by the Sydney Morning Herald, reported that Pay TV has upped their local content spending significantly.  However, it was pointed out that most of their investment was put towards expensive sports rights instead of spreading them between the different genres. Minister for Broadband, Communications and the Digital Economy, Stephen Conroy, quickly responded by saying at least 10 per cent of total programming expenditure needs to be put towards drama (Strachan, 2012).
While it is common knowledge that producing local content is costly, the paper found that the three commercial free-to-air televisions have no reasons to complain, bringing in top dollars with a combined annual revenue of between $3.5 billion and $4 billion, while earning more than $2 billion in the second half of last year from advertising alone. They also received a licence fee rebate of $250 million in the past few years and $53.5 million from the government to move off the spectrum they had already agreed to vacate.  With figures like these, it is only justifiable that the review has pushed for more Australian content over the television (McCreadie, 2012).
The convergence review also proposed further expenditures by the television industry to support the push for more Australian content. The review proposed a uniform content scheme which will require CSEs to invest a certain percentage of their revenue from Australian drama, documentary and children’s programs. Alternatively, they may also choose to contribute a percentage of its revenue to a ‘converged content production fund’ for reinvestment in traditional and innovative Australian content. In order to better support content production, the review also prosed a raise of the Producer Offset from 20 per cent to 40 per cent. Finally, it also recommended the creation of a converged content production fund, which will get direct funding from the government and may include spectrum license fees from broadcasting services and contributions from content service enterprises under the uniform content scheme. These proposed expenditures caused the current shadow minister for communications and broadband, Malcolm Turnbull, to lash out at the review and claimed that the requirements would impose an additional financial burden on enterprises which in many cases are struggling to remain profitable and viable (Turnbull, 2012). But is the television industry really struggling while research has shown that it is still the medium of choice for consumers and advertisers alike? Or, could it be that when the review finally discloses what the said percentages of contributions are that it would prove to be costly for the industry? This is yet to be unravelled in a few months.
Interestingly, this paper could not find the implications that these local content requirements would have in terms of quality of programs produced. One can only assume that pressurising digital television broadcasters as such could possibly force them to produce and air low quality programs in an attempt to meet the standards. For instance, do Australians really want locally produced television programs similar to the “Jerry Springer” show? The review seems to have failed to define what “quality” programming means and this only provides broadcasters a loophole through which they can cut costs while meeting the standards of the review. There is also another question that the review has failed to answer - Will the content standards stifle the development of emerging content service initiatives, even where those initiatives are housed within larger organisations that qualify as CSEs (Corrs Chambers Westgarth, 2012)?  This is yet another important question that the review will need to consider before implementing the changes.

New Standards Body

The final review proposed the establishment of two separate bodies – the first, a statutory regulator which will replace the current Australian Communications and Media Authority (ACMA) and secondly, an industry-led body which will oversee journalistic news and commentary across all platforms in the media and communications sector. 

Communications Regulator

The review recommends that the new regulator would –
“Be independent and operate at arm's length from the Government. Significantly, ministerial control of the regulator would be only through disallowable legislative instruments, not general directions.”
This regulator would take the form of a statutory corporation which will be managed by a board which will have the full powers to act within the limits of the law. It will be interesting, to see the persons who will be appointed to the “independent” board. Despite its “arm’s length” operation from the government, it would still be, according to the review –
“Held accountable for its decisions under existing parliamentary, judicial and administrative arrangements; for example, disallowance by Parliament, merits review by the Administrative Appeals Tribunal and judicial review…”
With the exception of news and commentary, the regulator would be responsible for all compliance matters relating to media content standards. This automatically begs the question – What would be classified as news and commentary? Does anyone who uploads a YouTube video discussing current affairs become subject to regulation, if suppose YouTube were to meet the CSE requirements?
The new regulator would also define the thresholds for CSEs, administer ownership rules, and ensure Australian and local content obligations are applied. With regards to the above, Free TV Australia addressed some concerns in their submission to DBCDE. Firstly, they wanted the review to give a clearer indication as to the relationship between the new regulator and the DBCDE in terms of policy making and also the Australian Competition and Consumer Commission (ACCC) in terms of competition rules. Secondly, Free TV refers to the statement in the review which says the new regulator would not be subject to:
“Unreasonable procedural requirements”
In its report, it wanted the review to explain how it will guarantee that there will indeed be procedural fairness, since the review failed to provide details (Free TV Australia Limited, 2012).
The regulator would also be required to set technical standards which will assist users in managing access to content, such as parental locks or age-verification systems. However it is yet to provide more details. While many details of this new regulator are yet to be revealed, it is difficult to determine the impact of this new regulator on TV in Australia.

News Standards Body

The new standards body would oversee all news and commentary on television and thus:
“Would administer a self-regulatory media code aimed at promoting standards, adjudicating complaints, and providing timely remedies”
However, once again the review failed to specify its reach of the regulation. For instance, would it regulate news and commentary content from overseas news agencies such as British Broadcasting Corporation (BBC) and Fox news, which currently air on free view and if so, how? Does that mean it would monitor news content and engage in censorship?
Because news and commentary play a vital role in any democracy, it is critical that journalistic standards in fairness, accuracy and transparency be applied regardless of the delivery platform. However, it is unclear as to how the news standards board would go about implementing its powers.

Broadcasting Spectrum

One of the benefits that the switchover to digital TV provides is the release of TV spectrum so they may be allocated for next generation mobile broadband services. This will also allow TV stations to broadcast on more than one channel using the same bandwidth. After the switchover, there will be 126 megahertz (MHz) of spectrum, in the 694-820 MHz range, available for use. This is what the Department of Broadband, Communications and the Digital Economy (DBCDE) refers to as the “digital dividend.” Of the 126 MHz, the 700 MHz is the most coveted of all. The International Telecommunications Union (ITU), which specifies the approved services to be used by certain radio frequency bands worldwide, allocated the 700 MHz band primarily for broadcasting, even though the band could also be used for fixed wireless and mobile services (International telecommunications Union, 2012). Traditionally, the entire 700 MHz band was used for analogue TV broadcasting. But with the switchover, its availability means telecommunications corporations could fight for a slice of the bandwidth as well.
So, why is the 700 MHz bandwidth so popular in the first place? This is because it possesses excellent propagation characteristics, such as being able to penetrate buildings and walls easily, and it covers relatively large geographic areas without unacceptable deterioration of the signal. This means that lesser base stations would be needed to serve a large area, thus providing for an efficient use of wireless networks (Marius, 2012). Therefore, it seems likely that most of this bandwidth will be snatched up by telecommunication companies. Indeed, in a report prepared in 2009, by Spectrum Value Partners for the Australian Mobile Telecommunications Association (AMTA, an association of mobile operators, handset manufacturers, retail outlets, network equipment suppliers and other suppliers to the industry), it indicated that Australia’s economy would be boosted by up to ten billion Australian dollars if at least 120MHz of useable spectrum from the digital dividend were allocated to mobile broadband uses (Access Economics Pty Limited, 2010).
In the final review, existing holders of commercial broadcasting licences would have their apparatus licences replaced by spectrum licences planned for the supply of broadcasting services.  The broadcasting licence fees would be replaced by annual spectrum access fees based on the value of the spectrum as planned for broadcasting use. However, this fails to answer an important question – How will the television industry, which will spend a substantial amount on spectrum be protected against inroads made by content service providers who avoid the use of such spectrum (Corrs Chambers Westgarth, 2012)?


In a survey done by Deloitte, of more than 2000 Australian consumers, with 63 per cent, TV was by far the most popular choice for entertainment, followed by the internet at 47 per cent and listening to music at 30 per cent (Davie, 2012). Hence, this paper decided to focus on the impact of the Convergence Review on television because contrary to popular belief, television is still the dominant media for Australians. Therefore, any changes made to the way it functions will impact Australian viewers directly. This paper chose to critically examine the proposed changes that would most impact the television industry, mainly the impact of the new regulators, the coinage of the term CSE, the content regulations and its associated financial contributions and finally the financial impact of spectrum licensing. The paper did not touch on media ownership as the ownership landscape in the television industry will not be as impacted as much as radio or internet will be. While there are many aspects of the review that were unclear, especially in terms of the actual figures associated with expenditures and also detailed ways in which the regulations will be implemented, the paper still managed to speculate on the various possible implications it would have on the television industry. One on hand, proponents of the review claim that the television industry will in fact be doing well despite facing competition from other media platforms and therefore, their expenditures are justified. On the other hand, there are major television networks which have expressed unhappiness with having to contribute to funds and licencing and to produce more Australian content, claiming that they are struggling to survive. Until the review comes into implementation it will not be possible to discuss the review’s impact in financial terms. The review is set to be implemented in three stages; however, no specific deadlines have been set by the DBCDE as yet.


Access Economics Pty Limited, 2010. Economic Contribution of Mobile Telecommunications in Australia, Canberra: Access Economics Pty Limited.
Australian Communications and Media Authority, 2012. Allocation of the 700 MHz (digital dividend) and 2.5 GHz bands, [Online] Available at: http://www.acma.gov.au/scripts/nc.dll?WEB/STANDARD/1001/pc=PC_312315 (Accessed 6 November 2012).
Australasian Business Intelligence, 2012. Net giant problem sidestepped, Virginia: COMTEX News Network, Inc.
Blumenau, J., 2011. Save Kids’ TV 2006 – 2011. [Online]
Available at: http://www.savekidstv.org.uk/wp-content/uploads/2011/05/SKTV-competitor-territory-research-post-final-updated-24.4.11.pdf
[Accessed 30 October 2012].
Boston Consulting Group, 2010. Socio-economic impact of allocating 700 MHz band to mobile in Asia Pacific, [Online] Available at: http://www.acma.gov.au/webwr/_assets/main/lib311973/ericsson_attachment%20a_ifc34-2010.pdf (Accessed 6 November 2012).
Bradshaw, S., 2012. Convergence Review: Meeting the Regulatory Challenge, [Online] Available at: http://www.corrs.com.au/thinking/insights/convergence-review-meeting-the-regulatory-challenge/ (Accessed 2 November 2012).
Clift, J., 2011. TV is still king: Analysis from Thinkbox's Payback 3 study. [Online] Available at: http://www.thinktv.com.au/media/TV_Insights/TV_is_still_king_Analysis_from_Thinkbox.pdf [Accessed 6 November 2012].
Corrs Chambers Westgarth, 2012. Convergence Review Final Report: More Regulation, or More Effective Regulation? [Online] Available at: http://www.corrs.com.au/publications/corrs-in-brief/convergence-review-final-report-more-regulation-or-more-effective-regulation/ (Accessed 4 November 2012).
Davie, M.. 2012. TV and New Media. [Online] Available at: http://rossdavie.com.au/tv-in-new-media-world/ [Accessed 6 November 2012].
Department of Broadband, Communications and the Digital Economy, 2012. Convergence Review, Canberra: Department of Broadband, Communications and the Digital Economy.
Free TV Australia Limited, 2012. Submission by Free TV Australia Limited, [Online] Available at: http://www.freetv.com.au/media/submissions/2012_0003_SUB_FINAL_Convergence_Review.pdf (Accessed 7 November 2012).
Holgate, B., 2012. The Australian Financial Review. [Online]
Available at: http://afr.com/p/national/quotas_at_odds_with_tv_digital_age_YxVJJ77m6YbxwFao8qF22I
[Accessed 1 November 2012].
International Telecommunications Union, 2012. Digital Dividend: Insights for Spectrum Decisions, Switzerland: International Telecommunications Union.
Kidman, A., 2011. How Badly Will Convergence Affect TV Networks? [Online] Available at: http://www.lifehacker.com.au/2011/02/how-badly-will-convergence-affect-tv-networks/ (Accessed: 6 November 2012).
Marius, M., 2012. What is so special about the 700 MHz band? [Online] Available at: http://www.ict-pulse.com/2012/10/special-700-mhz-band/ [Accessed 04 November 2012].
McCreadie, S., 2012. Media, Entertainment and Arts Alliance. [Online]
Available at: http://www.alliance.org.au/top-making-drama-about-regulation-sur-mccreadies-opinion-piece-in-the-australian
[Accessed 1 November 2012].
Strachan, J., 2012. The Sydney Morning Herald. [Online]
Available at: http://www.smh.com.au/entertainment/tv-and-radio/pay-tv-boosts-local-content-spending-20121031-28jdh.html
[Accessed 2 November 2012].
Turner, A., 2012. The Sydney Morning Herald Sun. [Online]
Available at: http://www.smh.com.au/digital-life/hometech/what-is-google-tv-in-australia-20120711-21uhd.html
[Accessed 31 October 2012].
Warburton, J., 2012. The Australian Financial Review. [Online]
Available at: http://afr.com/p/business/marketing_media/viewers_and_channels_wait_on_new_IsYe64jn31ObQfk6KmYByL
[Accessed 1 November 2012].

Big Data

Whoever said that “information is power” definitely was talking about big data. No one could have predicted that social media sites which were created for mere entertainment and social purposes could become a treasure chest full of valuable data about one’s very existence. However, while everyone knows what big data is, that it exists and where it is contained, there is much confusion as to what to do with it. Experts in the field of research have come to realize that their old methods of collecting and analyzing data is becoming less and less useful and individuals are becoming afraid that their personal information can be used against them. Many industries are also grappling with the fact that they are not equipped to handle the sheer volume of big data.
Despite the confusion, big data is being used in various industries in various ways, although reports suggest that so far, only about eight per cent of big data is being extracted and put to use. In their article, “The 6 Provocations of Big Data,” Danah Boyd and Kate Crawford put big data into perspective by identifying six challenges which social scientists need to be aware of when dealing with big data. Therefore, this paper, endeavors to delve deeper in each of the provocations to better understand the confusion surrounding big data.


As more and more data is being collected, curated and analysed, the term “Big data” is becoming the buzz word in every industry and institution today. In simple terms, big data comprises of datasets whose size is beyond the ability of typical database software tools to capture, store, manage and analyse (McKinsey and Company, 2011). However, the term “Big Data” does not merely refer to the size or volume of datasets. The International Data Corporation (IDC) defines big data as:
“Big data technologies describe a new generation of technologies and architectures, designed to economically extract value from very large volumes of a wide variety of data, by enabling high-velocity capture, discovery, and/or analysis” (Gantz & Reinsel, 2011)
Hence, the definition of Big Data also concerns a computational turn in thought and research. With the advent of social platforms like Twitter, Facebook, Google Earth, data sets now have expanded to include maps and images along with texts and numbers. Facebook for example, allows users to upload an endless amount of photos and videos and with an estimated 800 million users (Hancock, 2012) and Twitter is publishing about 500 million tweets per day, while organising them and storing them in perpetuity (Raftery, 2012), therefore, contributing significantly to creating Big Data. Technological advancements also allow just about anyone to have access to large datasets, which were otherwise made available exclusively to academic and scientific institutions. For instance, just about any type of data is accessible at data.gov, by anyone with an internet connection (Weigel, 2012).       
While proponents of Big Data celebrate it as creating value by enhancing productivity, improving organizational transparency, and forecasting models, sceptics see it as creating new social problems. In their article, “The 6 Provocations for Big Data,” Danah Boyd (Boyd) and Kate Crawford (Crawford) identify areas in which Big Data may create new issues in social science research. This paper thus, using those six provocations as a framework, will explore the challenges that big data presents social scientists.

Automating Research Changes the Definition of Knowledge

The scientific method is the process by which scientists, collectively and over time, endeavor to construct a reliable, consistent and non-arbitrary, representation of the world (Wolfs, n.d). To do this scientists built the scientific method around testable hypothesis, which are then tested to either falsify or confirm theoretical models of how of the world works. However, with the availability of Big Data, the theories developed through the scientific method could become obsolete, and in effect change the way the world is understood. The availability of Big Data and advanced technological tools can potentially generate more accurate results than specialists or domain experts who traditionally craft carefully targeted hypotheses and research strategies (Anderson, 2008). But in order to get more accurate results from these data, the scientific method as a whole needs to change. This is because large amount of data means more false correlations under the current types of scientific methods employed. For instance, with Big Data, one can easily get false correlations such as, “On Mondays, people who drive to work are more likely to get the flu.” While this assertion may be true under current scientific methods, it will not explain why it is true and if there is any causality. Thus, scientists now need to step outside their laboratories, to develop new methods of testing causality where they can no longer rely on control for variables (Edge.org, 2012). But, while data is being created in real time, the reality is, knowledge is going to outpace understanding. Therefore, data users are now able to predict outcomes , but without understanding what drove those outcomes (Weinberger, 2012).

Claims to Objectivity and Accuracy Are Misleading

As computational scientists begin to study society using quantitative methods, there is the danger, results from their research, will be accepted as facts rather than being open to interpretation. This is because the quantitative method comes with limitations, particularly with its use in social science. Since quantitative researchers need to carefully design their study even before data is collected, they need to know in advance what they are looking for. With Big Data, however, due to its volume and because it includes both unstructured and semi-structured data, it is impossible to know in advance what to look for. Since quantitative analysis deals with data in the form of numbers and statistics, it also ignores contextual detail, which Big Data is rich in.

Bigger Data Is Not Always Better Data

Bigger data is not necessarily better data simply because not all data is worth exploiting. With more data, there is an increased possibility of redundant data as well. Take for instance twitter updates and/or shares in the form of hashtags. When a particular update gets re-tweeted again and again, it creates a high volume of data in terms of the number of tweets. However, what one gets is more data that is being repeated to create Big Data, while the amount of information gathered from those data remains the same. Therefore, the value of information derived from Big Data is not dependent on the data size.  However, one may argue that while a retweet itself gives no further data as it is a copy of another tweet, the number of tweets and retweets brings a lot of insights, as many marketers know, about the popularity or importance of something.
There is also the issue of limitations inherent in sampling. No matter how much data one can gather, it will never be the entire population set. This is because Big Data is being created in real time and therefore, at any point in time, it is impossible to collect the entire population set. Hence, researchers are always going to be dealing with a sample size, even though it may be a relatively bigger sample size. This brings one to the next question which is, “Does a bigger sample size mean better information?” This is not necessarily the case. While a larger sample size may provide better estimates with a lower standard of error, its inherent problem of biasness may skew the results and this may be the case even if the samples were picked randomly. Suppose for example that a company operating in a certain industry has collected 'big data' on its customers in that country. If it wants to use that data to make assertions about its existing customers in that country, then its data may yield accurate results. If however it wants to draw conclusions about a larger population - potential as well as existing customers, or customers in another country, then it become important to consider to what extent the customers about whom data has been collected are representative - perhaps in income, age, gender, education, etc - of the larger population (Brown, 2012).

Not all Data is Equal

Once again, it is simplistic to assume that bigger amounts of data will provide better information using the same analytical tools used with small data. This is because, data is not always interchangeable and with Big Data, context is critical. Traditionally, data sets have generally been structured. So, the “context” of those data can be found within the records or the files which contain those data. However, Big Data includes high volumes of unstructured and semi-structured data. In fact, while an average sixty per cent of the data created today is unstructured, businesses are currently only able to capture just eight per cent of the unstructured data.This is because, the tools and techniques that have proved successful in transforming structured data into business intelligence and actionable information, simply do not work when it comes to unstructured data (Bank of America Corporation, 2012). For example, researchers of social network analysis study networks through data traces in articulated and behavioural networks. But the problem with this kind of data is that even though it provides valuable information, it does not necessarily represent the nature and complexity of social behaviours (Boyd & Crawford, 2011).

Just Because It’s Accessible Doesn't Make It Ethical

Before tackling the issue of ethics in Big Data, it is important to understand what ethics means. The Oxford dictionary defines ethics as,
“the branch of knowledge that deals with moral principles, governing a person’s behaviour or the conducting of an activity.” 
The definition itself highlights the main issue with the ethics of Big data – the conflicting  moral principals of the parties involved, mainly, the people who create the data and the users of those data. A huge amount of Big Data is created by people who do not understand how information about themselves is being used and for what reasons(Boyd & Crawford, 2011). While Big Data is ethically neutral, the use of Big Data is not. In his book, “Ethics of Big Data – Balancing Risk and Innovation,” Kord Davis (Davis) identifies four aspects of Big data ethics – Identity, Privacy, Ownership and Reputation. This paper will thus touch on those four aspects to understand the issue of ethics in Big Data.


Most social media sites such as Facebook and Google, function on the basis that each user has only one identity and hence, is portraying a mirror image of themselves in the virtual world. The founder and CEO of Facebook, Mark Zuckerberg believes in  a singular identity so much so that he said,
“Having two identities for yourself is an example of a lack of integrity…The days of you having a different image for your work friends or co-workers and for the other people you know are probably coming to an end pretty quickly (Newman, 2011).”
Indeed, Facebook and Google have real-name policies, which make it mandotary for its users to only use their true names online, failing which they may have their accounts deleted. For the purposes of using Big Data, real names are thought as being more useful, for example, in terms of advertising, where the advertiser can target relavent advertising if it knew who they are dealing with. It also helps researchers gather more meaningful and accurate data if they knew the real identities of a particular user. However, problems arise where on Twitter for instance, users are allowed pseudonyms and can have multiple accounts. Users of data will find it difficult to back track who the users are to derive any kind of useful information.
But, proponents of annonymity claim that real names deny users of freedom of expression and thus places people such as political activists and abuse survivors at risk (Heussner, 2012). One such proponent is Chris Poole, the founder of 4chan and Canvas, who argues that identity is prismatic, and that one chooses to act out various “selves” to different groups of people anyways and therefore, permitting a user to have multiple identities allows for more accurate  information about people in general. Supporting this notion is Universal Mccan’s ex Vice President, David Cohen, who said,
“There’s the use of pseudonyms to mask behaviour that we wouldn’t condone in the world of marketing and the use of masking that is simply another personification of your persona, along the lines of interests and passions.”
The latter, he says, could create additional consumer insights (Heussner, 2012).


The next aspect of ethics in Big Data is privacy ,where unlike the above mentioned identities, it is a black and white issue. In fact, the Associate Professor at the University of Colorado Law School, Paul Ohm, describes the issue best by saying that data can either be useful or perfectly annonymous but never both (Ohm, 2009).      The harvesting of large data sets from social media platforms clearly indicate privacy concerns. It is still unclear how major social media sites such as Twitter and Facebook use data created by their users, although through their friends’ suggestion tools and targetted advertsing, it is obvious that it is being used in some way. Before technological advancements, one could rely on annonymity, having used pseuodonyms, to have some relief that one’s privacy is still maintained when data about them is being collected. This is because organisations used methods of de-identification such as, anonymisation, pseudonymisation, encryption, key coding and data sharding to distance data from real identities, allowing analysis to proceed while maintaining privacy (Polonetsky & Tene, 2012). However, reasearch by computer scientists found that anonymised data can often be re-identified and attributed to specific individuals, thus causing huge privacy concerns (Narayanan & Schmatikov, 2008).


The third issue Davis identifies is the ownership of collected data and a good example to demonstrate the complexity of the issue is Wikileaks. Who does the information released on Wikileaks rightfully belong to? Does it belong to the governments,  the general public, Wikileaks or the newspapers which published them? Facebook for instance has its privacy policy set such that any information that anyone puts on their site belongs to them (Facebook.com, 2012). Basically, this means that one holds rights to their information or data until they choose to put them up on the internet which then changes the ownership altogether. While this may seem like a new problem, it in fact is an old one. If someone decides to confide in another about some personal information, then the other person is being given the right to do whatever he wants to do with that piece of information. He may choose to repeat the information to some other persons to pass it on, he may decide to collect that information together with other information that he already knows about that person to derive at some sort of a conclusion or he may simply choose to forget that piece of information. Same problem, but on a different platform and in a bigger scale. This is why many pundits argue that if one is afraid of their private data being misused, say for example in Facebook, then they simply should either not sign up for their service or put up posts that may get “leaked.”


The final aspect of ethics in big data has to do with reputation, which begs the question,
“Is data trustworthy?”
In March of 2012, TheMotleyFool.com published its top ten lists of “Big Data Stocks” where LinkedIn.com (LinkedIn) was placed in at the third position (Reeves, 2012). However, in May that year LinkedIn fell victim to hackers and put at risk, the data and privacy of more than six million of its users, thus jeopardising the website’s reputation (Barlow, 2012). With the advent of big data, data security becomes a major issue for data mining companies like LinkedIn. Another problem this brings is that by losing control of their data, LinkedIn users were stripped of their private identities on that site, thus putting their reputation at risk as well. In fact, LinkedIn is known to be the preferred social network for government agencies (Williams, 2012).
Gone are the days where one could act one way to a certain group of people and another to a different group of people without each group finding out. Now, as more and more information gets collected about someone, organisations and even groups of people that someone does not even know can make assumptions about a person. For instance, in 2011, Kurt Nordland was seeking workers compensation from his company and the insurance company paying his claim decided to look him up on Facebook, where they found pictures of him drinking beer and relaxing at the beach. Based on those pictures, the insurance company cancelled his payments, cut off his medical benefits and Nordland had to delay surgery to repair torn cartilage in his shoulder (Romero, 2011).

Limited Access to Big Data Creates New Digital Divides

When pundits talk about digital divide, they are generally referring to the issue of accessibly of technology. Boyd and Crawford argue that big data is not easily accessible. This is because only social media sites have access to these data and they have no obligation to make it available. For example, Twitter has created different grades of access to its data. The Gardenhose access level grants ten per cent of public statuses for free while requiring case-by-case approval by Twitter and the most popular Spritzer level provides any user with just one per cent of the public statuses. If money is not an issue, then the highest level of access called the Firehose grants users access to all of its data, although Twitter varies the price from time to time (Gannes, 2012). This grading system alone proves that not everyone gets equal access to data.
However, the new era of big data also heralds in a new kind digital divide, which refers to a divide between the people who are able to use data and those who cannot. Even if a researcher has access to big data, he must have the technical knowledge to collect and analyse the data – a skill that many in the social sciences lack and inadvertently favours the computational scientists (Boyd & Crawford, 2011).


Using the article by Boyd and Crawford as a framework, this paper made a deeper analysis of each of their provocations. Although big data is generally identified by its volume, it is the quality of the data that truly matters.  To be able to sift quality data however, one needs the technical skills and financial capability to collect them and analyse them. With the Twitter example, to have access to its Firehose, one needs to be able to afford to purchase the data in the first place. In addition, because big data includes a high percentage of unstructured and semi-structured data, such as images and texts, the research methodologies traditionally used in social sciences needs to change to suit the new type of data. Researchers also have to contend with correlations of data without understanding the causality as data is created in real time. Therefore, scientists do not have the luxury of time to understand causality. But of course, privacy concerns present a major roadblock to the availability and collection of those data. Creators of big data often do not understand how information about them is used and more and more people are pushing for better privacy policies on social media sites.  Another problem is that many people also use pseudonyms to hide their real identity so that information about them cannot be traced easily. However, technological advancements have allowed for the de-anonymisation of people’s fake identities to reveal their true self.  The digital divide also presents another problem in that most of the critical data is only available to the financially and politically elite with sites like Twitter charging a huge amount of money to purchase most of their data.
Although their article highlights major challenges with big data, one cannot deny that big data has significantly improved people’s lives. The IDC reported an increase of three hundred billion dollars in potential annual revenue to the American healthcare and a hundred billion dollars increase in revenue for telecommunications service providers. Technology research giant, Gartner, reported a sixty per cent potential increase in net margin in the retail industry while predicting a two hundred and fifty euro potential annual value to Europe’s public sector (DataArt, 2012). Therefore, while the potential of big data is enormous, the task of overcoming the six provocations identified by Boyd and Crawford are equally as big.


Anderson, C., 2008. The End of Theory: The Data Deluge Makes the Scientific Method Obsolete. [Online] Available at: http://www.wired.com/science/discoveries/magazine/16-07/pb_theory
[Accessed 5 November 2012].

Bank of America Corporation, 2012. Tech's 'Big Data' Drive. [Online]
Available at: http://wealthmanagement.ml.com/wm/Pages/ArticleViewer.aspx?title=techs-big-data-drive.
[Accessed 28 October 2012].

Barlow, R., 2012. LinkedIn Hacking: What You Need to Know [Online]. Available at: http://www.bu.edu/today/2012/linkedin-hacking-what-you-need-to-know/ [Accessed 3 November 2012].

Boyd, D. and Crawford, K., 2011 ‘6 Provocations of Big Data’ paper given at Oxford Internet Institute Decade in Time Symposium on the Dynamics of the Internet and Society.Oxford, England.

Brown, M., 2012. Big Data Blasphemy: Why Sample?. [Online]
Available at: http://smartdatacollective.com/node/47591
[Accessed 5 November 2012].

DataArt, 2012. Industries Set to Benefit from Big Data. [Online]
Available at: http://www.dataart.com/software-outsourcing/big-data/industry-benefits
[Accessed 4 November 2012]

Davis, K. (ed). 2012. Ethics of Big Data: Balancing Risk and Innovation. California: O'Reilly Media, Inc.
Edge.org, 2012. Reinventing Society in the Age of Big Data. [Online]
Available at: http://www.edge.org/conversation/reinventing-society-in-the-wake-of-big-data
[Accessed 4 November 2012].

Facebook, 2012. Statement of Rights and Responsibilities [Online]. Available at: https://www.facebook.com/legal/terms [Accessed 4 November 2012].

Gannes, L., 2012. Twitter Firehose Too Intense? Take a Sip from the Gardenhose or Sample the Spritzer [Online]. Available at: http://allthingsd.com/20101110/twitter-firehose-too-intense-take-a-sip-from-the-garden-hose-or-sample-the-spritzer/
[Accessed 4 November 2012].

Gantz, J. & Reinsel, D., 2011. The 2011 Digital Universe Study: Extracting Value from Chaos, Massachusetts: IDC Go-to-Market Services.

Hancock, C., 2012. Broadband is important but BIG DATA is the future!. [Online]
Available at: http://www.abc.net.au/technology/articles/2012/04/16/3478362.htm
[Accessed 22 October 2012].

Heussner, K.H., 2012. The Internet Identity Crisis [Online]
Available at: http://www.adweek.com/news/technology/internet-identity-crisis-137991 [Accessed 26 October 2012].

McKinsey and Company, 2011. Big Data: The next frontier for innovation, competition and productivity, s.l.: McKinsey Global Institute.

Narayanan, A. and Schmatikov, V., 2008 ‘Robust De-anonymization of Large Sparse Datasets’ paper given at IEEE Symposium on Security and Privacy.Washington, United States of America

Newman, J., 2011. Facebook comments Expose a Flaw in Zuckerberg's Vision [Online]. Available at: http://technologizer.com/2011/03/07/facebook-comments-zuckerberg-vision/ [Accessed 4 November 2012].

Ohm, P., 2009. Broken Promises of Privacy: Responding to the Surprising Failure of Anonymization. UCLA Law Review. 57, pp.1701.

Polonetsky, J and Tene,O. (2012). Privacy in the Age of Big Data: A Time for Big Decisions. Stanford Law Review. 64, pp.63.

Raftery, T., 2012. Sustainability, social media and big data. [Online]
Available at: http://theenergycollective.com/tom-raftery/138166/sustainability-social-media-and-big-data
[Accessed 3 November 2012].

Reeves, J., 2012. Top 10 Big Data Stocks: LinkedIn [Online] Available at: http://www.fool.com/investing/general/2012/03/06/top-10-big-data-stocks-linkedin.aspx [Accessed 4 November 2012].

Romero, R., 2011. Are insurance companies spying on your Facebook page? [Online] Available at: http://abclocal.go.com/kabc/story?section=news/consumer&id=8422388
[Accessed 2 November 2012].

Weigel, M., 2012. What is Big Data? Research roundup, reading list. [Online]
Available at: http://journalistsresource.org/studies/economics/business/what-big-data-research-roundup
[Accessed 9 October 2012].

Weinberger, D., 2012. Understanding big data vs. theory. [Online]
Available at: http://www.kmworld.com/Articles/Column/David-Weinberger/Understanding-big-data-vs.-theory-85784.aspx
[Accessed 29 October 2012].

Williams, M., 2012. Social Media Still Has Skeptics in Government [Online]. Available at: http://www.govtech.com/policy-management/Social-Media-Skeptics-Government.html [Accessed 1 November 2012].

Wolfs, F., n.d. Introduction to the Scientific Method. [Online]
Available at: https://sites.google.com/site/kapostase/home/fyr/use-of-the-scientific-method
[Accessed 30 October 2012].