Data collection

The need for data

3.4 Accurate and up-to-date data on the costs associated with discovery in Federal Court proceedings, and the extent to which discovered documents are used in the resolution of those proceedings, would provide a sound basis upon which to respond to concerns about the high costs of the discovery process in some matters and a sense that these costs are in some cases ‘disproportionate’—as noted in the opening paragraph of the Terms of Reference in this Inquiry.

3.5 It is not just the amount of money spent on discovery that has raised concern. Rather, it is the ‘low value for money’ that prompts criticism of the discovery process—in terms of the cost of discovery relative to the utility of discovered documents in the context of the litigation.

3.6 Statistical data on discovery costs in Australia—and research measuring the extent to which discovered documents are used in the disposal of litigation before Australian courts—has not been collected or recorded in a systematic or ongoing manner. As a consequence, accurate and up-to-date information to inform an assessment of the proportionality of discovery processes in federal courts is not readily available.

3.7 With a lack of accurate data there may be a distorting effect of perceptions as to cost, created by cases such as Seven Network Limited v News Limited (C7).^[1] In a submission to this Inquiry, Michael Legg remarked that:

Responses to the questions posed by the ALRC’s Discussion Paper may also be influenced by a view that the experience in the C7 litigation or that the New South Wales Supreme Court Chief Justice Spigelman’s anecdote of the flag-fall for discovery in a significant commercial dispute being often $2m, are the norm. We currently have no reliable evidence as to whether these examples are representative or are anomalies.^[2]

3.8 The collection of such data may provide an informative measure of the concerns associated with discovery in Federal Court proceedings. Such information may help to guide the direction of future reform in this area of civil litigation. It may also enable a basis for comparison to measure the effectiveness of the recommendations for reform in this Report, should they be implemented. Reliable statistics would be helpful to assess accurately whether these proposed procedures were successful in achieving the objective of reducing litigation expenses overall and achieve a greater level of certainty in the conduct of a discovery process—in terms of planning in advance the types of searches that are ‘reasonable’. For example, data which allowed a comparison of estimated discovery costs with the actual costs of discovery may indicate the level of certainty in a discovery process.

Existing findings

3.9 An extensive survey of the cost of litigation in the Federal Court conducted during the ALRC’sinquiry, Managing Justice: A Review of the Federal Civil Justice System, ALRC Report 89 (2000) (Managing Justice), asked solicitors to estimate the total legal costs of discovery for cases in the Federal Court. The results showed that the costs of discovery varied according to the complexity of the issues involved. For example, the range of costs for obtaining discovery from another party was $500–$750,000 for applicants and $200–$311,000 for respondents, while the range of costs for complying with discovery requests was $200–$400,000 for applicants and $300–$120,000 for respondents.^[3]

3.10 Since the Managing Justice report, the range of material potentially to be discovered has increased exponentially through advancing computer technologies, as noted above, with a significant increase in discovery costs. Electronic communications can be inherently expensive to discover, in part due to the cost of specialist service providers with expertise in computer technologies. For example, Lord Jackson’s Review of Civil Litigation Costs reported that typical service charges for e-discovery include: electronic document processing (extracting metadata, text, attachments etc, for use on a document review system) £250–£1,000 per gigabyte of data, document hosting on a review system at £20–£150 per gigabyte per month and a user access fee between £10–£100 per user.^[4]

3.11 Electronic discovery costs can also include expensive computer software and hardware. For example, the discovery of information stored on old backup tapes can require the reconstruction of outmoded hardware at great expense in order to read the tapes only to discover completely irrelevant information.

3.12 Another relevant study, of discovery practices in the United States in 1997, looked at the cost of discovery relative to the information needs of the case. This research found that most attorneys surveyed (69%) thought that the discovery generated by the parties was about the right amount needed for the fair resolution of the case.^[5]

What is measured and for what purpose?

3.13 Key questions in any data collection exercise are precisely what is to be captured, how is the data to be collected, for what purpose, and how is the data to be used for policy development and law reform. A further issue relates to any evaluative cycle or reflective phase of the process of data collection itself.

3.14 The collection of data on the proportionality of discovery costs would be continuous and incremental, rather than a one-off study over a finite period. Such data would be most meaningful when its collection is built into the justice system as a small but regular collation of relevant and relatable information. In this way, the evidence base of discovery data would become richer over time and the costs associated with its collection would be distributed over time and between all persons involved with civil litigation—such as: the parties, their lawyers, litigation service providers and the courts.

3.15 A range of data collection points would include, for example, information about: the anticipated costs of discovery; the actual costs of discovery; the damages, if any, awarded in the relevant litigation; whether the action was settled and, if so, whether it settled before or after discovery; the number of documents involved in discovery and the number of documents considered relevant to the actual resolution of the dispute; and so on.

3.16 An assessment of ‘proportionality’ could be considered from a number of perspectives with such information available. A driving concern in relation to discovery is the proportionality of discovery costs—particularly in terms of the extent to which discovered documents are used to facilitate the just disposal of litigation. Such a concern was taken up by the Access to Justice Taskforce which stated that:

The cost of discovery continues to be very high, and often disproportionate to the role played by discovered documents in resolving disputes.^[6]

3.17 The idea of proportionality, as outlined above, is a key reform principle in this Inquiry, as embodied in Federal Court objectives in the expression of the ‘overarching purpose’ provision, s 37M of the Federal Court of Australia Act, considered above. However, the collection of accurate and meaningful data on the proportionality of discovery processes is likely to present significant challenges. It will require the cooperation and input of all those involved in a civil proceeding, including the court, the parties, their lawyers, any litigation support service providers and financiers such as insurers or litigation funders. Establishing a central point for the collection of data from every participant in a civil proceeding, with respect to discovery costs, may present logistical issues including, for example, the protection of confidential information.

3.18 Quantifying the utility of discovered documents may also raise a particular challenge in the collection of this data. Recording the number of discovered documents tendered in evidence or relied upon at hearing may misrepresent the utility of discovery—since the objectives of discovery extend to clarifying the issues in dispute and testing the strength of each party’s case. Discovered documents, therefore, may have value in facilitating settlement of the proceeding or shortening the length of the trial by encouraging parties to agree on certain issues. On the one hand, this may suggest that measuring the extent to which discovered documents are actually used in the disposal of litigation would depend on the impressions of the parties or their lawyers, rather than exact numerical or monetary terms. On the other hand, empirical data as to when settlement occurred in proceedings, for example, could support an interpretation as to the value of discovery. Where data shows that proceedings settled after discovery, this might support further enquiry as to whether discovered documents were influential in the settlement of those cases. Alternatively, where data identifies proceedings that settled before discovery was given, this might support enquires as to the reasons for settlement in the absence of relevant documents.

3.19 The type of data that may assist to evaluate and track the proportionality of discovery processes in the Federal Court may include:

• the total litigation costs and the amount of costs associated with discovery, as well as the items of expenditure on discovery, for example, legal fees and court filing fees for discovery applications, the cost of time spent at trial considering discovered documents, the cost of litigation support services in the discovery process and the cost to the parties of employees engaged in a discovery process—this may indicate where costs are incurred in discovery, and those aspects which are most costly, in the context of litigation costs overall;

• the value of what is at stake for the parties in the litigation, for example, the amount of damages awarded in judgement, the sum of compensation paid by way of settlement, or the approximate value of non-pecuniary relief such as a declaration or injunction—this may provide context to discovery costs, as a proportion of the value of the case;

• the number of discovered documents that are tendered in evidence, and the number of documents relied upon at trial, as well as the judge’s impression of the extent to which discovered documents were crucial in determining the proceeding;

• whether settlement was achieved after discovery, and the parties’—and their lawyers’—impression of the extent to which discovered documents were crucial in resolving the dispute;

• whether certain issues in dispute were narrowed or agreed upon after discovery, and the parties’—and their lawyers’—impression of the extent to which discovered documents were crucial in clarifying or resolving those issues.

3.20 What all of this suggests is that an aspiration, or ‘overarching purpose’ of proportionality, while a useful guide or principle, is difficult to measure, and that there are dangers in relying upon assertions or anecdotes of disproportionate costs as a basis of propelling major shifts in practice—and law reform recommendations.

Data collection initiatives

3.21 The Australian Government Attorney-General’s Department’s Access to Justice Taskforce, A Strategic Framework for Access to Justice in the Federal Civil Justice System (Strategic Framework)^[7] identified the problem that ‘insufficient statistical data is available to make comprehensive decisions about access to justice’:^[8]

Statistics are inconsistently collected and reported, and significant gaps remain … Data is necessary not only to the institution to identify and act on problem issues, but also to inform analysis and understanding (undertaken by agencies, academics, the public) regarding the performance of the justice system generally.^[9]

3.22 In order to establish the appropriate framework for data collection, the Taskforce recommended that:

the Attorney-General’s Department should work with the federal courts, tribunals, and other justice services to develop an overarching data collection template to inform the necessary collection of data on a comprehensive, consistent basis.^[10]

3.23 The data collection template was to be based on the outcomes of a review of the efficiency of the courts and tribunals in the context of the civil justice system in Australia. The scope of the review would be ‘identification of relevant measures and data requirements necessary for ongoing monitoring of the justice system’.^[11]

Submissions and consultations

3.24 In the Consultation Paper, the ALRC proposed that the Australian Government should fund initiatives in the Federal Court to establish and maintain data collection facilities, to record data on the costs associated with discovery of documents, as well as information on the proportionality of a discovery process—in terms of the costs of discovery relative to the total litigation costs, the value of what is at stake for the parties in the litigation, and the utility of discovered documents in the context of the litigation.^[12]

3.25 The ALRC suggested that the Federal Court would be best placed to collect such data. However, the participation of the parties, their lawyers and others involved in the proceeding would be required to gather this data effectively. The ALRC also acknowledged that the Court may require additional funding to establish and maintain data collection facilities to measure the proportionality of discovery processes. Proportionality in this sense may be difficult to measure. The ALRC also noted that participants at a discovery seminar convened by the Australian Institute of Judicial Administration (AIJA) in 2007 suggested that the AIJA should undertake a research project to track how many discovered documents are in fact used in litigation.^[13] However, the ALRC understands that such research has not yet been undertaken.

3.26 All submissions that addressed this proposal supported it,^[14] and expressed considerable support—at the general level—for the proposal for data collection as a basis for supporting law reform, and not just in the field of discovery. However there were also comments about the difficulty of a data collection exercise of this kind.

3.27 The Litigation Law and Practice Committee of the Law Society of New South Wales, for example, considered that ‘the collection of empirical data is essential to the accurate assessment of the discovery process’.^[15] As noted in Chapter 1, Michael Legg, of the University of New South Wales, submitted that

In many ways proposal 3–7 is the most significant suggestion in the Discussion Paper as it would allow reform in the Federal Courts to be driven by fact rather than fashion.^[16]

3.28 Legg also commented that ‘many of the questions and issues … lend themselves to empirical study and would benefit from data collection facilities’.^[17]

3.29 The Law Council of Australia (Law Council) and the Australian Corporate Lawyers Association (ACLA) both expressed support in general terms for a careful analysis of the costs associated with discovery in the Federal Court,^[18] but added riders to their comments. The Law Council cautioned that :

the study must be undertaken by experts as the conclusions, if they are to be relied on, must be sound. If practitioners are involved in such a process, due consideration must be given to the time and cost involved in retrieving the data, particularly for small firms.^[19]

3.30 The difficulties that the exercise of data collection might involve were also singled out by ACLA:

it would be virtually impossible to obtain a sufficiently accurate data sample to make any worthwhile conclusions. Given that a significant majority of matters settle, it is unlikely that a meaningful sampling exercise could be undertaken without significant input from litigants.^[20]

3.31 Although the data collection project would be a challenging one, NSW Young Lawyers submitted that ‘the collection of empirical data is essential to the accurate assessment of the substantive impact of any scheme, and is a useful process in its own right’:

a data collection process could, at the very least, provide useful information on average discovery costs per document reviewed or reviewable. This information could lead to a degree of predictability and cross-comparability in large discoveries. Further, it may be difficult to predict in advance what uses new data may be put to, and it would be imprudent to miss the chance to collect such information.^[21]

3.32 As to what would be measured and for what purpose, the Public Interest Advocacy Centre (PIAC), argued that any such exercise should not just be about discovery:

PIAC is of the view that data collection about the discovery process in the Federal Court should be part of a comprehensive and ongoing review of the federal civil justice system as it is important that questions about the cost of discovery be weighed against issues such as equity and perception of justice.

In this respect, PIAC notes the [Access to Justice] Taskforce Report’s recommendation about the monitoring and review of the federal civil justice system was significantly broader: it recommended that the Productivity Commission undertake a review of the efficiency of the courts and tribunals and based on this review, the Attorney-General’s Department should work with the federal courts, tribunals, and other justice services to develop an overarching data collection template to inform the necessary collection of data on a comprehensive, consistent basis.^[22]

ALRC’s views

3.33 Developing the appropriate data collection process that would lead to meaningful conclusions about matters such as the cost of discovery in the context of the civil justice system and, for example, whether it is ‘proportionate’ or not, is a complex problem. The ALRC acknowledges that significant initiatives are in train in the field of data collection, in particular in consequence of the release of the Strategic Framework. There is considerable work to be done in terms of developing the proposed template for data collection, to secure agreement as to a common data set or dictionary of indicators, measures, terms and the kinds of matters that can be measured.

3.34 The ALRC considers that such initiatives will provide the appropriate basis for the development of data to inform future reform and to evaluate the existing practice in the federal courts. Data collection needs to be undertaken in such a coordinated and informed way—and properly resourced—and the ALRC commends the development of a data collection template to inform the necessary collection of data on a comprehensive, consistent basis, as recommended by the Access to Justice Taskforce.

3.35 Once the template is developed and the data collection project can commence on an informed basis, the ALRC understands that the data collection itself may impose an ongoing burden, for example on the Federal Court, and the matter of providing appropriate resources so that this can be undertaken systematically will need to be considered by the Australian Government at that time. However, the data collection template may be constructed in a way that builds on existing features of the justice system and distributes the cost across participants, thereby minimising the burden on individual contributors.