Waiver of consent

Operation of the National Statement

15.22 The National Statement recognises that there may be circumstances in which the use of identifiable information in research may be justified without complying with the requirement of individual consent from all participants. This can occur in epidemiological research but is also possible for human tissue research and genetic research.

15.23 The National Statement provides that HRECs may ‘sometimes’ waive consent after taking into account a number of factors. The matters that may be taken into account are:

  • the nature of any existing consent relating to the collection and storage of the sample;
  • the justification presented for seeking waiver of consent including the extent to which it is impossible or difficult or intrusive to obtain specific consent;
  • the proposed arrangements to protect privacy including the extent to which it is possible to de-identify the sample;
  • the extent to which the proposed research poses a risk to the privacy or well being of the individual;
  • whether the research proposal is an extension of, or closely related to, a previously approved research project;
  • the possibility of commercial exploitation of derivatives of the sample; and
  • relevant statutory provisions.[36]

Waiver of consent and tissues obtained for clinical purposes

15.24 Waiver of consent issues often arise where researchers propose to access and use material collected for therapeutic or diagnostic purposes, such as pathology samples.[37] The individuals from whom such samples have been taken may not have consented to the use of these samples in any research at all, let alone in unspecified research. Human tissue collections are discussed in detail in Chapter 19.

15.25 It is easier to argue that consent should be waived for new research on tissue originally collected for research purposes than for research on tissue originally collected for other purposes. In the latter case, there can be no doubt that research is an unrelated secondary purpose rather than a purpose that is related[38] (or directly related in the case of health information) to the primary purpose and within the reasonable expectations of the individual concerned.[39] On the other hand, requirements to seek consent to the use of archival tissue will often be difficult in practice because it may no longer be possible to contact the individuals to whom the samples relate.

15.26 Where tissue derived from clinical practices is used for the purposes of quality assurance, rather than for research, consent requirements may be avoided. The Office of the Federal Privacy Commissioner’s Guidelines on Privacy in the Private Health Sector state that an organisation’s quality assurance or clinical audit activities may constitute directly related secondary purposes, for which consent may not be required.[40] The Australian Health Ethics Committee (AHEC) has recently released guidance for institutions, HRECs and health professionals on when quality assurance in health care requires ethics review.[41]

15.27 Several submissions expressed concern about the use of clinical tissue. One submission, from a woman whose family is affected by Huntington’s disease, stated:

The collection and use of human tissue samples certainly should be regulated. Tissue samples could be used for research, but the samples should be de-identified first. People who have tests done for therapeutic reasons may not be aware that they have ‘interesting’ DNA. … When these tissue samples are used for genetic research, then that is when the researchers have crossed the ‘consent’ line.[42]

15.28 The Androgen Insensitivity Syndrome Support Group Australia (AIS Support Group) submitted that HRECs should never provide a waiver of consent under the National Statement[43] but recognised that such a prohibition would present considerable difficulties in some circumstances and, in particular, in relation to the use of samples initially obtained for therapeutic or diagnostic purposes.[44]

15.29 Privacy NSW raised the specific concern that the National Statement ‘effectively dispenses with the need for fully informed consent’ where a person’s genetic information is not initially collected for research purposes but is later used for research.[45]

De-identification and consent

15.30 As discussed in Chapter 16, the National Statement makes distinctions between identified, potentially identifiable and de-identified personal information or material.[46] The concept of de-identification is relevant in several ways to issues of consent to participation in research, and to the privacy and ethical implications of consent requirements.[47]

15.31 The National Statement provides that when consent is being sought from individuals for the collection of genetic material or information, they should be informed whether their genetic material and information will be used in an identified, potentially identifiable or de-identified form.[48]

15.32 Coded data is not ‘de-identified’ for the purposes of the National Statement because coding, by its very nature, is reversible.[49] Unlike the Privacy Act, the provisions of the National Statement continue to apply to de-identified material and information. However, where information is completely de-identified it may be impossible to seek consent to its further use from individuals.[50]

15.33 The extent to which coded data can be considered to be de-identified is not entirely clear under either the Privacy Act or the National Statement. For example, where an independent person (for example, a gene trustee) holds the code and it is not possible for the researchers to ascertain the identity of the individual concerned (without the assistance of the independent person), is the information de-identified so that the Privacy Act does not govern it, or does it depend on the circumstances and, in particular, the practices or policies of the gene trustee?

15.34 The Victorian Department of Human Services Genetics Advisory Committee considered that specific consent should not be necessary for ‘de-identified research’.[51] Other submissions also suggested that specific consent to the use of clinical samples in research should not be required where the samples are de-identified.[52] These views are consistent with conclusions of the United Kingdom Human Genetics Commission, which has stated that ‘it is acceptable to seek general consent in cases where there is to be irreversible or reversible anonymisation of data and samples’.[53]

Reform of the waiver of consent provisions

15.35 DP 66 asked whether changes are needed to the National Statement or the s 95 or s 95A Guidelines, in relation to waiver of consent by HRECs.[54] In this context, it is important to appreciate that the s 95 and s 95A Guidelines apply only to Commonwealth agencies and private sector organisations, respectively. In contrast, the waiver of consent provisions of the National Statement are intended to apply to all human research conducted in Australia, notwithstanding that the National Statement lacks legislative force (see Chapter 14).

15.36 Some submissions argued for significant reform of the waiver of consent provisions in the National Statement, the s 95 and s 95A Guidelines, or both. In relation to the current operation of the waiver of consent principles in the National Statement, the Centre for Law and Genetics submitted that:

These provisions may require rewriting to emphasise that, where commercial considerations are involved, or where the sample is identified or potentially identifiable (which is usually the case), these principles should not be used to avoid the primary requirement for participant consent. The current high levels of public support for medical research may be damaged irrevocably by misuse of these provisions.[55]

15.37 The Centre for Law and Genetics also stated that the original intent of these provisions was that such applications should be ‘exceptional’.[56] Privacy NSW expressed similar concerns about the waiver provisions, observing that the National Statement states simply that an HREC may take into account the listed matters.

This is not a mandatory requirement and would be difficult to enforce, nor are the listed factors sufficient to ensure that consent is protected in all but strictly defined and exceptional circumstances.[57]

15.38 Privacy NSW emphasised that explicitly defined and enforceable ethical standards should govern the circumstances in which consent may be waived in research involving genetic information and that consent should only be waived in very limited, closely defined, circumstances.[58]

15.39 A number of submissions shared the concerns expressed by Privacy NSW about the non-mandatory nature of the factors to be taken into account in making a waiver decision. The Office of the Federal Privacy Commissioner (OFPC) stated that:

In the interests of maintaining confidence in the HREC system, consideration could be given to making it obligatory for a HREC to consider those factors.[59]

15.40 Other submissions, including from those involved in the conduct or funding of medical research, considered that the existing consent requirements in the National Statement were adequate and appropriate.[60]

15.41 Jeremy Kenner, Ethics Coordinator at the Peter MacCallum Cancer Institute (PMCI), stated that the current guidelines ‘strike the necessary balance between the interests of the research community, research participants and the public exceptionally well’.[61] The NHMRC Research Committee made a similar point. The Committee noted that HRECs are intended to function as an ‘autonomous body which is particularly responsive to the environment and community group with which it is associated’.

Reducing or changing the autonomy of the HRECs by selecting out specific issues over which that body no longer has the final say is acceptable but only if the reason is well justified.[62]

15.42 The Department of Human Services Victoria Genetics Advisory Committee stated that:

The NH&MRC’s National Statement already contains specific and we believe adequate provisions dealing with the use of human tissue samples in research. We believe that the current system of approval and oversight by Human Research Ethics Committees has served the community well, and should not be replaced by a more restrictive system.[63]

15.43 In relation to the s 95 and s 95A Guidelines, Privacy NSW submitted that the existing framework does not adequately address the specific privacy issues in relation to the use of genetic information for research.

The Guidelines only operate where consent is not sought or obtained. They only apply to the collection and disclosure of identifiable personal information. Given the difficulty in truly deidentifying genetic samples, it would generally be appropriate to treat ‘de-identified’ genetic information in accordance with the principles contained in the Guidelines and the National Statement.[64]

15.44 Privacy NSW was also critical of the ‘public interest’ test under which the s 95 and s 95A Guidelines are approved by the Privacy Commissioner and under which HRECs determine whether to approve collection, use or disclosure of health information without consent. As discussed above, the test in each case refers to whether the public interest in research ‘substantially outweighs’ the public interest in privacy.[65]

It is of concern that this effectively utilitarian test does not address the substantial ethical issues … in relation to obtaining consent in relation to genetic information. The permissive scope of the public interest test, and its application by HRECs under conditions of limited oversight, also conflicts with the need for consistent and transparent principles that are applied regardless of where information is collected, used and disclosed.[66]

Survey of HRECs

15.45 In order to assess whether there is any need to reform the waiver of consent provisions of the National Statement to take account of any special characteristics of human genetic research, the Inquiry sought to understand more fully how these provisions operate in practice.[67]

15.46 In late 2002, the Inquiry sent questionnaires to the Executive Officers of 216 HRECs, requesting information about ethics review of human genetic research proposals considered between 1 January and 31 December 2002. The questionnaire asked for the number of human genetic research proposals reviewed, approved and denied. The questionnaire then asked for information about HREC consideration of, and decisions about, requests for access to genetic samples or information without consent—that is, about HRECs’ exercise of the authority given to them by the National Statement to waive consent and, in particular, about the reasons for waiving consent.

15.47 In response, the Inquiry received 82 completed questionnaires (a response rate of 38%). Twenty-six HRECs had considered 129 proposals involving human genetic research in the reporting period. Ten HRECs had considered 22 proposals that were not designed to obtain the consent of individuals to the use of their genetic samples or information. HRECs decided to waive consent in respect of 17 human genetic research proposals. This amounts to 13% of the human genetic research proposals considered by HRECs in the reporting period, and 77% of proposals where wavier was sought.

15.48 Consultations during the Inquiry identified that the circumstances in which HRECs waive consent typically involve the use of archival tissues originally collected for therapeutic or diagnostic purposes.[68] The Inquiry noted that waiver of consent may be sought where clinical samples have been collected from a cohort of individuals with serious illnesses, such as cancer. Researchers may be reluctant to seek consent because many of the subjects of the research may have died recently, or may be dying.[69]

15.49 The survey results provided confirmation of this general picture of the circumstances in which waiver of consent is considered by HRECs. At least seven of the research proposals for which waiver was granted involved the use of archival tissue originally taken for therapeutic or diagnostic purposes. The existence of prior prospective consent for research was a factor in at least five research proposals for which waiver was granted.

15.50 Given the limited nature of the survey, and the low response rate of HRECs, the Inquiry has concluded that there is insufficient information about the operation of the waiver of consent provisions of the National Statement to justify recommending specific reforms. However, as discussed below, the Inquiry does recommend augmented reporting requirements in relation to waiver of consent. Once these reporting requirements have been in operation for a period of time, it may be appropriate for the NHMRC to review this issue.

HREC reporting of waiver of consent

15.51 HRECs report to AHEC on various matters related to their membership and procedures.[70] These reporting obligations, and proposals for their augmentation, are discussed more generally in Chapter 17. HRECs are specifically required to report annually to AHEC on decisions made under the s 95 and s 95A Guidelines.[71] However, there is no requirement for HRECs to report on waiver of consent under the National Statement itself.

15.52 As discussed above, the s 95 and s 95A Guidelines have limited operation and apply only to Commonwealth agencies and private sector organisations respectively. Much human genetic research in Australia is conducted by universities, teaching hospitals and publicly funded research institutes. Further, the extent of the existing reporting requirements under the s 95 and s 95A Guidelines have been criticised. Privacy NSW stated that

it is not clear that the reporting and auditing processes in any of these instruments are sufficiently transparent and accountable to effectively protect people’s genetic information privacy. Accordingly there is a real risk that Sections 95 and 95A will facilitate rather than restrict the conduct of research involving human genetic information other than in accordance with strict ethical principles and practices.[72]

15.53 In DP 66, the Inquiry proposed that HRECs should be required to report annually to AHEC with respect to human genetic research proposals for which waiver of consent has been granted under the National Statement.[73] There was support for this proposal in submissions.[74] For example, the Department of Health Western Australia submitted that

waiver of consent should be allowed by an HREC only after detailed and complex consideration of issues relating to appropriate compliance with the National Statement, privacy legislation and privacy guidelines. As such, compliance reporting and the potential for audit of these decisions of the HRECs are both desirable.

If there are any ongoing concerns as a result of such audit, consideration should be then given to the development in guidelines of more defined circumstances under which such waiver of consent may be applied.[75]

15.54 Jennifer Fleming submitted that:

From a risk management perspective, where protocols involving waiver of consent have been identified as approved by HRECs it would be valuable for the HREC (in consultation with the researcher(s)) to provide a brief statement of justification for that waiver. An AHEC secure database could be held to include (i) responses to the Annual Compliance Reports and (ii) a brief listing of approved protocols where rationales for waiving consent have been endorsed. This would be valuable in terms of promoting transparency of practice, monitoring and education.[76]

15.55 The Inquiry also received a range of comments on the possible content of new reporting obligations. These submissions generally anticipated that reporting would focus on the reasons HRECs grant waiver and, in particular, would address the factors set out in the National Statement and s 95 and s 95A Guidelines.[77]

15.56 The Inquiry agrees with comments made in submissions that new reporting requirements will be of limited benefit, and merely add a new burden to HRECs, unless processes are put in place to ensure that AHEC analyses the information received and provides feedback to HRECs to assist with future ethics review.[78] The Queensland Government noted that

if [reporting of waiver] is implemented because of concerns regarding the practices or accountability of the HRECs in waiving consent, further consideration should be given to the processes of establishing and administering the HRECs to ensure compliance with national principles rather than merely adding an extra reporting burden.[79]

15.57 The general requirement to obtain consent in research is fundamental to the protection of interests in privacy and ethical treatment of an individual’s genetic samples and information. The Inquiry has concluded that the transparency and accountability of the present system would be better served by requiring HRECs to report specifically on all research proposals for which waiver of consent has been granted, and not just those for which waiver has been granted under the s 95 and s 95A Guidelines. This would enable AHEC to record, assess and monitor the operation of these provisions of the National Statement, and provide feedback to HRECs.

15.58 As an interim measure, the existing provisions of the National Statement could be used as the mechanism for extended reporting of waiver of consent. HRECs are currently required to provide information from their records to the NHMRC on request.[80]

15.59 The Inquiry also notes that the relevant provisions of the National Statement pre-date the enactment of the Privacy Amendment (Private Sector) Act 2000 (Cth) and the approval of the s 95A Guidelines. For this reason, the NHMRC should review the provisions of the National Statement relating to waiver of consent and reporting of decisions, to ensure they are consistent with privacy law.

15.60 The Inquiry further recommends that the NHMRC, in strengthening the level of training and other support provided to HRECs, in accordance with the recommendations in Chapter 17, should ensure that attention is given to the interpretation of the waiver of consent provisions and HREC decision making in relation to waiver.

Recommendation 15–1 The National Health and Medical Research Council (NHMRC), as part of its review of the National Statement on Ethical Conduct in Research Involving Humans (the National Statement) in the 2003–2005 triennium, should amend the National Statement to provide that Human Research Ethics Committees (HRECs) must report annually to the Australian Health Ethics Committee (AHEC) with respect to human genetic research proposals for which waiver of consent has been granted under the National Statement. Until such time as the National Statement has been so amended, the NHMRC should exercise its existing authority to request information from HRECs to require them to report annually to AHEC with respect to such human genetic research proposals.

Recommendation 15–2 The NHMRC, as part of its review of the National Statement in the 2003–2005 triennium, should ensure that the provisions of the National Statement relating to waiver of consent and reporting of decisions are consistent with privacy laws and, in particular, with guidelines issued under s 95 and s 95A of the Privacy Act 1988 (Cth).

Recommendation 15–3 The NHMRC, in strengthening the level of training and other support provided to HRECs in accordance with Chapter 17 of this Report, should ensure that adequate attention is given to: (a) the interpretation of the waiver of consent provisions of the National Statement; and (b) HREC decision making in relation to such waiver.

[36] National Health and Medical Research Council, National Statement on Ethical Conduct in Research Involving Humans (1999), NHMRC, Canberra [15.8].

[37] It should be understood that the vast majority of stored clinical samples are never used for research but are retained, in accordance with industry standards and guidelines, for quality assurance, medico-legal and other reasons.

[38] Research may be a related purpose where it is connected with the clinical care of the patient: ‘[t]here is no dividing line between clinical care and research and the clinician/scientist has a duty to his/her patient to conduct every possible test to find the cause and best treatment for a disease’: Australian Academy of Science, Submission G097, 21 January 2002.

[39] That is, in terms of Privacy Act 1988 (Cth) NPP 2.1. It has been suggested that more research is needed on community attitudes to the use of genetic information without specific consent: L Skene, ‘The Genetics Debate: Why Doctors Must be Heard’ (Paper presented at 2001 UMMS Lecture, 15 November 2001). The article refers to the unpublished results of a survey conducted by Patterson and Gillam at ethics committee seminars in which 29% of 177 respondents stated that they wanted to be contacted for consent to the use of their stored tissue in research, 40% if the research was on a disease that affected them and 52% if there was a possibility of commercialisation.

[40] Office of the Federal Privacy Commissioner, Guidelines on Privacy in the Private Health Sector (2001), OFPC, Sydney, 14–15.

[41] Australian Health Ethics Committee, When does Quality Assurance in Health Care Require Independent Ethical Review? (2003), AHEC, Canberra.

[42]Confidential Submission G051CON, 14 January 2002.

[43] Androgen Insensitivity Syndrome Support Group Australia, Submission G290, 5 January 2003.

[44] Androgen Insensitivity Syndrome Support Group Australia, Submission G106, 26 February 2002.

[45] Office of the Privacy Commissioner (NSW), Submission G118, 18 March 2002.

[46] National Health and Medical Research Council, National Statement on Ethical Conduct in Research Involving Humans (1999), NHMRC, Canberra, 9.

[47] Issues concerning the de-identification of genetic samples and the coverage of the Privacy Act 1988 (Cth) are discussed in Ch 8.

[48] National Health and Medical Research Council, National Statement on Ethical Conduct in Research Involving Humans (1999), NHMRC, Canberra [16.10(b)].

[49] Although one of the factors that may be taken into consideration by an HREC is determining whether consent should be waived is the ‘extent to which it is possible to de-identify the sample’—possibly suggesting that de-identification need not be complete anonymisation: Ibid [15.8], [16.13].

[50] However, consent may still need to be sought from collectivities where research involves a collectivity. See Ibid, Ch 8.

[51] Department of Human Services Victoria Genetics Advisory Committee, Submission G089, 24 January 2002.

[52] Association of Australian Medical Research Institutes, Submission G007, 27 April 2001; Human Genetics Society of Australasia, Submission G050, 14 January 2002; Confidential Submission G051CON, 14 January 2002.

[53] Human Genetics Commission, Inside Information: Balancing Interests in the Use of Personal Genetic Data (2002), London, 95.

[54] Australian Law Reform Commission and Australian Health Ethics Committee, Protection of Human Genetic Information, DP 66 (2002), ALRC, Sydney, Question 12–2.

[55] Centre for Law and Genetics, Submission G048, 14 January 2002.

[56] Ibid.

[57] Office of the Privacy Commissioner (NSW), Submission G118, 18 March 2002.

[58] Ibid.

[59] Office of the Federal Privacy Commissioner, Submission G294, 6 January 2003.

[60] Peter MacCallum Cancer Institute, Submission G028, 20 December 2001; Department of Human Services Victoria Genetics Advisory Committee, Submission G089, 24 January 2002; National Health and Medical Research Council Research Committee, Submission G128, 18 March 2002; Human Genetics Society of Australasia, Submission G267, 20 December 2002; Cancer Council Victoria Cancer Genetics Advisory Committee, Submission G195, 27 November 2002; NSW Health Department, Submission G303, 13 January 2003.

[61] Peter MacCallum Cancer Institute, Submission G028, 20 December 2001.

[62] National Health and Medical Research Council Research Committee, Submission G128, 18 March 2002.

[63] Department of Human Services Victoria Genetics Advisory Committee, Submission G089, 24 January 2002.

[64] Office of the Privacy Commissioner (NSW), Submission G118, 18 March 2002. The Law Institute of Victoria agreed with these views: Law Institute of Victoria, Submission G275, 19 December 2002.

[65]Privacy Act 1988 (Cth) ss 95(2), 95A(3); National Health and Medical Research Council, Guidelines Under Section 95 of the Privacy Act 1988 (2000), NHMRC, Canberra [3.2(b)]; National Health and Medical Research Council, Guidelines Approved Under Section 95A of the Privacy Act 1988 (2001) National Health and Medical Research Council [D.4].

[66] Office of the Privacy Commissioner (NSW), Submission G118, 18 March 2002.

[67] The NHMRC Research Committee referred to the importance, in this context, of knowing how many waivers have actually been approved by HRECs: National Health and Medical Research Council Research Committee, Submission G128, 18 March 2002.

[68] HREC Chairs and Officers, Consultation, Sydney, 20 June 2002.

[69] Ibid. In such circumstances, researchers may submit that it is ‘impossible or difficult or intrusive’ to obtain consent in terms of the National Statement: National Health and Medical Research Council, National Statement on Ethical Conduct in Research Involving Humans (1999), NHMRC, Canberra [15.8], [16.13]. However, the mere fact that distress may be caused may be insufficient to avoid the consent requirements of the Privacy Act, which requires that it be impracticable to collect health information without consent: Privacy Act 1988 (Cth) NPP 10.3(c). The OFPC emphasised that what may be considered to be impracticable is a ‘critical consent issue’: Office of the Federal Privacy Commissioner, Submission G294, 6 January 2003.

[70] See Australian Health Ethics Committee, AHEC HREC Annual Report Form, <www.nhmrc.gov.au/issues/pdfcover/annrpt.htm>, 20 February 2003.

[71] National Health and Medical Research Council, National Statement on Ethical Conduct in Research Involving Humans (1999), NHMRC, Canberra [2.46]–[2.48]; National Health and Medical Research Council, Guidelines Under Section 95 of the Privacy Act 1988 (2000), NHMRC, Canberra [3.4]; [4.1]–[4.3]; National Health and Medical Research Council, Guidelines Approved Under Section 95A of the Privacy Act 1988 (2001) National Health and Medical Research Council [D.6], [E.1]–[E.3].

[72] Office of the Privacy Commissioner (NSW), Submission G118, 18 March 2002.

[73] Australian Law Reform Commission and Australian Health Ethics Committee, Protection of Human Genetic Information, DP 66 (2002), ALRC, Sydney, Proposal 12–1.

[74] Law Institute of Victoria, Submission G275, 19 December 2002; Office of the Federal Privacy Commissioner, Submission G294, 6 January 2003; Department of Human Services South Australia, Submission G288, 23 December 2002; Cancer Council Victoria Cancer Genetics Advisory Committee, Submission G195, 27 November 2002; Children’s Cancer Institute Australia, Submission G221, 29 November 2002; Commonwealth Department of Health and Ageing, Submission G313, 6 February 2003; Centre for Law and Genetics, Submission G255, 21 December 2002; Centre for Genetics Education, Submission G232, 18 December 2002; Genetic Support Council WA, Submission G243, 19 December 2002; J Fleming, Submission G241, 20 December 2002; Department of Health Western Australia, Submission G271, 23 December 2002; Human Genetics Society of Australasia, Submission G267, 20 December 2002; Australian Biospecimen Network, Submission G238, 19 December 2002; Association of Genetic Support of Australasia, Submission G284, 25 December 2002; Australian Red Cross Ethics Committee, Submission G292, 6 January 2003; NSW Health Department, Submission G303, 13 January 2003; Office of the Health Services Commissioner Victoria, Submission G307, 17 January 2003. The AIS Support Group suggested that HRECs should be required to provide written advice to AHEC and to the HGCA within 28 days of making a decision to grant approval to conduct research with a waiver of consent: Androgen Insensitivity Syndrome Support Group Australia, Submission G290, 5 January 2003. The Department of Human Services South Australia suggested that HRECs should also report the number of genetic research proposals they receive. The proposal was opposed by Medicines Australia: Medicines Australia, Submission G302, 12 February 2003.

[75] Department of Health Western Australia, Submission G271, 23 December 2002.

[76] J Fleming, Submission G241, 20 December 2002.

[77] Centre for Law and Genetics, Submission G255, 21 December 2002; Office of the Federal Privacy Commissioner, Submission G294, 6 January 2003; Queensland Government, Submission G274, 18 December 2002.

[78] Queensland Government, Submission G274, 18 December 2002; NSW Health Department, Submission G303, 13 January 2003.

[79] Queensland Government, Submission G274, 18 December 2002.

[80] National Health and Medical Research Council, National Statement on Ethical Conduct in Research Involving Humans (1999), NHMRC, Canberra [2.47].