AustLII Home | Databases | WorldLII | Search | Feedback

Current Issues in Criminal Justice

Current Issues in Criminal Justice (CICrimJust)
You are here:  AustLII >> Databases >> Current Issues in Criminal Justice >> 2017 >> [2017] CICrimJust 1

Database Search | Name Search | Recent Articles | Noteup | LawCite | Author Info | Download | Help

Marchetti, Elena --- "Nothing Works? A Meta-Review of Indigenous Sentencing Court Evaluations" [2017] CICrimJust 1; (2017) 28(3) Current Issues in Criminal Justice 257


Nothing Works? A Meta-Review of Indigenous Sentencing Court Evaluations

Elena Marchetti[*]

Abstract

Indigenous-focused criminal justice programs are usually evaluated and studied without an adherence to, or acknowledgement of, Indigenous epistemologies, axiologies and ontologies. Indeed, many of the evaluations conducted of Australian Indigenous sentencing courts have relied on quantitative analyses of reoffending, finding little or no impact on recidivism, despite there being some evidence, derived mainly from qualitative analyses, that they have had an impact on strengthening informal social controls within Indigenous communities. This article uses published evaluations and impact studies of Indigenous sentencing courts as a case study to gain a better understanding of how these courts have been evaluated and researched, and how the methodological approaches used to study the courts may not properly capture the Indigenous-focused and community-building aims and goals of the programs. Employing a meta-review approach, the article examines why research that evaluates Indigenous-focused criminal justice programs should rethink how evaluations are framed and conducted when trying to determine ‘what works’.

Keywords: Indigenous-centric evaluation – Indigenous sentencing courts – Indigenous – sentencing – meta-review

Introduction

In recent times, there has been a greater emphasis on ensuring criminal justice programs are culturally sensitive and inclusive of Indigenous[†] epistemologies, axiologies and ontologies[‡] when they are specifically targeting Indigenous offenders or victims of crime. However, when it comes to evaluating Indigenous-focused programs it appears that little thought is given to how the measures and methods reflect Indigenous-centric values and knowledges. Evaluations of even the most well-considered Indigenous-focused criminal justice programs continue to find that such programs have little or no impact on outcomes such as recidivism rates (Beranger, Weatherburn and Moffatt 2010). But are these findings an accurate reflection of program ineffectiveness or the consequence of how the research was carried out? Are the programs achieving outcomes that are not measured by policy-driven evaluations, leaving policy-makers none the wiser about how Indigenous-focused criminal justice programs might assist in addressing Indigenous incarceration and over-representation in the criminal justice system?

Numerous studies and government policies have tried to curb Indigenous over-representation in the Australian criminal justice system, but as yet we have seen little change in incarceration and offending rates primarily because the reasons why Indigenous peoples so often come into contact with the criminal justice system are extremely complex (Morgan and Louis 2010). Recent Australian Bureau of Statistics data indicate that Indigenous peoples make up 3 per cent of the total Australian population (Australian Bureau of Statistics 2013), but in 2014 their rate of imprisonment was 13 times higher than the aged standardised imprisonment rate for non-Indigenous peoples and they accounted for 27 per cent of all sentenced prisoners (Australian Bureau of Statistics 2014). Indigenous disadvantage in the areas of health, education and employment, substance abuse, housing, and access to services, and the effects of intergenerational traumas such as the forced removal of children from their families, have all been identified as contributing to the disproportionate rates of Indigenous offending (Morgan and Louis 2010). Systemic and institutional racism also plays a part (Blagg et al 2005). There is no disputing the fact that focusing on ‘front-end’ (socio-economic and early prevention) policies is crucial in changing this disproportionality (Blagg et al 2005:166; Daly and Proietti-Scifoni 2009:8). However, ‘back-end’ (rehabilitation and criminal justice) policies are just as important if the aim is to provide people with a ‘second chance’ for change and avoid deepening their criminalisation. For example, Beranger, Weatherburn and Moffatt (2010:1) propose that ‘[e]fforts to reduce Indigenous over-representation in the criminal justice system should be focussed on offender rehabilitation and assistance in promoting compliance with court orders’ since reducing the rate of Indigenous reappearance in court has an impact on recidivism. There is, however, little sense in continuing to implement front- or back-end programs aimed at reducing Indigenous over-representation if we are unable to fully understand whether, and the extent to which, they are achieving their stated objectives. This article explores and critiques the manner in which Indigenous-focused criminal justice programs are typically evaluated and researched, by using Indigenous sentencing court evaluations and impact studies as a case study.

Government-directed research that monitors Indigenous over-representation in the criminal justice system ‘and evaluations of programs and initiatives that seek to “close the gap” between Indigenous and non-Indigenous people’ frequently relies on ‘secondary analysis of administrative data and national surveys’ (Putt 2013:1–2). Such research is often contracted to independent consultancy firms and requires specific work to be completed within limited timeframes and budgets. This leaves little opportunity for ‘collaborative and participatory research methods’ (Putt 2013:4), methods that have been recognised as better suited to a research agenda involving Indigenous participants. Indigenous scholars such as Tuhiwai-Smith (1999) and Moreton-Robinson and Walter (2009) have proposed post-colonial and decolonising methodology frameworks to describe how Indigenous-centric research could be carried out, but their work, although conceptually and theoretically informative and imperative, has not translated into different modes of evaluating criminal justice programs. Cunneen and Rowe (2014:52) note, ‘[p]ositivist approaches in criminology and criminal justice social work understand racial or ethnic over-representation as the result of essentially individualized factors that can be determined from aggregate populations’, which is in stark contrast to the ways ‘anti-colonial, post-colonial and Indigenous perspectives’ approach an analysis of offending rates.[§] For criminology and social work, ‘theor[ies] of colonialism and its effects’ tend to be omitted from evaluative and analytical frameworks (Cunneen and Rowe 2014:51). Additionally, Western positivist methods of evaluation often focus on measures and criteria that are difficult to apply within a non-mainstream setting and may not reflect Indigenous cultural values and aspirations for the programs being studied (Walter 2010). Ensuring Indigenous-focused criminal justice programs are evaluated in ways that take account of the specificities of the program, the cultural context of the innovative justice process and the limitations of using comparative court and offending history data is therefore of crucial importance (Daly and Proietti-Scifoni 2009:11; Cunneen and Rowe 2014).

Work along these lines has been carried out in health services (see, for example, the work of Wilson (2001), but not so much in criminal justice, although recognition of differences between Indigenous and non-Indigenous peoples’ epistemology, ontology, axiology and methodology does exist (for a detailed discussion of these differences see Cunneen and Rowe 2014). Indeed, government agencies are raising questions about how to adapt evaluation practices for Indigenous polices as evidenced by the Australian Productivity Commission’s roundtable in late 2012 to examine ‘Better Indigenous policies: the role of evaluation’. The roundtable sought to:

identify principles and practices to guide the better use of evaluations in Indigenous policy in the future — so that more high-quality evaluations will be undertaken and the evidence gained from them will drive policy improvements that benefit Indigenous people and the wider community (Australian Government Productivity Commission 2013:iii).

Presenters identified the following limitations associated with using more conventional methods of evaluation:

• Conventional evaluation methods fail to implement and adopt principles of ‘self-determination, governance and development’, which are principles that underpin the individual and collective rights contained in the United Nations Declaration on the Rights of Indigenous Peoples (Malezer 2013:72).

• Impact evaluations and research suffered from several limitations, including a lack of reliable data, an inability to utilise randomised control groups, statistical difficulties when it came to comparisons and determining causal links, and resource constraints in conducting the evaluation (Cobb-Clark 2013).

• Statistical data that focus on populations do not provide community-level information required to inform local Indigenous-controlled programs (Taylor 2013).

In Australia, there has been little or no attempt to understand what this all means in practice for evaluations seeking to measure the outcomes of Indigenous-focused criminal justice programs. Although it could be claimed Australia leads the world in court innovation when it comes to Indigenous sentencing court programs (Marchetti and Downie 2014), other colonised countries are ahead when it comes to ‘researching from an indigenous paradigm’ (Wilson 2001:175). For example, in New Zealand, the Social Policy Evaluation and Research Committee (‘SPEaR’), a cross-agency group established by the New Zealand Government in 2001, has established best-practice Māori guidelines to ‘assist in setting standards for good practice in New Zealand social policy research and evaluation’ (SPEaR and ANZEA 2007:3; see also SPEaR 2008). Independent evaluation contractors working for the Aotearoa/New Zealand government sector are expected to adopt the principles reflected in the guidelines when conducting their studies. But even researchers from other countries have noted that ‘[t]here are disappointingly few independent academic studies of the evaluation function and its impact on Indigenous communities’ (Abele 2013:196), and that ‘[a]lthough an evaluator or evaluation team may start with strong intentions of observing the advice and direction of ... guidelines, they often fall short of what is outlined as good practice’ (Roorda and Peace 2009:73–4).

To better understand to what extent conventional measures and methods can capture the Indigenous-centric aims and goals of culturally sensitive criminal justice programs, we must first understand and appreciate what previous evaluations of Indigenous-focused criminal justice programs have been able to achieve. Using published evaluations and impact studies of Indigenous sentencing courts as a case study, the following discussion employs a meta-review approach to: gain a better understanding of how these courts have been evaluated; uncover what methodological problems, if any, were encountered in conducting the evaluations; and summarise and critically review the findings of the studies. From this review, conclusions may be drawn about the efficacy of Indigenous sentencing courts beyond the Western scientific tradition of relying on a reduction in recidivism as the main outcome measure of ‘success’.

Indigenous sentencing court evaluations

‘Indigenous sentencing courts’ are courts established by way of an agreed practice or protocol in regional towns or cities that involve the usual participants in a conventional sentencing process (such as the judicial officer, defence lawyer and prosecutor), as well as respected members from the Indigenous community within which the court sits. They are governed by Australian state and territory criminal laws and procedures, rather than Indigenous customary laws. These courts were first established in Port Adelaide, a suburb of Adelaide, South Australia on 1 June 1999 and today they operate under varied legislative frameworks and with differing eligibility criteria in most states and territories around Australia (Marchetti and Ransley 2014). They were initially established to operate at a Magistrates’ (or lower) Court level but they have also been operating at a County (District) Court level in Victoria, and in Children’s Courts in Queensland, New South Wales, Victoria and the Australian Capital Territory. In addition, in South Australia, criminal courts at all levels can now convene an Aboriginal sentencing conference prior to sentencing, or an Aboriginal conference, which is convened out of court without the presence of a magistrate (Courts Administration Authority of South Australia 2012).

The courtroom setting is quite different to mainstream courts, and most jurisdictions have remodelled or built new court rooms to house the Indigenous sentencing courts. There is more focus on dialogue, so judicial officers no longer sit on an elevated bench, but rather around a circle or an oval bar table with the offender, their support person (if one has attended), Elders, the prosecutor and defence lawyer. The involvement of the Elders varies between courts, but in all courts the Elders will speak frankly with the offender. Practices among the courts vary, but there are some common elements: (1) the offender must be Indigenous (or in some courts, Indigenous or South Sea Islander) and must have entered a guilty plea or have been found guilty in a summary hearing;[**] (2) the offender consents to having the matter heard in the Indigenous sentencing court; (3) the charge must fall within the jurisdiction of the mainstream court of equivalent level; and (4) the Magistrate retains the ultimate power in sentencing the offender.

Over the past ten years there have been a number of evaluations and impact studies conducted considering the effect these courts have on various outcomes, such as recidivism, penalties imposed and strengthening Indigenous communities. It is not easy to compare the findings of all these studies as they are mostly jurisdiction specific, and because they all identify limitations in the manner in which the data were either collected or analysed. Many have used quantitative studies of reoffending, finding little or no impact on recidivism as a result of the introduction of such courts despite there being some evidence that they have had an impact on strengthening informal social controls within Indigenous communities (Fitzgerald 2008; Morgan and Louis 2010; Beranger, Weatherburn and Moffatt 2010; Borowski 2010, 2011). What makes research that evaluates the impact of Indigenous sentencing courts particularly difficult is that it is usually aligned with an assessment of whether the aims of the courts have been achieved, which, according to Stobbs and MacKenzie (2009:94) are ‘fairly broad’ and ‘may appear aspirational and ambitious’, often including claims relating to a reduction in reoffending or imprisonment rates.

Several researchers who have evaluated Indigenous sentencing courts regard a focus on recidivism as limited claiming that it should be viewed as just one measure of success (Harris 2004; Potas et al 2003; Cultural & Indigenous Research Centre Australia (‘CIRCA’) 2008). Additionally, recidivism analyses have been criticised for using inappropriate comparison groups and inadequate follow-up periods, for incorrectly using the number of matters or files instead of defendants, and for reaching conclusions based on insufficient reoffending data or without a comparative control group. Despite these limitations and despite evaluations of criminal justice programs identifying positive outcomes apart from a reduction in recidivism, policy-makers tend to rely on recidivism analyses when making funding decisions, leaving programs at risk of closure when such measures generate less than satisfactory results. Recent examples of this can be found in the de-funding of the Queensland Murri Courts in 2012 (although they were refunded in 2016 by the newly elected Labor Government) and of the Western Australian Kalgoorlie Community Court in 2015 as a result of ‘evidence’ that indicated these courts did not lead to a reduction in imprisonment or recidivism rates (Moore 2012; Banks 2015). The aim of this meta-review is to better understand what previous evaluations and impact studies can teach us about the practices of Indigenous sentencing courts and to critically assess the extent to which such studies can capture both the outcomes envisaged by the stated aims and goals of such courts, and other Indigenous aspirations for the courts.

Method

Meta-reviews

A meta-review is distinct from a meta-analysis (or systematic review), which is a statistical analysis often used in assessing clinical effectiveness in healthcare interventions. Examples of systematic reviews of criminal justice processes do exist, such as the review of correctional interventions, management policies and treatment programs to determine their effect on recidivism conducted by MacKenzie (2006). Such reviews assess the quality of the design and execution of each individual study, and then calculate a quantitative (statistical) estimate of the aggregate net benefit or aggregate net effect.

In contrast, the meta-review (or what is also referred to as an ‘integrated review’ in the health literature (Whittemore 2005; Whittemore and Knafl 2005)) conducted in this analysis relies on a narrative form of synthesis, similar to the review methodology adopted by Madell, Thom and McKenna (2012) in assessing studies of youth problem-solving courts and by Weed (2009) in examining the journeys reviewers have taken in the field of sports tourism. In essence, it is ‘a review of reviews’ that involves an examination of the methodologies used in the evaluations and an ‘interrogation of the explicit and implicit decisions made by reviewers’ about what are the aims and objectives of Indigenous sentencing courts (Weed 2009:615–6). Similar to the Weed study, this permits an exploration of the ‘conscious and unconscious decisions’ (2009:616) that were made by the evaluation and impact study researchers in using certain research methodologies and perspectives, and then considering how these decisions affected the findings and conclusions reached about the operation and influence of Indigenous sentencing courts. A statistical or systematic review was not possible because of the diversity of data collection and analytical methods used in the various Indigenous sentencing court evaluations and impact studies included in this analysis. However, Hawker et al’s (2002) and Whittemore and Knafl’s (2005) methods of systematically reviewing research using different paradigms and methods has been adopted as much as possible to ‘synthesize large amounts of disparate data’ and to ‘provide a database of reviewed articles’ and reports (Hawker et al 2002:1286).

Search strategy

Using the term ‘Indigenous sentencing courts’ and the specific names of such courts (for example, Koori Court, Murri Court, Nunga Court), a search of criminology and law databases (such as CINCH, Web of Science, Google Scholar and AGIS Plus Text) and a World Wide Web search was conducted to ensure all evaluations and relevant impact studies had been captured. References listed in articles and reports that had already been identified were also located and reviewed for completeness. Included in this analysis are evaluations and impact studies that have empirically measured one or more outcomes of Indigenous sentencing courts using raw data such as interview data, observations, court or police data, or case studies. Evaluations (as opposed to research) are usually (but not always) government funded and driven research endeavours used to determine the merit, effect or worth of a program, which ultimately influences policy and the allocation of resources (Tilley 2000). They are ‘intended to amass knowledge and understanding of a concrete activity — a project — and to give this understanding back to the project as feedback’, as well as being ‘systematic and repeatable, able to be followed up and criticized’ (Levin-Rozalis 2003:6). Evaluations tend to report findings that are specific to a project and correct at a particular point in time, although other conclusions can often also be reached and presented (Levin-Rozalis 2003:7). On the other hand, research per se advances broader knowledge and theory objectives, ‘by formulating generalized laws, with the broadest application of findings’ (Levin-Rozalis 2003:8). Impact studies that increase knowledge about offender or offending-centred aims, such as impact on recidivism and sentencing outcomes, have been included in this review as this is the type of information that usually influences government funding and policy reform. Additionally, only publicly available articles and reports were included.

All of the identified evaluations and impact studies of Indigenous sentencing courts were included in this review (apart from those listed in Table 2). As listed in Table 1, this comprised 16 evaluations and impact studies.[††]

Table 1: Evaluations and impact studies included in review

Authors and year of publication
Jurisdiction
Evaluation or impact study
Methodology
Potas et al 2003
NSW
Evaluation
Qualitative
CIRCA 2008
NSW
Evaluation
Qualitative[‡‡]
Fitzgerald 2008 [§§]
NSW
Evaluation
Quantitative
Daly and Proietti-Scifoni 2009
NSW
Impact study
Qualitative
Parker and Pathé 2006
Qld
Impact study
Qualitative
Morgan and Louis 2010
Qld
Evaluation
Qualitative/quantitative
Harris 2006
Vic
Evaluation
Qualitative/quantitative
Byles and Karp 2010
Vic
Impact study
Quantitative[***]
Borowski 2010, 2011
Vic
Evaluation
Qualitative/quantitative
Dawkins et al 2011
Vic
Evaluation
Qualitative/quantitative
Tomaino 2004
SA
Impact study
Quantitative[†††]
Marshall 2008
SA
Impact study
Qualitative
Bond and Jeffries 2012
SA
Impact study
Quantitative
Aquilina et al 2009
WA
Evaluation
Qualitative/quantitative
Research and Analysis Branch, Department of the Attorney General Western Australia 2014
WA
Evaluation
Qualitative/quantitative
CIRCA 2013
SA and NT
Evaluation
Qualitative/quantitative[‡‡‡]

Six studies are classified as impact studies, and the remaining 10 evaluations. The majority (12 of 16) have been conducted since 2008, reflecting the fact that in all jurisdictions Indigenous sentencing courts had been established by 2006 (the only jurisdiction that never established one was Tasmania).

One of the four studies in Victoria considered Children’s Koori Courts (in Melbourne and Mildura) (Borowski 2010, 2011) and another evaluated the County Koori Court (in Latrobe Valley) (Dawkins et al 2011); the other two evaluations focused on Koori Magistrates’ Courts (Harris 2006; Byles and Karp 2010). One of the South Australian studies considered Aboriginal Conferences (in Port Lincoln) (Marshall 2008), while the other South Australian studies (Tomaino (2004); CIRCA (2013); Bond and Jeffries (2012)) considered Nunga Courts. One Western Australian study focused on the Kalgoorlie Aboriginal Sentencing Court (Aquilina et al 2009) and the other on the Barndimalgu Family Violence Court (Research and Analysis Branch, Department of the Attorney-General Western Australia 2014), which uses a more culturally sensitive sentencing court process (akin to an Indigenous sentencing court) to address Aboriginal family violence in Geraldton.

A list of empirical studies that measured outcomes of Indigenous sentencing courts, which were reviewed but not included in this analysis, appears in Table 2, together with an explanation of why they were excluded. There are also five studies commissioned by government departments in the Australian Capital Territory, the Northern Territory, Victoria and Western Australia that have not been included in this analysis because there were not published or not finalised.

Table 2: Impact studies not included in review

Authors and year of publication
Jurisdiction
Reason for exclusion
Marchetti 2010
Qld and NSW
Not assessing outcomes or impact of the Murri and Circle Courts, but rather exploring the extent to which Indigenous sentencing courts can manage gendered power imbalances for intimate partner violence offending.
Marchetti 2014
Qld and NSW
Not directly assessing outcomes or impact as related to goals and aims of the courts, but instead considering how Murri and Circle Courts can deliver justice from the perspective of judicial officers, Elders, Community Representatives and Indigenous Court workers.
Marchetti 2015
Qld and NSW
Not assessing offending outcomes, but rather offender perceptions of justice. Although it did consider notions of fairness, it did not set out to evaluate one of the community building or offending/offender-centred aims of the courts.
McAsey 2005
Vic
Not assessing outcomes or impact of the Koori Courts, but rather conducting a critical assessment of whether the courts are addressing power imbalances in sentencing hearings for Indigenous offenders.

Quality of studies and analysis of data

Of the evaluations and impact studies included in this analysis, five are qualitative, four quantitative, and seven both qualitative and quantitative (evaluations and impact studies that predominantly focused on reporting descriptive statistics without necessarily having conducted statistical analyses were still categorised as quantitative). A consideration of the methodological rigour of the studies is important for assessing the reliability of the results. Due to the varying types of methodologies used an assessment of the quality of each study becomes a complicated exercise (Whittemore and Knafl 2005). There is ‘no gold standard for evaluating and interpreting quality in research reviews’; instead, assessing quality depends on the sampling frame and research purpose of the meta-review (Whittemore and Knafl 2005:549–50). For this study, the quality of the evaluations and impact studies were assessed using a modified version of the protocols developed by Hawker et al (2002). This included an assessment of the degree to which Indigenous input and understandings of the cultural specificities of Indigenous sentencing court processes were incorporated into the research design and informed the analysis of each of the studies. A list of the modified Hawker et al (2002) criteria applied to each study is set out in Table 3. The primary purpose of this assessment was not to ‘grade’ the studies, since even studies that are poorly designed and executed can still serve as a source of information. It was used to gain a deeper understanding of the cultural appropriateness of the research design and methodological approach used in each study, so as to make a determination of the reliability and adequacy of the results as compared to the court’s goals and aspirations. An independent researcher conducted a separate analysis of the quality of the studies using the criteria in Table 3. Our conclusions were compared and discussed until a consensus was reached. The ways in which the studies satisfied the criteria is presented in a descriptive format in the findings section of this paper (rather than presented here) since some of the issues raised during the quality assessment exercise go to the core purpose of this meta-review.

Once the studies were assessed for quality, their findings were recorded according to their methodological approach. That is, the qualitative, quantitative and mixed method studies were initially analysed as separate groups to better compare and assess the findings, and comparisons were later made across the different approaches. Methodological limitations specified in the studies were also recorded in detail.

Table 3: Modified Hawker et al (2002) quality assessment criteria


Good
Fair
Poor
Very poor
Not present in study
1. Clear statement of introduction and aims





2. Clearly stated method and adequacy of data collected (including selection of sample population)





3. Explanation of data analysis





4. Statement of limitations and ethical consideration





5. Manner in which findings and results are reported and explained





6. Degree of Indigenous input into study design





7. Degree to which specificities of Indigenous sentencing court processes have influenced study design





8. Usefulness of the study





Findings

Testing the aims and goals of the courts

Indigenous sentencing courts have both community-building and offending-centred aims. As stipulated in the reports and articles analysed for this study, the community-building aims of the courts focus on:

• increasing the participation and involvement of Indigenous community members, victims and offenders in the sentencing process (Potas et al 2003; Harris 2006; CIRCA 2008; Fitzgerald 2008; Borowski 2010; Byles and Karp 2010; Morgan and Louis 2010; Dawkins et al 2011);

• increasing the confidence of Indigenous communities in the court process and criminal justice system (Potas et al 2003; CIRCA 2008; Fitzgerald 2008; Marshall 2008);

• reducing barriers to accessing justice (Potas et al 2003; CIRCA 2008; Fitzgerald 2008);

• supporting and involving victims of crime in the sentencing process (Potas et al 2003; Tomaino 2004; CIRCA 2008; Fitzgerald 2008; Marshall 2008);

• increasing Indigenous community ownership in the administration of law (Harris 2006; Borowski 2010);

• promoting and increasing community awareness of codes of conduct and community standards of behaviour (Harris 2006; Borowski 2010);

• promoting and increasing the awareness of Indigenous sentencing courts (Harris 2006);

• creating a more culturally appropriate and comprehensible criminal justice system and sentencing process (Tomaino 2004; Marshall 2008; Acquilina 2009; Morgan and Louis 2010; Sentencing Advisory Council 2010); and

• improving relationships between the court and Indigenous peoples (Acquilina 2009).

Offending-centred aims focus on reducing reoffending and the over-representation of Indigenous peoples in custody (Potas et al 2003; Tomaino 2004; Parker and Pathé 2006; Harris 2006; CIRCA 2008; Fitzgerald 2008; Acquilina 2009; Borowski 2010; Sentencing Advisory Council 2010), but they also stipulate that the Indigenous courts aim to:

• provide more appropriate and transparent sentencing options and processes (Potas et al 2003; CIRCA 2008; Fitzgerald 2008; Acquilina 2009; Morgan and Louis 2010; Sentencing Advisory Council 2010);

• increase offender accountability and the accountability of Indigenous communities for Indigenous offenders (Harris 2006; Marshall 2008; Borowski 2010; Sentencing Advisory Council 2010);

• improve court appearance rates for Indigenous offenders (Tomaino 2004; Harris 2006; Parker and Pathé 2006; Borowski 2010; Sentencing Advisory Council 2010);

• increase compliance with court orders (Harris 2006; Parker and Pathé 2006);

• reduce the seriousness of reoffending (Borowski 2010);

• reduce deaths in custody (Tomaino 2004);

• encourage rehabilitation and support for changing offending behaviour (Potas et al 2003; Tomaino 2004; Harris 2006; Parker and Pathé 2006; Marshall 2008; Morgan and Louis 2010; Sentencing Advisory Council 2010); and

• enhance safety for all members of the community (Acquilina 2009).

As noted by Stobbs and MacKenzie (2009) these aims are wide-ranging and possibly too ambitious to expect of an innovative sentencing process, particularly when there is no suggestion that mainstream court processes attain such goals. The five qualitative studies used interviews, submissions, observations of hearings, case/court reports and departmental or court documents mainly to assess whether community-building aims had been achieved. Findings confirmed that offenders thought the Indigenous sentencing courts were fairer (Potas et al 2003; Harris 2006; Daly and Proietti-Scifoni 2009), that the presence of Elders increased confidence and respect for the sentencing process (Potas et al 2003; CIRCA 2008; Daly and Proietti-Scifoni 2009), that the courts strengthened the relationship between Indigenous communities and the criminal justice system (Daly and Proietti-Scifoni 2009), that community engagement in the sentencing process encouraged respect for Elders and a sense of community empowerment (Parker and Pathé 2006; Marshall 2008), and that the process was culturally sensitive, engendering more suitable sentencing options (CIRCA 2008; Marshall 2008; Daly and Proietti-Scifoni 2009).

Four of the qualitative studies did, to varying degrees, focus on offending-centred aims. Two studies — the Daly and Proietti-Scifoni (2009) study and the Potas et al (2003) Nowra Circle Court study — conducted reoffending analyses, the former using a ‘pathways to desistance’ framework and the latter using case studies and interview data to provide descriptive narratives of reoffending. The CIRCA (2008) study of the New South Wales Circle Courts incorporated the recidivism analysis conducted by Fitzgerald (2008) in its report, which found Circle Sentencing had no impact on recidivism. On the other hand, interview data collected by CIRCA (2008:60) found that there was ‘a conflict between people’s perceptions of the impact of Circle Sentencing on recidivism and the reality of the data’ presented in the Fitzgerald study. Although Parker and Pathé’s (2006:24) qualitative study of the Queensland Murri Courts acknowledged that adequate comparative mainstream court data was not available to measure the impact on recidivism, it did conclude that the court was ‘having success in regard to its objective of diverting offenders from prison’ and that Indigenous offenders had better attendance rates in the Murri Court than in the mainstream Magistrates’ Court.

All the mixed methods studies assessed both community-building and offending-centred aims, although the manner in which recidivism was tested varied. The main measures used were failure rate (frequency of reoffending post Indigenous sentencing court appearance), time taken to reoffend, and difference in seriousness of reoffending. The quantitative studies focused on offending-centred aims, particularly measuring the impact on recidivism, although the Tomaino (2004) study used interview data reported in higher degree theses, a conference presentation by the magistrate who initiated the Aboriginal (Nunga) Court in Port Adelaide, and a case study to confirm that the Aboriginal Courts in South Australia were having a positive impact on community-building aims, such as increased rapport between the offender, the Indigenous community and court players. Other offending-centred aims both the mixed methods and quantitative studies found were realised by the Indigenous sentencing courts included that they provided more culturally appropriate sentencing options (Acquilina 2009; Morgan and Louis 2010; Dawkins 2011; Borowski 2011; CIRCA 2013), improved court appearance rates and compliance with court orders (Borowski 2010:475; Morgan and Louis 2010:142), and encouraged offender rehabilitation (Acquilina 2009; Dawkins 2011).

Methodological limitations

Both the community-building and offender/offending-centred aims require comparative benchmarks to assess whether the Indigenous sentencing courts are improving the experiences and outcomes of Indigenous peoples when appearing before a sentencing court. However, when it comes to the community-building aims, little thought is given to the use of control or comparative groups (to measure equivalent indices in mainstream courts), and interviews are generally used to collect data. In contrast, the offender/offending-centred aims tend to be tested using statistical quantitative analyses and comparing findings with those generated using comparison groups sentenced via the mainstream court system. Only one of the four quantitative studies (Byles and Karp 2010:2) provided an acknowledgement that there were only limited inferences that could be drawn regarding the differences in sentencing outcomes between the Indigenous sentencing court and the comparative mainstream court, ‘and the factors underlying those outcomes’ due to the ‘relatively low number of Koori Court cases, compared to Magistrates’ Court cases’. The Fitzgerald (2008) study contained explanations of the various factors that limited further or varied analyses or sample selection in the footnotes of the report. It also acknowledged that the study only addressed one of the aims of the Circle courts and that there was nothing in the analysis ‘to suggest that circle sentencing is not meeting the other objectives’ (Fitzgerald 2008:7). The other two quantitative studies (Bond and Jeffries 2012; Tomaino 2004) paid little, if any, attention to the limitations of their studies. Ultimately, however, the Fitzgerald (2008) and Bond and Jeffries (2012) studies, in particular, presented the logic underpinning their statistical analyses without acknowledging the existence of similar problems as expressed in the Byles and Karp (2010) study. The findings are presented as objective ‘truths’ that rely on positivist paradigms that measure outcomes based on aggregate populations without considering how differences in social, economic, cultural and political contexts located at each court site might have an impact on the results. As Rynne (2015:103) notes, ‘[r]esearch that fails to consider ways of living, learning and knowing when interpreting statistical analysis is potentially an exercise in a culturally shared illusion that conceals what is essentially an exercise in subjective inquiry under an accepted cloak of objectivity’. Qualitative research that focuses on canvassing local knowledge, on the other hand, is often viewed as being based on more subjective evaluative criteria, since it has no ‘cloak of objectivity’ under which it can hide.

Drawing on the work of Payne (2007), Daly and Proietti-Scifoni (2009) identify two other obstacles to measuring reoffending. The first is that, more often than not, it is difficult, if not impossible, to link a court’s database to criminal justice databases maintained by police and corrections so that there is no under- or over-estimation of reoffending. Second, the criteria by which an offender is allowed to appear before an Indigenous sentencing court need to be considered when selecting cases for a comparative group of mainstream court offenders. Four out of the seven mixed methods studies (Morgan and Luis 2010; Borowski 2010, 2011; Dawkins et al 2011; CIRCA 2013) gave considerable attention to the limitations of their reoffending analyses, focusing particularly on issues such as a lack of accurate administrative court or police data and inappropriate comparison groups. Only two out of the total studies (Daly and Proietti-Scifoni 2009; CIRCA 2013) mentioned that ethics approval from an appropriate Human Research Ethics Committee had been obtained.

Extent of Indigenous participation in the research and understanding of court operations

Cultural safety in research involving Indigenous peoples requires the prioritisation of Indigenous research methodologies and values (Leeson, Smith and Rynne 2016), which embodies notions of spirit and integrity, reciprocity, respect, equality, survival and protection, and responsibility (National Health and Medical Research Council 2003; National Health and Medical Research Council, Australian Research Council, and Australian Vice Chancellor’s Committee 2007 (updated May 2015); see also Australian Institute of Aboriginal and Torres Strait Islander Studies 2012). With these principles in mind, critical Indigenous studies scholars have advocated for the use of a ‘strengths-based’ rather than a ‘vocabularies of deficit’ approach when planning and conducting research concerning Indigenous peoples (Blagg 2008; Leeson, Smith and Rynne 2016). This entails focusing on how the program being studied supports and increases the strength, resilience and wellbeing of Indigenous communities, rather than simply assessing the extent to which Indigenous-focused programs may (or may not) reach benchmarks set by mainstream processes.

For evaluations and impact studies of Indigenous sentencing courts to meet cultural safety research standards, there must be Indigenous democratisation (Brown et al 2016) of the research process and an in-depth understanding of the cultural specificities of Indigenous sentencing court processes. Without this, evaluations and impact studies of these courts ‘may be understood as culturally inappropriate and a form of deep colonisation where data are collected without reference or understanding of the histories and policies of oppression that may impact on the way research is understood, collected and analysed’ (Leeson, Smith and Rynne 2016:9).

Only one of the quantitative studies (Byles and Karp 2010:3) made mention of the need to properly understand the operation of the Indigenous sentencing courts and acknowledged that each of the courts in that jurisdiction operated ‘slightly differently, given differences between Koori communities and in locations and personnel’. The report prepared was not an evaluation of the Koori Court in Victoria, but an exercise in providing the Sentencing Advisory Council with ‘a statistical profile of accused persons appearing before the Koori Court, the offences heard and sentencing outcomes’ (Byles and Karp 2010:2). The report presented comparable data from the mainstream Magistrates’ Court only where it made sense to do so. Recognising that comprehensive comparisons were not possible, the report notes that any inferences drawn from such comparisons ‘are tentative and are based to a significant degree on meetings the Council has had with court staff and other stakeholders’ (Byles and Karp 2010:2). The findings in this report, which focused on types of offences heard in the Koori Court and types of sentencing orders made for different types of offences, aimed to construct a detailed picture of how particular Indigenous offenders are sentenced in the different types of courts, rather than assessing the efficacy of Koori Court sentencing outcomes according to a mainstream court benchmark.

Indigenous involvement in the research design or data collection phases of the studies reviewed ranged from interviews, consultations and the use of focus groups with Elders, community representatives and Indigenous court workers (Harris 2006; Morgan and Louis 2010; CIRCA 2013) to the use of a research advisory committee comprising Elders and other key stakeholders involved with the courts (Potas et al 2003; Borowski 2010, 2011; Dawkins et al 2011), and the employment of Elders to assist with interviews with offenders (Daly and Proietti-Scifoni 2009). All of these studies used either a mixed methods or qualitative research approach, which generated more contextualised findings, but which, in the case of two (Harris 2006; Potas et al 2003), did not guarantee a high standard of methodological rigour.

Testing Indigenous-centric aims and goals

Research that used interviews with magistrates, Indigenous sentencing court workers, defence lawyers, prosecutors and Elders or Community Representatives to determine what they believed could be achieved from a process that involves Indigenous community participation (Marchetti 2014:350) found that ‘notions of justice go beyond the need to simply punish and reform an offender’s behaviour’, but encouraging offender accountability and rehabilitation was more important than punishing an offender. Accountability and rehabilitation occur because of the presence of the Elders or Community Representatives who bring community cultural values and knowledges into the sentencing space. Their presence also ensures greater procedural justice since their participation provides courts with a better understanding of an offender’s circumstances and makes the hearing more accessible and comprehensible for offenders. Marchetti (2014:360) found that reconnecting offenders to their community and culture and, as a consequence, strengthening cultural empowerment were important goals of Indigenous sentencing courts for Indigenous interview participants. When conducting research on Indigenous sentencing courts, such narratives and perspectives should inform the ambit of the research focus; that is, a theory-driven approach to evaluation should be preferred. Instead of simply asking ‘what works’, there needs to be an awareness and understanding of the cultural context and specificities within which the intervention works or, more specifically, how Indigenous sentencing courts involve ‘the interplay of individual and institution, and of structure and agency’ (Pawson and Tilley 1997:xiii). As Pawson and Tilley (1997:160) argue, ‘programs ... are complex social organizations’, requiring theory-driven understandings of ‘what works for whom in what circumstances’ (1997:161).

The program theory underpinning Indigenous sentencing courts, although partially reflective of therapeutic jurisprudence and restorative justice principles and practices, is more concerned with ‘transforming racialised relationships and communities’ (Marchetti and Daly 2007:443). This means that oversimplified versions of program theory used in evaluations that view Indigenous sentencing courts primarily ‘in terms of effects, with little attention paid to how those effects are produced’ (Astbury and Leeuw 2010:364) will fail to capture the full effect of the courts on Indigenous offenders, and their victims and communities. How this translates in practice can be illustrated using recidivism as the dependent variable. Since the program theory underpinning Indigenous sentencing courts centres on the involvement of Elders and Community Representatives in the sentencing process, an explanatory evaluative approach would, for example, take into account the possibility that offender behaviour might only change if an individual has a particular relationship or inherent degree of respect for certain Elders or Community Representatives in the community. In comparing the effect on reoffending of appearing before an Indigenous or a mainstream sentencing court, this cultural relational factor would need to be controlled for in order to attain an Indigenous-centric and more nuanced understanding of how the courts work in certain contexts for particular individuals. An example of a recidivism study that undertook an Indigenous-centric theory-driven approach is Daly and Proietti-Scifoni (2009). The authors acknowledge that their methodology entailed more of a critical analysis, ‘with the research team comprising both Indigenous and non-Indigenous staff’, rather than adopting an ‘Indigenous methodology’ (Daly and Proietti-Scifoni 2009:25). This study used desistance theory and a qualitative methodology that included the employment of an Aboriginal Elder to assist with interviews of offenders and two Indigenous undergraduate students to review court files and other information relating to the offenders who had appeared before the Nowra Circle Court. There had also been a close relationship of trust built with the Indigenous and non-Indigenous court workers at the court site in which the project was located. The interview instrument was developed within a desistance framework and with a detailed knowledge of the processes surrounding the operation of Indigenous sentencing courts. As a result, the findings provided a more nuanced (albeit more complicated) understanding of how the Indigenous sentencing courts might support an offender’s desistance from offending. The authors argue for using a mixed methods design when measuring the impact of an intervention on reoffending, rather than a quantitative approach. Daly and Proietti-Scifoni state:

In light of the small number of matched cases in the BOCSAR sample (n = 68), Fitzgerald (2008) would have been on a better track by comparing these Circle and conventional court cases in greater depth. These could have been examined with reference to offending trajectories and types of offences subject to police targeting (e.g., driving and public order). Such an approach would shed light not only on comparing Circle and conventional court outcomes, but also provide a better understanding of Aboriginal pathways into and out of crime, more generally (2009:109).

When the CIRCA (2008) and Fitzgerald (2008) evaluations are read together (as they should be as they were commissioned as part of the same study), policy-makers are left with a broader understanding of the complexities of why Indigenous sentencing courts may or may not impact on reoffending. Although Fitzgerald correctly concludes that involving Indigenous peoples in the sentencing of Indigenous offenders ‘is not enough, by itself, to produce a reduction in reoffending’ (2008:7), and recommends combining circle sentencing with other cognitive behaviour or substance abuse programs, interviews with offenders reported in the CIRCA (2008) study confirm that circle sentencing changed the behaviour of offenders who reconnected with their culture and strengthened their relationships with Elders and other family members. Reading the two studies together suggests that there is more at play than may be apparent from a ‘black box’[§§§] evaluation (Astbury and Leeuw 2010).

Conclusion

This meta-review is not intended to suggest that only evaluations or impact studies that use certain methodologies can appropriately capture the cultural and political workings of an Indigenous-focused criminal justice program. Instead, it illustrates that linear chains of causality and positivist research paradigms, although methologically rigourous, may not ensure we gain the knowledge required to address Indigenous over-representation in the criminal justice system. When designing the methodology to be used, it is imperative that the people involved in conducting the Indigenous-focused program are consulted, to better understand the contexts within which program mechanisms work. All too often, this does not occur, and the cultural complexities that drive and influence many Indigenous-focused criminal justice programs are lost and not integrated into the measures that are used to determine a program’s ‘success’. As in Queensland’s Murri Courts and Western Australia’s Kalgoorlie Community Court, this can lead to ill-informed policies which open — rather than close — the gap in Indigenous over-representation in the criminal justice system.

Acknowledgements

This research was supported under the Australian Research Council’s Future Fellowship funding scheme (project number FT140100313). I am particularly grateful for the guidance and assistance of Dr Kate Williams and Fabienne Else in analysing the data.

Legislation

Magistrates’ Court Act 1989 (Vic)

References

Abele F (2013) ‘Is Evaluation a Tool for Social Justice? Reconiliation? Control? — Reflections on the Canadian Experience in Indigenous Affairs’ in Australian Government Productivity Commission (ed), BetterIndigenous Policies: The Role of Evaluation, Commonwealth of Australia, 183–204

Aquilina H, Sweeting J, Liedel H, Hovane V, Williams V and Somerville C (2009) Evaluation of the Aboriginal Sentencing Court of Kalgoorlie, Shelby Consulting

Astbury B and Leeuw F (2010) ‘Unpacking Black Boxes: Mechanisms and Theory Building in Evaluation’, American Journal of Evaluation 31(3), 363–81

Australian Bureau of Statistics (2013) 3238.0.55.001 — Estimates of Aboriginal and Torres Strait Islander Australians, June 2011, Australian Bureau of Statistics <http://www.abs.gov.au/ausstats/abs@.nsf/

latestProducts/3238.0.55.001Media Release1June 2011>

Australian Bureau of Statistics (2014) 4517.0 — Prisoners in Australia, 2014: Imprisonment rates, Australian Bureau of Statistics <http://www.abs.gov.au/ausstats/abs@.nsf/Lookup/by Subject/4517.0~2014~Main Features~Aboriginal & Torres Strait Islander prisoner characteristics~10007>

Australian Government Productivity Commission (2013) Better Indigenous Policies: The Role of Evaluation Roundtable Proceedings, Australian Government Productivity Commission <http://www.pc.gov.au/__data/

assets/pdf_file/0003/123069/better-indigenous-policies.pdf>

Australian Institute of Aboriginal and Torres Strait Islander Studies (2012) Guidelines for Ethical Research in Australian Indigenous Studies, Australian Institute of Aboriginal and Torres Strait Islander Studies

Banks A (2015) ‘Aboriginal Court Gets the Chop’, The West Australian (Osborne Park), 14 August 2015 <https://au.news.yahoo.com/thewest/wa/a/29258150/kalgoorlie-aboriginal-court-gets-the-chop/>

Beranger B, Weatherburn D and Moffatt S (2010) ‘Reducing Indigenous Contact with the Court System’, NSW Bureau of Crime Statistics and Research: Crime and Justice Statistics Bureau Brief 54, 1–4

Blagg H (2008) Evaluation of Red Dust Role Models: Assessing the Performance of red Dust Role Models in Their Work with Young People and Their Communities on Remote Aboriginal Communities in the Northern Territory, Red Dust Role Models <http://www.reddust.org.au/assets/files/review/Evaluation-Document.pdf>

Blagg H, Morgan N, Cunneen C and Ferrante A (2005) Systemic Racism as a Factor in the Over-representation of Aboriginal People in the Victorian Criminal Justice System: Report to the Equal Opportunity Commission and Aboriginal Justice Forum of Victoria, Equal Opportunity Commission and Aboriginal Justice Forum of Victoria

Bond C and Jeffries S (2012) ‘Indigenous Sentencing Outcomes: A Comparative Analysis of the Nunga and Magistrates Courts in South Australia’, Flinders Law Journal 14, 359–82

Borowski A (2010) ‘Indigenous Participation in Sentencing Young Offenders: Findings from an Evaluation of the Children’s Koori Court of Victoria’, Australian and New Zealand Journal of Criminology 43, 465–84

Borowski A (2011) ‘In Courtroom 7 — The Children’s Koori Court at Work: Findings From an Evaluation’, International Journal of Offender Therapy and Comparative Criminology 55(7), 1110–34

Brown D, Cunneen C, Schwartz M, Stubbs J and Young C (2016) Justice Reinvestment: Winding Back Imprisonment, Palgrave Macmillan

Byles D and Karp T (2010) Sentencing in the Koori Court Division of the Magistrates’ Court: A Statistical Report, Sentencing Advisory Council

Cobb-Clark DA (2013) ‘The Case for Making Public Policy Evaluations Public’ in Australian Government Productivity Commission (ed), Better Indigenous Policies: The Role of Evaluation, Commonwealth of Australia, 81–91

Courts Administration Authority of South Australia (2012) Courts Administration Authority: Home, Courts Administration Authority of South Australia <http://www.courts.sa.gov.au/Community/Pages/Aboriginal-Programs.aspx>

Cultural & Indigenous Research Centre Australia (2008) Evaluation of Circle Sentencing Program: Report, New South Wales Attorney General’s Department

Cultural & Indigenous Research Centre Australia (2013) Evaluation of Indigenous Justice Programs — Project A: Aboriginal and Torres Strait Islander Sentencing Courts and Conferences, Final Report, Cultural and Indigenous Research Centre Australia

Cunneen C and Rowe S (2014) ‘Changing Narratives: Colonised Peoples, Criminology and Social Work’, International Journal for Crime, Justice and Social Democracy 3(1), 49–67

Daly K and Proietti-Scifoni G (2009) Defendants in the Circle: Nowra Circle Court, The Presence and Impact of Elders, and Re-Offending, School of Criminology and Criminal Justice, Griffith University

Dawkins Z, Brookes M, Middlin K and Crossley P (2011) County Koori Court: Final Evaluation Report, Clear Horizon Consulting

Fitzgerald J (2008) ‘Does Circle Sentencing Reduce Aboriginal Offending?’, Crime and Justice Bulletin: Contemporary Issues in Crime and Justice, New South Wales Bureau of Crime Statistics and Research, 1–12

Harris M (2004) ‘From Australian Courts to Aboriginal Courts in Australia — Bridging the Gap?’, Current Issues in Criminal Justice 16(1), 26–41

Harris M (2006) ‘A Sentencing Conversation’: Evaluation of the Koori Courts Pilot Program October 2002–October 2004, Department of Justice Victoria

Hawker S, Payne S, Kerr C, Hardey M and Powell J (2002) ‘Appraising the Evidence: Reviewing Disparate Data Systematically’, Qualitative Health Research 12(9), 1284–99

Leeson S, Smith C and Rynne J (2016) ‘Yarning and Appreciative Inquiry: The Use of Culturally Appropriate and Respectful Research Methods when Working with Aboriginal and Torres Strait Islander Women in Australian Prisons’, Methodological Innovations 9, 1–17

Levin-Rozalis M (2003) ‘Evaluation and Research: Differences and Similarities’, The Canadian Journal of Program Evaluation 18(2), 1–31

McAsey B (2005) ‘A Critical Evaluation of the Koori Court Division of the Victorian Magistrates’ Court’, Deakin Law Review 10(2), 655–85

MacKenzie DL (2006) What Works in Corrections? Reducing the Criminal Activities of Offenders and Delinquents, Cambridge Press

Madell D, Thom K and McKenna B (2012) ‘A Systemic Review of Literature Relating to Problem-Solving Youth Courts’, Psychiatry, Psychology and Law 20(3), 412–22

Malezer L (2013) ‘Challenges in Evaluating Indigenous Policy’ in Australian Government Productivity Commission (ed), Better Indigenous Policies: The Role of Evaluation, Commonwealth of Australia, 69–79

Marchetti E (2010) ‘Indigenous Sentencing Courts and Partner Violence: Perspectives of Court Practitioners and Elders on Gender Power Imbalances During the Sentencing Hearing’, Australian and New Zealand Journal of Criminology 43(2), 263–81

Marchetti E (2014) ‘Delivering Justice in Indigenous Sentencing Courts: What this Means for Judicial Officers, Elders, Community Representatives, and Indigenous Court Workers’, Law & Policy 36(4), 341–69

Marchetti E (2015) ‘An Australian Indigenous-Focused Justice Response to Intimate Partner Violence: Offenders’ Perceptions of the Sentencing Process’, British Journal of Criminology 55(1), 86–106

Marchetti E and Daly K (2007) ‘Indigenous Sentencing Courts: Towards a Theoretical and Jurisprudential Model’, Sydney Law Review 29, 415–43

Marchetti E and Daly K (2016) ‘Indigenous Partner Violence, Indigenous Sentencing Courts, and Pathways to Desistance’, Violence against Women (OnlineFirst 15 September 2016), 1–23

Marchetti E and Downie R (2014) ‘Indigenous People and Sentencing Courts in Australia, New Zealand and Canada’ in SM Bucerius and M Tonry (eds), The Oxford Handbook on Ethnicity, Crime and Immigration, Oxford University Press, 360–85

Marchetti E and Ransley J (2014) ‘Applying the Critical Lens to Judicial Officers and Legal Practitioners Involved in Sentencing Indigenous Offendes: Will Anyone or Anyting Do?’, UNSW Law Journal 37(1), 1–33

Marshall J (2008) Port Lincoln Aboriginal Conferencing Pilot: Review Report, Office of Crime Statistics and Research

Moore T (2012) ‘Diversionary Courts Fall Victim to Funding Cuts’, Brisbane Times (online), 13 September 2012 <http://www.brisbanetimes.com.au/queensland/diversionary-courts-fall-victim-to-funding-cuts-201209

12-25sj5.html>

Moreton-Robinson A and Walter M (2009) ‘Indigenous Methodologies in Social Research’ in M Walter (ed), Social Research Methods, Oxford University Press, 95–109

Morgan A and Louis E (2010) Evaluation of the Queensland Murri Court: Final Report, Australian Institute of Criminology

National Health and Medical Research Council (2003) Values and Ethics: Guidelines for Ethical Conduct in Aboriginal and Torres Strait Islander Research, Commonwealth of Australia

National Health and Medical Research Council, Australian Research Council and Australian Vice Chancellors’ Committee (2007, updated May 2015) National Statement on Ethical Conduct in Human Research, Commonwealth of Australia

Parker N and Pathé M (2006) Report on the Review of the Murri Court, Queensland Department of Justice and Attorney-General

Pawson R and Tilley N (1997) Realistic Evaluation, SAGE

Payne J (2007) ‘Recidivism in Australia: Findings and Future Research’, Research and Public Policy Series, Australian Institute of Criminology

Potas I, Smart J, Brignell G, Thomas B and Lawrie R (2003) Circle Sentencing in New South Wales: A Review and Evaluation, Judicial Commission of New South Wales

Putt J (2013) ‘Conducting Research with Indigenous People and Communities: Brief 15’, Research Brief <http://www.indigenousjustice.gov.au/briefs/brief015.pdf>

Research and Analysis Branch Department of the Attorney General Western Australia (2014) Evaluation of the Metropolitan Family violence Court and Evaluation of the Barndimalgu Court: Evaluation Report, Government of Western Australia, Department of the Attorney General, Policy and Aboriginal Services Directorate

Roorda M and Peace R (2009) ‘Challenges to Implementing Good Practice Guidelines for Evaluation with Māori: A Pākehā Perspective’, Social Policy Journal of New Zealand (34), 73–89

Rynne J (2015) ‘Assessing the Prison Experience for Australian First Peoples: A Prospective Research Approach’, International Journal for Crime, Justice and Social Democracy 4(1), 96–112

Social Policy Evaluation and Research Committee (SPEaR) (2008) SPEaR Good Practice Guidelines 2008, SPEaR <http://www.spear.govt.nz/documents/good-practice/good-practice-guidelines-june-2008-final-version.pdf>

Social Policy Evaluation and Research Committee (SPEaR) and Aotearoa New Zealand Evaluation Association (ANZEA) (2007) Report on the SPEaR Best Practice Māori Guidelines Hui 2007, SPEaR <http://www.spear.govt.nz/documents/good-practice/spear-bpg-maori-final-report-anzea.pdf>

Stobbs N and Mackenzie G (2009) ‘Evaluating the Performance of Indigenous Sentencing Courts’, Australian Indigenous Law Review 13(2), 90–105

Taylor J (2013) ‘Data for Better Indigenous Policy Evaluation: Achievements, Constraints and Opportunities’ in Australian Government Productivity Commission (ed), Better Indigenous Policies: The Role of Evaluation, Commonwealth of Australia, 119–30

Tilley N (2000) ‘Doing Realistic Evaluation of Criminal Justice’ in V Jupp, P Davies and P Francis (eds), Doing Criminological Research, SAGE Publications Ltd, 97–113

Tomaino J (2004) Information Bulletin: Aboriginal (Nunga) Courts, Office of Crime Statistics and Research

Tuhiwai Smith L (1999) Decolonizing Methodologies: Research and Indigenous Peoples, Zed Books Ltd

Walter M (2010) ‘The Politics of the Data: How the Australian Statistical Indigene is Constructed’ International Journal of Critical Indigenous Studies 3(2), 45–56

Weed M (2009) ‘Progress in Sports Tourism Research? A Meta-review and Exploration of Futures’, Tourism Management 30, 615–28

Whittemore R (2005) ‘Combining the Evidence in Nursing Research: Methods and Implications’, Nursing Research 54(1), 56–62

Whittemore R and Knafl K (2005) ‘The Integrative Review: Updated Methodology’, Methodological Issues in Nursing Research 52(5), 546–53

Wilson S (2001) ‘What is Indigenous Research Methodology?’, Canadian Journal of Native Education 24(2), 175–9


[*] Griffith Law School, Griffith University, 170 Kessels Road, Nathan Qld 4111, Australia. Email: e.marchetti@griffith.edu.au.

[†] In this article ‘Indigenous’ is used to refer to both Aboriginal and Torres Strait Islander Peoples. There is much cultural and language diversity not only between Aboriginal and Torres Strait Islander Peoples, but also among various communities and societies of both groups, which will be differentiated as required.

[‡] Wilson describes these terms as encompassing a person’s thoughts or knowledge (epistemology), a person’s ethics or values (axiology) and a person’s belief system or way of being (axiology). He makes the point that Indigenous research ‘needs to reflect Indigenous contexts and world views’ (Wilson 2001:176), meaning Indigenous epistemologies, axiologies, ontologies and methodologies. He considers the research methodology to be about ‘how you are going to use your ways of thinking (your epistemology) to gain more knowledge about your reality’ (2001:175). Using all four aspects creates research that ‘comes from an Indigenous paradigm rather than an Indigenous perspective’ (2001:176).

[§] Cunneen and Rowe (2014) describe a Western positivist methodological approach as one that uses a scientific paradigm to separate, quantify, measure and statistically analyse individualised factors, while viewing these factors and the analysis as unrelated to the ‘social, economic, cultural and political set of relations’ within which they are embedded (2014:52). When researching questions related to the over-representation of Indigenous peoples within the criminal justice system, a Western positivist methodology also separates individualised factors from ‘the historical experience of colonisation and dispossession’ (2014:52).

[**] When Community Courts operated in the Northern Territory, there was no restriction requiring the offender be Indigenous. In Victoria, the Koori Court will also have jurisdiction to deal with an offence where the defendant ‘intends to consent to the adjournment, under section 59 of the Criminal Procedure Act 2009, of the proceeding to enable him or her to participate in a diversion program’: Magistrates’ Court Act 1989 (Vic), s 4F(1)(c)(iii).

[††] A further study by Marchetti and Daly (2016) was published after this analysis was conducted but prior to publication of this article and for this reason was not included in the analysis.

[‡‡] The report contains the findings from the Fitzgerald (2008) evaluation, but this did not form part of the evaluation conducted by CIRCA, which is why the CIRCA study has been classified as ‘qualitative’. The CIRCA (2008) report does, however, contain the complete evaluation (that is, both the CIRCA and Fitzgerald components of the evaluation) commissioned by the New South Wales Attorney General’s Department.
[§§] The Fitzgerald (2008) study is one part of the evaluation commissioned by the New South Wales Attorney General’s Department and it should be read in conjunction with the CIRCA (2008) report.
[***] This study presents descriptive statistics about the Koori Court, but not statistical analyses.
[†††] This study contained a number of descriptive statistics, but no statistical analyses.
[‡‡‡] The quantitative component was conducted by the South Australian Office of Crime Statistics and Research (‘OCSAR’) for the evaluation of the South Australian programs. The report also included descriptive statistics provided by the Data Warehouse division of the Northern Territory Department of Justice and findings from the Local Knowledge Community Court review (which was not finalised and released publicly).

[§§§] A ‘black box’ evaluation measures the effects of a program without exploring ‘how those effects are produced’ (Astbury and Leeuw 2010:364).


AustLII: Copyright Policy | Disclaimers | Privacy Policy | Feedback
URL: http://www.austlii.edu.au/au/journals/CICrimJust/2017/1.html