• 1. Introduction

  Over the past decades, scholars from many different disciplines have analysed and commented on the work of the European Court of Human Rights (ECtHR or Court), making it one of the most researched international courts. In announcing a 2022 blog symposium that focused on three recent monographs on the European Convention on Human Rights (ECHR or Convention), ECHR scholar Eva Brems briefly elaborated on how much the study of the Convention has changed over the years (Brems, 2022). Brems recalled that when she was writing her LLM thesis on the margin of appreciation doctrine in 1995, ‘the analysis of all relevant case law and literature was something that could easily be done during a one-month winter term’. The manageability of the body of case law of the ECtHR and the available literature allowed for this. Today, however, ECHR researchers are confronted with an entirely different situation. Like Brems explains, the sheer number of cases as well as the amount of literature makes it much more difficult to stay on top and research the Convention system. Indeed, studies on the ECHR have matured into a separate research field in which scholars from across and beyond Europe participate.
  The amount of case law and the available scholarship are not the only things that have changed over the years. The way the Convention system and the work of the ECtHR is studied has diversified and novel approaches are still appearing. Originally, legal scholars relied heavily upon highly qualitative and classical-doctrinal methods to study the Court’s case law, often through interpretative analyses of small samples of case law. In addition to this, there have also been in-depth historical, philosophical and theoretical studies on the foundations and meaning of human rights and their interpretation by the ECtHR or the development of the Convention and the Court over time. Increasingly, such ‘traditional’ ECHR scholarship has been complemented by empirical studies conducted by political and other social scientists, who have shown interest in the Convention system. More recently, technological innovation and the availability of deep-text and big-data analysis allow teams of scholars to research large sets of case law for patterns in the Court’s reasoning or correlations between the use of certain terminology and external developments such as political or backlash. Consequently, at present, the Convention system is not researched by using one particular method only. On the contrary, a rich and continuously expanding methodological toolbox is available for scholars who set out to research the Convention system.
  In view of the special issue’s central theme, ‘progress in legal scholarship’, our objective for this contribution is to unpack the methodological toolbox of the ECHR scholar as it has expanded over the past two decades, with a focus on methods that are of importance to legal scholars. By doing so, we do not only aim to show the richness of ECHR scholarship, but we also want to provide more insight into the different ‘tools’, including their application, value and potential weaknesses. To these ends, we start by discussing historical, theoretical and philosophical methods (Section 2) and the methods for ‘classical’ analysis of the Court’s case law (Section 3), followed by empirical qualitative and quantitative, statistical and machine learning research methods (Section 4). For each different group of methods, we set out what kind of legal research questions they can help to answer and offer examples as well as discuss benefits and possible pitfalls. In the concluding section (Section 5), we show common threads and share some general remarks on progress in ECHR research.
  A few caveats apply in relation to the focus and scope of this article and the methods we used in writing it. First, we decided to focus on research methods rather than approaches. This means that we do not deal specifically with inter-, multi- and transdisciplinary research. The focus on methods rather than approaches further means that we do not pay separate attention to comparative legal research, even though such research is frequently carried out in the ECHR context. The reason for this is that comparative legal research can be done using many different methods, ranging from functional legal comparisons from a qualitative perspective to comparing experiences and perceptions that have been brought to light via empirical methods.¹x Van Hoecke (2015), for example, held that ‘[r]esearchers get easily lost when embarking on comparative legal research. The main reason being that there is no agreement on the kind of methodology to be followed, nor even the methodologies that could be followed’ (p. 1). We discuss these methods on their own, but think the particular and challenging element of comparison deserves a separate study.
  In writing the current article, we primarily relied on our own knowledge and experiences as legal ECHR researchers. We discuss the methodologies we are aware of as a result of our own studies, and our illustrations derive from the contributions we have come across when researching the Convention system. We have been looking into English-language sources only, even though important work on the ECHR system is published in many other languages, both within and outside Europe. Moreover, the emphasis of our research was on publications that appeared in the most recent two decades. We are aware that this forms an important limitation of our contribution: although we are three researchers with a different research focus and different backgrounds and views, we lack expertise in certain fields and we may have missed out on certain (historical) developments or innovative new work. We have strived to compensate for some of these limitations by relying on insights drawn from a workshop organized in May 2022 by Utrecht University and the University of Oslo on the subject of researching the ECtHR.²x We thank all workshop participants for their valuable contributions during the different panel sessions. In particular, we would like to express our gratitude to our co-organizers of the international workshop, Mads Andenæs and Antoine Buyse, for their invaluable input and ideas. This has certainly broadened our horizon and added to our insights into the richness of the methods debate in the ECHR field. Nevertheless, we cannot do justice to all important, valuable and interesting scholarly publications in this vast field of study. Our paper therefore should be read as being an exploratory starting point for further study rather than as an in-depth, comprehensive and final discussion of methods in the field of legal ECHR research.

• 2. Historical, Theoretical and Philosophical Methods

  This section focuses on historical, theoretical and philosophical methods for studying the Convention system. These methods probably fall somewhat between the more classical (legal doctrinal) analysis of ECtHR case law and qualitative and quantitative empirical research methods as we discuss them in the subsequent sections. At the same time, we will show below that they cannot be viewed completely separately from each other in the sense that a combination of, for example, theoretical and philosophical methods and more classical analysis of ECtHR case law is often conducted and may be necessary to answer particular research questions regarding the Convention system.

  2.1 Historical Methods

  In general terms, historical research has been described as ‘developing an understanding of the past through the examination and interpretation of evidence’ and ‘[using] that evidence to develop an interpretation of past events that holds some significance for the present’ (Diana Hacker, cited in Hutchinson & Duncan, 2012). A somewhat similar definition is that historical research offers ‘a rich description of an earlier era or contrasting legal regime’, thereby ‘satisfying the criteria within the fields of anthropology or history in use of sources, triangulation, and contextualization’ with the aim of illuminating ‘differences, choices, or continuities when compared with contemporary domestic practice’ (Minow, 2013). It follows from these definitions that a key feature of historical studies is that they offer context to present-day practices. This is particularly relevant for the Convention system, which has a rich and complex institutional history. For that reason, knowledge of and insight into this history is crucial for understanding current features of, or developments within the Convention system. For instance, the current interpretation and application of the subsidiarity principle, one of the two guiding Convention principles, cannot be understood without knowledge of the coming into being of the Convention system and the historical development of the role of the Court and the relationship between the Court and the States. Similarly, a historical account of the role of a State in the drafting of the Convention or the parliamentary debate leading up to the ratification of the Convention can help to explain the stance taken by that particular State in current discussions on the role and legitimacy of the Court. An eminent example of a study providing such historical insights on the basis of extensive historical research, is a book by Bates on the evolution of the Convention (Bates, 2010).³x For a similar, although broader, multidisciplinary approach, see e.g. Lambrecht (2022). For a historical account of, but a different perspective on, the origins of the Convention system, see also Duranti (2017). In this book, Bates provides a detailed historical account, based on intensive archive studies, of the coming into being of the Convention system up to the creation of a permanent court in 1998, including reasons for the establishment of the Convention system, the attitude of States at the time of ratification and landmark judgments of the first fifty years that are still important today.⁴x There are also examples of historical studies on a particular Convention right or aspect of the Convention system, see e.g. Johnson (2016) and the different contributions in Aust & Demir-Gürsel (2021).

  2.2 Theoretical and Philosophical Methods

  Like historical methods, theoretical and philosophical methods are about providing context. However, in general, the aim in using these methods is rather different in the sense that theoretical and philosophical methods are more about critically contextualizing the case law of the Court than about offering an in-depth historical understanding. In other words, theoretical and philosophical methods are often used to deal with more normative or critical questions instead of descriptive or explanatory questions.

  2.2.1 The Notion of ‘Theory’; Internal and External Frameworks

  Before discussing such theoretical and philosophical approaches in more detail, it is important to say a bit more about legal theory. Lieblich defines theory as the ‘general intellectual framework through which we think about law, or a certain legal question … the prism through which we analyse or assess a question’ (Lieblich, 2021). This shows that there is a difference between the notion of ‘theory’ as it is used in social sciences and in legal research. As has been explained by Taekema, in the social sciences, the theoretical framework provides the support for a descriptive or explanatory question, advancing possible explanations or causes that need to be investigated in empirical work (Taekema, 2018). In legal research, in her view, theory can be considered to have a different function. The research questions legal researchers aim to answer are often evaluative and normative in nature instead of descriptive or explanatory. Consequently, for legal researchers the theoretical framework also needs to provide the basis for the evaluation or proposed solution, making the theoretical framework more a normative framework instead of an explanatory theoretical framework (Lieblich 2021; Taekema, 2018). Importantly, however, when it comes to researching the Convention system, both understandings of the notions can play a role. It is certainly possible to conduct philosophical and theoretical studies to provide insight into and explain the foundations and meaning of human rights and their interpretation by the ECtHR, without taking a normative perspective per se. Nevertheless, in this section we focus on the use of theoretical frameworks as normative frameworks.
  In this respect, the question may arise where such normative frameworks come from. Westerman has argued that ‘the theoretical framework commonly used by scholars who engage in doctrinal analysis is made up from the legal system itself’ (Westerman, 2011). According to Westerman, taking such a view means that ‘the function of theory, namely to provide a guideline and perspective from which the object can be described in a meaningful way, is exercised by the legal system itself’. In other words, an internal framework is used for assessing the law. Westerman’s view is, however, not the most common one. Often, the theoretical or normative framework for assessing the law can be seen to be an external framework, consisting of theories that find their origin in, for example, the social sciences or philosophy.⁵x See further on this, including the difference between an internal and external framework, Taekema (2018). Legal philosophy can play an important role in this respect as it can be described as a collection of insights regarding central or fundamental concepts and principles, or the ideas behind the legal order (Taekema & van der Burg, 2020).⁶x For a structured overview of the added value of legal philosophy in the context of legal doctrinal research see Taekema & van der Burg (2022).
  A very clear example of how such an external approach can be used to study the Convention system is a recent and important monograph by Heri (Heri, 2021).⁷x For less recent examples, see e.g. Dembour (2006), Letsas (2007). In this monograph, Heri assesses the treatment of vulnerability by the Court in the light of philosophical-theoretical conceptions of vulnerability, thereby combining classical analysis of case law with a theoretical approach. According to Heri, the Court’s current approach to vulnerability consists of ad hoc protection of certain individuals or groups, raising questions about the meaning of vulnerability and the criteria for describing someone as vulnerable. To avoid oversimplifying an applicant’s circumstances, vulnerability must, moreover, be a fleshed-out concept with real standards that are applied and tested in each individual case. Heri’s objective thereby is to further develop the concept by relying on philosophical conceptions of vulnerability. In the end, this helps her create a conceptual framework for vulnerability, which can guide the Court in improving its application of the notion in its case law.

  2.2.2 Critical Studies

  Another prominent example of an external approach is a monograph by Theilen on consensus reasoning employed by the Court (Theilen, 2021). Theilen critically examines the Court’s case law, particularly its references to European consensus, on the basis of human rights theory, including critical international legal theory. The latter distinguishes Theilen’s study from Heri’s work on vulnerability. More specifically, whereas Heri relies solely on a philosophical-theoretical framework, Theilen also uses critical theory to assess the Court’s case law. Critical theory, such as feminist legal theory, queer theory, or third world approaches to international law,⁸x For a detailed description and overview of such theories, see Bianchi (2016). aim to reveal concealed or disregarded ‘underlying power structures’, thereby attempting to ‘unravel underlying policy choices, inconsistencies and inequalities’ (Hirsch Ballin, 2020). Thus, the key feature of critical approaches is to ‘expose the relations between law and power’ (Lieblich, 2021). Consequently, they are often more concerned with social or political reform.⁹x Theilen, for example, sees themselves as ‘interested in social transformation beyond merely incremental adjustments to the status quo’ and held that from that perspective ‘it’s not enough to make suggestions for how the ECtHR could change its reasoning, its role or self-image. We also need to ask ourselves whether we want to pin our hopes of social transformation on the possibility of those changes within the ECtHR occurring’ (Strasbourg Observers [5 May 2022]. Rethinking European Consensus, Reimaging human rights. An interview with J. Theilen, https://strasbourgobservers.com/2022/05/05/rethinking-european-consensus-reimagining-human-rights/ (last accessed 23 November 2022). However, research based on such critical perspectives can also help to expose important aspects of the Court’s case law that are not always illuminated when a classical legal analysis is conducted (Taekema & van der Burg, 2022). To illustrate, although Theilen aims not only to analyse and interpret the Court’s case law, but also to ‘denaturalize current social arrangements so as to open up imaginative space for social transformation’ by providing a new approach to consensus reasoning, their analysis of the Court’s reference to European consensus provides in itself important new insights into the consensus reasoning used by the Court.

  2.3 Benefits and Pitfalls

  The recent studies by Heri and Theilen are also interesting and important as legal (human rights) research is sometimes criticized for not being explicit regarding the chosen theoretical and/or normative framework (Bianchi, 2016; Hutchinson & Duncan, 2012; Taekema, 2018). McInerney-Lankford, for example, has stated that

  greater attention must be paid to the choices and values implied in human rights legal research and the normative claims it makes, as well as the values and assumptions it bases those upon. (McInerney-Lankford, 2017)

  Expressly combining theoretical and philosophical methods with classical legal analysis, as Heri and Theilen have done, can help to address this criticism. An additional value is that work such as theirs provides a different perspective to understanding the Court’s case law and the Convention system in general, thus allowing to answer a wider range of research questions than single-method research could support.
  Nevertheless, there are also some pitfalls in conducting this type of research. When relying on critical, theoretical or philosophical methods, account must be taken of the fact that theories having their basis in social or political philosophy can be rather abstract, necessitating the need for significant ‘translation’ and further refinement if they are used to assess concrete legal developments (Taekema, 2018). This also touches on the fact that legal researchers may need to have sufficient philosophical training or background to be able to give a good account of philosophical arguments or theories.¹⁰x For this reason, Taekema & van der Burg (2020), for example, wrote an article in which they explain three general philosophical methods that, in their view, are most relevant and feasible for doctrinal legal research. Naturally, and as further discussed in the following pages, this also applies to the methods discussed in the remaining part of this article. Methods, including theoretical and philosophical methods, should only be relied upon if they are necessary and suitable for answering the research question and can be applied in a methodologically sound fashion.

• 3. ‘Classical’ Case Law Analysis

  3.1 Deductive and Inductive Case Law Analysis

  An important part of the scholarly work on the Court consists of – what can be called – ‘classical’ analysis of the Court’s case. Very roughly this means that a sample of judgments and decisions is studied qualitatively and systematically to answer a research question that is related to the case law. Scholars often use this method to discover patterns, definitions, standards and developments in the Court’s reasoning.¹¹x Studies into the general principles of the ECHR generally make use of this method; see e.g. Gerards (2023). In some studies, they do so by combining the method with certain hypotheses or research questions that have been derived from theoretical literature or social or political sciences. For example, based on political science insights and theories about the separation of powers, Leloup hypothesized in a recent study that the Court will take a particular stance in cases related to the role of national courts vis-à-vis the national legislature. He then conducted a qualitative case law analysis (QCA) to test the hypothesis, which helped him to arrive at a deeper understanding of the Court’s approach (Leloup, 2021). In studies such as Leloup’s, thus, a mostly deductive approach is taken.
  Other scholars use a more inductive approach, studying the case law with the aim of categorizing or making a typology of concepts or types of argument. For example, scholars such as Loven, Beijer and Lavrysen have studied the Court’s case law to see if different categories of positive obligations can be distinguished according to their nature, to the type of cases in which the Court imposes them, to their procedural consequences, and so on (Beijer, 2017; Lavrysen, 2016; Loven, 2022). Similarly, Gerards has analysed samples of judgments to detect and categorize different uses of process-based review (Gerards, 2017). As the Court’s case law is in constant flux and the Court’s approach to how it decides its cases is subject to change, such inductive, qualitative analyses of a wide sample of case law can be useful to discover certain developments in its approach. The method can be used to find out, for example, how the Court gives shape to its proportionality or balancing test in concrete cases (Smet, 2017), whether the margin of appreciation still has the function that is traditionally imputed to it (Gerards, 2018), how the Court uses its ‘consensus reasoning’ (Dzehtsiarou, 2015; Senden, 2011), whether the Court has changed its approach towards certain States (Çalı, 2016, 2018), how the Court makes use of certain sources in its argumentation (Glas, 2017), when the Court finds that the family life interests of migrants who are convicted criminals must stand in the way of their expulsion to a third country (Hilbrink, 2017), or how the Court deals with the protection of civil society and civic space (Buyse, 2019). Hence, classical, qualitative analysis of case law can help to make sense of the abundant case law of the Court and may offer a better insight in the development and changes in the case law over time.

  3.2 Developments in Classical Case Law Analysis – Managing Case Law Selections

  The approach taken towards these ‘classical’, qualitative forms of case law analysis has evolved over the years. This is not in the least due to the sharp increase of the volume of case law. Where early ECHR scholars could (and necessarily had to) work with very small samples of case law, as also observed by Brems (Brems, 2022), nowadays the Court hands down more than 1,000 judgments each year.¹²x See ECHR Overview 1959-2021, www.echr.coe.int; in the period 1999-2021, the average was 1,028 cases, while in the entire period between 1959 and 1999, the total number of judgments handed down was just 837. Moreover, for some research purposes it may be important to also include inadmissibility decisions, which come in even greater numbers; although the Court does not distinguish between pre- and post-1998 decisions, the grand total of all inadmissibility or strike out decisions handed down between 1959 and 2021 is 901,168 (i.e. 14,535 each year). Making selections of case law to be studied is therefore inevitable, and selecting a relevant sample deserves significant attention. Today, most scholars give insight in the number of cases they have studied, justify their choice of the period of time for which a selection was made, as well as explain a number of other selection criteria. Scholars often use the ECtHR’s digital database HUDOC as a basis for a first selection of cases, for example according to certain Convention provisions (e.g. Art. 10 ECHR) or based on a text search (e.g. entering search terms such as ‘narrow margin of appreciation’, ‘public figure’, ‘expulsion’, etc.) (e.g. Arnardóttir, 2017; Gerards, 2020). Usually they also make use of tools that are offered by the Court’s Registry to facilitate the selection of potentially relevant cases, in particular the classification system developed to indicate the ‘importance level’ of a judgment or decision. Although this classification system is not fully reliable, it is mostly undisputed that cases classified as ‘key cases’ and ‘level 1 cases’ are the most important and interesting from a legal perspective, and therefore can usefully be included in a sample (e.g. Arnardóttir, 2017).¹³x For the ‘importance levels’, see ECtHR. HUDOC FAQ: Frequently asked questions, available at https://www.echr.coe.int/Documents/HUDOC_FAQ_ENG.pdf (last accessed 23 November 2022). It can be wise to also include a number of cases of a lower importance level as a kind of ‘control group’, however. Many scholars further (or alternatively) use scholarly literature and handbooks to identify core judgments and decisions that will be of relevance to their analysis and then may make use of the ‘snowball method’ to find more (Brems & Lavrysen, 2015). This method also entails that they check the various judgments and decisions that are referred to in the core judgments and decisions to see if these judgments and decisions contain further references, and so on, until they have reached a saturation point. Finally, to arrive at manageable sample sizes, scholars increasingly choose to select ‘case studies’, which are usually understood to be thematically grouped cases that – based on a study of literature and theory or a pilot study of cases – promise to be illustrative or representative of certain phenomena or developments (e.g. Loven, 2022).

  3.3 Developments in Classical Case Law Analysis – Growing Towards Empirical Research

  In the past, it seems that some scholars, based on their prior knowledge and understanding of the case law, mainly selected a few cases as examples and suggested that these were representative for the Court’s case law as a whole (e.g. Alkema, 1988).¹⁴x Alkema explained his limited choice as follows: ‘Completeness would require more than the aid of a word-processor; it would call for a thorough rereading of all the case-law. That is not feasible. Instead, I will briefly mention just a few decisions illustrating Drittwirkung and showing some of its limitations and a few related aspects.’ In more recent years, the analyses have become more methodical and systematic, and the method has increasingly grown similar to some of the empirical research we discuss in Section 4. Some researchers examine hundreds of cases to see how certain concepts or standards are used, cataloguing characteristics of these cases or using coding to make sense of them (e.g. Hilbrink, 2017; see further Hall & Wright, 2008). In a further phase, they study the various categories, classifications, interpretations or codes to see if they reveal any significant patterns that confirm or reject a hypothesis, disclose a certain development, help to create a typology, and so on.
  More recently, Qualitative Case Law Analysis (QCA) has become en vogue, which allows not only for creating insight in the case law selection and the categories distinguished by the researcher, but also for controllable coding and even for statistical analysis of the coded cases (e.g. Mol, 2021; see further e.g. Schreier, 2014). QCA has a tendency to be deductive, especially when a codebook with a set of coding instructions is drafted based on available theories and hypotheses. For the inductive type of case law analysis that many ECHR scholars find valuable, it is not well-suited. The reason for this is that in inductive case law analysis it is not possible to make use of a predefined codebook; rather, codes and classification are found and developed during the research process. Moreover, the results still need significant interpretation and explanation, which sets the method apart from some of the methods discussed in Section 4.2.

  3.4 Pitfalls

  It can be seen from the earlier text that ‘classical’ case law analysis has a value of its own in discovering and describing developments and patterns in the Court’s case law, but there are also many vulnerabilities. The greatest challenge is that the strongly interpretative activity of reading, analysing and classifying case law hardly can be undertaken in a fully objective manner (Scheinin, 2017). Researchers, especially when working on their own, can easily fall victim to confirmation bias. In addition, they may read the cases from their own political or ideological perspective without that being clear to the outside reader, or they may disagree on contextual factors that may explain certain outliers or developments in the Court’s case law (McInerney-Lankford, 2017). Indeed, a recent polemic between scholars has demonstrated how differently certain turns of phrase and arguments can be understood in the Court’s judgments and dissenting opinions when the authors have different views of certain core Convention notions or a conflicting understanding of certain historical and political developments (Helfer & Voeten, 2020, 2021; Stone Sweet et al., 2021a, 2021b).
  Theoretically it would be possible to conduct replication studies to check the interpretations provided in a study, but in practice, this is seldom done. As part of the polemic mentioned earlier, for example, some replication has been tried, facilitated by the original study being open about the cases studied and the interpretations and codes used. However, because of the conflicting views just mentioned, the result mainly has been that there are now juxtaposed readings of the same case law. Moreover, systematic replication studies also are not generally considered a useful activity; they are time-consuming and for scholars there is little to be gained from them. When scholars do not have full confidence in the reliability or quality of another scholar’s findings, they seem to prefer to produce a qualitative counter-study of their own or study a different sample of case law.

• 4. Empirical Methods

  As noted previously, qualitative analytical research methods lie at the heart of much of the research undertaken by ECHR scholars and form the bulk of the research into the Court’s case law. Over time, scholars also started to make use of a variety of qualitative empirical research methods that can capture the experiences, behaviours, perceptions or attitudes from Court officials and from other actors and persons engaged with the Court (Webley, 2010). As observed elsewhere, such empirical methods in general, and interviewing in particular, have gained prominence and ‘the appetite for empirical work [in the field of legal studies] has grown rapidly’ (Jaremba & Mak, 2014). Often, Convention scholars combine these methods with more ‘traditional’ methods such as those discussed in Sections 2 and 3. For example, Çalı et al. have relied on empirical methods and used external frameworks as offering benchmarks in developing a particular theory on the legitimacy of the ECtHR (Çalı et al., 2013). Furthermore, empirical methods can be useful to explain or give meaning to the outcomes of other types of studies, including those that use classical qualitative as well as quantitative analysis. For that reason, we offer a brief overview of the various empirical methods that are (increasingly) frequently used in studying the Convention system. Section 4.1 is devoted to the often-used qualitative empirical methods of observation and interviewing, while Section 4.2 turns to the use of statistical information and machine learning methods.

  4.1 Observation and Interviewing

  4.1.1 Function and Use

  As a research tool, ‘observation’ can help achieve a rich understanding of the Court and its practices. For example, it allowed Yildiz to construct a ‘legal culture-based explanation’ for why court room practices, that is, public hearings, (continue to) differ between the European Court and the Inter-American Court of Human Rights (Yildiz, 2019). While previous scholarly work discusses the different historical, institutional and political contexts of the two regional human rights systems, Yildiz added to this by attending and observing public hearings. This method allowed her to elucidate how these distinct contexts have influenced court room practices, in particular the roles afforded to victims and civil society organizations, and why such practices have persisted over the years.
  In addition to attending public hearings, Yildiz conducted in-depth, semi-structured interviews with various actors at the two respective Courts to further explain the ‘way these Courts function [and] their cultural and institutional ethos’. Like Yildiz, many others have resorted to interviews to obtain an ‘inside’ view of how the Court system operates (e.g. Murray, 2020). To mention just a few examples out of many, Hodson has analysed a sample of the Court’s case law to uncover the frequency and type of NGOs that act as legal representatives for applicants. In a second research stage, she interviewed actively litigating NGOs to further understand their motivations and practical experience relating to Strasbourg litigation (Hodson, 2011). Similarly, interviews with Convention insiders have been conducted to build a ‘user’ perspective of the pilot-judgment procedure (Kindt, 2018) or to ascertain ‘first-hand’ perspectives on the remedial practices of the Court (Donald & Speck, 2019). Other interview-based studies have been conducted in an effort to better understand the friendly settlement procedure of the Court (Fikfak, 2022) and to uncover the use and perceived impact of Rule 9(2) Communications submitted as part of the execution process of the Court’s judgments (Erken, 2021).
  While some scholars like Hodson target a particular group for their interviews, others can be seen to include a wide range of actors in their research design to obtain an all-round view of a particular phenomenon under study. Single papers may draw on interviews with ECtHR Judges, members of the Court’s Registry and lawyers that bring cases to the Court (Fikfak, 2022; Yildiz, 2019), while others include a variety of domestic actors, like national judges and politicians who engage with the Court (Çalı et al., 2013). Others, yet again, rely on the input from staff from the Department for the Execution of Judgments of the ECtHR, State representatives and civil society actors (Erken, 2021).

  4.1.2 Benefits and Pitfalls

  The added value of these empirical research methods is clear: they allow researchers to get a glimpse into the ‘black box’ that is the Court, and gain a greater understanding of its functioning, procedures and views of those working within the system. Observations and interviews help to ensure that theory and practice are brought closer together and assessments of the workings of the system as well as possible solutions to improve it are well grounded. Again, however, there are some pitfalls related to this type of empirical research. One of these is its subjective nature; both the interviewer or observer and the actors included in the study are at its very centre and as such can influence the outcomes of the study to a great degree. Another researcher who sets out to study a similar phenomenon in a similar way might achieve different results or may receive different answers to their questions. Additional challenges are related to selecting interviewees and finding sufficient respondents, drafting good interview questions, conducting interviews in a proper manner, transcribing and coding them adequately, and describing and analysing the results (Webley, 2010). Moreover, there are limits to the extent to which the findings are representative for the large issue under study. Associated with this is the difficult question of when and why a researcher may decide that enough data was gathered. Hence, the often-repeated mantra about being explicit and transparent about research choices made particularly applies to the use of such qualitative empirical research methods.¹⁵x It is good practice, for instance, to publish interview questions as a separate annex to increase transparency of the study done and to allow other research to build on the findings. This is, for instance done by Kindt (2018).

  4.2 The ‘Quantitative Turn’ in ECHR Scholarship and ‘Predicting’ the Court’s Judgments

  4.2.1 Statistical and Machine Learning Methods

  The most recent and perhaps most innovative addition to the methodological toolbox of the ECHR scholar is the use of quantitative and predictive research methods. Such methods equip researchers to systematically and quantifiably analyse trends in the Court’s ever-increasing and voluminous case law and to draw generalized conclusions. This allows ECHR scholars to consider the bigger picture; to look at the forest instead of a number of trees. In addition, machine learning-driven research increasingly allows researchers to not only analyse, model, categorize and explain the Court’s case law, but even to predict the Court’s future judgments.
  In order to explain for what research purposes legal scholars can use quantitative research methods, it may be useful to compare this approach to the use of qualitative empirical methods. Quantitative research can be described as allowing researchers to uncover frequencies, relations, correlations and comparisons that are observable and measurable (Webley, 2010). Although the two types of empirical case law analysis can and are easily combined, for reasons of clarity, we focus only on quantitative analysis in the present section – examples of qualitative methods have been discussed in Sections 3 and 4.1.
  Generally speaking, statistical information about the performance of the Court is highly relevant for ECHR scholars. In order to assess the workings and effectiveness of the Court, it is vital to have a sense of the Court’s performance that can be expressed in numbers. Which States are frequently brought before the Court, and which States are frequent violators of Convention rights? What type of systemic problems give rise to repetitive violations and what is the status of the execution of judgments? Such questions can be answered by means of relying on statistical data offered by the ECtHR itself, or through datasets that have been built by researchers themselves from the HUDOC database. For her study on friendly settlements, for instance, Fikfak created a major dataset that includes all the friendly settlements before the Court from the 1980s to 2020. Conducting a statistical analysis of the sample, she has shown how this procedure mostly favours the States that most frequently violate the Convention (Fikfak, 2022).
  Studies such as the one conducted by Fikfak still mostly rely on manually collected and coded data and result in descriptive statistics.¹⁶x This general observation was made in Medvedeva et al. (2019). The age of big legal data now also allows researchers to use computerized techniques to collect case law and machine learning techniques to study the available case law data. To illustrate, Medvedeva et al. have written a computer programme that can analyse the texts of ECtHR judgments and categorize them (Medvedeva et al., 2019). In addition to being very useful in conducting quantitative analysis of case law to discover patterns or for purposes of categorization, as mentioned earlier, such machine learning programmes can even train a computer to offer predictions about the Court’s future case law. So far, this has allowed the researchers to predict with 75 per cent accuracy whether the Court will decide that a particular article of the ECHR was violated, or not. The little robot Juri gives a ‘face’ to this new technique to study the Court. Based on the prediction algorithm written by the researchers, Juri publishes its predictions every month as to whether it expected the Court to find a violation in a particular case (Medvedeva et al., 2020).¹⁷x See https://jurisays.com/.

  4.2.3 Benefits and Pitfalls

  Statistical and machine learning methods can be of great value in searching, categorizing and discovering patterns and developments in the Court’s case law. For example, to be able to predict the outcome of a case before the Court, an algorithm must be trained on large amounts of information of previous judgments to allow it to learn which factors played a role in reaching that outcome. In using this information, machine learning can help identify such factors and can come to a more detailed categorization of judgments based on new patterns it discovered (Medvedeva, 2022). This may ultimately improve the starting point of the qualitative work of many ECHR scholars and reduce their need to study each and every case individually.
  However, it is important to remain aware of the limitations of these research methods. Most importantly, quantitative data and research are sometimes thought to be fully neutral and independent, especially when compared to qualitative research. Yet, individual choices are still made when collecting data, coding and or training an algorithm, which inevitably influence results. In particular, researchers will make choices about the data to be used (e.g. a sample (Hodson, 2011)) or the entire dataset (e.g. Cliquennois & Champetier, 2016) and what is included or excluded in the dataset (e.g. all cases or merely the English-language judgments, leaving those only available in French aside, hence excluding most cases against certain countries, such as France, Belgium and Luxembourg). As Fikfak has explained, sometimes this may lead to uneven results. Such odd outcomes raise obvious questions about what the algorithm in the study was being “fed”, i.e. what decisions it was learning from in order to establish the relevant predictors, how these decisions were selected and whether cases against [a certain State] were over-represented. (Fikfak, 2021)

• 5. Conclusion

  In this contribution we have aimed to unpack the toolbox of scholars studying the Convention system and the case law of the ECtHR. This toolbox proves to be richly filled with a wide array of tools, and its contents have been added to continuously over the past years. Historical, philosophical, theoretical and ‘classic’ deductive and inductive case law research have dominated the field for a long time, but nowadays it can be seen that these methods have been supplemented by many more empirical methods, ranging from QCA and observations or interviewing to statistical analysis and machine learning.
  We submit that there is great value in the richness of the toolbox. ‘Old-fashioned’ tools such as classical, QCA still have great value for answering research questions related to understanding the Court’s case law, to detecting trends and developments in the judgments, and to identifying, typifying and defining certain standards, concepts and tests. Similarly, historical, philosophical and theoretical methods continue to be important for offering descriptive and normative frameworks to understand and criticize the Court’s work and to offer contextual and conceptual insights that are much needed in explaining developments in its approach. Newly added tools such as observation and interviewing are equally of great value, as they can give in-depth and qualitative insights in the perspectives and experiences of actors involved in the system and thus in the everyday (dis)functioning of certain procedures or practices. Finally, although the brand-new tools of statistical research and machine learning still have to be tried and tested, it is clear that they have significant potential in assisting ECHR researchers to do part of the categorization and pattern-discovering work that now is still often done manually.
  Based on these findings alone, it might be expected that we see nothing but promise and progress in ECHR research. Nevertheless, we also have pointed out some weaknesses, limitations and pitfalls for all of the methods discussed. Some general issues relate to difficulties in data(set), sample and case-study selection, to risks of bias and subjectivity, to problems related to a lack of procedural and substantive insight and context-sensitivity in reading and interpreting the case law of the Court, and to the need for clarity and transparency to be offered about a scholar’s normative preferences or the theoretical or philosophical traditions on which that scholar builds. Although these pitfalls certainly exist, it seems to us that over the past years, significant progress has been made in this respect. We have seen that many studies pay much explicit attention to matters of choice and application of methods and offer clarity and openness about the samples, sources and analytical or theoretical approach. More recently, moreover, the increasing popularity of systemic methods of case law analysis helps to address – to a certain degree – the challenges of interpretative subjectivity.
  Surely many more challenges can be mentioned than those addressed in our contribution. In particular, we expect that scholars may hesitate to try a new method because of a lack of knowledge or experience, and thus may not be able to answer certain research questions for which they are needed. Similarly, we see that certain important research questions necessitate a combination of tools that is impossible for a single or monodisciplinary researcher to handle, and teamwork may be needed. There is therefore much need of proper training of both junior and senior scholars in the application of the different tools and in understanding their values and limitations. In addition, we think there is great value in an open exchange of information and experiences in using the various methods, in offering and receiving constructive criticism to the approaches used and, where needed, in building teams of scholars with different methodological expertise to work together on research projects.

References

• Alkema, E.A. (1988). The third-party applicability of the Convention or ‘Drittwirkung’ of the European Convention on Human Rights. In F. Matscher & H. Petzold (Eds.). Protecting Human Rights: The European Dimension. Studies in Honour of Gérard J. Wiarda (pp. 33-45). Köln, Carl Heymanns Verlag 1988).

• Arnardóttir, O. M. (2017). The ‘procedural turn’ under the European Convention on Human Rights and presumptions of Convention compliance. International Journal of Constitutional Law, 15, 9-35. doi:10.1093/icon/mox008.

• Aust, H. P. & Demir-Gürsel, E. (Eds.) (2021). The European Court of Human Rights: Current challenges in historical perspective. Cheltenham: Edward Elgar Publishing.

• Bates, E (2010). The evolution of the European Convention on Human Rights. From its inception to the creation of a permanent court of human rights. Oxford: Oxford University Press.

• Beijer, M. P. (2017). The limits of fundamental rights protection by the EU. The scope for the development of positive obligations. Antwerp: Intersentia.

• Bianchi, A. (2016). International law theories. An inquiry into different ways of thinking. Oxford: Oxford University Press.

• Brems, E. (2022, May 3). Introduction to blog symposium: three recent monographs on the European Convention on Human Rights. Strasbourg Observers. https://strasbourgobservers.com/2022/05/03/introduction-to-blog-symposium-three-recent-monographs-on-the-european-convention-on-human-rights/ (last accessed 23 November 2022).

• Brems, E. & Lavrysen, L. (2015). ‘Don’t use a sledgehammer to crack a nut’: Less restrictive means in the case law of the European Court of Human Rights. Human Rights Law Review, 15(1), 139-168.

• Buyse, A. C. (2019). Why attacks on civic space matter in Strasbourg: The European Convention on Human Rights, civil society and civic space. Deusto Journal of Human Rights, 2019, 13-37. doi:10.18543/djhr-4-2019.

• Çalı, B. (2016). From flexible to variable standards of judicial review: the responsible domestic courts doctrine at the European Court of Human Rights. In O. M. Arnardóttir & A. C. Buyse (Eds.), Shifting centres of gravity in human rights protection. Rethinking between the ECHR, EU, and National Legal orders (pp. 144-160). Milton Park: Routledge.

• Çalı, B. (2018). Coping with crisis: whither the variable geometry in the jurisprudence of the European Court of Human Rights. Wisconsin International Law Journal, 35, 237-276.

• Çalı, B., Koch, N. & Bruch. A. (2013). The legitimacy of human rights courts: A grounded interpretivist analysis of the European Court of Human Rights. Human Rights Quarterly, 35(4), 955-984. doi:10.1353/hrq.2013.0057.

• Cliquennois, G. & Champetier, B. (2016). The economic, judicial and political influence exerted by private foundations on cases taken by NGOs to the European Court of Human Rights: Inklings of a new Cold War? European Law Journal, 22(1), 92-128. doi:10.1111/eulj.12171.

• Dembour, M. (2006). Who believes in human rights?: Reflections on the European Convention. Cambridge: Cambridge University Press.

• Dzehtsiarou, K. (2015). European consensus and the legitimacy of the European Court of Human Rights. Cambridge: Cambridge University Press.

• Donald, A. & Speck, A. (2019). The European Court of Human Rights’ remedial practice and its impact on the execution of judgments. Human Rights Law Review, 19, 83-117. doi:10.1093/hrlr/ngy043.

• Duranti, M. (2017). The conservative human rights revolution. European identity, transnational politics, and the origins of the European Convention. Oxford: Oxford University Press.

• Erken, E. (2021). Non-governmental organisations and national human rights institutions monitoring the execution of Strasbourg judgments: An empirical perspective on Rule 9 communications. Human Rights Law Review, 21(3), 724-751. doi:10.1093/hrlr/ngab007.

• Fikfak, V. (2022). Against settlement before the European Court of Human Rights. International Journal of Constitutional Law, doi:10.1093/icon/moac087.

• Fikfak, V. (2021, May 19). What future for human rights? Decision-making by algorithm. Strasbourg Observers. https://strasbourgobservers.com/2021/05/19/what-future-for-human-rights-decision-making-by-algorithm/ (last accessed 23 November 2022).

• Gerards, J. H. (2017). Procedural review by the ECtHR – a typology. In J. H. Gerards & E. Brems. Procedural review in European fundamental rights cases (pp. 127-160). Cambridge: Cambridge University Press.

• Gerards J. H. (2018). Dealing with divergence. Margin of appreciation and Incrementalism in the case-law of the European Court of Human Rights. Human Rights Law Review, 18, 495-515. doi:10.1093/hrlr/ngy017.

• Gerards, J. H. (2020). Abstract and concrete reasonableness review by the European Court of Human Rights. European Convention on Human Rights Law Review, 1, 218-47.

• Gerards, J. H. (2023). General principles of the European Convention on Human Rights (2nd rev ed.). Cambridge: Cambridge University Press.

• Glas, L. R. (2017). The European Court of Human Rights’ use of non-binding and standard-setting Council of Europe documents. Human Rights Law Review, 17(1), 97-125. doi:10.1093/hrlr/ngw043.

• Hall, M. A. & Wright, R. F. (2008). Systematic content analysis of judicial opinions. California Law Review, 96(1), 63-122. doi:10.15779/Z38R99R.

• Helfer, L. R. & Voeten, E. (2020). Walking back human rights in Europe. European Journal of International Law, 31(3), 797-827. doi:10.1093/ejil/chaa071.

• Helfer L. R. & Voeten, E. (2021). Walking back dissents on the European Court of Human Rights: A rejoinder to Alec Stone Sweet, Wayne Sandholtz and Mads Andenas. European Journal of International Law, 32(3), 907-914. doi:10.1093/ejil/chab058.

• Heri, C. (2021). Responsive human rights. Vulnerability, ill-treatment and the ECtHR. London: Hart publishing.

• Hilbrink, E. (2017). Adjudicating the public interest in immigration law. A systematic content analysis of Strasbourg and Luxembourg case law on legal restrictions to immigration and free movement (Doctoral Dissertation). Vrije Universiteit Amsterdam, Amsterdam.

• Hirsch Ballin, E. (2020). Advanced introduction to legal research methods. Cheltenham: Edward Elgar Publishing.

• Hodson, L. (2011). NGOs and the struggle for human rights in Europe. Oxford: Hart Publishing.

• Hutchinson, T. & Duncan, N. (2012). Defining and describing what we do: doctrinal legal research. Deakin Law Review, 17(1), 83. doi:10.21153/dlr2012vol17no1art70.

• Jaremba, U. & Mak, E. (2014). Interviewing judges in the transnational context. Law and Method.

• Johnson, P. (2016). Going to Strasbourg. An oral history of sexual orientation discrimination and the European Convention on Human Rights. Oxford: Oxford University Press.

• Kindt, E. (2018). The pilot judgment procedure at the European Court of Human Rights: An evaluation in the light of procedural efficiency and access to justice (Doctoral dissertation). Ghent University, Ghent.

• Lambrecht, S. (2022). Convention through states’ eyes. Embedding of the European Convention on Human Rights in State Parties (Doctoral dissertation). Antwerp University, Antwerp.

• Lavrysen, L. (2016). Human rights in a positive state. Rethinking the relationship between positive and negative obligations under the European Convention on Human Rights. Antwerp: Intersentia.

• Leloup, M. (2021). The impact of the fundamental rights case law of the European Court of Human Rights and the European Court of Justice on the domestic separation of powers (Doctoral dissertation). Antwerp University, Antwerp.

• Letsas, G. (2007). A theory of interpretation of the European Convention on Human Rights. Oxford: Oxford University Press.

• Lieblich, E. (2021). How to do research in international law? A basic guide for beginners. Harvard International Law Journal Online, 62, 42-67.

• Loven, C. M. S. (2022). Fundamental rights violations by private actors and the procedure before the European Court of Human Rights. A study of verticalised cases. Antwerp: Intersentia.

• McInerney-Lankford, S. (2017). Legal methodologies and human rights research: challenges and opportunities. In B. A. Andreassen, H. O. Sano, & S. McInerney-Lankford (Eds.), Research methods in human rights. A handbook (pp. 38-67). Cheltenham: Elgar Edward Publishing.

• Medvedeva, M. (2022). Identification, categorisation and forecasting of court decisions (Doctoral dissertation). Rijksuniversiteit Groningen, Groningen.

• Medvedeva, M., Vols, M., & Wieling, M. (2019). Using machine learning to predict decisions of the European Court of Human Rights. Artificial Intelligence and the Law, 28, 237-266. doi:10.1007/s10506-019-09255-y.

• Medvedeva, M., Xu, X., Wieling, M. & Vols, M. (2020). Juri says: Prediction system for the European Court of Human Rights. In Legal Knowledge and Information Systems: JURIX 2020: The Thirty-third Annual Conference, Brno, Czech Republic, December 9-11. IOS Press.

• Minow, M. (2013). Archetypal legal scholarship: a field guide. Journal of Legal Education, 63(1), 65-69.

• Mol, C. (2021). The child’s right to participate in family law proceedings according to International and European Human Rights Law. Silenced, heard, or represented? (Doctoral dissertation). Utrecht University, Utrecht.

• Murray, R. et al. (2020). Righting wrongs: The dynamics of implementing international human rights decisions (Special issue). Journal of Human Rights Practice, 12(1), 1-243. doi:10.1093/jhuman/huaa005.

• Scheinin, M. (2017). The art and science of interpretation in human rights law. In B. A. Andreassen, H-O. Sano & S. McInerney-Lankford (Eds.), Research methods in human rights: A handbook. Cheltenham: Edward Elgar Publishing.

• Schreier, M. (2014). Qualitative content analysis. In U. Flick (Ed.), The SAGE handbook of qualitative data analysis. Los Angeles: SAGE.

• Senden, H. C. K. (2011). Interpretation of fundamental rights in a multilevel legal system. An analysis of the European Court of Human Rights and the Court of Justice of the European Union. Antwerp: Intersentia.

• Smet, S. (2017). Resolving conflicts between human rights. The judge’s dilemma. Routledge.

• Stone Sweet, A., Sandholtz, W., & Andenaes, M. (2021a). Dissenting opinions and rights protection in the European Court: A reply to Laurence Helfer and Erik Voeten. European Journal of International Law, 32(3), 897-906. doi:10.1093/ejil/chab057.

• Stone Sweet, A., Sandholtz, W., & Andenaes, M. (2021b) Walking back dissents: A reply to Helfer and Voeten. EJILTalk! https://www.ejiltalk.org/walking-back-dissents-a-reply-to-helfer-and-voeten/ (last accessed 23 November 2022).

• Taekema, S. (2018). Theoretical and normative frameworks for legal research: putting theory into practice. Law and Method, 1-17.

• Taekema, S. & van der Burg, W. (2020). Legal philosophy as an enrichment of doctrinal research part I: introducing three philosophical methods. Law and Method, 1-19. doi:10.5553/REM/.000046.

• Taekema, S. & van der Burg, W. (2022). Legal philosophy as an enrichment of doctrinal research part II: the purposes of including legal philosophy. Law and Method, 1-19. doi:10.5553/REM/.000065.

• Theilen, J. (2021). European consensus. Between strategy and principle. The uses of vertically comparative legal reasoning in regional human rights adjudication. Baden-Baden: Nomos Verlagsgesellschaft.

• van Hoecke, M. (2015). Methodology of comparative legal research. Law and Method, 1-35. doi:10.5553/REM/.000010.

• Webley, L. (2010). Qualitative approaches to empirical legal research. In P. Cane & H. Kritzer (Eds.). Oxford handbook of empirical legal research (pp. 926-950). New York: Oxford University Press.

• Westerman, P. (2011). Open or autonomous? The debate on legal methodology as a reflection of the debate on law. In M. van Hoecke (Ed.), Methodologies of legal research. Which kind of method for what kind of discipline? (pp. 87-110). London: Hart Publishing.

• Yildiz, E. (2019). Enduring practices in changing circumstances: A comparison of the European Court of Human Rights and the Inter-American Court of Human Rights. Temple International & Comparative Law Journal, 34, 309-338.

Noten

• 1 Van Hoecke (2015), for example, held that ‘[r]esearchers get easily lost when embarking on comparative legal research. The main reason being that there is no agreement on the kind of methodology to be followed, nor even the methodologies that could be followed’ (p. 1).

• 2 We thank all workshop participants for their valuable contributions during the different panel sessions. In particular, we would like to express our gratitude to our co-organizers of the international workshop, Mads Andenæs and Antoine Buyse, for their invaluable input and ideas.

• 3 For a similar, although broader, multidisciplinary approach, see e.g. Lambrecht (2022). For a historical account of, but a different perspective on, the origins of the Convention system, see also Duranti (2017).

• 4 There are also examples of historical studies on a particular Convention right or aspect of the Convention system, see e.g. Johnson (2016) and the different contributions in Aust & Demir-Gürsel (2021).

• 5 See further on this, including the difference between an internal and external framework, Taekema (2018).

• 6 For a structured overview of the added value of legal philosophy in the context of legal doctrinal research see Taekema & van der Burg (2022).

• 7 For less recent examples, see e.g. Dembour (2006), Letsas (2007).

• 8 For a detailed description and overview of such theories, see Bianchi (2016).

• 9 Theilen, for example, sees themselves as ‘interested in social transformation beyond merely incremental adjustments to the status quo’ and held that from that perspective ‘it’s not enough to make suggestions for how the ECtHR could change its reasoning, its role or self-image. We also need to ask ourselves whether we want to pin our hopes of social transformation on the possibility of those changes within the ECtHR occurring’ (Strasbourg Observers [5 May 2022]. Rethinking European Consensus, Reimaging human rights. An interview with J. Theilen, https://strasbourgobservers.com/2022/05/05/rethinking-european-consensus-reimagining-human-rights/ (last accessed 23 November 2022).

• 10 For this reason, Taekema & van der Burg (2020), for example, wrote an article in which they explain three general philosophical methods that, in their view, are most relevant and feasible for doctrinal legal research.

• 11 Studies into the general principles of the ECHR generally make use of this method; see e.g. Gerards (2023).

• 12 See ECHR Overview 1959-2021, www.echr.coe.int; in the period 1999-2021, the average was 1,028 cases, while in the entire period between 1959 and 1999, the total number of judgments handed down was just 837. Moreover, for some research purposes it may be important to also include inadmissibility decisions, which come in even greater numbers; although the Court does not distinguish between pre- and post-1998 decisions, the grand total of all inadmissibility or strike out decisions handed down between 1959 and 2021 is 901,168 (i.e. 14,535 each year).

• 13 For the ‘importance levels’, see ECtHR. HUDOC FAQ: Frequently asked questions, available at https://www.echr.coe.int/Documents/HUDOC_FAQ_ENG.pdf (last accessed 23 November 2022). It can be wise to also include a number of cases of a lower importance level as a kind of ‘control group’, however.

• 14 Alkema explained his limited choice as follows: ‘Completeness would require more than the aid of a word-processor; it would call for a thorough rereading of all the case-law. That is not feasible. Instead, I will briefly mention just a few decisions illustrating Drittwirkung and showing some of its limitations and a few related aspects.’

• 15 It is good practice, for instance, to publish interview questions as a separate annex to increase transparency of the study done and to allow other research to build on the findings. This is, for instance done by Kindt (2018).

• 16 This general observation was made in Medvedeva et al. (2019).

• 17 See https://jurisays.com/.