School of evaluators (CLACSO – FOLEC)

This virtual seminar, which launches the “School for Evaluators,” offers a space for reflection and training on the complex processes of scientific and academic evaluation, focusing on current dynamics and the challenges faced by both evaluators and researchers. At a time of profound re-evaluation of traditional evaluation methods, this initiative aims to address contemporary dilemmas, exploring new approaches and fostering open dialogue among those who perform these key roles in scientific development.

Through the participation of experts in scientific and academic evaluation, the seminar will provide updated theoretical and methodological tools that will allow for critical analysis of existing evaluation practices. Furthermore, it will highlight the importance of incorporating inclusive and critical approaches that take into account issues of gender, territorial equity, interdisciplinarity, and the recognition of knowledge as a public good, particularly from a Latin American and Caribbean perspective.

In this vein, the seminar will revisit and expand upon discussions on open science and responsible academic evaluation, aiming to foster openness, collaboration, reflection, and participation in the production and dissemination of scientific knowledge. It will present debates, dilemmas, theoretical and methodological perspectives, experiences, and policies related to open access, open academic communication, alternative evaluation practices (including open evaluation), and the social mobilization of knowledge, addressing the needs of universities and science and technology organizations in the region.

This proposal arises within the framework of the continuous efforts of CLACSO and its FOLEC program to generate a space for dialogue between national, regional and global actors, promoting the development of an academic and scientific evaluation that responds to the principles of open science and that values ​​the social relevance of scientific knowledge.

The inaugural seminar of the “School of evaluators” offers a space for reflection and training on the complex processes of candidate evaluation and performance evaluation of researchers, allowing participants to address current dilemmas, recent proposals, and discussions surrounding traditional forms of evaluation that are currently under review and being questioned.

This initiative arises from the need to expand the spaces for exchange between evaluators currently in their roles and those aspiring to fill these fundamental positions, which are essential for the normal development of science. It takes into account the current wealth of tools and debates surrounding practices that must be revisited, as they send signals that impact the careers of academics. Policymakers and managers of evaluation, as well as scholars in this field, are also welcome to participate. 

Discussions surrounding differentiated evaluation criteria according to disciplines and interdisciplinary approaches, the recognition of numerous declarations usually foreign to the discussion of disciplinary evaluators (DORA, BOAI, Leiden Manifesto, the Helsinki Initiative, Toluca Manifesto, CLACSO-FOLEC Declaration), the emergence of new categories that evaluators take for granted but whose historicity and etymology they are unaware of when they have to refer to them (knowledge transfer, relevance or pertinence of the research) require a space that is offered here through the mediation of experts in the field who have a critical view regarding forms of evaluation that are intended to be modified.

Disqualifying practices, even denigrating ones, colonial practices, lack of gender perspectives, racist practices, lack of perspectives of positive discrimination for temporary or permanent disability, illnesses, periods of gestation, care tasks and others are common issues that have marked evaluative practices and that need to be reversed.

Experience within the system shows that neither expertise in a subject, nor years of experience training researchers, nor having been evaluated for many years provides evaluators with the necessary tools to evaluate competently. Most evaluators who are new to evaluation committees use their own experience as those evaluated as a foundation for their practice and are eager to learn when provided with information on the aforementioned topics.

We believe that this experience will be an enriching one and represents a stage of maturity for CLACSO-FOLEC after several years of contributing and engaging in dialogue from a Latin American and Caribbean perspective with actors from around the world on the topics that will be addressed at the school.  

Since its inception, CLACSO-FOLEC has undertaken various activities aimed at creating a shared agenda, with a high level of consensus, on the challenges facing evaluation policies. In partnership with researchers, specialists, and leading experts in the field, it has produced numerous diagnostic and policy documents, policy summaries, and research project results that have provided tools to strengthen different components of responsible academic evaluation in the region, in line with international trends in Open Science. At the same time, forums, events, and training seminars have been organized. And in 2022, the FOLEC Declaration of Principles, “A New Academic and Scientific Evaluation for Socially Relevant Science in Latin America and the Caribbean,” was adopted during the 9th Latin American and Caribbean Conference on Social Sciences of CLACSO. 

At this stage, FOLEC aims to launch the “School for Evaluators.” The goal is to create a platform for dialogue among regional and global stakeholders, science and technology organizations, researchers, public policy decision-makers, evaluators in the exercise of their functions, professionals and technicians from the public sector involved in the design and management of Science, Technology and Innovation (STI) instruments, and all those interested in undertaking these fundamental tasks for the development of science.

The seminar aims to:

• To introduce, deepen and update the most recent international and regional debates on open science, in dialogue with current trends in responsible academic evaluation.
• To critically analyze contemporary practices of scientific evaluation, questioning their limitations and challenges.
• Sharing updated theoretical and methodological tools to strengthen academic and scientific evaluation.

• The performative nature of scientific evaluation.
• Evaluative cultures.
• Denaturalizing the categories of excellence and quality in evaluation: contextualizing merit and prestige
• Evaluation and Open Science: initiatives, principles, policies and infrastructures
• Weighting by solving development problems: social relevance and social impact.
• The participation of non-academic agents in scientific evaluation.
• The weighting in the evaluation of the link with extra-academic agents.
• Technology transfer, social transfer and knowledge mobilization.
• Modes of production and circulation of scientific knowledge: co-production, use, usability and social appropriation of knowledge.
• Responsible, situated and relevant evaluation.
• Evaluation and internationalization: scientific centers and peripheries.
• Evaluation and geographical equity: territorial, institutional and disciplinary asymmetries.
• Evaluation and gender: perspectives, debates and experiences
• Bibliometric evaluation and indicators: design, uses, weaknesses and potential  
• The quantitative-qualitative debate in scientific evaluation. 
• Emotions and the conditions of intellectual production and their weighting in the processes of scientific evaluation.

Researchers from all scientific disciplines, officials and those responsible for public policies in Science, Technology and Innovation (STI), as well as those involved in evaluation processes in higher education institutions in Latin America and the Caribbean, professionals and technicians from the state sector involved in the design and management of STI instruments, scientific evaluators in universities and national Science and Technology organizations, and anyone interested in the topic or in playing key roles for scientific development in the region.

In this edition, the seminar will be taught in Spanish. 

The seminar will be held virtually and synchronously on Thursdays at 4:00 PM (GMT-3). Recordings of the classes can be accessed later, according to each student's availability.