Know Your Impact through Action Research: Evidence-based Evaluation of University Teaching in SKILL.de

Evidence-based Evaluation, as a part of Accountability Movement, has already affected many institutions, i.e. governmental organizations, health care, corporations, etc. The same applies to Higher Education: society is asking for verification of the effectiveness and efficacy of instructional practices provided by instructors. Considering the autonomy and authority that instructors always enjoyed at universities, Know Your Impact, as a part of SKILL.de, could be a viable solution. It provides the instructors with skill, knowledge and instruments which are needed to systematically evaluate their own classroom, with the goal of improving their own teaching and students’ learning. Action Research empowers its practitioners to engage in a continued professional development by researching their own practice and enacting knowledge produced by themselves from the evidence. 

I. The challenge: Know-Your-Impact (empirically)

Although it is quite accepted among practitioners in higher education to conduct research in their own fields, when it comes to conducting research on their own teaching and classroom, it sounds foreign to them. Such a sharp distinction between research and teaching has led to an impressionistic, individualistic and unsystematic approach towards teaching, in which almost ‘everything works’, and its consequences are evident in some statistic (i.e. 40% of students who enter STEM majors never graduate). Evaluation is used both as a source of measurement and rich feedback, providing suggestions for changes and improvement. Currently, there are some evaluation measures in higher education, i.e. students’ rating of their instructors, teacher evaluation via standardized tests, or teachers’ perception about their own effectiveness. However, most of them seems to be either inadequate, posing a threat to instructors‘  autonomy, or bringing almost no effect or change in the teaching of the practitioners. 

There is a difference between evaluation conducted on the teachers (by an outsider or so-called expert), and evaluation conducted by and for the teachers themselves. Research indicates that if a human being is involved in a process and conduct it himself, there is a higher chance of change in behavior, attitude and knowledge (learning-by-doing) compared to listening passively to the advice of an expert telling him what works. 

II. Action Research in SKILL.de

The goal of Action Research in SKILL.de is to enable instructors to become independent, professional problem-solvers who can find relevant, evidence-based and context-sensitive solutions to the daily problems they encounter in their classrooms. 

Action Research acknowledges the reality of higher education: instructors have other responsibilities: teaching, research, supervising, conference presentations, meetings, cooperations, etc. Therefore, efficiency is of paramount importance: evidence-based evaluation should be manageable, i.e. it should not take too much time, resources, and energy. Furthermore, it should be within the control of instructors (something that they can change and improve it) and they should be passionate about it. 

III. Procedure: Professionalization through Evaluation

1. It starts with ‘organizational review’. Every discipline has its own objectives, structure, and challenges. We should consider such contextual factors carefully; otherwise, the end-solution will be irrelevant. Together, we identify an area of focus (e.g. effectiveness of a digital technology) or a problem (i.e. students are not motivated, quality of homework is poor, lack of transfer of computational problem-solving to real-life situations, etc). 

2. We will develop a joint design to collect data empirically. The instructor receives intensive trainings about systematic data collection.

3. The instructors collect, analyze and interpret data based on scientific methods. They get extensive individual coaching and feedback through all stages.

4. The instructor takes the role of a reflective practitioner: based on the evidence and analysis, he or she develops an action plan, aiming at improving learning based on his or her understanding and experience gained.  

As a part of the SKILL.de project, Action Research aims at promoting life-long learning: instructors who research their own practice can adapt better to changes and challenges posed by technological development and digitalization. Furthermore, SKILL.de mandates access to continued professional development opportunities. Course Transformation Program, developed by Carl Wieman at the Stanford University, is introduced to provide SKILL.de instructors with one of the most successful instructional intervention (Active Learning) we know to date. 

Project Lead: Prof. Dr. Jutta Mägdefrau

Project Execution: Dr. Sima Caspari-Sadeghi

Publications:

• Forster-Heinlein, Brigitte & Mägdefrau, Jutta (2023, in press).  Action Research in der Hochschullehre: Drei Beispiele aus dem Fach Mathematik. Empirische Pädagogik (3).
• Caspari-Sadeghi, Sima; Mille, Elena; Epperlein, Hella; Forster-Heinlein, Brigitte (2023): Democratizing Assessment for Sustainable Learning in Mathematics Instruction: Giving Students Agency in Classroom Research. In: Institute of Electrical and Electronics Engineers (IEEE), Global Engineering Education Conference (EDUCON). Kuwait: IEEE. Pp. 1–3.
• Caspari-Sadeghi, Sima (2022): Teaching Expertise in Context: How to Evaluate Teacher’s Situated Cognition? In: Higher Education Evaluation and Development, Vol. 16, No. 1. Bingley, UK: HEED. Pp. 1–5.
• Caspari-Sadeghi, Sima; Mille, Elena; Epperlein, Hella; Forster-Heinlein, Brigitte (2022): Stimulating Reflection through Self-Assessment: Certainty-based Marking (CBM) in Online Mathematics Learning. In: Mathematics Teaching Research Journal, Vol. 14, No. 2. Ney York, USA: MTRJ. Pp. 145–156.
• Caspari-Sadeghi, Sima; Forster-Heinlein, Brigitte; Mägdefrau, Jutta; Bachl, Lena (2021). Student-generated Questions: Developing Mathematical Competence through Online-Assessment. In: International Journal for the Scholarship of Teaching and Learning, Vol. 15, No. 1, Article 8. Statesboro, USA: CTE. Pp. 1–5.
• Caspari-Sadeghi, Sima; Forster-Heinlein, Brigitte; Mägdefrau, Jutta; Bachl, Lena (2021): Sustainable e-assessment in mathematics instruction. In: Kollosche, David (Ed.): Proceedings of the Eleventh International Mathematics Education and Society Conference, 1. Hamburg, Germany: tredition. Pp. 57–60.