Evaluation is a dynamic feature of any educational process. Ιt begins before any teaching and learning activity, continues during the whole activity and finishes after the end οf the activity. In the case of the prevention of early school leaving, evaluation is strongly related to the pupils' needs assessment before their engagement in a learning activity in order to identify their learning difficulties. Throughout evaluation teachers can identify pupils' learning difficulties and point out what kind of intervention is appropriate for each pupil in order to prevent early school leaving. Teachers need to be acquainted with the modern meaning of evaluation, evaluation types and forms, evaluation models, techniques and tools in order to become able to use them effectively in their school classes to achieve, among other educational objectives, the prevention of early school leaving.
In this Module, an attempt is made to approach educational evaluation in a theoretical and practical way.
The first Chapter is intended to present and analyze the meaning of evaluation, its distinctions and its necessity in the modern educational reality. In the second Chapter a presentation of the types of evaluation is made and in the third one various forms of evaluation are described. In the fourth Chapter the main evaluation methods are presented and in the fifth Chapter the evaluation subjects are analyzed. In the sixth Chapter a description of the evaluation techniques is made and in the seventh one an analysis of the meaning and content of the meta-cognitive evaluation is attempted. The Module is completed with the eighth Chapter, which includes an analysis of the meaning of the skills that are necessary for a successful issue of any educational activity.
Chapter 1: Semantic Clarifications
Determination of the meaning of Evaluation
Evaluation, as part of the discipline of education, has only recently started to develop in the education bibliography, given that, as a term, it had been closely connected with guidance and discipline, as well as with control and imposition of sanction on both earners and teachers.
There is no consensus among theorists as far as the essence and content of the term “evaluation” is concerned. This is natural for any new term, especially when such a term refers to a new scientific area. According to the view expressed by Sanders (1987), which has been adopted by Webster & Stufflebeam (1978), evaluation is the designation of the value of a thing, a person or a situation.
Guba (1969) mentions three common directions in the content of the evaluation definitions found in the relevant bibliography: a) measuring, b) agreement between goals and practices, c) scientific judgment. Suchman (1967) refers to a distinction between the concepts of evaluation and evaluative research. Evaluation, he maintains, is the judgment process regarding a product, a procedure or a programme, a process which does not necessarily require as a prerequisite the use of systematic procedures or inferences of such a judgment. Evaluative research presupposes the use of research scientific methods and techniques aimed at the evaluation of persons, situations or procedures.
According to Stufflebeam et al (1971), evaluation is the procedure of designed collection and provision of information meant to make selection of alternative solutions easier ( Dimitropoulos, 1999).
According to Galloway (1975) “evaluation is a very broad term, which refers to a continuing procedure of information collection, appreciation of such information and decision-making”.
The term “appreciation” refers to the estimate of the value of a thing or a procedure and substitutes the term “measuring” which does not fully render the psychological and mental dimensions of the procedure he describes. Special importance in that differentiation is acquired by the quality of the “measure”, which is used as an evaluation unit. ( Dimitropoulos, 1999: 25).
The terms examination and marking are related to the evaluation of the learners’ performance at school and the expression of the results in a specific numerical value (Dimitropoulos, op. cit.).
Measuring is the procedure whereby the size or the amount is determined as related to some predetermined measuring unit. More specifically, it is a systematic quantitative designation of properties or characteristics of a subject or an individual. According to Dimitropoulos (1999), measuring refers to a systematic designation of results or characteristics with the use of some kind of assessment measure. Guba and Lincoln (1989) state that the evaluation was based to a great extent on measuring and, in some cases, it was identified with it. This is owed to the scientific character of the measuring procedures.
For the purpose of this work, we consider the term evaluation as the broadest one among all the comparable terms (appraisal, measuring, marking), i.e. as a term serving as an “umbrella”. In other words, by the term evaluation we mean: “the procedure intended to determine as systematically, validly, reliably and objectively as possible the operability and the outcome of a teaching and educational activity, as related to its objectives, but, also, following a specific method” (Constantinou,2004:15).
In view of the above, it becomes clear that for an evaluation procedure to be complete the following conditions should be met: a) its general and special goals should be set, b) the evaluation subjects should be determined, c) the entity that will undertake its implementation should be determined, d) the conditions in which such evaluation will take place should be defined, and e) the tools that will be used in its implementation should be defined.
Evaluation Distinctions
Worthen & Sanders (1987) distinguish evaluation in informal and formal.
Informal is the evaluation that is made by everybody on a daily basis and for a variety of reasons and under a variety of circumstances. The circumstances are related to the daily needs and activities of individuals, while such evaluations are, as a rule, non intentional and subjective.
Formal evaluation is intentional, conscious, and is made by individuals specialized in its materialization. It is carried out in an organized and systematic way with the use of specific tools and techniques, such as questionnaires, interviews, and observation grids (Dimitropoulos, 1999).
Necessity of Evaluation
Over the last few years, at a European level, great interest is shown in the development and implementation of evaluation procedures, for two main reasons: Firstly, it has been ascertained, that there is a need for evaluation of measures regarding institutions, activities, programmes, persons, groups. Secondly, there are direct and indirect pressures felt by public and private agencies like UNESCO and OECD which finance educational activities and programmes, and are interested in the best utilization of the resources they make available ( animate and material).
The need for evaluation procedures in the field of education is reduced to three different levels: the financial, the psychological-pedagogical and the practical (Dimitropoulos, 1999:33).
At the financial level the problem arises from the scarcity of resources and goods. Education absorbs huge funds which have to be invested in a rational way. Consequently, efforts are made to ensure that each fund invested in the educational filed yields the best possible results. The analysis of cost-result constitutes an evaluation method which is aimed at ensuring the aforementioned effectiveness. (Dimitropoulos,1999:34)
The cost-result analysis is a technique whereby the cost and the benefit of any educational investment is correlated. The cost-result analysis is a basic tool, which is used in evaluating educational programmes (Monk,1990, Thomas,1990). The goal of such implementations is to provide reliable information to education designers in relation to the cost-result ratios of the various educational programmes, which may be utilized in every procedure of relevant evaluative assessment of investment options faced by politicians (Karatzia- Stavlioti & Lambropoulos, 2006:137).
The results from the cost-result analysis constitute an important ingredient of any educational policy. In particular, the analysis of the cost effectiveness indicator is all the more becoming a useful and helpful tool in configuring any educational policy, given that, through the relevant calculations, it is easy to estimate and understand the returns from an investment and compare them with those from a different one (Karatzia- Stavlioti & Lambropoulos, 2006: op.cit).
At the psychological-pedagogical level the need for evaluation lies in the idea of understanding and promoting the learning process. That evaluation dimension is directly related to the use of an effective teaching method and the safeguarding of ways that will, on the one hand, promote learning and, on the other hand, will mitigate waste of time, money and effort. Within that framework, the easier adaptation of learners to the educational environment is taken for granted, as well as the easier adaptation of the environment to the needs and characteristics of the individual learners (Dimitropoulos, 1999: 35).
At the practical-administrative level, the need lies in addressing the practical, administrative, educational or other problems, such as the appraisal-selection of personnel, the promotion or selection of learners, the educational planning, and the occupation of posts in the educational hierarchy. Irrespective of the factors that gave rise to the movement for a systematic evaluation and which now push towards its fuller organization and systematization, it is a fact that we now go through a period of evaluation euphoria and this euphoria is expected to get intensified within the next few years.
Undoubtedly, we are now running the period of evaluation systematization as, both, countries of the developed world and countries of the developing world, engage upon overall evaluations of their education systems with a view to a more rational distribution of human and material resources.
Related link:
http://users.thess.sch.gr/salnk/arthra/arthra36.htm
Chapter 2: Kinds of Evaluation
Descriptive Evaluation
Within that form of evaluation, learners are globally addressed. Learners are described not only by their output and performance in the lessons involved, but, also, by their characters, their individualities, their social class communities, their games, their activities and, generally, by the expression of their personalities.
The above description begins with the positive personality points of the learner and goes on gradually to the points where the learner is behind or shows serious weaknesses. For such a description to be complete, it should be detailed. It is not enough to state that the learner is good in any subject X, but reference should also be made to the learner’s performance in the individual dimensions of the specific subject (e.g. oral speech, written speech, oral expression, written expression etc.). Within the framework of descriptive evaluation, evaluated are not just the knowledge acquired by the learner, but, also, the learner’s skills acquired through involvement in the specific subject, as well as the attitudes and behavior adopted by the learner in relation to the subject in question.
That type of evaluation gives a reinforced image of the learner’s progress, together with evaluative judgments and precious educational observations. It is obvious that the usefulness of that form of evaluation is important. Its implementation is difficult, as it requires special knowledge, a lot of time and, when a preset scale is not used, it has inherent subjectivity risks. In spite of the objective difficulties its implementation presents, this type of evaluation is, perhaps, one of the best evaluation types (Frangoulis, 2008).
Evaluation using standard designations
This type of evaluation is in accordance with natural evaluation, i.e. the kind of evaluation taking place every day in our lives, in our relationships with other people. It is specific and comprehensive. The weakness of such a type of evaluation is that in very few words there are a lot and usually varied thoughts included. Thus, one-word characterizations run the risk of obscurity and misinterpretation. For example, the adjective “good” is differently interpreted by each learner or teacher.
For that type of evaluation to be effective, it has to be continuous and materialized in a systematic way. Through an evaluation with the use of standing characterizations, the positive points of the learner’s efforts are marked, the negative points are found out and ways for their remedy are proposed. Under such circumstances, the learner’s anxiety is mitigated and evaluation acquires essence and content (Frangoulis, 2008).
Evaluation with the use of a numerical scale
It concerns the type of evaluation whereby learners are evaluated with the use of a specific scale, usually one of ten grades, twenty grades or even one hundred grades. In such an evaluation form, the evaluation result is expressed with use of one number alone which is used as an indicator of the learner’s performance in relation to some specific subject. With scales of greater ranges, one would think that a more accurate evaluation can be achieved. (Frangoulis, 2008).
However, research work has shown that more error possibilities exist in the broader scales. Nevertheless, in cases where smaller range scales are used an error acquires larger dimensions because in such scales learners of great learning divergences are squeezed.
Marking by means of numeral scales gives the impression that it concerns an easy case and that it shows accurately and explicitly a learner’s learning condition. In reality, however, that view is not valid, as evaluation by means of numeral scales represents the expression of an assessment in relation to the range of the graded scale involved and lacks qualitative – descriptive elements, which can be accurately and explicitly expressed within the framework of evaluation by means of a descriptive scale.
Related link:
http://www.pi-schools.gr/download/publications/epitheorisi/teyxos1/e1%20172-186.doc
Chapter 3: Evaluation Forms
Initial or Diagnostic Evaluation
This form of evaluation is implemented before the start of any educational programme or a teaching unit in order to establish the terms and conditions that may affect positively or negatively the materialization of the educational goals, whereupon needs and priorities will be determined. This form includes mainly procedures contributing to the determination of the learner’s standard of knowledge, abilities and attitudes so that the subject, content and teaching pace may be adjusted accordingly Mavrogiorgos, 2006: 278).
It is advisable to carry out a diagnostic evaluation even at the start of each teaching unit or hourly lesson. This form of evaluation can be carried out by the teacher by giving the learners a test of knowledge or abilities or by a thorough discussion with them, whereby what they know in relation to the specific teaching subject can be found out. Such form of evaluation will help the teacher organize the subject matter in a better way, distribute the time available in a correct way, adopt adequate teaching techniques and respective supervising means, which will be functionally related to the subject and the goals of his teaching work.
Interim or Formative Evaluation
By the term interim or formative evaluation we understand the kind of evaluation that takes place during the implementation of a programme or a teaching unit and it is aimed at improving the internal operation and the expected results of that programme or that teaching unit ( Karalis, 1999:128).
Through formative evaluation what is expected to be achieved is a feedback for the teachers involved in the implementation of the programme or the teaching unit involved, with the information that is necessary in any corrective action for improved implementation of the teaching work. Formative evaluation is characterized by:
Identification of any divergences in relation to the initial design of the programme and formulation of proposals for the achievement of the initial goals.
Identification of possible alternative solutions in relation to those that have been followed and which have been characterized as inadequate.
A continuous “dialogue” between the teacher/s and the participants in the programme or the teaching unit.
In case divergences are found out, the initial choices are reassessed and the right conditions are created for the improvement of those points where the evaluation procedure showed existing problems. ( Mavrogiorgos, 2006: 279).
Summative or Reporting Evaluation
By summative or reporting we mean the evaluation that is carried out after the end of any educational programme or teaching unit and is aimed at drawing conclusions and documented evidence in relation to the value of the programme or the teaching unit involved. Summative evaluation is characterized by:
The formulation of final conclusions in relation to the outcome of the educational programme or teaching unit.
The formulation of judgments in relation to the achievement of the programme’s goals and identification of the factors that contributed to the configuration of the final product.
Comparison of the quality of the implementation and results of the programme or the teaching unit involved with another relevant programme regarded to be a “standard” programme. ( Karalis, 1999:129).
Final evaluation does not lend itself so much to the learners’ feedback as it does to that of the educational work, which will be the basis for the reexamination of approach methods with a view to their improvement. In such a case a “summative evaluation” becomes “formative evaluation” for future implementations.
Important indicators in such cases are:
- The views and assessments of the learners as to the adequacy of the methods, techniques, means, communication and cooperation framework and organization adopted.
- The extent and degree to which the learners can apply and transfer what they have learned to the real world in similar situations (Mavrogiorgos, 2006: 279).
Internal - External Evaluation
A secondary distinction of evaluation types is that which is based on the position held by the agent of the evaluation. When the agent of the evaluation comes from the organization implementing the programme or is related to it, the evaluation produced is characterized as internal, whereas, when the agent of the evaluation does not come from the organization implementing the programme or is not related to it, the evaluation produced is characterized as external (Karalis,1999:133).
In the case of internal evaluation the agent of the evaluation:
- Knows the educational environment wherein the programme is implemented, as well as the special conditions prevailing therein.
- Can readily gather information relating to the method and procedure used in the programme implementation.
- Is more interested in the development of the educational environment within which the programme was implemented and, therefore, is more interested in the programme evolution.
- Is in a position, upon the completion of the programme, to apply the evaluation conclusions, directed towards an improved programme.
In the case of external evaluation, the agent of the evaluation:
- Has no dependence on the educational environment, wherein he/she attempts the evaluation, nor is he/she related to the staff of the programme.
- Is not bound by the evaluation results or by the programme’s evolution.
Being outside the agency, within the framework of which he/she attempts to implement the evaluation procedure, he/she is in a position to configure a more global opinion regarding the programme under evaluation.
Related link:
http://www.ecedu.upatras.gr/didinfo/eishghseis_DIDINFO08/DIDINFO08_373_381.pdf
Chapter 4: Evaluation Models
The term evaluation model defines an integrated approach to the evaluation of any programme, which, apart from the speculative framework, includes proposals regarding the scopes and individual goals of the evaluation, its implementation range, the value and use of the results, the roles of those practicing the evaluation and the roles of the participants in the programme, the research methods and techniques and, lastly, the scopes and axes of the evaluation (Karalis, 1999:135).
Although certain evaluation models are more compatible with certain evaluation types or even with certain research methods and techniques, the selection of an evaluation model depends to a great extent on:
The scope and goals of the evaluation.
The characteristics of the programme under evaluation (programme type, learners’ characteristics).
The institutional framework and the specifications in force for the implementation of the evaluation procedure.
Reciprocal Evaluation
By proposing the reciprocal evaluation model, Robert Stake aimed at widening the participation of factors, within an education programme, in the evaluation procedures and decision-making. This kind of evaluation, which is based on reciprocity, has the following characteristics:
- It promotes renegotiation of the programme involved, during its course and on the basis of the emerging problems.
- Is focused more on the programme actions and not on the programme’s initial design.
- It responds to the participants’ demands for updating and information in relation to the programme’s course.
- It regards “success” or “failure” of a programme as relative meanings which are defined on the basis of value systems which have to be stated in the reports of the final judgments. (Dimitropoulos, 1999).
Existing situation: At this level, we refer to the situation current at the start of the educational intervention. In order to determine that situation, we analyze the characteristics of the learners (knowledge, skills, expectations, attitudes), teachers and those directly or indirectly involved in the educational procedure.
Intervention: At this level, the educational intervention is described with special emphasis on the interaction between the various programme factors.
Results: At this level, we describe the results that emerged from the implementation of the programme and, mainly, the results referring to the learners (knowledge they acquired, skills they developed and attitudes they adopted).
In designing and implementing any educational programme or teaching unit based on the reciprocal evaluation model, the agent of the evaluation:
Seeks the broader possible participation of all the programme participants, by providing information to them during the implementation of the programme.
Encourages the participants to express their judgments regarding the improvement of the programme and the learning procedure.
Analyses the relation: existing situation --> intervention --> results with a view to determining the crucial points for the evolution of the programme.
Formulates final judgments, taking into consideration the varied criteria set by the programme participants (Karalis, 1999:138).
Four-Level Evaluation
The model of evaluation at Four Levels was proposed in 1959 by Donald Kirkpatrick and referred to the evaluation of the educational programmes. The four levels proposed by Kirkpatrick regarding the evaluation of educational programmes or teaching units are as follows: 1) the response level, 2) the learning level, 3) the behaviour level and 4) the results level.
The response level concerns the response of the learners of the programme regarding the content and the goals of the programme and their stands against the programme’s individual functions. A positive stand on the part of the participants does not necessarily mean that the programme’s goals have been achieved.
The learning level is defined as the degree to which the learners change attitudes, increase their knowledge and improve or develop their skills by their participation in the programme. The evaluation of learning requires the designation of goals for each programme or teaching unit under evaluation ( Καραλής, 1999:140).
The behavior level concerns the change in the learners’ attitude on account of their participation in the programme. The changes in the behaviour of the learners acquire greater importance if in the two preceding evaluation levels they were negative for the programme. As long as the behaviour is revealed in one of the criteria that determine the extent of the programme’s success, the description of the existing organizational and institutional climate of the programme’s materialization constitute basic elements which are made available to the programme’s designers and teachers during the preparation and materialisation phases.
The results level is assessed on the basis of indicators referring to the contribution of the learners to the programme’s goals and concern the quality of its implementation, as well as to the achievement of its goals. The procedure of results assessment takes place at the end of the programme and adjudicates the effectiveness of the specific educational programme or teaching unit.
Within the framework of evaluation at four levels utilized for the evaluation of education programmes or teaching units utilized, we realised that the evaluation procedure was determined to a great extent by the needs of the agency implementing the programme concerned, whereas the entity practicing the evaluation functioned rather as a consultant and less as a factor of improvement or modification of the training intervention itself. In this specific model, special emphasis is given on the cost-effectiveness analysis and especially on the identification of the extent to which the goals of the programme or the teaching unit have been achieved. These elements have made the model of evaluation at four levels the model that is mostly used in evaluating educational programmes or teaching units ( Karalis, 1999:140).
CIPP Evaluation Model ( Context, Input, Process, Product)
The CIPP model was first proposed in 1976 by Daniel Stufflebeam and formed the first comprehensive proposal for the evaluation of educational programmes. With various individual improvements and revisions, this model has been used over the last few years by many educational organisations and has become a subject of dialogue and concern in the field of evaluation of programmes.
The acronym CIPP comes from the words Context, Input, Process, and Product and concerns the four distinct model phases, which are described as follows:
The context evaluation phase concerns the recording of the strong and weak points of any education programme or teaching unit and the determination of the actions that should be taken towards improvement. When probing into the needs of population-goal, the identification of possible interaction between the programme and the broader environment, as well as the identification of the difficulties and problems that may possibly emerge, constitute basic goals of the context evaluation. An important ingredient is the examination of the response of the programme’s goals to the needs and special characteristics of the learners. ( Karalis, 1999:141).
The input evaluation is targeted at the configuration of a strategy which will be followed during the implementation of the programme or the teaching unit. The individual actions of this phase include evaluation of the varied strategies, creative utilization of similar successful educational practices and identification of the limitations imposed by the results of the preceding phase. The scope of this phase is the selection of the most adequate strategy, as related to both, the needs, the interests and the special characteristics of the learners’ group and the limitations imposed by the broader organizational and social framework (Karalis, 1999:op.cit.).
The process evaluation concerns the follow-up, the assessment and the continuous supervision aimed at the achievement of the goals and the strategy of the educational programme of the teaching unit. The scope of the process evaluation is the feedback of the participants in the learning process in relation to: a) the extent of achievement of the goals of the programme or the teaching unit, as these were set by the preceding phase and b) the effective use of the resources available. Recording and ascertaining the problems that were not foreseen in the course of the preceding phase and the identification of weaknesses, which lead to inadequate completion of the individual actions, constitute fundamental aspects of the process evaluation.
The product evaluation is targeted at the assessment of the results of the programme or the teaching unit and includes the recording and analysis of the positive and negative effects shown during its implementation process. Decisions relating to the continuation, improvement, extension or discontinuation of any education programme form the end of the product evaluation.
Evaluation based on the aforementioned evaluation model may be formative or summative. In the former case, emphasis is given to the support of the participants in the programme designing and implementation procedures. In the latter case, emphasis is given to the recording of actions and the assessment of the results. (Karalis, 1999:142).
Related links:
http://dspace.lib.uom.gr/handle/2159/3438
Chapter 5: Evaluation Subject
Evaluation of any educational programme may refer to all the aspects of the educational process or to some of them. What I to be evaluated evaluate in an educational programme depends to a great extent on the goals set for the evaluation process, in combination with the axes to be evaluated in a more thorough way. The main factors usually evaluated in an educational programme or teaching unit are: a) animate factors, b) non animate factors and c) the programme’s learning results.
Animate Elements of Evaluation
The main animate factors evaluated in an educational programme or teaching unit are the teachers and the learners.
The teachers are evaluated as to: a) their knowledge; b) the relationships they developed with their learners; and c) the learning climate they created during the implementation of the lesson involved.
In relation to the knowledge axis, some examples of questions that may be asked are:
- Did the teacher have an in-depth knowledge of the subject he/she was expected to teach?
- Was he/she adequately prepared in relation to the subject he/she was expected to teach?
- Was he/she looking into the educational needs of the learners during the course of the lesson?
- Was he/she directing the learners satisfactory towards sources of knowledge for further exploration of the educational subject?
In relation to the axis of relationships, the following questions could be asked:
- Did the teacher develop equal communication and cooperation relationships with thw learners’ group?
- Did he/she give the opportunity to the learners to express their emotions and thoughts in an accurate and clear manner?
- Did he/she understand the needs and aspirations of the learners?
- Did he/she take into consideration the special learning characteristics and culture of the learners?
In relation to the learning climate the following questions could be asked:
- Did the teacher take advantage of experiential educational techniques during the course of the lesson in order to make the learners as fully productive as possible?
- Did he/she set as the focus of educational procedure the learners and their special characteristics?
- Did he/she use the discovery course towards learning?
- Did he/she promote critical thinking?
- Did he/she create equal communication relationships with the learners?
Learners are evaluated in relation to: a) the knowledge they have acquired, the skills they have developed and the attitudes they have adopted as a result of their participation in the specific learning subject, b) the utilization of modern learning principles during the lesson, c) the communication climate and the relationships they developed with their peers during the lesson.
In relation to the knowledge they have acquired, the skills they have developed and the attitudes they have adopted, the following questions could be asked:
- Did the learners acquire knowledge in subjects relating to the content of the learning unit?
- Did they develop skills in subjects relating to the content of the learning unit?
- Did they adopt attitudes-behaviours in subjects relating to the content of the learning unit?
In relation to utilization of modern learning principles during the learning activity, the following questions could be asked:
- Was the learning during the lesson autonomous or dependent?
- Did the learners make good use to a considerable extent your experiences during lesson?
- Were the learners at the focal point of the learning process during the learning activity?
- Was the finding-discovering course towards knowledge followed?
- Was the learners’ role active during the teaching process?
In relation to the communication climate and the relationships they created with their peers, the following questions could be asked:
- Did you develop equal communication relationships with your peers during the lesson?
- Did the learners have the chance to express openly their emotions and thoughts?
- Did the learners boldly express their training needs and expectations from the lesson?
- Did the learners understand the needs and aspirations of their peers?
- Did the learners make good use of active exchange of messages during their communication with their peers?
Non-Animate Elements of Evaluation
The main non animate factors evaluated in a training programme or a teaching unit are: a) the design of the programme or unit, b) the techniques and means used for their implementation.
In relation to the design of the programme or teaching unit the way in which the overall design of the programme or unit is examined, as well as the individual thematic interrelations. The teacher may evaluate the design of the lesson in all the phases of its course (beginning-middle-end).
Examples of questions that could be raised are:
- Have the needs of the learners needs been identified?
- Have the educational goals of the lesson been accurately and explicitly expressed?
- Are the learning activities connected with the goals of the lesson?
- Is the time available adequate for the completion of the learning activities?
In relation to the techniques and means used for the implementation of the lesson, what has to be examined is the extent to which the educational techniques and means activated the learners and aroused their interest, enriched the ways in which knowledge is approached, connected the new knowledge and skills acquired through participation in the learning process with daily practice.
Examples of questions that can be raised in evaluating the educational techniques and means are:
- Was there any use of educational techniques and lessons that aroused the learners’ interest during the lesson?
- Was there any use of appropriate educational techniques and means according to the teaching goals, learning content and special characteristics of the learners?
- Were the educational means used connected with the learning content and the special characteristics of the lesson?
Learning Products
Effectiveness of any educational programme or teaching unit is evaluated in connection with: a) the learners’ response to the content of the learning process, b) the knowledge and skills adopted by the participants as a result of their participation in that process and c) the behaviour adopted by the learners as a result of their participation in the educational process.
In relation to the learners’ response to the content of the learning procedure, what is to be evaluated are the reactions, impressions and emotions of the learners with regard to the content and the methods used. Moreover, the degree of the learners’ satisfaction from the content of the learning procedure has to be identified.
In relation to the knowledge and skills adopted by the participants as a result of their participation in the learning procedure the facts evaluated are the knowledge acquired, the improvement of abilities and the change in the attitudes that have resulted from their participation in that process. In other words, what is checked is the extent of success in achieving the goals that had been set during the learning procedure.
In relation to the behaviour of the learners, the evaluation concerns the degree of improvement or modification of their behaviour in respect of the content of the learning procedure through their involvement therein.
Examples of questions that could be asked in relation to the learning results are:
- Have the learners been satisfied with their participation in the learning procedure concerned?
- Have they acquired knowledge and developed skills relating to the content of the teaching unit?
- Is there any change in the learners’ behaviour as a result of their participation in the learning procedure?
Related articles:
Related links:
http://www.aea-europe.net/
http://www.reva-education.eu/
Chapter 6: Evaluation Τechniques
Questionnaires
Questionnaires provide information on the evaluation concerning the response of the learners to the programme as a whole. The questionnaire includes questions of open and closed type or graded scale and may be filled in before the start of the programme, during its implementation and upon its completion.
As an education technique, it is regarded to be the most frequent one used in evaluating educational programmes. Moreover, it is considered as an easy and efficient method for the collection of data in respect of any education programme and the response of learners thereto. The questions may cover one or more axes, depending on the goals being investigating, the duration of the programme and the time available for evaluation (Mavrogiorgos, 2006:291).
Diagnostic Tests, Educational Activities
The diagnostic tests given during and at the end of the educational programme are used to indentify the learners’ learning progress.. The tests may vary in form and content. They may include learning activities of an experiential nature (e.g. playing parts, group work) as a supporting material in determining the extent to which the programme’s learning goals have been achieved.
As stated by Kirkpatrick, evaluation by means of diagnostic tests gives information not only regarding the achievement of the goals, but, also, on the teachers’ effectiveness, a fact that helps the teachers themselves to improve elements of their teaching practice. In other words, through the diagnostic tests, the following scheme is utilized: I act <--> I observe <--> I reflect <--> I revise (Kirkpatrick, 1994:42-43).
Observation
The education technique of observation provides information regarding the programme’s development, the achievement of its goals, the part played by the teacher therein, the relationships of the learners between each other and the teacher participating in the programme. Observation may take place both, in the classroom and in the field. In terms of time, it may take place before and after the learners’ training - especially if its objective is the assessment of knowledge and behavior. Deviation from the initial programme design may be identified through observation. Subsequent specific action may then be taken for the improvement of the programme concerned/ a particular programme.
Patton (1990) indicates that through direct observation the observer can understand the framework wherein the functions and activities of the educational programme take place and thus can draw useful conclusions. Observation, as a tool of data gathering, can also be combined with other methods and techniques for the achievement of reliable results. (Karalis,1999:32).
Tape-recording of an educational meeting
Tape-recording helps in recording educational incidents, as well as in further analyzing and processing such incidents. Tape-recording can show the development of interaction between teacher and learners during the implementation of the programme. It can, also, yield considerable information on the way the learning process is materialized, the educational methods and techniques followed the part played by the teacher, and the relationships developing between the learners. By listening to the tape, the teacher can identify revisable elements in the teaching process, find out weaknesses or omissions and reflect with a critical on the possibilities of further improvement.
Critical hearing of the recorded incidents and analysis of those incidents require a “code of hearing and analysis” which ensures an analysis focused and carried out in a systematic way.The tape analysis technique may be combined with the observation technique (Mavrogiorgos, 2006:290).
Video-recording of an educational meeting
The advantages of videoing educational meetings lie in the fact that there is image as well as sound. This reveals the places of the learners in the classroom, their reactions, their movements, the changes in their moods, their emotions and the frequency in which such changes take place, their body language
This technique may be embarrassing the participants, particularly if they are not aware of its implementation or if they are not familiar with such procedures. Therefore, the learners need to be informed about the implementation of the videoing in a timely manner and, also, there is need for preparation in how to address their emotional expresses. The initial unpleasant feelings can be dealt with in a satisfactory manner if a climate of cooperation, mutual respect and trust has been ensured beforehand ( Mavrogiorgos, 2006:290).
Interviews given by learners
Interviewing is a technique whereby the interviewees are asked to freely express their views on issues relating to the educational programme they followed, such as in relation to its content, its organisation, the method in which it was implemented, the knowledge they acquired, the skills they developed as a result of their participation. In the course of the interview, a relationship is created between the interviewer and the learners (Dimitropoulos, 1999:194).
The interview criteria may take either of the following two forms: 1) the form of personal contact ( face-to-face), 2) the form of telephone contact.
The manner in which the interview is organised and carried out may be: 1) standardized- structured, 2) free.
Standardized – structured is the interview whose course and content are preset and the procedure is based on the existing interview plan (Dimitropoulos, 1999:195).
Free interview may take a variety of forms along an axis of the interviewee’s freedom, from an interview with a delimited framework within which the interviewee may move to an interview of complete freedom of moves. Even a free form of interview needs an overall preparation of the framework of the questions to be asked Dimitropoulos, 1999: op.cit.).
Related link:
http://ec.europa.eu/education/index_en.html
Chapter 7: Meta- cognitive Evaluation
Semantic Clarifications
Either we refer to the evaluation of an educational system or to the evaluation of a part thereof or even to the evaluation of an educational programme or an educational unit, the question always raised is that regarding the quality, the correctness and the usefulness of the evaluation in question.
In other words, the evaluation itself becomes the subject of the evaluation (Dimitropoulos, 1999:332).
It cannot be taken for granted that every evaluation system is correctly designed. It cannot be taken for granted either that every evaluation procedure, however well planned, is correctly implemented and that the evaluation results are successful in any case. To ensure that nothing of the above will occur, metacognition procedures are implemented.
The term “metacognition”, was first used by Flavell (1976) and established thereafter in the relevant bibliography when referring to the consciousness and knowledge of a person regarding the procedures of thought he/she follows, as well as his/her ability to plan, foresee, direct and evaluate his/her thoughts (Matsangouras, 1997:75).
The scope of metacognition activities, which have been developed within the framework of critical mind, is to help those participating in an educational activity to become aware of the manner in which they think when trying to carry out a cognitive task, It is also to establish their resolution to deal with difficult cognitive tasks, and support their self confidence in their cognitive abilities (Frangoulis, 2003:287-288).
Metacognition evaluation appears at the end of an educational programme or a teaching unit and gives the wrong impression that, as a procedure, it takes place at the end of the training activity. As a matter of fact, the questions involved in a metacognition evaluation are asked throughout the implementation of any programme and not only at its completion. Thus, a continuing evaluation of the evaluation procedure itself is ensured, either it takes place at the start or during or at the end of the programme.
The content of Meta- cognitive Evaluation
The content of metacognition evaluation is related to the kind of knowledge, skills and attitudes acquired by the learners as a result of their participation in any learning procedure (Matsangouras, 1997).
The knowledge concerned refers to the procedures used by any individual, as well to how and when he/she can use them, either in order to analyze or to address a difficult situation or to set and materialise his/her personal pursuits.
Metacognitive skills are distinguished into two categories. The first category includes the simplest metacognitive skills, such as that of self correction, while the second includes more complicated skills, such as the systematic guidance of the thinking process in solving problems (Bruer,1994:280).
The attitudes are connected with critical mind and for them to be expressed, they have to become subject of systematic teaching. Examples of such attitudes are the passion for clarity and accuracy, readiness to reexamine issues, pedantry in performing a task, respect and search for different views (Matsangouras, 1997).
Requirements for an effective implementation
Valid and reliable implementation of any evaluation procedure is based on the use of preset subjects, criteria, evaluation type, approach, methods and techniques, incorporated in a specific procedure. (Dimitropoulos, 1999).
The agent of the evaluation concerned, in order to be sure that he/she performed the evaluation task in a methodically flawless manner, should be able to answer the following questions:
- Was the evaluation procedure correctly designed?
- Was it correctly incorporated in the programme and at the appropriate time?
- Was evaluation being implemented throughout the duration of the programme?
- Were the right special subjects identified and analyzed into their individual dimensions?
- Were the scopes of evaluation expressed in the right way and at the appropriate time?
- Were the criteria correctly used in the evaluating process?
- Was the right procedure followed?
- Was the agent of the evaluation procedure the right agent for the right task?
- Was the right method used?
- Was the analysis of data performed in the right way?
- Did the agent of the evaluation give the right interpretations?
- Were errors avoided during the implementation of the evaluation procedure?
- Was the evaluation report drafted in a methodically flawless way? (Dimitropoulos, 1997:163).
Related links:
http://www.alfavita.gr/ASEP/asep20061007b.php
Chapter 8: Development of Skills
Semantic Clarifications
In accordance with the
CEDEFOP(2002), skill is defined as the knowledge and experience that a person has and which are necessary for the performance of a specific duty and ability is defined as the proven capacity of an individual to utilize the know how, skills, qualifications or knowledge in tackling successfully both, familiar and new situations and requirements. A skill usually concerns the performance of jobs learned through practice without great depth required regarding the logic behind them.
On the basis of the aforementioned differentiations, we propose the use of the term “social skills”, according to the current social requirements.
Nevertheless, there is still confusion in the use of the term “social skills”. In reviewing the bibliography, it is hard to discover a specific and clear-cut picture of the total of skills under the term “social”, since it is encountered in many variants such as: conveyable skills, basic skills, emotional skills. However, irrespective of the absence of homogeneity in the use of the term, there is an indisputable nucleus of specific skills shared by every definition concerning the sense of “social” (
Papadakis & Frangoulis, 2006).
Forms of Learners' Skills
One of the main groups of skills included in the term “social” is that of communication skills, which allow a learner to express clear and convincing messages and to have active listening (Goleman, 2000). It is precisely because the modern learning and training conditions involve many learners in one single task, the ability of the learners to communicate with each other in an effective way in order to understand what needs to be done in every situation constitutes a key point.
The “repertoire” of social skills includes even those of cooperation and team work. In other words, one should be in a position to cooperate with others towards the achievement of common goals and, at the same time, be able to foster and preserve cohesion in the team for the achievement of those goals (Goleman, 2000). The modern role of learners includes considerable time in team works and efforts. The training of individuals on how they should operate within a team is particularly important.
The ability in tackling disputes and effective negotiations (Goleman, 2000), constitutes a social skill and is regarded to be invaluable in day-to-day practice, since, even if the learners have developed their communication and cooperation skills, they very often come face-to-face with conflicts, crises and disputes with fellow-learners, who have completely different opinions about the manner in which a case should be approached or resolved.
In addition to the aforementioned social skills, another skill that has to be developed by all learners is that of accepting and tackling the ongoing changes in the practices of their tasks. New training methods are applied and new training techniques are tried aiming at a more effective learning process. Therefore, they have to show a positive attitude each time a change is planned and be able to accept it in a creative way in order to be productive. Consequently, flexibility and adjustability acquire a new value (Goleman, 2000).
Together with acceptance of change comes the social skill of the learner to keep learning and be able to learn how to learn. Of fundamental importance is the achievement of skills allowing individuals to keep themselves busy with further learning, which is the basis of personal completion and active participation in the learning process.
Additional necessary ability is that of organizsation. Current learning conditions require learners to be able to set their priorities right, organise their study effectively and distribute their time in a way that will allow them to meet deadlines.
Of course, none of the aforementioned skills alone can ensure the effectiveness of the learners. Skills are set in groups. For one to achieve a distinctive performance one should have a combination of skills and not just one or two (Goleman, 2000). This is what makes their development effort even more difficult, as systematic practice is required in many and varied situations.
In addition to the skills that learners should command for an effective learning climate, analogous activities should, also, be set by the teachers.
Quality Indicators for the assessment of Social Skills
The quality indicators are a simple or complex, usually, quality expression correlating the various structural elements or parameters of education and its procedures and helping in the decision-making is aimed at its improvement (Savelson et al., 1987). They constitute a basic tool in the procedure ensuring quality in a learning process. They provide accurate data regarding the achievement of the educational procedure goals and necessary information in order to render the learning procedure more functional and effective at certain points required Vassilou-Papageorgiou, 2004). Moreover, the use of such indicators makes the standardisation of the results easier and helps comparisons, where comparisons are necessary (Van den Berghe, 1997). The determination of quality indicators supports reflection and discussion about the desired results among the organizsers of any learning procedure .
In the field of Education, over the last few years, an effort is made to set criteria – quality indicators. Whereas, during the preceding years reference was made to the scopes, goals of the learning procedures, as well as to the educational results at the level of qualifications, the interest now is focused on learning results and the evaluation of the learning procedures (Vassilou- Papageorgiou, 2004).
The teachers themselves give special importance to the teaching quality provided to them by the school; namely to what they learn, how they learn, by means of what methods (European Commission,2002). Targeting at the assessment of the skills that the teachers develop through their participation in educational activities, we proceeded with the design and development of the following quality indicators.
As quality indicators for communication, we propose:
- Equivalent communication between learners and teachers
- Communication frequency between learners and teachers
- Use of multiple communication forms (verbal, non verbal)
- Presence of appropriate communication environment
- The degree to which the needs of the learners are met
- The creative utilizsation of disputes
- The creation of an active listening climate among those participating in the learning procedure ( teachers- learners)
As quality indicators for team work and cooperation, we propose:
- The interaction degree between the members of the educational team (learners- teachers)
- The awareness of membership property on the part of the individuals forming the educational team
- The presence of common goals among the members of the educational team
- The acceptance degree of the operation rules governing the educational team on the part of its members
- The development of collective awareness spirit on the part of the members of the educational team
- The extent to which the training needs and expectations of the members of the educational team (learners – teachers) are met
As quality indicators in respect of dispute tackling, we propose:
- The extent to which the presence of a problem is understood
- The basic ingredients consisting the causes which generated a problem
- The degree of contribution to the resolution of a dispute
- The initiatives undertaken for the resolution of the dispute
- The selection of the best possible solution addressing the dispute
- The time spent for the resolution of the dispute
As quality indicators for the acceptance and tackling changes, we propose:
- The extent to which changes in the social and wider environment are accepted
- The extent to which changes in the school environment are accepted
- The time spent for adjustment to evolutions and new data
As quality indicators for the organizsation we propose:
- The determination of basic priorities in the learning procedure
- The hierarchy of the learning activities (necessary activities, useful activities, supplementary activities)
- The extent to which the goals have been achieved in the time limits provided
An example of a
Scale for the assessment of student’s social skills
Related links:
http://www.messaggiamo.com/el/attraction/17145-6-great-ways-to-improve-your-social-skills.htm
Chapter 9: Bibliography
Chapter 10: Final Test
Members
Webmaster: