Diskusi 7 EFL Curriculum and Materials Development

 Please discuss:

• What is materials evaluation?
• What criteria should be considered for the evaluation process?
• How to conduct material evaluation.
• The principles and techniques for evaluating materials.
• A framework to set the criteria to achieve a decision considering materials.

Jawaban 1:

Material evaluation means the process of evaluating educational resources, such as books and other printed media. The term material evaluation describes initiatives to gauge the worth of materials. By evaluating how well a set of educational resources affects the learners who use them, the value of the contents is determined (Renandya & Richard, 2004). In other words, a material evaluation is done to see whether the material will be easy for learners to use and enjoyable for them to utilize.

Criteria should be considered to the evaluation process are:
a. Physical quality
The suggested resources are dependable and available to students.
b. Content/format
The suggested resources are appropriate for the learners' cultural background context, truthful, authentic, and up to date.
c. Social considerations
The recommended reading materials steer clear of bias based on gender, social class, religion, races, and sexual orientation.
d. Instructional design
Materials must be consistent with the learning philosophy, link to the curriculum, and force students to comprehend them in order to meet the learning objectives.
e. Qualifications of developers
The materials have been developed and validated by qualified people.
5. Cost
The materials are affordable in terms of cost or budget.

How to conduct material evaluation
There are several steps in conducting materials evaluation. Below are the descriptions.
a. Pre-evaluation
The pre-test is the initial stage of the material evaluation process. It is done by asking pre-test participants to review the contents using the developed criteria. The information about the learning materials should be included in the draft, together with the goals, title, contents, language, formats, and other components.
b. Peer-test
This phase involves requesting a coworker to review the items using the developed criteria.
c. Revision and finalization
To complete this section, follow these steps: read and analyze the questionnaire, learner comments, and peer pre-test results; assess the results of the pre-test and group them into areas that require development; add the information and suggestions for improvement to the draft master; talk with peers in groups about the topics that will be revised and how to do so; decide on a final materials.
d. Conducting the evaluation
This step involves inviting participants, providing copies of the materials and curriculum units, asking the facilitators to observe, use, and demonstrating the materials, talking about the participants' opinions of the materials, providing the questionnaire and requesting participation from everyone, and analyzing and listing the responses.

The principles for evaluating materials
Cunningsworth (in Soepriyatna; 2019) outlines four principles for material evaluation. Here are descriptions of each.
a. Correspondent with the learners; need and fit to the course goals and objectives
The goal and objectives of the course will decide the type of instructional materials that can be employed to meet the needs of language learners.
b. Equip learners to use language effectively for their own purposes
The course materials should take into account the program's requirements in order to teach students what they must learn in order to use the language for communication.
c. Facilitate a learning process
When evaluating a piece of material, it is important to ensure that it offers learners the chance to learn in a variety of ways while also taking into account their various learning strategies and styles.
d. Function as a support for learning, mediating between learners and the target language
Materials should offer examples for students to follow, exercises for them to practice, games to encourage language learning, and explanations and examples to help them comprehend how language functions.

A framework to set the criteria to achieve a decision considering materials
The evaluation framework includes:
a. Brainstorming a list of universal criteria of evaluation
Creating the universal standards for materials that might be used comes before creating the evaluation criteria for materials. It can begin with general information and progress to a simple set of questions before evaluating the resources.
b. Subdividing some of criteria
This phase is carried out by first looking at the general criteria and then breaking it down into sub-lists. Following that, evaluation statements and questions can be created.
c. Monitoring and revision
The statements and questions should be monitored and amended when they are created.

Jawaban 2:

          According to Soepriyatna (2017), the evaluation of materials and media necessitates the use of a criterion framework in order to make decisions regarding the selection, modification, or creation of course materials. In this case, the teacher endeavors to think of the right choice heading a problem to a solution which is represented in the goal and objectives set before. These two items, the goal and objectives, eventually function as the key guidelines for providing and/ or generating the most appropriate materials to achieve them. Further, the teacher selects learning sources that best meet the needs of their students and also their schemata dealing with their previous proficiency level when it comes to teaching English. The abundance of existing materials that is accessible available today makes the right choice that expects the teacher to make informed and proper decisions while choosing course books and supporting materials. Then, modification of instructional materials is always needed by committing to adaptation and adoption of materials that are finally suitable to the student’s needs and potential. The next step is that the teacher needs to strive for robust, flexible course material creation that can primarily be developed in a wide variety of settings (formal-informal, synchronous-asynchronous, theoretical-practice based, etc.). The main point behind all this explanation is that optimal learning experiences head to required achievement sensibly with the goal and the objectives.
          The criteria should be considered for the evaluation process comprise appropriate material selection for literature that is related to student goals and needs, learning styles, the background of students ‟ literary and students‟ culture, linguistic proficiency levels, language teaching methods, and even classroom context and the potential of materials for generating motivation and interest (Gailea, Syafrizal, and Indasari, 2018). Firstly, goals and needs are influential for each. The needs can be viewed from the problems dealing with the student’s desired cognitive state and psychological aspects such as perception, motivation, behavior, attention, interest, etc. Then, the student’s preferable learning styles also determine the materials and their task completion in order to achieve the goal and the objectives. Afterward, the student’s prior knowledge revealing their linguistic proficiency levels becomes a considerate view in material evaluation. Besides, material evaluation impacts design of language teaching methods that gradually and continuously facilitate, increase, and develop the student’s achievement. Lastly, classroom context and psychological consideration also give the actual influence and sensible antecedent to hold an ideal teaching learning process supported by the student’s compliant feeling in interacting with the materials and their task completion effectively.
          Regarding the ways of material evaluation, Soepriyatna (2017) proposes three steps as before revealed by some experts such as Cunningsworth (1995), Ellis (1995, 1997), Tomlinson (1998, 2003), McGrath (2002), Mukundan (2009), and Tomlinso and Masuhara (2004). They are presented below.
a. Pre-use evaluation
The essence of this step requires the teacher to predict the most appropriate materials whose values are adequate and useful to achieve the goal and the objectives.
b. Whilst (in)-use evaluation
The main point for this step lies in the actual measurement of using the selected materials through the student’s performance on task completion and exercise interaction whether those are done effectively or not.
c. Post-use evaluation
This step can be seen from the actual effects in terms of strengths and weaknesses within a durable period of using those selected materials to have reliable information.
          Then, the principles and techniques for evaluating materials are oriented to Cunningsworth’s (1995) outlines as cited in Soepriyatna (2017). Basically, material evaluation can:
a. correspond with the students’ needs and fit with the course goal and objectives
This step is initiated with the students’ need assessment. On the basis of the result, the goal is broken down into specified course objectives. Then, teaching materials is directly related to the goal and relevant objectives by committing predication, application, and evaluative effect measurement.
b. equip the students to use language effectively for their own purposes
The teaching materials can facilitate communicative purposes for which are suitable with the program and its assessment.
c. facilitate a learning process
The selected materials relevant with learning activities can facilitate the students’ learning in different ways and optimize different types of the students’ learning styles, and strategies.
d. function as a support for learning, mediating between the students and the target language
The selected materials may finally indicate some learning progress such as students’ activeness and interest in learning and interacting with them so those materials can promote language learning, explanation, and examples to understand how language works.
          Additionally, Soepriyatna (2017) intend to give evaluation framework of the teaching materials encompassing some criteria below.
a. Brainstorming a list of universal criteria of evaluation
The universal criteria must be evaluated here in terms of duration, accessibility, authentic materials suitable with the students’ background, avoidance of social unrest problems, representation of curriculum being applied, fulfilment of validity, and reachable budgeting.
b. Subdivide some of criteria
The evaluated materials based on the universal criteria are then broken down in details in the considerate category of gender, age, sexual orientation, social status, social class, and many other subdivide criteria in order to conduct evaluation purposively, accurately, and objectively.
c. Monitoring and revision
Strength and weaknesses of the selected materials for the category of universal and more specified criterial must be monitored and revised starting from the process of prediction, application, and evaluative effect measurement.

Reference
Gailea, Nurhaedah, Syafrizal, and Indasari, Iin. 2018. Materials Selection in Teaching English Skills for Teachers of Senior High School in Serang City. JELTS Vol. 1 No. 2. https://jurnal.untirta.ac.id/index.php/JELTS/article/view/7736.

Soepriyatna. 2017. Buku Materi Pokok MPBI5204/3SKS/Modul 1-9: EFL Curriculum and Material Development. Tangerang Selatan: Universitas Terbuka.

Jawaban 3:

Material evaluation means evaluation process of teaching materials including learning media that may take a form of book, other printed forms such as: newspaper, label, flyers, posters, brochures, video, and audio recordings, computer software, and digital content which are used as part of the language teaching process. (Soepriyatna, 2017). Based on the statement, the materials evaluation includes all the facilities that we use in the learning process. Evaluation helps the teachers to discover the needs of the students.
Criteria should be considered for the evaluation process
According to Soepriyatna (2017), the criteria are developed to achieve a decision considering materials to be selected, adapted or developed. It is important that the definitions of the criteria are understood within a broader context, and read in conjunction with other principles and guidance on how to conduct evaluations in ways that will be useful and of high quality.

Criteria should be considered to the evaluation process are:
1. Relevance
The extent to which these materials are used in the learning process both in terms of the purpose and benefits of using certain materials
2. Coherence
materials must always undergo improvements and changes as well as updates on new things
3. Effectiveness
The content materials recommended are effective,
4. Efficiency
The materials mustt be efficient, usefu,l saving time and cost
5. Impact
The use of materials must have a positive impact and create interest for students to take part in learning
6. Sustainability
the use of materials must be sustainable in the sense that they must continue to be used , ability to exist and develop without depleting natural resources for the future.
• How to conduct material evaluation.
Evaluating materials in course design is important to decide whether a learning material to support the leaning activities. there are several steps in conducting materials evaluation:
1. A systematic evaluation is based on a specification of objectives, principles and procedures adopted or embedded in materials. A systematic evaluation is best done in a longitudinal fashion, which has three stages: pre-use evaluation, whilst-use evaluation and post-use evaluation.
2. An internal evaluation focuses on the internal accountability of the materials, e.g. the theoretical assumptions behind the materials; the intended objectives and the extent to which the objectives have been realized; the justification of language selection and grading; the design of activities and tasks.
3. An external evaluation examines the extent to which a set of materials meets the needs of a particular group of learners, the syllabus, and the examination, i.e. whether a set of materials is suitable for a group of students.

The principles and techniques for evaluating materials.
There are two principles in materials evaluation: effectiveness and efficiency
• Effectiveness principle: Is a course book effective in meeting the needs of the learners
• Compare what the learners knew and were able to do before they used the course book with what they know and are able to do after they have used the book;
• Efficiency principle: Does a course book meet the needs of the learners more effectively than some alternative course books?

A framework to set the criteria to achieve a decision considering materials.
In order to make materials evaluation more operable and easier to conduct, many frameworks or checklists have been developed.
1. Initial question: contains questions about the purpose of the material and its content,
How do you expect the material to be taught to students in class?
then confirms whether the material is the only resource in language learning in the classroom
2. Learners and materials: analyze student needs and interests, suitability of the material with the language learning approach, the suitability of the material with the teaching and learning process in the classroom, material design features for classroom work and finding students' criteria for good material

Jawaban 4:

• The definition of evaluation can be explained literally or literally. Linguistically, evaluation comes from the English word "evaluation" which means an assessment or assessment. Literally, evaluation is a process to determine a certain thing or object based on certain goals to achieve certain goals.
Evaluation of materials to collect and combine data with standard goals to be achieved so that it can be used as a basis for decision making.
In terms of the company, evaluation is the process of measuring the effectiveness of the strategy implemented to achieve the company's goals. The results of the next evaluation will be used as the next program analysis.

• If you want to do a learning evaluation, of course you have to have a few things that must be planned so that the results obtained are maximized. The following are things that the teacher must pay attention to in conducting an evaluation:
1. Needs Analysis
In this case, the analysis is carried out by the teacher to assist them in identifying needs and making it easier for them to determine the priority scale for solving them. The analysis will be carried out by the teacher as in determining the needs of students, both individually and in groups.
2. Determine the Purpose of the Assessment
In addition, the teacher must also determine the purpose of the assessment. The purpose of the assessment must be formulated clearly and unequivocally, because it plays an important role in determining the direction, scope of teaching materials, learning models to be used, and the character of the assessment tool. The purpose of the assessment should be formulated according to the type of assessment that the teacher will conduct, such as formative, summative, placement, or diagnostic assessments. The formulation in the assessment objectives must pay attention to the domain of student learning outcomes.
3. Identifying Competencies and Learning Outcomes
Teachers must identify competencies and student learning outcomes in accordance with existing competencies in the applicable curriculum, starting from competency standards, basic competencies, student learning outcomes to learning indicators.
4. Assemble the Grid
In this case, the preparation of the grid prepared by the teacher is related to assessments that are relevant to the subject matter that has been conveyed by the teacher to students. The function of the grid is as a guide for writing questions or assembling questions in student tests. The grid must also be arranged based on the syllabus, so the teacher must first analyze the syllabus before compiling the grid.
5. Develop Instrument Draft
The draft of the assessment instrument prepared by the teacher can be in the form of a test or non-test. In the form of a test, it means that the teacher must make questions with clear and focused questions. While in the form of non-test, the teacher can make it in the form of questionnaires, observation sheets, interview activities, and study documentation.
6. Trial and Problem Analysis
This activity aims to find out which questions need to be changed, corrected, even questions that must be discarded, as well as which questions are good for further use. Good questions are questions that have been tested and revised based on empirical and rational analysis.
7. Revise and Assemble Questions
If the questions have been tested and analyzed, then the next step is to make revisions that are in accordance with the proportion of the level of difficulty and distinguishing power. Thus, there will be questions that can be corrected from the delivery of language, the subject matter, or must be completely revised.

• A good way to evaluate students, a teacher must do several things:
First , the teacher must know his students by taking an approach approach. Then, the teacher must know the student's family, then the teacher must know the characteristics of the student, and many other things as long as it involves judging in the form of kindness to his students.
In evaluating students, a teacher should have their own or different ways, sometimes the teacher uses practical methods and there are also teachers who use texts such as exercises, daily tests, mid exams, and semesters.

• Some of the evaluation principles and evaluation techniques that serve as a guide for a teacher:
1. Continuity
Evaluation in learning is not only done during the mid-semester or end-semester exams. More than that, if the teacher wants to see changes in the grades of students, it must be done continuously. This means that from the stage of preparing the lesson plan to reporting it must be monitored continuously.
2. Comprehensive
Not infrequently some teachers only focus on the cognitive aspects of their students. In fact, two other aspects, namely cognitive and affective, play a major role in the learning evaluation process. As a teacher, it is not only required how students can understand a material. Teachers are also required how to shape the character of good students so that they can have a positive impact on their lives. Therefore, a good learning evaluation is carried out from the learning process to the learning outcomes of students.
3. Cooperative
In fact, the learning evaluation process carried out must coordinate with various elements that contribute to student development. Starting from the principal, subject teachers, homeroom teachers, parents, to administrative officers. In fact, it is highly recommended to work with the students themselves. Why? Because this is intended so that all elements involved in the evaluation of learning feel appreciated for the cooperation carried out.
4. Objective
Assessment of results in the evaluation of learning must be objective. This means that subjective factors such as the teacher-student relationship and the feeling factor because they don't have the heart or others should not be included in the evaluation. If the student gets a bad grade, it means that the score must be included with the provision of notes to motivate students and notification to parents.
5. Practical
The principle of learning evaluation must be practical. This means that these activities must save costs, time, and energy. This principle emphasizes the ease of the teacher in compiling an assessment instrument that is easy to use not only for himself, but also allows other teachers to use it. Along with this practicality, do not lose sight of the essence of the learning evaluation itself, which is to achieve the optimality of the learning objectives.

• The general framework for making evaluation decisions includes as follows:
a. Evaluation as a professional judgment.
b. Evaluation as measurement.
c. Evaluation as an analysis of the suitability between performance and goals,
work goals or standards.
d. Decision-oriented evaluation.
e. Responsive or goal-free evaluation.

Jawaban 5:

Materials evaluation is a process that involves determining the (potential) worth of a collection of learning materials (Tomlinson, 2003). In other words, material evaluation is a principled method of gathering valuable information about the specific target substances in order to pick and/or develop them in a trustworthy and legitimate manner.

What criteria should be considered for evaluation process?

An evaluation focuses on the materials' users and makes judgments about their effects, and no matter how structured, criterion-referenced, and rigorous an evaluation is, it will inevitably be subjective (Tomlinson, 2003).

Evaluation attempts to quantify some or all of the following:

1.    the materials' appeal to the students

2.    the materials' credibility in the eyes of students, teachers, and administrators

3.    the materials' validity, make sure that “is what they teach worth teaching?”

4.    the materials' dependability, “will they have the same effect on different groups of target learners?”

5.    the materials' ability to engage both students and teachers

6.    the materials' ability to motivate learners

7.    the materials' flexibility, such as the ease with which a teacher can adapt the materials to a specific context.

8.    How to conduct material evaluation?

In conducting material evaluation, the first step that we need to do is pre testing of the material. We give the facilitator a copy of the coursework unit of the material and guide them to utilize the review material in accordance with the teaching and learning process outlined throughout the curriculum design unit. The questionnaires are then distributed to the students, who must complete them. Pre-test results should be analyzed afterward by compiling questionnaire responses, and based on observation and analysis, the areas that need improvement should be identified.

The next step is peer pre-test, clientele should include material developers/facilitators and school teachers. Request that one facilitator demonstrate the draft in accordance with the teaching learning process established in the curricular unit, and observe the demonstrator's performance and the clientele's reaction. Then, distribute questionnaires to all clients and ask them to fill them out. Then, among the clientele, discuss the overall impression of the draft. Take note of everything said. After conducting the pre-test, analyze the results by aggregating the comments in the questionnaire, and identify and list the points that need to be improved based on the analysis. Request that one facilitator demonstrate the draft in accordance with the teaching learning process established in the curricular unit, and observe the demonstrator's performance and the clientele's reaction. Then, distribute questionnaires to all clients and ask them to fill them out. Then, among the clientele, discuss the overall impression of the draft. Take note of everything said. After conducting the pre-test, analyze the results by aggregating the comments in the questionnaire, and identify and list the points that need to be improved based on the analysis.

The evaluation principles and techniques are based on Cunningsworth's (1995) outlines, as cited in Soepriyatna (2017). Essentially, material evaluation could indeed:

1.    Correspondent with the learners; need and fit to the course goals and objectives. Teaching materials must be directly related to the goal and pertinent objectives of a course in order to achieve them. A goal is divided into specific course objectives based on the outcome of an assessment of students' needs and abilities, as well as their ability to meet them.

2.    Equip learners to use language effectively for their own purposes. When used for appropriate program and assessment purposes, the teaching materials can help with communication.

3.    Facilitate a learning process. It is critical to provide students with the tools and resources they need to learn in a variety of ways. Resources can help students learn in a variety of ways and maximize a variety of learning styles and strategies; an expert's assessment can help you decide what works best for you.

4.    Function as a support for learning, mediating between learners and the target language. Students should be given examples to emulate, exercises to practice, games to encourage language learning, and explanations and examples to help them understand how language works. Students' activeness and interest in learning and interacting with the materials may indicate some learning progress.

The evaluation framework based on Soepriyatna (2017):

1.    Brainstorming a list of universal criteria of evaluation. Requisites that would be applicable to all language learning materials used by any learners, anywhere.

2.    Subdivide some of criteria. The general criteria are examined, and the sub-lists are then broken down. The evaluation process can then be guided by the evaluation statements and questions.

3.    Monitoring and revision. It means that instead of using checklists, each practitioner can use or create their own set of evaluation criteria.

Thank you very much.   

Refference

Soepriyatna. 2017. Buku Materi Pokok MPBI5204/3SKS/Modul 1-9: EFL Curriculum and Material Development. Tangerang Selatan: Universitas Terbuka.

https://www.researchgate.net/profile/Farangis-Shahidzade/publication/349485391_How_to_evaluate_language_teaching_materials/links/6032c8bea6fdcc37a84248bc/How-to-evaluate-language-teaching-materials.pdf 

Jawaban 6:

Materials evaluation is a systematic assessment process that determines the potential value(s) of materials on learners in connection to their goals (Tomlinson, 2003). To put it another way, materials evaluation refers to a methodical procedure that offers helpful information about the intended materials so that they can be chosen and/or developed in a trustworthy and valid manner.

What criteria should be considered for the evaluation process?
In terms of focusing on the criteria that should be considered for the evaluation process, we must first establish a framework from which a set of criteria can be constructed to evaluate the materials. We also need to take into account why we are analyzing the materials. Hence it will restrict the focus to the elements or criteria utilized in this Evaluation. For example, we may establish the framework suggested by Tomlinson (2003) that consists of the following:

·       Universal (driven by SLA theory): e.g., are the materials motivating?

·       Local (related to the context): e.g., are the materials culturally acceptable in the context?

·       Media-specific (e.g., audio or computer): e.g., is the sound quality of the audio materials good?

·       Content-specific (e.g., exam or English for Specific Purposes (ESP)): e.g., do the materials replicate the types of real-world tasks the target group will need to do?

·       Age-specific: e.g., are the visuals likely to appeal to children?

How to conduct materials evaluation
According to Tomlinson (2003), materials evaluation can be conducted through 3 processes:

1.    Pre-use Evaluation: it is designed to forecast a material's potential performance (predictive).

2.    Whilst-use Evaluation: it entails determining the worth of materials while using them or seeing them in use. As opposed to pre-use Evaluation, it can be more unbiased and trustworthy because it uses measurement as opposed to prediction. It cannot be said to measure what is occurring in the learners' minds, however, as it is only capable of measuring what is visible.

3.    Post-use Evaluation: it gives a retrospective evaluation of material and is also used to determine whether or not to use the same material again in the future. Due to its ability to assess how the materials actually affect their users, post-use Evaluation is perhaps the most beneficial and instructive method of Evaluation.

The principles and techniques for evaluating materials.
Tomlinson (2003) believed that it was crucial for evaluations to be guided by a set of principles and that the evaluator(s) should have stated these principles before the Evaluation. Any evaluation should begin with a reflection on the evaluator's practice, followed by the articulation of the evaluator's learning and teaching theories.

A framework to set the criteria to achieve a decision considering materials
According to Tomlinson (2003), the most effective way to consider an evaluation is to:

1.    Make certain that more than one evaluator is present (reliability issues)

2.    Examine the criteria to ensure that the interpretation is equivalent

3.    Answer the criteria separately and apart from the other evaluator(s)

4.    Concentrate on a large-scale evaluation of a typical unit for each level (and then check its typicality by reference to other units)

5.    Each criterion should be assigned a score (with some sets of criteria weighted more heavily than others)

6.    Remark at the end of each category

7.    At the end of the Evaluation, average the scores of each evaluator for each criterion, category of criteria, and set of criteria.

8.    Take notes on the evaluators' comments

9.    Write a joint report

What is suggested above, according to Tomlinson (2003), is a very thorough and organized approach to materials evaluation. However, there is also a drawback of this approach which is time-consuming, yet this is still necessary for major evaluations from which significant decisions will be made.

References
Tomlinson, B. (2003). Materials Evaluation. In B. Tomlinson (Ed.), Developing Materials for Language Teaching. London: Continuum.

Jawaban 7:

1. Definition Material Evaluation
Language learners may employ language teaching resources to aid the acquisition of the target language (Tomlinson & Masuhara (2018:2). In addition, Tomlinson (2013) defines materials assessment as the process of determining the worth (or prospective value) of a collection of instructional resources (p. 21). Materials assessment is essential to the success of a course since it allows us to examine the course materials' compatibility with the course goals and students' needs.

According to Soepriyatna (2019:7.3), material and media evaluation need a framework of criteria to make a decision, whether to pick, change, or produce course materials. Recognizing that a variety of materials are accessible on the market, material assessment should be governed by a set of rules. It is necessary to analyze course materials to ensure that those selected contribute to achieving course goals. The purpose of a materials assessment is to identify the best appropriate course materials.

2. Criteria should be considered for the evaluation process
Soepriyatna (2019:7.3) states that in materials evaluation, a framework to set the criteria should be identified. The criteria are developed to achieve a decision considering materials to be selected, adapted or developed. Moreover, the purpose of the evaluation should be considered to focus on particular criteria. Criteria and the purpose will become the guideline in evaluation. The criteria can range from universal to specific criteria.
There are some general criteria for materials evaluation proposed by Learning Resources Evaluation Guideline in Soepriyatna (2019: 7.12) that can be used as a reference.

a. Physical Quality
The materials recommended are durable and accessible for learners.
b. Content Format
The materials recommended are accurate, authentic, current and suitable to or the learner's cultural background context.
c. Social Considerations
The materials recommended avoiding inequality of gender bias, status, religion, race and sexual orientation.
d. Instructional Design
The materials have to relate to the curriculum and consistently the learning philosophy. In addition, the instructions require students to force themselves to understand the materials and achieve the objectives.
e. Qualifications of Developer
The materials have been developed and validated by qualified people
f. Cost
The materials are reasonable regarding the cost or budget.

3. How to conduct a material evaluation.
Soepriyatna (2019:7.24-7.28) urges that conducting materials evaluation can be done after the criteria are developed. As the criteria have been set, it is just a matter of conducting the evaluation. There are several steps in conducting materials evaluation, pre-test peer test, revision, duplication, and evaluation.
a. Pre-evaluation
A pre-test is the first step in conducting the material evaluation. This should be done to see if the criteria could be used to evaluate materials and to see if the materials can be evaluated or not. Pre-test in materials evaluation is done by asking the pre-test takers to check the material by using the criteria developed.
b. Peer-test
The other step to be conducted beforehand is doing a peer-test. Ask a colleague to evaluate materials by using the criteria designed.
c. Revision and finalization
After evaluating the materials used in the teaching-learning process, a revision needs to be made for finalization. To revise it, the following steps should be done:
- Read and analyze the questionnaire and comments from learners and peer pre-test.
- Interpret the data from the pre-test and put them into points that need improvements.
-Put the data and improvement points into the draft master.
- Discuss the points that will be revised in groups (with peers) and how to Revise.
- Make the final materials.
d. Conducting the Evaluation
Several things should be prepared before the material evaluation is conducted. Here are the examples; arranged place and time, assigned roles (facilitator, note-taker), duplicate material (hard copy, each person gets one), curricular unit (syllabus, as the guide), questionnaire and equipment (PC, LCD, projector, tape-recorder, etc.)

4. The principles and techniques for evaluating materials.
Cunningsworth in Soepriyatna (2019:7.7-7.8)  outlines four principles for material evaluation.
Principle-1:
Correspondent with the learners ' needs and fit the course goals and objectives.
The result of the needs assessment is converted into course goals and objectives. "The learners need to improve their academic presentation skills" is formulated into the course goal, "To improve learners' academic presentation", which may be broken into more specified course objectives.
Principle-2:
Equip learners to use language effectively for their own purposes. Materials should be evaluated to convince that they contain what the learners need to learn. The materials should accommodate the needs of the program to provide what the students need to learn so that they can make use of the language for their communicative purposes, be it personal, professional, or academic.
Principle-3:
Facilitate a learning process.
Evaluating materials should be conducted to guarantee that the materials take account of students' needs as learners and facilitate their learning process. Evaluation should ensure that t materials provide opportunities for the learners to learn in different ways, catering to different types of students' learning styles and strategies. Materials should also be evaluated to guarantee that they will motivate the learners when working with those materials. Varieties of topics and activities, and lively and well-material presentations will likely motivate the learners.
Principle-4:
Function as a support for learning, mediating between learners and the target language.
Material evaluation should be able to control the relationship between learners and the target language. Materials should facilitate learning. Materials should provide models for the students to learn, exercises for the students’ practice, activities to promote language learning, and explanations and examples to understand how language works.

5. A framework to set the criteria to achieve a decision considering materials.
In materials evaluation, Tomlinson in Soepriyatna (2019:7.12-7.15)  proposes a framework starting from 1) Brainstorming a list of universal criteria of evaluation; 2) Subdividing some of the criteria, 3) Monitoring and revising a list of universal criteria.

The criteria are then used to develop the instrument of evaluation. There is some instrument that can be developed. The most popular instruments used in materials evaluation are questionnaires, checklists and guides.
a. Brainstorming a list of universal criteria for evaluation
The first step before developing the criteria for evaluating materials is to devise the universal criteria of materials that might be applied. The brainstorming process can start from general information to evaluate the materials to a simple list of questions.
b. Subdividing some of the criteria
The next step after developing universal criteria is subdividing them to make the evaluation precise. Take a look at the universal criteria, Social Consideration. This universal criterion can be subdivided into sub-list that contains Gender, Age, Sexual orientation, Social status, and Social class.
c. Monitoring and revising
Once the questions or statements are formulated, those questions or statements should be monitored and revised.

Thank you for the opportunity

References.

Tomlinson, B. 2013. Developing materials for language teaching (2nd ed.). Bloomsbury Publishing.

Tomlinson, B., & Masuhara, H. 2018. The complete guide to the theory and practice of materials development for language learning. John Wiley & Sons, Incorporated.

Soepriyatna. 2019.  EFL Curriculum and Material Development. Tangerang Selatan: Universitas Terbuka

Jawaban 8:

1. What is materials evaluation?
Materials evaluation is a procedure that involves measuring the value (or potential value) of a set of learning materials. It involves making judgements about the effect of the materials for the students’ progress in learning English.
2. What criteria should be considered for the evaluation process?
As you evaluate evaluation process, the first things you normally consider are
• is it accurate?
• is it relevant? (Does it provide information that students need to complete your assignment?)
• is it understandable? (Is it at a level appropriate to your student’s current understanding of the topic or concept?)

3. How to conduct material evaluation.
a) To identify the popular and successful features of existing materials, which will help us to produce better materials.
b) To choose suitable materials for language.
c) To adapt the materials to make them better suit the needs of the students and the needs of teaching.
d) To identify how successful the materials used in the classroom are.
e) To examine whether the materials fulfil the prescribed course
f) To examine the extent to which materials permit students to achieve learning objectives.
g) To identify whether the designs of the materials are suitable for a task-based syllabus.

4. The principles and techniques for evaluating materials.
1) Materials should be current
2) Materials should be relevant
3) Materials should be varied
4) Materials should be relevant.

5 .A framework to set the criteria to achieve a decision considering materials.
a. Pre-evaluation
Pre-test is the first step in conducting material evaluation. This should be done to see if the criteria could be used to evaluate materials and to see if the materials can be evaluated or not. . Pre-test in materials evaluation is done by asking the pre-test takers to check material by using the criteria developed.

b. Peer test
Peer test is used for assessing the students’ capability in learning process. Empower students to take responsibility for and manage their own learning.

C. Revision and finalization
After evaluating the materials that will be used in the teaching learning process, a revision needs to be made for finalization. To revise it, the following steps should be done:
v Read and analyze the questionnaire and comments from learners and peer pre-test
v Interpret the data from the pre test and put them into points that need improvement
v Put the data and improvement point the draft master
v Discuss the points that will be revised in group ( with peerz) and how to revide.
v Make the final materials.

6. Conducting the Evaluation
There are several things that should be prepared before material evaluations conducted. Here are the examples :
1) Arranged place and time
2) Assigned roles ( facilitator, note-taker)
3) Duplicate material ( hard copy, each person gets one)
4) Curricular unit ( syllabus, as the guide)
5) Questionnaire
6) Equipment (PC, Laptop, LCD/ infocus, tape recorder, etc.)

Soepriyatna. 2017. Buku Materi Pokok MPBI5204/3SKS/Modul 1-9: EFL Curriculum and Material Development. Tangerang Selatan: Universitas Terbuka.

https://iu.pressbooks.pub/online2020/chapter/instructional-materials
https://blogs.brighton.ac.uk/materialsdip/category/materials-evaluation

Jawaban 9:

1. Materials evaluation may be defined as a procedure or a systematic appraisal measuring the potential value(s) of materials on learners in relation to their objectives (Tomlinson 1998, 2003). In other words, materials evaluation means a principled process of providing useful information about the targeted materials in order to select and/or develop them in a reliable and valid approach.

2. Criteria to be considered for the evaluation process:
a. Clarity in the formulation of learning objectives (does not cause multiple interpretations and contains learning outcomes behavior).
b. Selection of learning materials (according to the objectives and characteristics of students).
c. Organizing learning materials (coordinate, systematic material, and conformity with time allocation).
d. Selection of learning resources or media (according to the objectives, materials, and characteristics of students).
e. Clarity of learning steps which include the introductory step, this activity, and closing.
f. Detailed scenarios or learning steps (each step is reflected in the strategy or method and time allocation).
g. The suitability of the technique with the learning objectives.
h. Completeness of evaluation instruments (questions, answer keys, and scoring guidelines).
Based on the evaluation criteria above, an evaluation instrument is then made. Learning to plan evaluation instruments include learning components (including learning indicators), learning materials, learning resources (including media and aids), learning methods (including approaches, strategies, techniques, or learning activities), and learning evaluations.

3. Conducting materials evaluation can be done after the criteria are developed. As the criteria have been set, it is just the matter of conducting the evaluation. There are several steps in conducting materials evaluation :
a. Pre-evaluation
b. Peer test
c. Revision and finalization
d. Duplication
e. Conducting evaluation

4. There are two principles in materials evaluation: effectiveness and efficiency (Ellis 1998 in Tomlinson 1998).
- Effectiveness principle: Is a course book effective in meeting the needs of the learners? To answer this question, the evaluator needs to compare what the learners knew and were able to do before they used the course book with what they know and are able to do after they have used the book.
- Efficiency principle: Does a course book meet the needs of the learners more effectively than some alternative course books? To investigate efficiency, it is necessary to compare the learning gains evidenced by using one course book with the gains evidenced by another course book. A framework to set the criteria to achieve a decision considering materials

5. The evaluation framework includes:
a. Brainstorming a list of universal criteria of evaluation
Creating the universal standards for materials that might be used comes before creating the evaluation criteria for materials. It can begin with general information and progress to a simple set of questions before evaluating the resources.
b. Subdividing some of the criteria
This phase is carried out by first looking at the general criteria and then breaking it down into sub-lists. Following that, evaluation statements and questions can be created.
c. Monitoring and revision
The statements and questions should be monitored and amended when they are created.

Thank You,

Reference:
- Soepriyatna. 2017. Buku Materi Pokok MPBI5204/3SKS/Modul 1-9: EFL Curriculum and Material Development. Tangerang Selatan: Universitas Terbuka.

Jawaban 10:

What is materials evaluation?
Material evaluation is the process of evaluating teaching materials used as learning resources such as of books, other printed forms such as: newspapers, food labels, pamphlets, posters, brochures, video and audio recordings, computer software, and digital content (Soepriyatna, 2019). It is one step in syllabus designing, because it is conducted based on students’ situation and needs, it can be conducted before, while and after it is used (Soepriyatna, 2019). Material evaluation is the process of measuring the (potential) value of various learning materials (Tomlinson, 2003).

What criteria should be considered for the evaluation process?
Based on Learning Resources Evaluation Guidelines in Soepriyatna (2019) there are some criteria should be considered in evaluating process as follows:
- Physical Quality (durable and accessible for learners)
- Content/Format (accurate, authentic, current and suitable for the learners)
- Social Considerations (avoid inequality of gender bias, social status, religion, races, etc.)
- Instructional Design (relate with curriculum and consistency)
- Qualifications of developer (validated by qualified people)
- Cost (reasonable regarding to the cost or budget)

How to conduct material evaluation.
In conducting material evaluation Soepriyatna (2019) mentioned three step which can be conducted as follows:
- Pre-Use Evaluation
In this case, teacher subjectively perceived the prospective values of the material to the course, and This type of evaluation is very important, particularly, in the process of materials selection.
- Whilst (In)- Use Evaluation
The value of materials is evaluated. It measures the value of the materials whilst observing or using them.
- Post-Use Evaluation
It is the most important and valuable, it can measure the actual effect of material. this type of evaluation is helpful and useful for identifying the points of strength and weakness that emerge over a period of using the coursebooks (Cunningsworth in Soepriyatna, 2019).

The principles and techniques for evaluating materials.
Curningsworth (in Soepriyatna, 2019) proposed 4 principles for evaluating materials as follows:
- Principle 1: Correspondent with the learners’ needs and fit to the course goals and objectives.
- Principle 2: Equip learners to use language effectively for their own purposes.
- Principle 3: Facilitate a learning process, and the material must be relevant to learning and can facilitate student learning
- Principle 4: Function as a support for learning, mediating between learners and the target language.

A framework to set the criteria to achieve a decision considering materials.
According to Soepriyatna (2019) several evaluation frameworks are as follows:
- Brainstorm a list of universal evaluation criteria. This step is to devise the universal criteria of materials that applied.
- Subdivide some of criteria. To subdivide them to make the evaluation precise.
- Monitoring and revision. Once a question or statement is formulated, it must be monitored and revised

References:

Tomlinson, B. 2003. Developing materials for language teaching. London:Continum

Soepriyatna. 2019. EFL Curriculum and Materials Development. Tangerang Selatan, Banten: Universitas Terbuka

Jawaban 11:

A. What is materials evaluation?
Materials evaluation is a procedure that involves measuring the value (or potential value) of a set of learning materials. Materials evaluation is one part of a complex process. And once material is selected, can only be evaluated after class implementation and feedback.

B. What criteria should be considered for the evaluation process?
1. Physical Quality: materials should be durable & accessible
2. Content Format: materials should be accurate, authentic, current, and suitable for the learners’ cultural background context.
3. Social Consideration: materials should avoid inequality of gender bias.
4. Instructional Design: materials should relate with the curriculum, consistent with the learning philosophy, facilitating students to understand them, and achieve the learning objectives.
5. Qualification of the Developers: materials are developed and validated by qualified persons.
6. Cost: materials should be reasonable to the budget cost.
7. Content: materials should be current, accurate, and supporting the curriculum, appropriate with the students and integrating with the real-world situation.
8. Age: materials should accommodate different age group and include age integrated activities.

C. How to conduct material evaluation?
1. Pre-Evaluation: it is done by asking the pretest takers to check materials by using the developed criteria.
2. Peer Test: is conducted by asking a colleague to evaluate materials by using the designed criteria.
3. Revision & Finalization:
• Make a preparation
• Read and analyze the result
• Put data into master draft
• Discuss points to be revised in groups
• Make final materials
4. Conducting the Evaluation
• Invite participants
• Distribute the materials and curricula
• Ask facilitator to observe, use, and demonstrate the materials
• Discuss opinions
• Distribute questionnaires
• Analyze and list up data from questionnaires and comments

D. The principles and techniques for evaluating materials
Principle 1
• The materials should correlate with the learners needs (NA) and fit with the course goal and objectives
• The NA is inserted into Goal and Objectives, and they will determine the teaching materials to cover the NA of the learners
Principle 2
• The materials should accommodate the needs of the program that reflects from the students needs to use in real life.
Principle 3
• The materials should guarantee to take account of the students needs and facilitate the learning process.
Principle 4
• The materials should be able to control the relationship between learners and the target language.

E. A framework to set the criteria to achieve a decision considering materials.
1. Brainstorming a list of universal criteria of evaluation
The universal criteria must be evaluated here in terms of duration, accessibility, authentic materials suitable with the students’ background, avoidance of social unrest problems, representation of curriculum being applied, fulfilment of validity, and reachable budgeting.
2. Subdivide some of criteria
The evaluated materials based on the universal criteria are then broken down in details in the considerate category of gender, age, sexual orientation, social status, social class, and many other subdivide criteria in order to conduct evaluation purposively, accurately, and objectively.
3. Monitoring and revision
Strength and weaknesses of the selected materials for the category of universal and more specified criterial must be monitored and revised starting from the process of prediction, application, and evaluative effect measurement.

Referensi:
Soepriyatna. 2017. Buku Materi Pokok MPBI5204/3SKS/Modul 1-9: EFL Curriculum and Material Development. Tangerang Selatan: Universitas Terbuka.

Jawaban 12:

What is materials evaluation?
Material should be evaluated to make sure that the chosen ones will help achieve course objectives. Well, material evaluation is the prosses of teaching materials including learning media that may take a form of book, brochures, video or audio recording, computer software which are used a part of language teaching process (soepriyatna, 2019).

What criteria should be considered for the evaluation process?
(1)Physical Quality is the materials recommended are durable and accessible for learners.
(2) Content Format is the materials recommended are accurate, authentic, current and suitable to or the learner's cultural background context. (3) social Considerations is the materials recommended avoiding inequality of gender bias, status, religion, race and sexual orientation. (4) instructional Design is the materials have to relate to the curriculum and consistently the learning philosophy. In addition, the instructions require students to force themselves to understand the materials and achieve the objectives. (5) qualifications of developer is the materials have been developed and validated by qualified people (6) cost is the materials are reasonable regarding the cost or budget.

How to conduct material evaluation.
there are several things that should be prepared before material evaluation is conducted, they are pre evaluation, peer test, revision and finalization, conducting the evaluation.

The principles and techniques for evaluating materials.
according to Cunningsworth’s (1995) cited in soepriyatna (2017). The outlines for principles for material evaluation. (1) Correspondent with the learners; need and fit to the course goals and objectives, (2) Equip learners to use language effectively for their own purposes. (3) Facilitate a learning process (4) Function as a support for learning, mediating between learners and the target language

A framework to set the criteria to achieve a decision considering materials.
(1) Brainstorming a list of universal criteria for evaluation
The first step before developing the criteria for evaluating materials is devise yhe universal criteria of materials that might be applied then the process of brainstorming can start from general information to evaluate the materials
(2) Subdivide some of criteria the next step is subdivide them to make the evaluation precise. Take a look the universal criteria, social consideration that contains gender, sexual orientation, social status, social class.
(3) Monitoring revision
Once the questions or statements are formulated, those questions or statements should be monitored and revised.

Jawaban 13:

1. material evaluation is aimed at seeking the most fitted materials to achieve course objectives. It is in line with Tomlinson 2003:66, materials evaluation means evaluation process of teaching materials including learning media that may take a form of books and other printed forms.

2. According to Renandya & Richard 2004;1, the criteria that should be considered for the evaluation process are the appeal of the materials of learners, the validity of the materials, the ability of the materials to interest the learners and the teacher, the ability of the materials to motivate the learners, the potential learning value of the materials, the assistance given to the teacher in preparation, delivery, and assessment and flexibility of the materials.

3. According to Cunningsworth, 1995; Ellis 1995,1997; Tomlinson1998,2003; McGrath 2022; Mukundan 2009; and Tomlinson and Masuhara 2004, There are three categories of when material evaluation can be conducted; pre-use evaluation, whilst (in)-use evaluation and post-use evaluation. Pre-use evaluation, in this category course designers or teachers subjectively perceive the prospective values of the material to the course, Whilst (in)-Use Evaluation, in this category the value of the materials is evaluated whilst observing or in-using them. This evaluation observes the performance of learners on task and exercises. Post-Use Evaluation is the most important and valuable type of evaluation because it can measure the actual effects of the materials on the users and provide reliable information.

4. The principles and techniques for evaluating materials.
Principle 1; corespondent with the learners need and fit to the course goals and objectives.
Principle 2; equip learners to use language effectively for their own purposes.
Principle 3; evaluating materials should be conducted to guarantee that materials take aaccount of students needs as learners and facilitate their learning process.
Principal 4; materials evaluation should be able to control the relation between learners and the target language.

5. A frameworl to set the criteria to achieve a decision considering materials.
*Brainstorming a list of universal criteria of evaluation.
The process of branstorming can start fro general information to evaluate the materials to simple list of
question.
*Subdividing some of criteria.
This universal critetion can be subdividing into sub-list that contains Gender, Age, Sexual orientation, Social
status and social class.
*Monitoring and revision
Tomlinson (2004) provides guidelines to monitor and revise the question develop. Here is the guideline
-Is each question as evaluation question?
-Does each question onlu ask one question?
-Is each question answerable?
-Is each of question free from dogma?
- Is each question reliable in the sense that orher evaluators would interpret it in the same way?

Jawaban 14:

1. Materials evaluation can refer to either before the program begins or after or both. Here, materials evaluation refer to a before- program evaluation of published textbooks. This is motivated by the need to choose materials that will be relevant and appropriate for a particular group of learners and also by the need to identify specific aspects of the materials that require adaptation

In materials evaluation the evaluator identifies a set of criteria that are used to reach a decision regarding which book to adopt and how it needs to be adapted.

Instruments used in evaluating materials.
The instruments used in evaluating materials or textbooks are questionnaires, checklists and guides.

2. Criteria used in Materials Evaluation
In evaluating materials, we need to put a framework from which a set of criteria can be developed. In Addition, we have to consider the objective of evaluating materials. This will limit the focus to aspects or criteria used in this evaluation. In our context, the focus is on materials as one of the sources in the teaching and learning a foreign language. In line with this purpose, the frameworks.

1. Analysis of the target situation of use
(the cultural context, the
institution, the course, the teachers, the learners). The framework used: Develop criteria based on
the framework
2. Match and Evaluation:
How appropriate are the criteria or aspects used to the target situation of materials use ?
3. Action:
Adopt the materials Reject the materials Adapt the materials Supplement the materials.


3. Principles in Materials Evaluation
Many evaluations are impressionistic, or at best are aided by an ad-hoc and very subjective list of criteria (Tomlinson, 2003c).
In Tomlinson’s view (2003), it is very important that evaluations are driven by a set of principles and that these principles are articulated by the evaluator(s) prior to the evaluation.
In this way greater validity and reliability can be achieved and fewer mistakes are likely to be made.
In developing a set of principles it is useful to consider the following.

The Evaluator’s Theory of Learning and Teaching
All teachers develop theories of learning and teaching which they apply in their classrooms.
Many researchers (e.g. Schon, 1983) argue that it is useful for teachers to try to achieve an articulation of their theories by reflecting on their practice.
Edge and Wharton (1998, p. 297) argue that reflective practice cannot only lead to “perceived improvements in practice but, more importantly, to deeper understandings of the area investigated.”

In a similar way, Tomlinson (2003b) argues that the starting point of any evaluation should be reflection on the evaluator’s practice leading to articulation of the evaluator’s theories of learning and teaching.
In this way, evaluators can
a) make overt their predispositions,
b) make use of them in constructing criteria for evaluation,
c) be careful not to let them weight the evaluation too much towards their own bias and
d) learn a lot about themselves and about the learning and teaching process.

3. Principles in Materials Evaluation There are two principles in materials evaluation: effectiveness andefficiency (Ellis 1998 in Tomlinson 1998). Effectiveness principle: Is a course book or material effective in meeting theneeds of the learners? Compare what the learners knew and were able to dobefore they used the course book or material with what they know and areable to do after they have used it; Efficiency principle: Does a course book or material meet the needs of thelearners more effectively than some alternative course books? Compare thelearning gains evidenced by using one course book or material with the gains evidenced by anothermaterial.

4.
The framework suggested by Brown (1986): • Approach
• Syllabus
• Techniques (ways of presenting materials)
• Exercises

The framework suggested by Littlejohn (in Tomlinson, 1999)
a. Publication (physical aspects of the materials)
1. Place of the learner’s materials in any wider set of materials
2. Published form of the learner’s materials
3. Subdivision of the learner’s materials into sections
4. Subdivision of sections into sub-sections 5. Continuity (or coherence)
6. Route (or the order of materials)
7. Access ( availability of contents lists, wordlists, indexes etc)

Jawaban 15:

Material evaluation is one step in syllabus designing. It is aimed at seeking the most fitted materials to achieve course objectives. It is conducted based on analysis of students’ situation and needs.
Materials evaluation is a procedure that involves measuring the. (potential) value of a set of learning materials (Tomlinson, 2003).

The criteria should be considered for the evaluation process :
a. Physical quality
The suggested resources are dependable and available to students.
b. Content/format
The suggested resources are appropriate for the learners' cultural background context, truthful, authentic, and up to date.
c. Social considerations
The recommended reading materials steer clear of bias based on gender, social class, religion, races, and sexual orientation.
d. Instructional design
Materials must be consistent with the learning philosophy, link to the curriculum, and force students to comprehend them in order to meet the learning objectives.
e. Qualifications of developers
The materials have been developed and validated by qualified people.
5. Cost
The materials are affordable in terms of cost or budget.

To conduct material evaluation:
1. Pre-evaluation
2. Peer-test
3. Revision and Finalization
4. Conducting the Evaluation

The principles and technique for evaluating materials:
Principle 1 : correspondent with the learners’ need, and fit to the course goals and objectives.
Principle 2 : equip learners to use language effectively for their own purposes.
Principle 3 : facilitate a learning process.
Principle 4 : function as a support for learning, mediating between learners and the target language.

A framework to set the criteria to achieve a decision considering materials:
In making the materials evaluation more operable and easier to conduct, there are some frameworks or checklists have been developed. They are:
1. Initial question:
It consists of questions about the purpose of the material and its content,
then confirms whether the material is the only resources in language learning in the classroom.
2. Learners and materials:
Teacher should analyze the student needs and interests, suitability of the material with the language learning approach, the suitability of the material with the teaching and learning process in the classroom, material design features for classroom work and finding students' criteria for good material.
Thank you

Reference:
Soepriyatna. 2019. EFL Curriculum and Materials Development. Tangerang Selatan. Universitas Terbuka.
Tomlinson, B. (2003c). Materials evaluation. In B. Tomlinson (ed.), Developing Materials for Language Teaching. London: Continuum, 15–36.

Jawaban 16:

1. What is materials evaluation?
     The process of evaluating instructional materials includes learning media such as books, other printed materials like newspapers, food labels, flyers, posters, brochures, video and audio recordings, computer software, and digital content that are utilized in the instruction of foreign languages. The term "material evaluation" describes initiatives to gauge the worth of materials. What measured in materials evaluation is “the value of a set of learning materials by making judgments about the effect of materials on the people using them” (Renandya & Richards, 2004:1). Material evaluation is conducted to decide if the material will enable the learners to use it without difficulty and enjoy the experience of using it.

2. What criteria should be considered for the evaluation process?
     There are some general criteria for materials evaluation:

1.    Physical Quality. The materials recommended are durable and accessible for learners.

2.    Content/Format. The materials recommended are accurate, authentic, current and suitable for the learner’s cultural background context.

3.    Social Consideration. The materials recommended avoid inequality of gender bias, social status, religion, races and sexual orientation.

4.    Instructional design. The material have to relate with curriculum and have consistency with the learning philosophy.

5.    Qualifications of developer. The materials have been developed and validated by qualified people.

6.    Cost. The materials are reasonable regarding to the cost or budget.

3. How to conduct material evaluation.
     There are three categories of when material evaluation can be conducted:

1.    Pre-use evaluation. In this part, course designer or teacher subjectively perceive the prospective values of the material to the course.

2.    Whilst (in)-use evaluation. In this part, the value of the materials is evaluated whilst observing or in-using them

3.    Post-use evaluation. In this part, may be the most useful evaluation since it is conducted after the students have had reasonable time using the materials.

4. The principles and techniques for evaluating materials.
     Curningsworth (in Soepriyatna, 2019) outlines four principles for material evaluation.

1.    Principle 1: correspondent with the learners need, and fit to the course goals and adjective

2.    Principle 2: equip learners to use language effectively for their own purposes.

3.    Principle 3: the materials take account of students’ needs as learners and facilitate their learning process.

4.    Principle 4: function as a support for learning, mediating between learners and the target language.

5. A framework to set the criteria to achieve a decision considering materials.
     According to Soepriyatna (2019) there are some evaluation frameworks. They are:

1.    Brainstorm a list of universal evaluation criteria. This step is to devise the universal criteria of materials that applied.

2.    Subdivide some of criteria. To subdivide them to make the evaluation precise.

3.    Monitoring and revision. Once a question or statement is formulated, it must be monitored and revised

Reference:
Soepriyatna. 2019. EFL Curriculum and Materials Development. Tangerang Selatan, Banten: Universitas Terbuka

Jawaban 17:

1. Material evaluation is a step in the material development cycle and revising or perfecting the material to match the expected competencies.
2. a. Assessment is directed to measure the achievement of competence/learning objectives.
b. The assessment uses the reference criteria; that are based on what students can do after following the learning process, and not to determine a person's position in his group.
c. The planned system is a system of continuous assessment. Sustainable in the sense that all indicators are billed, then the results are analyzed to determine the basic competencies that have been possessed and those that have not, as well as to find out the difficulties of students.
d. The results of the assessment were analyzed to determine follow-up. Follow-up in the form of improvement of the next learning process, remedial programs for students whose competency achievement is below the criteria of completeness, and enrichment programs for students who have met the criteria of completeness.
e. The assessment system must be adapted to the learning experience taken in the learning process. For example, if learning uses a field observation task approach, the evaluation must be given both to the process (process skills) such as interview techniques, as well as the product/result of conducting field observations in the form of the required information.
3. Needs Analysis, determining Assessment Objectives, identifying Competencies and Learning Outcomes, compiling Grids, developing Instrument Drafts, Testing and Problem Analysis, Revision and Assembling Questions.
4. The principle of the assessment must provide results that are acceptable to all parties, both those who are assessed, those who assess, and other parties who will use the results of the assessment.
Techniques for evaluating materials are compiling an evaluation plan for results, collecting data, verifying data, processing and analyzing data, providing interpretation and, following up on evaluation results, test techniques, and objective tests.


Reference: Soepriyatna. 2017. MPBI5204 : EFL Curriculum and Material Development. Tangerang Selatan: Universitas Terbuka.

Jawaban 18:

The process of selecting, adapting, or developing teaching and learning materials including some textbooks and media is called material evaluation according to Richards and Schmidt (2002). Furthermore, material evaluation is conducted to decide if the material will enable the learners to use it without difficulties and enjoy the experience when using it (Renandya & Richards, 2004).

Some criteria to be taken into account in the evaluation process are the physical quality of the material that should be durable and easy to get by the learners; the content or format should be accurate, authentic, current, and suitable for the learners’ cultural background context; the material reflects sensitivity to gender, social status, religion, races, and sexual orientation; the material is able to support and be consistent with provincial and local program curriculum outcomes; the material should be developed and validated by professional personnel, and the material’s price should be reasonable.

There are 4 steps in conducting a material evaluation.

1.    Pre - Evaluation
The first step is done to determine whether the criteria can be utilized to evaluate materials and whether the materials can be reviewed. Pre-test takers are requested to check the material using the criteria set. The more complete the draft, the better the outcome. The draft should include information about the chosen learning resources, such as the objectives, title, contents, language, formats, and other features. The created criteria will be placed in the questionnaire as points that the evaluators must fill out (learners and peers).

2.    Peer-test
The second stage is to ask a teammate to evaluate materials using the designed criteria.

3.    Revision and finalization
This step is done by reading and examining the questionnaire, learner comments, and peer pre-test; interpreting the pre-test data and categorizing it into areas that require improvement; putting the facts and suggestions for improvement into the draft master; discussing in groups (with peers) the points that will be revised and how to revise them; and producing the final materials.

4.    Conducting the evaluation
In this step, a meeting will be held, and a certain time and place must be informed to the participants (learners or peers or certain necessary parties). All necessary equipment needed for this meeting should be prepared earlier. Then, the participants will be given some evaluation materials, curricula, and questionnaires. The participants are expected to give their feedback about the materials given. Finally, comments or opinions, and data from the questionnaire will be recorded and analyzed.

According to Cunningsworth (1995) there are 4 principles for material evaluation:

1.    Correspondent with the learners’ need, and fit to the course goals and objectives
First, we need to have a needs assessment then find out the outcomes and turn them into course goals and objectives to serve as the foundation for material evaluation. In other words, course goals and objectives will dictate which instructional resources can be employed to meet the demands of language learners.

2.    Equip learners to use language effectively for their own purposes.
Materials should be examined to ensure that they include the information that learners require. The resources should meet the program's needs in order to deliver what the students need to learn so that they can utilize the learning language for personal, professional, or academic objectives.

3.    Facilitate a learning process
The evaluation should be able to draw students into group and cooperative learning while also providing for individual growth; it should promote hands-on activities and an applied approach to learning; it should encourage students to question, think, react, reflect, and decide in ways that develop critical-thinking and decision-making skills; and it should offer choice and flexibility, as appropriate, to meet needs related to individual aptitudes, abilities, learning styles, multiple intelligences, and so on.

4.    Function as a support learning, mediating between learners and the target language.
The materials should be supportive of continuous learning by the individual and provide for both formative and summative assessment/evaluation as appropriate.

Evaluation framework to set the criteria to achieve a decision considering materials includes:

1.    Brainstorming a list of universal criteria for evaluation
The first step before developing material evaluation criteria is to design universal material criteria that can be applied. The brainstorming method might begin with general information to evaluate the materials and end with a simple set of questions.

2.    Subdivide some of the criteria
After developing universal criteria of evaluation, we need to subdivide it into a sub-list. Then we can create some questions or statements to develop the evaluation.

3.    Monitoring and revision
According to Tomlinson (2004), the questions we have should be monitored or revised by determining whether each question is an evaluation question, whether each question only asks one question, whether each question is answerable or free of dogma, and whether each question is reliable for other evaluators to interpret in the same way.

Reference:

Evaluation and Selection of Learning Resources: A Guide, 2008 Prince Edward Island
Department of Education

Soepriyatna. 2017. Buku Materi Pokok MPBI5204/3SKS/Modul 1-9: EFL Curriculum and Material Development. Tangerang Selatan: Universitas Terbuka.

Jawaban 19:

Materials are used in a class to facilitate the learning and teaching process. They include course books and media, so material evaluation is an activity to ensure that course books and media fulfill the needs of targeted students or programs. Three processes selected, adapted, and developed during the evaluation process should be considered. Those processes can be done entirely when course goals and objectives are clear. The goals and objectives will guide the process.
Conducting material evaluation needs three stages:
1. Pre-use evaluation is a stage in which the course designers or teachers select the materials. They predict the potential value of using the materials. As Ellis (1997) and Mukandan said (2009), " predictive evaluation is carried out to determine if the materials are adequate for use."
2. Whilst in-use evaluation is evaluation while using the materials. This stage is the follow-up after the prediction since it will measure the use of materials directly. The final result of observation will decide whether the materials can be used again, including which activities work/ not and how to modify the materials to make them more effective for future learning.
3. Post-use evaluation: this is the most crucial stage because it can measure the actual effects and provide reliable information. Canningsworth in Supriana said that this type of evaluation is helpful and valuable for identifying the points of strength and weakness that emerge throughout using the course books. (2017, p.7.6).
The materials evaluation should be conducted based on certain principles; the reason is to convince that the evaluation's result matches the course's objectives. Canningswoth in Soepriyatna outlines four principles:
1. Correspondent with the learners' needs and fit the course goals and objectives.
2. Equip learners to use language effectively for their purposes.
3. Facilitate the learning process, and ensure that they provide opportunities to learn differently,
4. Function as a support for learning, mediating between learners and the target language.
To achieve the decision considering materials undertakes procedures, there are some frameworks required, (1) brainstorming a list of universal criteria of evaluation, (2) subdividing some of criteria, (3) monitoring and revising a list of universal criteria.

Reference:
Soepriyatna. 2017. MPBI5204 : EFL Curriculum and Material Development. Tangerang Selatan: Universitas Terbuka.

Jawaban 20:

What is materials evaluation?
Material assessment refers to the process of evaluating instructional materials, such as books, other printed materials including food labels, flyers, posters, and brochures, as well as video and audio recordings, computer software, and digital content that are used in language instruction. Evaluations of materials are efforts to quantify the worth of materials. By drawing conclusions about how the use of the resources affects the users, the value of a set of learning materials is assessed. To put it another way, a materials review is carried out to determine whether the material will make it possible for learners to use it without difficulty and to enjoy the experience.

What criteria should be considered for the evaluation process?
1. Physical quality = the materials recommended are durable and accessible for students.
2. Content/format = the materials recommended are accurate, authentic, current and suitable for students; cultural background context
3. Social consideration = the materials recommended avoid inequality of gender bias, social status, religion, races and sexual orientation
4. Instructional design = the materials should be relevant with curriculum and have the consistency with learning phylosophy
5. Qualification developer = the materials should be developed and validate by qualified people
6. Cost = the materials are reasonable to the cost.

How to conduct material evaluation.
After the criteria are created, it is possible to evaluate the materials. It is simply a matter of conducting the evaluation now that the criteria have been established. Pre-testing, peer-testing, revision, duplication, and conducting the evaluation are the five processes in conducting a materials evaluation.
By asking pre-test participants to check the material using the stated criteria, pre-test materials are evaluated.
When evaluating materials, you can conduct a peer exam by requesting a colleague to do so using the established criteria.
revision and finalization: following the completion of the two steps above, a revision is required for finalization.
Lastly, performing the evaluation

The principles and techniques for evaluating materials.
1. Correspondent with the learners need and fit the course goals and objectives
2. Equip learners to use language effectively for their own purposes
3. Facilitate a learning process
4. Function as a support for learning, mediating between learners and the target language

A framework to set the criteria to achieve a decision considering materials.
a. brainstroming a list of standard evaluation criteria
Prior to developing the criteria for material evaluation, universal standards for materials that might be employed must first be established. Prior to analyzing the resources, it might start with broad information and go on to a short list of questions.
b. subdiving the requirements
In this phase, the general criteria are first examined, and then the sub-lists are created. Evaluation statements and questions can then be developed.
c. Monitoring and revision
When they are developed, the statements and questions should be reviewed and modified.

Jawaban 21:

• What is materials evaluation?
Materials evaluation is high consideration on the learning materials, lesson, content, media and other supporting printed forms which are used in teaching and learning process. Materials evaluation is conducted based on the goals and objectives, so it is a process where need analysis is done and seek the most suitable materials in order to reach a course objectives.

According to Tomlinson (1998, 2003), materials evaluation can be a procedure or a systematic appraisal measuring the potential value(s) of materials on learners in relation to their objectives.

So, it can be concluded that evaluation is the process of selecting, analyzing and deciding the learning materials in the course design whether it is usable, useful, meaningful or not. That’s why the institution or school must involve colleagues as evaluators, parents representatives, school boards, media specialist, learners and other relevant experts in conducting material evaluation.
Some steps in conducting materials evaluation are pre-test, peer test, revision and finalization, duplication, and materials evaluation.

• What criteria should be considered for the evaluation process?
Many experts stated their different ideas about the criteria in conducting the evaluation process:
Richards (2001) convinced that goals and objectives must be consideration in conducting the evaluation. He mentioned three roles in conducting the evaluation, namely materials, teachers, and students.
• How to conduct material evaluation.
Materials evaluation is conducted through three steps,: pre-test peer test, revision, duplication and conducting evaluation.
Pre test is doing by having questionnaire, peer test is possible done by asking colleagues to evaluate materials. Revision is performed as the materials are ready to use in learning process, and certainly the materials needs revising and finalizing. Conducting the evaluation needs time and place arrangement, roles, copies of materials, course outline, questionnaire and equipment.
• The principles and techniques for evaluating materials.
Cunningsworth (1995) stated four principles for material evaluation:
1. Correspondent with the learners ‘need’ and fit the course goals and objectives
2. Equip learners to use language for their own purposes effectively
3. Facilitate a learning process
4. Function as a support for learning, mediating, between learners and target the language.

• A framework to set the criteria to achieve a decision considering materials.
Universal criteria of materials is possible to do, like to have a physical quality, content, social consideration, instructional design, qualification, and the cost in order to achieve a decision in considering materials.

Reference :
Soepriyatna.2019. EFL Curriculum and Material Development. Tangerang Selatan. Universitas Terbuka
www.researchgate.net

Jawaban 22:

1.Materials evaluation is a procedure that involves measuring the(potential) value of a set of learning materials (Tomlinson, 2003c).It involves making judgements about the effect of the materials onthe people using them.In the other word, Material evaluation means a principled process of providing useful information about the target material in order to select and or develop them in reliable and valid approach.By evaluating learning materials,we as a the teacher can make decisions on selecting the appropriate coursebook.Moreover, evaluation can familiarize the teacher with the potential weaknesses and strengths of materials.Material evaluation can be another way of action research in developing our understanding of the functions of materials.“The primary function of evaluation is to assess the suitability of
materials for a given teaching and learning context” (Mishan &
Timmis, 2015, p. 66).
2.The criterias in evaluating the materials
• Physical quality
The materials recommended are durable and accessible for leaners.
• Content / format
The material recommended are accurate, authentic, current, and suitable for the leaner’s culture background content .
• Social consideration
The materials recommended avoid in equality of gender bias,sosial status, religion, race, and sexual orientation
• Instruksional design
The materials have to relate with curriculum and have consistency with the learning philosophy.In addition the instruction required student to force themselve to understand the materials and achieve the learning objective.
• Qualification of developer
The materials have been developed and validate by qualified people .
• Cost
The materials are reasonable regarding to the cost or budget.

3.How to conduct materials evaluation
A.pre-use evaluation
The teacher predicts the potential value of the following materials in this category:
• Context-free reviewing for journals
• Context-influenced reviewing for publishers with their intended audience
• Dependent on context, chosen for use in a certain class 2013 Tomlinson
It is "fundamentally a subjective rule of thumb action," according to Tomlinson (2013). However, checklists and more thorough criterion-referenced evaluations can also be employed, according to McGrath (2002). This should lessen subjectivity and provide a more trustworthy, principled, rigorous, and organized conclusion for judgments.
B.In-use Evaluation
This category gives the materials a more unbiased and trustworthy perspective. Because it will be able to consider the materials being used and the instantaneous responses and impacts, it has the ability to employ measurements rather than just relying on projections. An in-use evaluation can measure the following:
• Simplicity of directions;
• Clarity of layout;
• Text comprehension;
• Credibility of assignments;
• Achievability of assignments; etc
This does not imply that there are no restrictions on this kind of evaluation. It has the ability to judge both the criteria that can be seen and the impact of the information on short-term memory. Due to the delayed effect of instruction, it cannot, however, claim to evaluate either effective learning or what is occurring in the learner's brain.
C. Post-evaluation
The most effective (though least used) method for evaluating the suitability and prospective affordances of resources for your classroom is definitely this category. This is due to the fact that it is not practical or economical for most institutions. A post-evaluation would require effort and knowledge to correctly complete. If properly implemented, it may be possible to observe immediate changes in motivation, significance, and realizability. Additionally, it might look at the long-term results of lasting learning and application and provide feedback.
The advantage of a post-evaluation is that it can assess actual learning outcomes using a variety of methods, including testing what the learning materials have taught, testing what the learners can do, and testing these methods through interviews, questionnaires, criterion-referenced user evaluations, among others. Such information would offer solid and trustworthy input for decisions to be made regarding the use, adaptation, or replacement of the materials. However, care is still advised because learning is not a precise science and a number of factors, including the effectiveness of the teacher, parental support, exposure to language outside of the classroom, and intrinsic drive, may have an impact on the results.
4.The principles and technique for evaluating materials.
Many researches have conducted materials evaluation by considering the basic principles to evaluate the materials appropriately.Those principles are:
a.Considering of Cognitive level
Cognitive level is one of the criteria used to evaluate the textbook beside its availability on proficiency level (Assaly & Smadi, 2015). Cesur & Ozisler, 2019 suggest that material developers should provide students more support and variety in writing, reading, listening, and speaking activities and tasks in the textbook to engage students cognitively. Features of the book should also be chosen according to their language level, success score and expectation (Cesur & Ozisler, 2019). Therefore, the involvement of thinking skills as the representation of the cognitive aspect through the textbook selection and evaluation should be given more consideration due to its importance in developing critical thinking of students, their process of language acquisition, and their cognitive engagement (Margana & Widyantoro, 2017).
b.Affective engagement of students
Beside the cognitive engagement, the materials evaluation should consider the principles of affective engagement of students. In line with this, Simsek, M & Dundar, E (2018) revealed that the list of coursebook they evaluated absences criteria can be lack of support for creative thinking and cultural diversity that does not engage students’ interest at all.To overcome such an issue, creating creative instructional can be done through familiarity of the domains being designed.On the other hand, many coursebooks also contains many dry and dull texts, do not engage the interest of foreign students and contain repetitive format . In this case, the innovative materials can be a choice to employ new, different and better materials in terms of its content and the tasks being provided (Bouckaert, 2015). Therefore, a textbook should also provide the students with more autonomy and self-studying to the students to stimulate their motivation and interest and the content should capture students’ attention (Cesur & Ozisler, 2019).

c.Furthermore, the notion of authenticity as the principle to evaluate the materials could be an essential aspect need to be considered. Gilmore (2007) argues that authenticity as the criteria for evaluating materials means that it is designed to provide the students with real language to deliver the messages by taking the context of the real audience into consideration.

5.The framework to set criteria to achieve a decision considering materials.

a.Brainstorming evaluation criteria (Cunningsworth 1995,Tomlinson1999, 2003, McGrath 2002, Tomlinson and Masuhara,op. cit., and Dörnyei, 2007). Dörnyei (op. cit.) argues that the first step is that researchers should let their imagination go free and create as many as potential items they can think of and he refers to should articulate a list of universal criteria i.e. the criteria is
applicable to any language material anywhere for any learner.They derive from language learning principles and provide the fundamental basis for any materials evaluation. However, not
everyone would go about this in the same way.

b. Subdividing the criteria (Tomlinson 2003 and Tomlinson and Masuhara, op. cit.).
c. Monitoring and revising the criteria (Cohen, Manion and Morrison 2007; McGrath op. cit.; Tomlinson 2003; and Tomlinson and Masuhara, op. cit.). McGrath (op. cit.), for example, argues that the best way to check the transparency of the criteria and to see whether they work in the way they were intended is to try them
out.
d. Classifying the criteria into categories (Cunningsworth op. cit.;Tomlinson, 1999, 2003; Cohen, Manion and Morrison, 2000,2007; McGrath op. cit.; and Tomlinson and Masuhara, op. cit.).
McGrath (op. cit.) indicates that during considerations of layout, it is necessary to think about the ordering of the criteria and 294 categories. He maintains that user convenience and logical
interrelationships need to be taken into account.
e. Articulating specific criteria related to the medium of the materials(Tomlinson 1999, 2003 and Tomlinson and Masuhara, op. cit.).
f. Localising criteria (Tomlinson 1999, 2003 and Tomlinson and
Masuhara, op. cit.).
REFERENCE:
Soepriyatna. 2017. MPBI5204 : EFL Curriculum and Material Development. Tangerang Selatan: Universitas Terbuka.
https://www.researchgate.net/publication/337474246_Material_evaluation_in_English_language_classrooms_at_secondary_level

Jawaban 23:

Materials evaluation is a procedure that involves measuring the value (or potential value) of a set of learning materials. It involves making judgements about the effect of the materials on the people using them.
The evaluation process must consider the following criteria:
1. the learning materials' appeal to the students;
2. the credibility of the materials to learners, teachers and administrators;
3. the validity of the materials (i.e., is what they teach worth teaching?);
4. the reliability of the materials (i.e., would they have the same effect with different groups of target learners?);
5. the ability of the materials to interest the learners and the teachers;
6. the ability of the materials to motivate the learners;
7. the value of the materials in terms of short-term learning (important, for example, for performance on tests and examinations);
8. the value of the materials in terms of long-term learning (of both language and of communication skills);
9. the learners’ perceptions of the value of the materials;
10. the teachers’ perceptions of the value of the materials;
11. the assistance given to the teachers in terms of preparation, delivery and assessment;
12. the flexibility of the materials (e.g., the extent to which it is easy for a teacher to adapt the materials to suit a particular context);
13. the contribution made by the materials to teacher development;
14. the match with administrative requirements (e.g., standardization across classes, coverage of a syllabus, preparation for an examination).

There are three steps to conduct the material evaluation. They are Pre-Use Evaluation, Whilst (in)-Use Evaluation, Post-Use Evaluation.
In conducting the material evaluation, we can do the following procedures:
1. Invite the participants.
2. Distribute a copy of the material and curricula unit to the participants.
3. Ask the facilitators to observe, use, and demonstrate the material based on the curricula given.
4. Discuss the participants’ opinions about the material.
5. Distribute the questionnaire and ask all participants to fill in.
6. Analyze and list up the data from questionnaire and comments.
There are two principles in materials evaluation: effectiveness and efficiency (Ellis 1998 in Tomlinson 1998).
A framework to set the criteria to achieve a decision considering materials.
1. Brainstorm a list of universal criteria of evaluation.
That is to devise the universal criteria of materials that might be applied
2. Subdivide some of the criteria
That is to subdivide them to make the evaluation precise.

Jawaban 24:

Materials evaluation is the process of evaluating the material including media covers printed like course books and handouts, or non-printed ones such as real objects, realia, or computer-based media. It is conducted before the material is used, while they are used, or after they are used to ensure that the materials are suitable for the course/program. It refers to the efforts done to measure the value of the materials.

There are some general criteria for materials evaluation:

1.    Physical quality. Recommended materials should be durable and accessible for learners.

2.    Content/format. Recommended materials should be accurate, authentic, current and suitable for the learner’s cultural background context.

3.    Social considerations. Recommended materials avoid inequality of gender bias, social status, religion, race, and sexual orientation.

4.    Instructional design. The materials used should relate to curriculum and have consistency with the learning philosophy.

5.    Qualifications of Developer. Recommended materials should be developed and validated by qualified people.

6.    Cost. The cost / budget of the materials should be reasonable.

There are three categories to conduct materials evaluation:

1.    Pre-use Evaluation. At this stage, course designer making predictions about the potential value of materials on people who use them. This stage is important in the process of materials selection.

2.    Whilst (In)-Use Evaluation. At this stage, materials evaluation is conducted whilst observing or in-using them. This evaluation observes the performance of learners on task and exercises. At this stage, retrospective evaluation is also done. It is conducted to re-evaluate the materials while they are in use to determine if the materials work well.

3.    Post-Use Evaluation. It is the most important and valuable stage of evaluation. It can measure the actual effects of the material on the users and provide reliable information.

Materials evaluation must be conducted based on certain principles:

1.    Fulfil the learners’ need and appropriate with the course goals and objectives. The result of needs assessment is converted into course goals and objectives then becomes the bases for material evaluation.

2.    Accommodate learners to use language effectively for their own purposes. The materials should meet the students’ need so they can make use the language for their communication purposes.

3.    Facilitate the learning process. Materials evaluation has to ensure that the students are motivated and facilitated through materials used.

4.    Function as a support for learning, mediating between learners and target language. Materials should provide models for the students for the students to learn, exercises for the students to practice, activities to promote language learning, explanation, and examples to understand how language works.

A framework to set the criteria to achieve a decision considering materials, as follow:

1.    Brainstorming a list of universal criteria of evaluation. Devising the universal criteria for materials that might be used. Begins with general information and progress to a simple set of questions before evaluating the resources.

2.    Subdividing some of criteria. This step started by first looking at the general criteria and then breaking it down into sub-lists. Following that, evaluation statements and questions can be created.

3.    Monitoring and revision. The statements and questions formulated should be monitored and revised.