Guide and Template Walkthrough

You may note that of the examples provided, only the Brewing program followed the suggested simple table format for section 1.1. You may also notice significant variations among the completed sections in terms of detail, and format. The recommended format is exactly that: a recommendation. It is understood that specific disciplines may need to present their material in a manner other than the format recommended, therefor It is possible to successfully complete the Self-Study Report using a format other than the one recommended. That said, the process will be much more straight forward and efficient for all concerned if the Program Review Team follows the recommendations provided in the guides and this resource.

A note to consider when you start completing the Self-Study Report. The Program Review team are all very familiar with the discipline and there may be a tendency to use jargon, or acronyms when answering the questions on the template. Throughout the Program Review Process, the Self-Study Report will pass through the hands of many well-intentioned people who support this process and have no training or insight into the specifics of the discipline. In many cases these people will be asked to approve or vote to approve advancing the Self-Study Report to the next phases of the process. If the reviewers do not understand the jargon, acronyms or writing in general, they send the report back to the team to clarify any items in question. This can significantly extend the Program Review Process. To ensure as quick and efficient a process as possible, please be explicit, describe things clearly and avoid, where possible, jargon, acronyms or insider language in the Self-Study Report.

This brings us to The Educated Person section of the Program Review Guide. This section of the guide provides prompts to consider when describing what a graduate from your program may look like, how they may behave and what they need to think and know for moving forward in their career or education. By nature, some programs may be more focused on the development of the graduate for employment in their career area. Other programs may focus on preparing their graduate for future study, and other programs may be more focused on developing the skillsets required to prepare the graduate for an enriched civic and personal life they want. Some programs will focus on all of these areas to varying degrees. The Program Review Guide #2 refers to these as pathways to employment, future study and an enriched civic and personal life. The Program Review Team will consider the prompts, document their responses in the Self-Study Report Template section 2.1, and use the information to populate the Career Pathways Map.

Please keep in mind that the prompts suggested are not comprehensive. There may be additional discipline specific considerations regarding a graduate’s future that are not reflected in the prompts. It is our recommendation to address each provided prompt for “Pathways to Employment” individually in section 2.1 of the Self Study Report before adding any discipline specific prompts. Once any additional prompts are recorded the Program Review Team may move on to the next pathway.

What to Report: 2.1 Pathways for Graduates [Self-Study Report Template – Page 4]

Reporting Guidelines: KPU Program Review Guide 2: Curriculum Guide

For that section, produce a short narrative for each of the following pathways:

Pathways to employment, including professional competencies that graduates acquire, and occupations they are qualified to enter.
Pathways to further study, descripting the most common study options that graduates pursue, and the career options this makes possible.
Pathways to an enriched civic and personal life, describing the essential skills a graduate of the program would have.

It also provides the basis for developing the Career Pathways Map, discussed next.

Examples of Graduate Pathways

The Career Pathway Map is a simple visual representation of the types of careers typical for graduates, future programs of study, or potential careers for graduates who have met the Program’s Learning Outcomes. The Career Pathways Map can help the Program Review Team identify potential opportunities beyond the typical which will provide a more comprehensive perspective on the Knowledge, Skills, and Attitudes program graduates may need to possess. The Career Pathways map will be added to the Self-Study Report as Appendix A.

See several examples of Career Pathways Maps in the box below. You will notice that some programs have provided extensive multipage documents for their Career maps, and others have provided a single page. Additionally, some have provided a visual representation while others have provided a written representation. Again, the guides’ recommendations are just recommendations. If there is a discipline specific reason that requires the Program Review Team to deviate from the recommendation, this will be considered. Again, the people who are reviewing these documents for approval may not understand the rational for the deviation which may lead to some confusion and a return of the Self-Study Report for clarification. The more closely you emulate the recommendations, the more efficient the process will be for all involved.

What to Report: Career Pathways Map [Self-Study Report Template – Page 4]

Reporting Guidelines: KPU Program Review Guide 2: Curriculum Guide

Produce a visual representation, in graphic or table form, of the Career Pathways Map, for the appendix of the Self-Study Report, including:

The typical occupations for graduates of the program, and whether this is entry level, or more advanced.
Possible occupations suitable for someone who has achieved the Program Learning Outcomes.
Pathways to further study, and potential careers to which these pathways would lead.

The Map is a communication tool that can help students understand the options available to them after graduating from the program. Examples of Career Pathway Maps are provided in Appendix A.

Examples of Career Pathways Map

KPU has established a series of essential skills. All programs at KPU are expected to develop these skills in our graduates. These skills are clearly articulated in policy AC9. The current iteration of policy AC9 indicates seven essential skills. There is a new version of policy AC9 which lists 12 essential skills required by all programs at KPU. For this resource we will use the 12 essential skills as outlined by the emergent AC9 policy. The essential skills required by KPU policy AC9 include:

Creative Thinking and Problem-Solving Skills
Oral skills
Interpersonal Skills
Teamwork and Leadership Skills
Personal Management & Entrepreneurial Skills
Writing Skills
Reading and Information Skills
Visual literacy
Mathematical Skills
Intercultural skills
Technological Skills
Citizenship and Global Perspective

KPU policy a C9 also articulates the skills required by the ministry. The Ministry that oversees Advanced/Post-Secondary Education (currently Ministry of Post-Secondary Education and Future Skills) conducts a survey of graduates each year. These surveys assess if graduates self-identify as having obtained a series of knowledge skills and abilities. Each institution is required to achieve at least 85% of graduates reporting that the program helped them attain these skills. While program review guide document #2 suggests that there is one survey measuring for seven skills, there are in fact 3 separate surveys each measuring for their own set of skills. The three surveys include #1 the baccalaureate graduate questionnaire, #2 the diploma, associate degree, and certificate student outcomes survey, and #3 the BC apprenticeship student outcomes survey. There is a significant overlap between the skills identified in the program review guide #2 and the skills identified in these three ministry surveys. In this resource we will include all the skills identified in all three surveys. This will better inform your Self-Study report and allow for a more comprehensive understanding of ministry requirements 4 KPU credentials. The skills that the ministry surveys measure for include:

Writing clearly and concisely
Speaking effectively
Reading and comprehending material appropriate to your field
Working effectively with others
Analyzing and thinking critically
Resolving issues or problems
Learning on your own
Numeracy [baccalaureate survey, and apprenticeship survey]
Ability to conduct research appropriate to your field of work [baccalaureate survey]
Using tools and equipment appropriate to your field of work [apprenticeship survey]

Reviewing these lists show that seven of the skills identified appear on each list. To be comprehensive and address all of the skills identified on both lists, our recommendation is to detail how the program addresses each of the twelve skills identified by KPU’s policy as well as the three Ministry skills that are not addressed for a total list of fifteen skills. Those would be:

Creative Thinking and Problem-Solving Skills
Oral Skills
Interpersonal Skills
Teamwork and Leadership Skills
Personal Management & Entrepreneurial Skills
Writing Skills
Reading and Information Skills
Visual Literacy
Mathematical Skills
Intercultural Skills
Technological Skills
Citizenship and Global Perspective
Learning on Your Own
Ability to Conduct Research Appropriate to Your Field of Work
Using Tools and Equipment Appropriate to Your Field of Work

The above will be recorded in section 2.2 of the Self-Study Report in section Skill Development. This resource recommends listing the first skill, then addressing in some detail how the program develops this skill, and continuing with the next skill until the list of 15 skills is exhausted.

See several examples of essential skills write ups in the box below. You will notice that some programs have listed each skill individually followed by a detailed explanation of how the program develops this skill, while others list all the skills in total and then describe how the program develops all the skills. Again, the guides recommendations are just recommendations and if there is a discipline specific reason to deviate from the recommendations provided in the guide then please do so and provide the rationale for this deviation. Please be reminded that those who are viewing the self-study report will be expecting the skills to be listed individually followed by a description of how the program develops these skills. The more closely you emulate the recommendations the greater the likelihood that your Self-Study report will be approved to proceed without the need for clarification.

What to Report: 2.2 Essential Skill Development [Self-Study Report Template – Page 4]

Reporting Guidelines: KPU Program Review Guide 2: Curriculum Guide

The assessment of how well the program is designed to address the essential skills should be reported in Chapter 2 of the Self-Study Report template, in section 2.2, Essential Skill Development. While doing the assessment, identify recommendations for improvements that may be required to better address the essential skills and clearly articulate these in Section 2.2.

Examples of Essential Skills

The Program Curricula section of the program review guide #2, describes reviewing your existing curricula and collecting information that will inform your curriculum map. The program review guide #2 discusses learning outcomes at the program level, at the course level, and how to proceed if your program doesn’t have outcomes. This resource will cover all the topics introduced by the program review guide #2 in a more detailed and more granular manner. It is our hope that by covering this material in this way we will better prepare program review teams to develop accurate and effective curriculum that will inform their programs curriculum maps for this review process and for use in the future.

Before we begin it is important to note that this resource is designed to support program review teams in evaluating current curricula. Most programs at KPU do not have the curricula required to develop a curriculum map. Even those programs with existing program learning outcomes may find that their program learning outcomes are not structurally sound, effective, or current. Focusing this resource to support program review teams in developing current and effective learning outcomes ensures a well-developed curriculum map for future use.

Stated simply, Learning Outcomes are statements that describe the learning that students can demonstrate at the end of an educational activity like a lesson, course, or program. Learning outcomes contextualize the knowledge, skills, and attitudes developed for everyone associated with the educational activity. This includes students, instructors, and administrators.

Quality learning outcomes tend to follow a specific structure. This structure contains the following four components.

The stem of a sentence that focuses on student ability. Most often this sentence stem will resemble something like:
“At the end of the educational activity, the student will be able to:”.

Well-structured learning outcomes will include a verb that can be measured or observed and will indicate the level of learning that has occurred. When considering a verb to be used in a learning objective it can be helpful to review Bloom’s taxonomy of learning. Bloom’s taxonomy list verbs in order of learning competency from introductory through to advanced for each of the cognitive, psychomotor, and affective domains of learning. These domains of learning generally refer to knowledge skills and attitudes respectively. Try to avoid verbs like know or understand as they are difficult to observe or measure.

A well-structured learning objective also includes the specific knowledge, skill, or attitude (KSA) to be displayed by the student.

Finally, a well-structured learning objective includes an indication of how that knowledge skill or attitude will be displayed in a measurable or observable way.

Put together a well-structured learning objective will resemble the following:

At the end of the educational activity the student will be able to (verb) play (KSA)a G major chord (How KSA will be displayed in a measurable or observable way) with their left hand on the piano.

Quality learning outcomes are also meaningful. This means a quality learning objective doesn’t just look at the material covered in the educational activity, rather a quality learning objective focuses on the integration, synthesis, and application of the knowledge, skills, and attitudes to a context broader than that they were learned in. It is possible for a learning objective to be well structured but not meaningful, or meaningful but not well structured. A high-quality learning objective will be both well-structured and meaningful to the educational activity.

Examples of Common Learning Outcome Problems and Solutions

Problem Learning Outcomes	Better Learning Outcomes
Not Student Centered	Student Centered
Different theories of economic development will be explored through lectures, readings and assignment	Students will name each theory of economic development and describe the key characteristics that distinguish theory.
Not Measurable	Measurable
Students will understand foreshadowing.	Students will be able to identify examples of foreshadowing in English literature and incorporate foreshadowing in their own works.
Unclear	Clear
Students will be able to analyze plants.	Students will be able to analyze climate characteristics of various plants making appropriate recommendations for green roof projects in specific cities.
Not Concise	Concise
Students will describe the process to weld aluminum in a shop, in the open air, in the rain, in the cold and in extreme heat, paying attention to the specific material preparation and equipment required for tig welding, mig welding, and arc welding with recently prepared material versus material prepared more than two weeks ago.	Students will weld aluminum successfully in the environments they are likely to experience.
Task Based Inflexible	Outcome Based Felxible
Students will be able to demonstrate on an injection pad the 8 steps to give an injection.	Students will be able to demonstrate the 8 steps used to administer an injection.

Learning outcomes exist at as many levels as there are learning activities. As a result, it can be quite challenging to develop guidelines that account for the variety of learning outcomes. this resource will focus on 3 levels of learning outcomes, learning outcomes at the lesson level, course level, and program level.

Lesson Learning Outcomes (LLOs) maintain the structure and purpose described above. LLOs speak to what a student will be able to do or know at the end of the lesson. These outcomes tend to be very specific, dealing with discrete ideas, tasks or attitudes that will become the foundation of knowledge on which future broad understanding is built. They should not be confused with Learning Tasks which are the discrete things a student must master to achieve the learning objective. For example, a lesson learning outcome could be as follows:

Upon completion of the lesson the student will be able to (verb) play a (KSA) G major chord (How KSA will be displayed in a measurable or observable way) with their left hand on the piano.

Upon completion of the lesson the student will be able to play a G major chord with their left hand on the piano.

While learning tasks for that lesson would be:

to play the note G.
to play the note B.
to play the note D.

Course Learning Outcomes (CLOs) are probably the most familiar as most of the courses taught at KPU have learning outcomes baked into the course outline. Similar to LLOs, CLOs speak to what the student will be able to do or know at the end of the learning activity, and in this case that learning activity is a course. As this is the end of the course, the CLOs should reference the integration of material taught in lessons into themes or broader understanding and its application in a measurable or demonstrated manner. For example, in a class on European history from 1918 – 1945 LLOs may be “by the end of this lesson the student will be able to discuss discrete occurrences of economic, political or social importance”, while the CLOs might be “by the end of this course, the student will be able to synthesize the impact of relevant occurrences in Europe between 1918 and 1945 and apply them in a cogent description of how these factors led to World War Two.”

Program Learning Outcomes (PLOs) are probably the most nebulous to understand although they are just learning objective like LLOs and CLOs. PLOs state what the learner should be able to demonstrate doing or knowing in a measurable way at the end of their program of study. While a CLO consolidates the learning from all the lessons, the PLOs consolidate the learning from the courses. The learner should be able to use this consolidated information in measurable and demonstrable ways. Coming back to our example of a class on European History from 1918 – 1945, where:

LLO – By the end of this lesson the student will be able to discuss discrete occurrences of economic, political or social importance. (small font

CLO – By the end of this course, the student will be able to synthesize the impact of relevant occurrences in Europe between 1918 and 1945 and apply them in a cogent description of how these factors led to World War Two.

PLO – By the end of this program, the student will be able to interpret past and contemporary events in broader historical contexts to explain the complex process by which societies have evolved.

While this resource has demonstrated that the more specific lower-level Learning Outcomes inform the higher-level Learning Outcome, it is important to note that this is not the order that they are developed. When developing learning outcomes, it is the highest level, or Program Level that is the starting point for development. It is important to deal with the general before moving to the specific.

The purpose of this process is to review the existing program, assess that it’s meeting what the institution claims it is meeting and verify that it is still relevant to industry, students and society. Before embarking on any changes as a result of your program review please consult with the Office of the Provost and Vice-President Academic at oProCurriculum@kpu.ca. The Provost’s Office can advise on the potential regulatory or policy implications of program changes – including the determination as to whether the Provost might need to consult the Ministry of Post-Secondary Education and Futures Skills about particular significant program change proposals.

If the program review team finds that the program in its current form is lacking in any way, then the program review team should document this in section 2.3 of the self-study report and include this information as a recommended revision for the future. The one exception to this is Program Learning Outcomes (PLOs).

Programs at KPU will find themselves in one of three situations. These situations include:

No PLOs exist for the program.
Current PLOs exist for the program.
Irrelevant or Inadequate PLOs exist for the program.

The best approach for each of these scenarios will be discussed below.

You may notice some programs at KPU were designed and approved without program learning outcomes (PLOs). This is not unusual as the requirement for PLOs is fairly recent. If after reviewing the FPP, and any existing previous program review documents, no PLOs are found, and no statements that could function as PLOs are found then the program is able to develop their own PLOs as described in this document. If you have any questions around if the wording in your FPP or previous program review could be used as PLO’s, please reach out to the Teaching and Learning Commons for an additional opinion. All new PLOs must be documented in section 2.3 of the self-study report, along with a recognition that they were used for the current curriculum map, submitted as appendix B, and a recommendation that these PLOs be used for the future.

Many programs at KPU will have some language around program learning outcomes, program level learning outcomes, program competencies, etcetera. Any information like this must be collected by the program review team and assessed for relevance, currency, effectiveness, and structural integrity. If any of this exists for the program that the program review team is assessing, they will find themselves in one of the two following groups. Again, if there are any questions around this information please reach out to the Teaching and Learning Commons for support.

The program review team assesses these existing PLO’s and determines that they both meet and are effective in describing the objectives and the requirements of the program, then the program review team may use these PLOs in their curriculum map. If necessary, the PLO’s may update their punctuation or grammar, but they may not be altered to change their meaning at all.

If the program review team finds themselves in this group, skip ahead to section 5.1, getting ready for Curriculum review. While it is possible to skip to this stage, we recommend completing the following step 1 to 4 as well. Having a list of graduate competencies will help keep the program review team in alignment moving forward. Developing category titles from KSA’s will allow you to verify the validity of your PLOs. Both of these are worthwhile exercises that will provide value to the program review process, even if you were not reporting on them.

Some approved programs may find that they have PLOs documented in the full formal program proposal that do not meet the structural standard or share the intended functions of PLOs described in this document. In this case, this process recommends revising existing PLOs, where possible, to be current, effective, and structurally sound or writing new PLOs where revision isn’t possible. All revised and new PLOs must be documented in section 2.3 of the self-study report, along with a recognition that they were used for the current curriculum map, submitted as appendix B, and a recommendation that these PLOs be used for the future.

If the program review team finds themselves in this group, please proceed through the remainder of this document in the order it’s provided. This will help you develop structurally sound, effective and meaningful PLOs for your program.

Much of the detail around changes to PLOs falls into the realm of governance and is outside the scope of this resource. Any questions regarding governance must be directed to the Office of the Provost Curriculum.

Generally speaking, Program Learning Outcomes are developed from the intended graduate competencies of a particular program. Graduate competencies are the Knowledge, Skills and Attitudes (KSAs) a student who has completed a program of study are expected to have. Knowledge usually involves specific discipline related information, for example, knowing behaviorism, knowing ohms law, or knowing the stochiometric ratio for petroleum to combust. Skills can include discipline specific information such as the ability to weld aluminum, calculate pi to 5000 decimals, and brew a batch of beer. Skills can also transcend discipline, for example, communication skills, leadership skills, and the ability to work in teams. Attitudes speak to a graduate’s disposition to discipline related topics, for example ethical practices, respect for worksite dangers, or a commitment to social justice.

Graduate competencies can be imposed, or motivated. Imposed Competencies refer to the external requirements for a program. These can be requirements for membership with a regulatory body, ministry requirements for graduates, or institutional requirements. The Essential Skills outlined by policy AC9 and detailed earlier in this resource, are examples of imposed competencies. Motivated Competencies are competencies that are wanted by the student or the instructor. Generally, these KSAs will be in areas that contribute to; a profession or vocation, intellectual or interpersonal development, and allow students to fully participate in society. One category of competencies that is seldom addressed is the category that relates to the future of the discipline.

Imposed Competencies	Motivated Competencies
– KSA = Industry requirements – KSA = Regulatory bodies – KSA = KPU Essential Skills, AC9 – KSA = Further Education – KSA = Civic and Personal Life	Educator focused – KSA = UNSTG competencies – KSA = Areas the student may not consider – KSA = Guiding the future of the discipline Student Focused – KSA = Career, education, social

Instructors of a discipline will have a particular insight into that discipline and the direction it takes in the future. Often these speak to things like developing ideas around sustainability, ethical practice or commitment to social justice. When developing Graduate Competencies, it is best to be as comprehensive as possible. Forty identified KSAs is good, eighty is better. Even if your program has existing and effective PLO’s, it is a good idea to maintain an indication of your program’s graduate competencies. Having a list of graduate competencies can help frame future discussions and help keep departmental direction focused. We have found using an affinity diagram to be an effective means of generating a comprehensive list of program graduate competencies.

An Affinity diagram is used to group identified KSAs. They work best when developed by a group of people. The intention of the affinity diagram is to develop as many graduate competencies as possible and group these KSAs into categories. Gather the program review team and invite any other interested faculty members or instructional staff. Provide each member of the group with a writing implement and many pieces of notepaper or post-its. Present the group with one central question to be answered. This question will be something along the lines of “For this program to be successful a graduate should know, do, feel or have been exposed to what?”. It might be helpful to display the categories of imposed and motivated competencies to prime the group members. Each member of the group will think up as many answers to this question as possible. They will document one answer per note paper or post it note, then move on to their next idea on a new page. Try to keep the suggestions simple, no more than three to five words per notepaper or post it note. It is important that the group members not limit themselves, at this point of the process no suggestion is a bad suggestion. The more ideas suggested and the more diverse those ideas are, the more comprehensive a list of graduate competencies will be developed.

Keep the group members writing constantly for 10 minutes. Once complete have the group members distribute their notes on a surface visible to all group members. As the group members review each other’s suggestions have them group ideas together based on the similarity of those ideas. In a short period of time several larger categories of graduate competencies will become evident. Usually a natural title for the category is indicated in the suggestions. Place that note paper at the top of the group to act as a title. If no title is found, discuss among the group what an appropriate title is and add that to a new note paper or post it and place it at the top of the category. While the group members are reviewing the suggestions, they may have questions about the meaning of certain suggestions. The group should discuss among themselves the meaning of any suggestion in question and come to an agreement on the meaning. If it turns out that one suggestion has multiple meanings add that suggestion to multiple categories by duplicating the suggestion on a new piece of notepaper or post it note. Once the majority of suggestions have been grouped, reorganize into a table with the category titles at the top and the graduate competencies from that title listed below. Once arranged these graduate competencies and category titles can be recorded in the “KSA-PLO” sheet of the spreadsheet available here. In the first column of the spreadsheet list all the graduate competencies the group was able to generate. In the second column list the category titles and have them overlap the graduate competencies in that category. See the diagram to the right for an example.

With your graduate competency KSAs developed and the category titles identified, you can now begin developing your Program Learning Outcomes. You will use the process described above for developing learning outcomes to develop your PLOs. While the structure will be the same, it is important to understand the role the PLO plays in your curriculum.

The program learning outcomes represent the single most enduring understandings you want your graduates to retain. As a program review team discuss, “Ultimately, once the program is over, what is the most essential thing you want your graduates to be able to do, or know, or feel?”. As a reminder, learning outcomes describe the demonstration of a graduate competency (KSA); reviewing your competency category titles is a good place to start developing an answer to the above question.

As these learning outcomes describe the application of the KSA following program completion, PLOs should describe broad observable behavior reflective of a perspective only available after completing the credential. As a result, PLOs tend to be quite broad as they integrate learning from all the courses, synthesized with theory, and practice, and applied to real-world or hypothetical situations.

There is no hard and fast rule regarding the number of PLO’s required for any program, the correct number is the number of PLOs required to describe the enduring understandings of a particular program. That said most degrees will have 12 PLOs plus or minus 2. Most diplomas we’ll have six PLOs plus or minus 2, and most certificates will have 3 PLO’s plus or minus 1. Again, this is not a hard and fast rule, and if your program review team deviates from these numbers with a strong rationale that is completely appropriate.

Writing learning objectives is challenging and it is unlikely the Program Review team will develop structurally sound and meaningful PLOs on its first try. This is expected and the Teaching and Learning Commons is available to provide support for developing and reviewing a program’s PLOs so they are reflective of the Program Review team’s intentions.

Examples of Graduate Pathways

In addition to the individualized support provided by the Teaching and Learning Commons we have provided a document (found here) to support the development of PLOs with many interdisciplinary examples of existing structurally sound and meaningful PLOs.

Working with the program review team, develop your first draft of PLO’s. Focus on replicating the recommended structure of the PLOs. Once your structurally sound PLO’s have been developed, review them for meaning. If you would like the Teaching and Learning Commons’ support, this would be the time to reach out for it. When you have completed the final draft of your PLOs, document them in the “KSA-PLO” sheet of the spreadsheet previously provided (available here). Replace the PLO number title with your PLO statement. Continue this across the top row until each of your PLOs represent the title of a column. See the diagram to the right for an example.

With the programs PLO’s developed and recorded it is important to verify that these PLOs do address the graduate competencies identified for the program. To do this, the program review team will review each individual KSA category title, moving from one PLO to the next, indicating with an “X” in the column if the PLO addresses the KSA category title. See the diagram to the right for an example.

When each KSA category title has been assessed against each PLO and the relevant intersections have been marked, the program review team is able to assess PLO validity.

Ideally each KSA Category title will be represented relatively equally across all PLOs. If one or more KSA category titles are not represented by any PLO, then the PLOs may be too narrowly focused. If one or more KSA category titles are overrepresented by the PLOs or are represented by every PLO, then the PLOs may be too broad. Each PLO should also be equally represented across their intersection with the KSA category titles as well. If there are any glaring over or under representations of either the PLO’s or the KSA category titles, the Program Review team should review the PLOs to see if they can be revised to broaden or narrow their focus. When the Program Review Team have finalized the PLOs, they can be documented in section 2.3 of the Self-Study report.

See below several examples of other programs’ accepted submissions for PLOs in the box below. You will notice that one program has listed PLOs followed by a discussion on their relevance, another program has compared their PLOs with the KPU Essential Skills outlined in policy AC9, and the last program has a cursory comparison of PLOs against graduate competencies. None of this is required here. A list of PLOs is all that is required. You may also notice that the examples show PLOs discussed in section 2.2. All these examples were using a previous version of the Program Review Process. The current version of the Self-Study report only requires a list of PLOs. The more closely you emulate the recommendations the greater the likelihood that your Self-Study report will be approved to proceed without the need for clarification.

What to Report: 2.3 Curriculum Assessment [Self-Study Report Template – Page 4]

Reporting Guidelines: KPU Program Review Guide 2: Curriculum Guide

Program Learning Outcomes

The PLOs are reported in the sub-section labeled “Program Learning Outcomes.” List each PLO, numbering them for ease of reference. Indicate whether these are new, because none existed before, or modified from the most recent approved PLOs. If modified, please include the most recent approved PLOs as well as the recommended revised ones that were used for the curriculum review. These recommended changes should be the result of the curriculum mapping. The reasons for the recommended changes to the PLOs should be provided.

Provide the results of the assessment about whether the PLOs are correct:

Describe what PLOs are missing, if any, and why it’s needed. Missing PLOs can be added to the Curriculum Map to determine whether they are already being addressed through the existing CLOs.
Identify any PLOs that aren’t relevant, and explain why.
If PLOs have been changed, explain why.

You will have an opportunity to obtain feedback on the PLOs when the Office of Planning and Accountability conducts the surveys of faculty, students, alumni and the Discipline/Sector.

Once the program review team has verified the validity of the PLOs, then they can begin to develop their curriculum map by mapping Course Learning Outcomes (CLOs) with PLOs.

With new verified and validated PLO’s, the Program Review team is positioned to cross reference their course learning outcomes with their program learning outcomes. To begin, the Program Review team will transfer their program learning outcomes from the top row of the “KSA-PLO” sheet to the top row of the “CLO–PLO” sheet of the supplied spreadsheet, available here. Once complete, the program review team will document the courses required for the credential in the first column. While not required It will help the curriculum assessment if the courses are documented in roughly the order the student is expected to complete them. Information regarding credential requirements, including the courses required to complete the credential, are available in your formal program proposal (FPP), most recent program review or revision documents, or in the university’s academic calendar (available here). The Program Review team will list the course learning outcomes for each course in column two, expanding or reducing the CLO spaces as needed. See the diagram to the right for an example. Learning outcomes for the courses required by your credential are available through “KPU one” in Course leaf Course Inventory Management (CIM) available here.

a) Electives

Most program credentials offer opportunities for elective courses. In some cases, an elective option will involve choosing one course from two program specific options, and in other cases an elective option could be any one of hundreds of three-credit courses at the postsecondary level. A program’s curriculum map will quickly become too complex and unwieldy to be effective if the credential includes broad elective options. For this reason, we recommend including only elective options that are discipline specific or are binary in nature (for example a choice between CMNS 1140 and ENGL 1100).

b) Shared Core, Foundation, and Fundamentals

Some faculties and departments have a group of courses required for all credentials offered by that faculty or department. if the program being reviewed is offered through a faculty or department with a shared required group of courses, then the program review team will include these courses in their curriculum map. In some cases, these shared groups of courses come with their own program learning outcomes (PLOs). If this is the case then the program review team will need to discuss changing the number of PLOs they have developed for their program. The outcome of this discussion may be different for each program.

a) Assessing PLOs by CLOs

Once the program review team has come to agreement on the courses to be included and has documented these and the course learning outcomes in the “CLO–PLO” sheet of the supplied spreadsheet, (available here), they may begin assessing the intersection between CLOs and PLOs. The process is very similar to the one followed to verify PLOs validity. The program review team will review each individual course learning outcome against each PLO, and if the PLO is addressed by that CLO, they will mark an X in the column. Once assessed, the program review team will move to the next PLO, and continue this way until all PLOs have been exhausted for the individual CLO. Once all PLOs are exhausted, the program review team will move down to the next CLO and begin assessing against the PLOs again. When all the CLOs have been assessed against all the PLOs, the program review team can review the curriculum map for gaps, or redundancies. Gaps occur when a PLO or CLO are underrepresented by the intersections. A redundancy occurs when a PLO or a CLO are overrepresented by the intersections. Please see the diagram to the right for an example. Once again, we are looking for a relatively equal distribution of intersections between PLOs and CLOs. If one PLO or one CLO is an outlier in terms of number of intersections, the program review team should review this outlier to determine if it is an issue that needs to be addressed, or an expected distribution of the program. This should be documented in section 2.3 of the Self-Study report.

b) IDA Assessment

Once the program review team is confident in the distribution of intersections between the CLOs and the PLOs, they can move on to the IDA assessment. The IDA assessment looks at the level of the material germane to the PLO being introduced. There are three levels in this assessment, I or introductory, D or developing and A or advanced. This assessment involves reviewing each intersection of CLO and PLO to determine if the material described in the CLO is introducing the PLO, developing the ideas of the PLO, or presenting an advanced understanding of the PLO. The program review team will review each intersection of CLO and PLO marked with an X, determine its level, and replace the X with an I, D, or A as appropriate. Please see the diagram to the right for an example.

c) Curriculum Map

Once all the X’s have been replaced with an I, D, or A, then the program review team has developed a curriculum map that can be reviewed for the Program Review process.

What to Report:

The Curriculum Map will be included in the Self-Study Template as Appendix B.

All Curriculum Maps will be included as Appendix B.

Examples of Curriculum Maps