Paradigms in Physics

Final Evaluation Report of Work Spanning 1997-2000

Margaret L. Niess

Professor of Science and Mathematics Education

Oregon State University

The Paradigms in Physics NSF funded project was a three year redesign of upper division physics curriculum and instruction at Oregon State University. By necessity and design, the project included both formative and summative evaluation measures in order to assess the project's readiness for widespread dissemination.

Prior to 1997, the OSU physics curriculum was typical of traditional university physics curricula. The curriculum represented the discipline as subdivided into separate sub-disciplines; courses representing the sub-disciplines were taught separately as a long sequence of courses, two to three quarters in length. Some sequences (Electromagnetism, Classical Mechanics and Mathematics Methods) were taught in the junior year and some sequences (Quantum Mechanics and Thermodynamics/Statistical Mechanics) were taught in the senior year. In this model, individual faculty members taught courses/sequences independently with little or no regard for work that was undertaken in other courses/sequences. The curriculum represented a discrete, linear approach to learning physics. Little opportunity was provided for establishing connections among the various sub-disciplines. With the courses/sequences distributed over the junior and senior years, students were confronted with an immediate increase in level of difficulty upon entering the junior level work. No time was allotted for students to build in expertise in this curriculum; no time was purposely designed for students to build their analytical and problem solving skills in the discipline of physics.

The redesign of the curriculum for 1997 focused on the development of students' comprehension by cultivating their analytical and problem solving skills and establishing connections between the content of different sub-disciplines. A new spiral-approach curriculum was envisioned as a double-tiered course of study where students were to consider the main topics twice: first emphasizing analytical skills, then emphasizing deductive skills and disciplinary integration.

In addition to the redesign of the content of the physics curriculum, the project proposed significant changes in the instruction for the junior and senior years. Rather than courses offered in the traditional university 11-week, three credit courses with three hours of lecture, the project redesigned the instruction into smaller modules of study - three week, two credit paradigms, where students concentrated on one theory course at a time (instead of the traditional two or three), meeting on hour on Monday, Wednesday, and Friday, and two hours on Tuesday and Thursday. The longer times provided opportunities for alternative teaching strategies such as group projects and discussions, group problem solving sessions, regular labs, computer labs, and individual investigations. A one-week Preface at the beginning of the term was provided to introduce students to tools to support their work during the following three Paradigms of that particular term. The junior year (first offered Fall 1997) included three one-credit optional prefaces and nine two-credit (three per term) case study Paradigms spanning many of the principal examples typically developed in the deductive sub-disciplines in a traditional physics curriculum. The senior year (first offered Fall, 1998) was followed by five single-term Capstone courses that systematically presented the usual deductive systems of physics: classical mechanic, electromagnetism, quantum mechanics, mathematical methods, and statistical physics.

Ultimately, this curriculum redesign was more extensive than a simple redesign of course content or sequencing. With the non-traditional scheduling of courses (within a traditional university schedule) and the extensive restructuring of the content, this redesign was more complex, impacting how professor presented the revised courses content, how students had to approach learning, and what resources were available for supporting both the professors and the students. For that reason much of the evaluation over the three-year period focused on formative evaluation efforts to guide the development.

1. Formative Evaluation Efforts

The formative evaluation efforts are considered in three separate areas:

- Curriculum

- Faculty and Instruction

- Students

A reflection over the three years of evaluations resulted in important considerations for any dissemination efforts.

1.1 Curriculum. The initial design of the curriculum was subjected to a modified Delphi external review by expert physics faculty throughout the US summer, 1997. While eleven professors agreed to participate in this review, only eight returned the initial survey. These eight, six from universities similar to OSU and two from universities unlike OSU) were involved in the three Delphi rounds that resulted in a compilation of their perspectives on the curriculum. While the majority were in agreement with the proposed curriculum, two questions elicited some concern for consideration. The questions that elicited the concerns were:

§ Does the program utilize a good set of topics?

§ Does the flow of topics appear reasonable?

In essence the reviews were divided on these two questions, suggesting that the faculty review the set of topics and the flow of those topics. All comments and responses were summarized and provided to the faculty for their consideration as they began to implement the redesigned curriculum. Appendix A provides the full review.

As the program moved into the implementation phase, both of these concerns were at the forefront of consideration. Paradigm 1 was eventually switched with Paradigm 2. (This report retains the names of Paradigm 1 and Paradigm 2 in order for consideration of all the evaluation data.)

While the evaluator did not have specific data with respect to other content considerations, data collected continued to indicate that faculty needed to consider the content of specific paradigms and the ordering of the paradigms and capstones (e.g., E&M and Optics Capstones). It will be critical that any dissemination effort clearly identify any outstanding issues in the curriculum and the background that has directed those concerns.

1.2 Faculty and Instruction. Beginning Fall, 1997 faculty began the instruction with the junior year Paradigms courses. Given the significant changes, the faculty met often throughout the year to collaborate on the progress of the redesign effort. While the faculty initially wished to focus on the curriculum, the instructional aspect and concerns for students shifted the focus of the meetings. The fall meetings focused on the effectiveness of the teaching strategies and the rhythm of the intensive implementation. Near the middle of the year, the faculty were able to return to the issues of the content and continuity. The discussions of pedagogy were novel in the department and were a critical part of the redesign.

The second year of teaching allowed professors the opportunity to reconsider the individual courses and the collection of courses. A significant change was made for the first term of the second year. Based on the formative feedback and as previously mentioned, the decision was made to reorder the paradigms by switching Paradigm 1 with Paradigm 2. By the end of the funding, the Fall term Paradigms continued to cause some concern for both students and faculty.

More subtle changes were made in individual courses. Faculty relied on the feedback from the first year to reconsider their specific presentations, student expectations and evaluations. Student evaluations demonstrated overall more satisfaction with the courses the second year than in the first year. This result was not unexpected in that faculty were better prepared for questions, having had experience with similar questions. Student satisfaction continued to improve with the third year as the faculty taught the courses for a third time (in most cases). Analysis of the data with respect to this observation suggested several concerns for any dissemination of this curriculum.

§ Instructional Design. The current curriculum and instructional redesign is based on primarily a single faculty member's interpretation of the course (that is, each course was implemented by a single faculty member). These faculty worked over three years identifying a course plan that, with each year, became clearer as to the goals and objectives along with the instructional design to support students in meeting those goals and objectives. Part of this solidification, however, is clearly an "individual's" instructional design. Care must be taken to clearly describe not only the curriculum for the course, but also the instructional design incorporated, as replication efforts are considered.

§ Faculty Preparation. As with any change, faculty were required to invest additional time - time not typically allocated to faculty in preparing to teach a class. With the significant instructional and curricular redesign, any faculty new to the redesigned physics curriculum must have additional time allocated for implementation.

§ Resources. The majority of physics textbooks and resources are written within the traditional model for the upper division physics curriculum. Faculty in the Paradigms project were required to develop many written materials for this redesigned curriculum. Any consideration of dissemination outside the current institution will require that these materials be made available in a form that is understandable given the course structure.

§ Graduate Teaching Assistants. The Paradigms project was a significant shift from the traditional format for student learning. Continued analysis of student perspectives demonstrated a reliance of course assistants (Graduate Teaching Assistants). This redesign supported an increase in the number of assistants available for both faculty and student support. From the student data, "TA's were extremely helpful (a gift form God!) (see Appendix B page 5)." From the faculty perception, however, it may have been that the GTA's "buffered" the faculty from problems students were having. Certainly by the end of the project, students had indicated that "the most helpful instructional strategies" were the lectures, homework and group work; whereas at the beginning they had included faculty office hours as one of the most helpful instructional strategies (see Appendix B page 5). Any dissemination effort must provide for the support of both faculty and students as they adjust to the new framework for both instruction and curriculum. But, there must be a consideration of the balance that is desired for optimal progress of the students and of faculty understanding the needs of students.

1. 3 Students. The discussions of the faculty were supported by extensive data collection from students as they participated in the courses. Table 1 provides an outline of student data collected and reports prepared to provide information for the formative evaluation. The extensive student input to the redesign assisted in the ultimate identification of factors for consideration in any dissemination of the curriculum.

- Learning to learn in a non-traditional course structure: adjusting to intensity and pacing. The Paradigms’ modular approach (with three-week focus for each module) required students to learn in a manner different from their traditional mode of instruction for the courses they took in the lower division courses and non-physics upper division courses. From the student data, the Fall term of the Paradigms continued to be more troublesome for the curriculum redesign than any of the others terms. The students were required to adjust how they learned physics as well as how they integrated those expectations among their other concurrent traditional 11-week, three hours per week courses. The pace of the paradigms was frustrating for students. By the second term, students seemed to adjust by developing strategies for dealing with the pace and the requirements for learning in a non-traditional course. As one student indicated, “I learned how to learn.” Any dissemination consideration of the modular format of this physics curriculum and instructional approach will need to consider student support for the change.

Early in the project, the faculty were prepared to reconsider whether some or all of the Paradigms might be better suited to being taught concurrently in a lower-intensity mode, more like the traditional scheduling structure. By the end of the funded project, no changes were undertaken. Dissemination efforts may well have to also be prepared for a consideration of returning to the lower-intensity mode if unable to achieve the results of the initial implementation.

Caution must be taken in making adjustments in the structure however. It may be that the structure used in this project resulted in some of the outcomes of this particular project. For example, the intensity and pace may have required more attention to the expectations. Students may have made gains in problem solving skills because of the added concentrated attention.

§ Learning to learn in a non-traditional course structure: adjusting to different instructors. The modular approach with different instructors for each module resulted in various important side benefits of the program. Students were required to adjust to a new instructor every three weeks. This adjustment required additional student learning that was problematic for them until they had had the instructors more than once. As they indicated one of the major obstacles in the program was “getting used to the new system of three or four professors each quarter for each Paradigm. We did not know what they wanted and what they expected.” Another students indicated that “adjusting to the new system, four or five professors in the first term and half, [meant I had to adjust] the work load from previous years.” With the different instructors, however, a variety of learning styles were met in one term. Where one student indicated “I learn by lecture so the style is important,” another expressed a request for “more demonstrations and labs.” Some instructors used projects while others used examinations to assess student understanding. “I like it being project-based not final-based.”

Dissemination efforts will need to consider the effect of having one instructor versus a different instructor for each module. Other institutions who offer the paradigms in the format with different instructors will need to consider the effects of students learning to learn with a variety of instructors in a short period of time. On the other hand, there may have been benefits from learning form a variety of instructional perspectives that supported the results of this project.

§ Learning to learn in groups. With the focus on problem solving and project-based efforts, students were required to learn in groups. By the end of the project, group work continued to be a variable of interest because while several students would list group work as one of the best instructional strategies, others described groups as a hindrance to their learning. Dissemination efforts will need to consider this variable more carefully to determine how best to prepare students for learning in groups if the nature employed in this implementation.

§ Coping with homework. With the intensity of the courses in the three-week format, homework due everyday was the norm rather than the exception. This intensity frustrated the students for two reasons: (1) lack of time to work on the ideas and think through the ideas; (2) not having enough information to complete, or even start, the homework. The second reason was particularly problematic as students attempted to begin the homework in advance but, because of the progress in the lectures, had difficulty attempting solutions to the problems without an understanding of the content and processes. Eventually, students had to juggle the time for doing the work in conjunction with their other more traditional courses where more time was allotted for completion of homework and more information provided prior to completion. While it was not possible in this project to ascertain, future consideration might investigate whether student success with homework is at a cost to other courses outside the physics curriculum (other undergraduate course success and completion).

Table 1. Outline of Data Collection, 1996-2000

Year/Term	Data Sources	Purpose, Reports
1996-1997
Fall	Email	Traditional Curriculum Student Baseline
Winter	Email	Traditional Curriculum Student Baseline
Spring	Student Interviews	Traditional Curriculum Student Baseline
Year	§ Student Interviews § Faculty Evaluations	§ Interview Report § Traditional Student Descriptions
1997-1998
Fall	§ Junior questionnaire § Preface 1 Observations, questionnaires and student interviews § Paradigm 1 Observations, email, interviews § Paradigm 2 Observations, email § Paradigm 3 Observations, email § Seniors (baseline) email	§ Fall Report § Paradigm 1 report § Paradigm 2 report
Winter	§ Junior questionnaire § Preface 2 Observations, email § Paradigm 4 Observations, email, interviews § Paradigm 5 Observations, email § Paradigm 6 Observations, email § Seniors (baseline) email	§ Winter Report § Paradigm 4 report § Paradigm 5 report
Spring	§ Paradigm 7 Observations, email, interviews § Paradigm 8 Observations, email § Paradigm 9 Observations, email § Mechanics (Observations)	§ Paradigm 7 report § Paradigm 8 report
Year	§ Juniors Interviews, Faculty Evaluations § Seniors Interviews	§ End-of-Year Paradigm § End-of-Year Baseline Report
1998-1999
Fall	§ Junior Questionnaire § Senior Questionnaire § Preface 1 Email § Paradigm 2[1] Email § Paradigm 1 Email § Paradigm 3 Email § Seniors Email	§ Fall Paradigm Report § Fall Capstone Report
Winter	§ Junior Questionnaire § Senior Questionnaire § Preface 2 Email § Paradigm 4 Email § Paradigm 5 Email § Paradigm 6 Email § Seniors Email	§ Winter Paradigm Report § Winter Capstone Report
Spring	§ Junior Questionnaire § Paradigm 7 Email § Paradigm 8 Email § Paradigm 9 Email § Mechanics Email	§ Spring Paradigm Report
Year	§ Paradigms Interviews § Paradigms Faculty Evaluations § Capstone interviews § Capstone Faculty Evaluation	§ Interview Report § Student Description § Interview Report § Student Description
1999-2000
Fall	§ Junior Questionnaire § Senior Questionnaire § Preface 1 Email § Paradigm 2[2] Email § Paradigm 1 Email § Paradigm 3 Email § Seniors Email	§ Work on compiling comparison
Winter	§ Junior Questionnaire § Senior Questionnaire § Preface 2 Email § Paradigm 4 Email § Paradigm 5 Email § Paradigm 6 Email § Seniors Email	§ Work on compiling comparison
Spring	§ Junior Questionnaire § Paradigm 7 Email § Paradigm 8 Email § Paradigm 9 Email § Mechanics Email	§ Work on compiling comparison
Year	§ Paradigms Interviews § Capstone interviews	§ Comparison of Juniors: 97-98, 98-99 and 99-00 - see Appendix B § Comparison of Seniors: 98-99 and 99-00 - see Appendix C

2. Summative Evaluation Efforts

The summative evaluation efforts focused on the progress of students in the redesigned physics curriculum in comparison to students in the baseline traditional physics curriculum. Efforts to prepare this comparative profile was complicated by a number of factors: number of students in the baseline who signed the Human Subject release; access to only one year of baseline students; number of students in the redesigned curriculum who signed the Human Subject release; discrepancy between quality of student backgrounds from 1997-1998 to 1998-1999 to 1999-2000. Attempts to obtain long-term experiences beyond graduation resulted in only one student response. Both quantitative and qualitative information was gathered to present a summative evaluation of the redesigned curriculum.

The summative evaluation results are provided as:

§ Quantitative: Physics GRE

§ Qualitative factors

- Analytic problem solving abilities

- Integration of mathematics and physics

- Acceptance of varied notation

- Spiral nature of curriculum enhances learning

Future consideration of implementation of the redesigned program must consider these results as tentative, potential outcomes since they were identified in an experimental setting where the curriculum and instruction was shifting through extensive formative evaluation efforts.

2.1 Quantitative: Physics GRE

One indicator of the effect of the redesigned physics curriculum was the Physics Graduate Record Examination since many students typically continue with graduate study following the completion of their undergraduate degree. Table 2 provides both the data collected and a graphical representation of these data. While numbers are limiting factors and student background academic achievement was not available, the data do indicate that students completing the Paradigms undergraduate physics program have increased scores of the test. Of particular note in interpreting this information is that the Paradigms 2000 class did have a higher GPA during the upper division coursework than the Paradigms 1999 class.

Further examination of the GRE scores of students by comparing the Physics GRE scores with the pre-Physics GPA indicated that the Paradigms improved the support for the learning of physics for average and below average students. In the traditional program, students struggling early tended to withdraw, changing to other majors. However, similar average and below average students in the Paradigms program were more often retained and supported in their continued work with physics at no apparent expense to the above average students. In the Paradigms, students quickly recognized the importance of working together, both the strong and the weak students. And, their work was continuous over the term with courses changing every three weeks. The extensive group work appeared to contribute to a stronger support mechanism for average and below average students, students who typically need additional support to engage in the processes.

Table 2. Performance of Seniors on the Physics GRE

Baseline 1998	Paradigms 1999	Paradigms 2000
420	400	500
490	440	510
520	470	510
550	480	540
550	490	570