COLLABORATIVE LEARNING IN GENERAL CHEMISTRY WITH CQI MONITORING AND EVALUATION

 

Leonard S. Kogut * and Todd Luxton **

Penn State Beaver Campus, Monaca, Pennsylvania 15061

lenk+@pitt.edu

 

ABSTRACT

This paper describes the impact of mandatory collaborative learning in a second-semester college General Chemistry course.  A Continuous Quality Improvement (CQI) team guided the collaborative learning process and a student intern assisted in the design, management, and operation of the course.  Evaluation of the collaborative learning process by students revealed mixed results.  Evidence from "one-minute papers" and standard University course evaluation points to a lack of student time on task (study) as a causal factor.  The paper provides information on the role of both the intern and CQI team as well as analysis of factors which contributed to the somewhat less than desired outcome of this project.  A list of suggestions aimed at other instructors contemplating use of collaborative learning in chemistry concludes the paper.

 

Keywords: Collaborative Learning, Introductory, Teaching Techniques, Continuous Quality Improvement.

 

_____________________________________________________________________________

Presented at 14th Biennial Conference on Chemical Education, Clemson University, August 1996.

* Author to whom correspondence should be addressed. Present address: University of Pittsburgh, PA 15260

** Student intern for the Schreyer Institute for Innovation in Learning.

_____________________________________________________________________________

 

INTRODUCTION

Many college chemistry instructors are now using cooperative learning in their classrooms.  Some recent applications include the training of teaching assistants, enhancement of communication and student retention in General Chemistry, facilitating or improving laboratory instruction, and teaching Organic Chemistry, Analytical Chemistry, and Chemical Engineering [1-6].  I previously reported on a highly successful application of cooperative learning in a first semester course in General Chemistry in which students with poor academic preparation outperformed students in another course with better academic backgrounds [7].

 

During the Spring term in 1996, a student intern, the students in the second semester, 3 credit, General Chemistry "lecture" course (Chemistry 13), and I participated in the cooperative learning project described in detail below.  Unlike many of the studies cited above, however, and despite my previous success with this method in a similar course, Chemistry 13 was not as successful as I had anticipated.  This paper describes the course, the nature of the cooperative learning, and other facets of the project and list recommendations, based on lessons absorbed from Chemístry 13, which other instructors might consider to enhance success in using cooperative learning.

 

PURPOSE/GOALS

Many students come to Penn State Beaver campus with unrealistic expectations in terms of the level of difficulty of college course work and the amount of time needed to successfully study and learn course material.  Surveys in General Chemistry and Pre-Calculus Math during the Fall term in 1995 found that students spend under 4 hours per week on outside of class studying in each course — approximately one hour of study per credit.  Primary goals of this collaborative learning experience in Chemistry 13, were to:

 

(1)  Increase student time on task.

(2)  Introduce students to the concept that they can (and should) learn by studying with other students.

(3)  Encourage students to assume more responsibility for their learning by converting from a passive (lecture) to active learning mode (group discussion).

(4)  Use a Continuous Quality Improvement (CQI) team composed of students to observe collaborative learning, provide feedback, and develop recommendations for improving collaborative learning.

(5)  Extend the success of a previous collaborative experience in a first semester course in General Chemistry to the next course in the sequence.

 

DESCRIPTION: COLLABORATIVE LEARNING

Each student belonged to a collaborative learning team which met each week for at least one hour outside of class.  Teams formed based primarily on student class and work schedules, so student availability determined which students belonged to a given team.  At these meetings, teams discussed homework and completed other assignments including sample exams, critical thinking exercises, and problems requiring each student to provide specific information or data which was previously distributed to only that student but necessary for a collective solution.

 

Learning teams completed weekly Group Meeting Report Forms in addition to their assignments and returned these to a paid student intern who had taken both semesters of General Chemistry during the previous year, and who had participated in collaborative learning as well as CQI.  In addition, each of the eleven learning teams used another form weekly to submit two questions proposed by them for inclusion on the next major exam.  The intern posted these assignments in a "resource room" available to all students.  Two of the ten quizzes during the semester and 20% of the fìnal exam also required group participation.  A typical 50-minute class included 30-35 minutes of lecture based on a set of specific learning objectives for each chapter in the text followed by 10-20 minutes of group discussion or problem solving.  Each student received the learning objectives for the term on the first day of class.  In some classes the lecture mode took less than 30 minutes and in all classes some collaboration among team members took place.  The lectures were designed to be somewhat interactive and used a discovery or question mode, usually involving requests to specific learning teams for a response.

 

DESCRIPTION: CQI

Ten students volunteered.  They participated in CQI training at a beginning of semester two-hour session conducted by the Penn State Human Resources Development Center.  The intern served as CQI facilitator and the instructor was the team leader.  Ten evening CQI sessions took place and focused initially on providing direct feedback concerning operation of the collaborative learning process.  Each CQI member participated in a collaborative learning team and some were on the same team.  There were several learning teams without representation in the CQI group.  The CQI team also guided the design and implementation of four "one-minute papers" used to monitor class opinion and gather feedback about collaborative learning [8].  One of these papers discovered the critical, negative impact of a Physics course on the study habits of students in Chemistry 13 and is treated in the "Project Assessment" section below.  The intern and instructor used the results of the one-minute papers to design appropriate course strategies and to assess progress of the collaborative learning.

 

ROLE OF THE INTERN

The intern attended classes, assisted in the design of group assignments, collected and summarized one-minute papers, and served as facilitator for the CQI team.  The intern and instructor met weekly to review the progress of CQI, discuss problems, and, as indicated above, to map strategy for class activities for the next week.  The Schreyer Institute for Innovation in Learning at Penn State provided periodic training sessions for the intern.  The intern also kept a journal of course activities and experiences to serve as a foundation for future interns.

 

PROJECT ASSESSMENT

It would be comforting to report absolute success in this project but more realistic to summarize the results as mixed.  There were encouraging bits of evidence that, for some students, collaborative learning was successful.  Originally, each student was to complete an "Individual Perception Sheet" every two weeks.  These one-page forms were to probe levels of participation, learning, and enjoyment as well as monitor the development of group dynamics.  The CQI team, however, indicated that general class sentiment was that there was too much "paperwork" and each student completed this form only three times — during the 3rd week, 7th week, and 11th week of the 15 week semester.  The data in Table 1, a summary of responses to the student perception sheet, reflect a somewhat positive experience.

 

Table 1. Summary of data: individual perception sheet responses

 

I learned ...

              None

             Some

              Lots

# of students

% of class

# of students

% of class

# of students

% of class

Week 3

3

5.2

38

66.7

16

28.1

Week 7

2

4.6

36

81.8

6

13.6

Week 11

3

5.2

43

75.4

11

19.4

 

I partecipated ...

              None

             Some

              Lots

# of students

% of class

# of students

% of class

# of students

% of class

Week 3

0

0

17

29.8

40

70.2

Week 7

2

4.5

27

61.4

15

34.1

Week 11

0

0

18

31.6

39

68.4

 

I enjoyed ...

              None

             Some

              Lots

# of students

% of class

# of students

% of class

# of students

% of class

Week 3

3

5.3

32

56.1

22

38.6

Week 7

11

25.0

24

54.6

9

20.4

Week 11

4

7.0

36

63.0

17

29.8

 

Although two teams failed to submit the 7th week information, it appears that despite a mid-semester lag, most students learned, virtually all participated, and most enjoyed the experience to at least a moderate degree.  This data also revealed that only 2 of the 1l teams collectively had negative attitudes toward collaboratíve learning.  As indicated above, students also expressed opinions or provided data via the four separate "one-minute papers".  SeeTable 2 for the content of all of the one-minute papers.

 

Table 2. Questions asked in one-minute papers

_____________________________________________________________________________

CHEM  013 — "one-minute" paper # 1

1.  How many hours per week are you studying Chem 13?

2.  Are you enrolled in Physics 201 this semester?        Yes        No

3.  How many hours per week are you studying Physics 201?

4.  How would you rate the collaborative learning process at helping your understanding of the course material thus far?

1

2

3

4

5

very ineffective

neutral

very effective

5.  Thus far the quizzes have been:

1

2

3

4

5

very easy

average

very difficult

6.  Thus far my learning team has been:

 

1

2

3

4

5

very unhelpful

neutral

very helpful

 

CHEM  013 — "one-minute" paper # 2

1.  Describe the first 2 group meetings compared to the last 2 meetings with respect to productivity.

1

2

3

4

5

far less

less

same

more

far more

2.  Describe the first 2 group meetings compared to the last 2 meetings with respect to socialization.

1

2

3

4

5

far less

less

same

more

far more

3.  Make any comments you think are constructive.  Thank you.

 

CHEM  013 — "one-minute" paper # 3

1.  If given a chance would you break away from your group?  Why?

2.  Would you like to work in pairs of students for a week instead of in larger groups?  Why?

3.  Would you like to work on a project involving two learning teams for a week?  Why?

 

CHEM  013 — "one-minute" paper # 4

1.  List two things about your group that add to the success or ability of the group.

2.  List two things about your group that limit the success or ability of the group.

_____________________________________________________________________________

 

On the first of these, students reported the collaborative learning process as only mildly effective, 2.59 on a scale of 5.00, among students also enrolled in Physics and 2.81 on a scale of 5.00 among students not enrolled in Physics.  Of the 58 students originally participating in Chemistry 13, 27 were also enrolled in the 4 credit Physics 201.  What is ironic, if not conflicting, is that those enrolled in Physics rated the learning team in Chemistry 13 more helpful (3.67 out of 5.00) than those not in Physics 201 (3.41 out of 5.0,0).  Comments from this one-minute paper also reveal one of the possible detriments to the success of collaborative learning in this course.  Students did not spend sufficient time studying chemistry to achieve optimal performance.  The students in Physics 201 spent less time studying for Chemistry 13 than those not in Physics 201 (4.50 hours per week vs. 5.32 hours per week). 

 

In addition, the students in Physics spent more than twice as many hours on that course outside of class than on Chemistry 13 (9.65 vs. 4.50 hours per week).  Even when adjusted for the difference in the number of credits (Chemistry is 3 credits, Physics is 4), the data reveal that students enrolled in Physics 201 spent more time on Physics than Chemistry.  Random interviews of students determined that the Chemistry study total included the time at collaborative learning outside of class which was at least 1 hour per week.  From an instructor's viewpoint this amount of student effort is not conducive to success and explains, perhaps, some of the frustration students expressed on various feedback forms about the level of difficulty and need for more "teaching" by the instructor. The collaborative learning was therefore unsuccessful at attaining the goal of increasing time spent studying, at least early in the semester.

 

The second "one-minute paper" sought to monitor growth of group productivity and "socialization".  On a scale of 5.00 the average of 2.95 indicates productivity remained constant over the first 4 to 5 weeks of the semester as did socialization (3.09).  This was the data that also revealed two of the learning teams were experiencing difficulty in group dynamics.  The intern intervened to visit these teams in an unsuccessful attempt to help these teams become more effective.

 

In the third "one-minute paper" students were asked if they preferred to break away from their group.  Only 5 of 48 respondents said "no".  On the second question, 43 of 48 respondents said they would like to work in pairs for a week instead of with the entire team.  On a third question, 43 of 48 said they would not like to work on a project in which teams were combined.  Clearly, there is evidence here past the mid-point of the semester that students seemed comfortable in their groups but that perhaps the groups were too large.

 

The fourth "one-minute paper" sought behaviors that added to group success or hindered it.  Most students gave two responses in each category.  The frequency of the meaningful comments was:

 

Added to Success

 

Limited Success

Worked Well Together (22)

Disinterest (18)

Help Each Other (17)

Off-Subject Discussion (16)

Friendship (12)

Schedule Conflicts (13)

Dedication (10)

Personality of Members (8)

Discussion/Communications (5)

Lack of Understanding of Informations (7)

Doing Homework Before Meeting (4)

Domineering Members (4)

Others (5)

Lack of Preparation (4)

Others (4)

 

As part of the University Student Rating of Teaching Effectiveness (SRTE), administered during the 14th week, students also expressed opinions about the course apart from the one-minute papers. The comments and their frequency were:

 

Positive

 

Negative

Collaborative learning was good. (6)

Working in groups. (11)

I gained experience in collaboration. (2)

Organization of the course. (7)

Study groups were good. (2)

Too much material. (6)

Tests too hard. (5)

Collaborative learning in class was ineffective, need more lecture. (5)

We had to adiust to a totally new concept of learning—doing some "stuff" on our own. (5)

Mandatory group work. (2)

Others (5)

 

The instructor and intern reviewed the data and feedback and discussed it thoroughly.  That students in general still did not spend appropriate amounts of time on task is apparent.  There is also evidence from the "one-minute papers", group meeting reports, and intern and CQI observations to support a view that some students were immature and disinterested.  Both the intern and instructor also felt that even the CQI group was too large and size hindered its effectiveness.  However, the design of the one-minute papers and general feedback on learning team behavior were useful and productive.

 

Finally, the SRTE scores for this course were the lowest in the 25 year career for this instructor.  This result prompted even further discussion and reflection.  One factor that seems to account for the negative feedback from students is the abrupt change in teaching style they experienced from the first semester to the second chemistry courses.  Almost all of the students took either Chem 012 (3 credit) or 017 (5 credit) under the same instructor during the previous semester.  These courses are identical in content but differ in the number of class hours per week.  During the first semester, the courses used a predominantly lecture-oriented protocol with individual students responsible for their own studies.  SRTE scores were excellent in these courses, suggesting that students were unprepared for such a change or that they did not care for collaborative learning.  A second interpretation is that perhaps the course was poorly administered and the instructor skill level was not up to previous high standards.  This is an alternate explanation with which the instructor disagrees.

 

CONCLUSION

Several recommendations for refinement of the collaborative learning process emerged, based on the experiences of Chemistry 13:

 

(1)  Introduce collaborative learning in the first course in the sequence before trying it in the second.

 

(2)  Limit collaborative learning team size to 4 or 5 members and, when possible, encourage diversity in skill-level, gender, and ethnic background.  The two learning teams which were least productive consisted of groups who knew each other before they formed and which were composed in one case of students with uniformly better than average skills and in the other, uniformly lower than average skills.

 

(3)  Do more team-building exercises early in the semester.  These need not involve chemistry.

 

(4)  Dedicate class time each week to discussion of group progress and responding to team questione about the material or difficulties of group dynamics.

 

(5)  "Sell" collaborative learning more vigorously but stress that it be done voluntarily.  In this project it was mandatory.

 

(6)  Have one member of each collaborative learning team on the CQI team.

 

(7)  Train students in time-management skills.  Some method must be found to encourage appropriate study time.

 

(8)  Adopt testing methods supportive of group interaction as well as individual student preparation and responsbility. I now have learning teams submit a group answer to each quiz after students have completed it individually and I count both scores.  This also has the effect of validating the collaborative learning process because most team scores are higher than the individual scores.

 

(9)  Scale down the project.  In retrospect, for example, it appears that in-class student collaboration could have been toned-down, both in frequency and intensity.

 

(10)  Realistically assess the intellectual capabilities of the class in general to better match strategies used in the course to the ability of the students.  An after-the-fact comparison of SAT scores for the students in Chemistry 13 with those of students in the Chemistry 17 course which was the object of the previous study in 1994 determine that, based on these SAT scores, the earlier students were stronger in academic quality.

 

SAT SCORES

Course

        No. of students

Verbal

Math

Total

Chem 012, Fall 1994

44

465

572

1037

Chem 017, Fall 1994

38

443

530

973

Chem 013, Spring 1996

57

440

510

950

 

This comparison is surprising because students are placed into Chemistry 017 based on lower ability in Chemistry and Math according to well-tested Penn State placement testing.  The 1996 class in Chemistry 013 had a equal mixture of students from Chemistry 017, a 5 credit course for students with low placement scores, and the normal 3 credit course for students with adequate placement scores (Chem 012), which would suggest the Chemistry 013 class should be somewhat higher in ability than a typical Chemistry 017 section.  Note the higher SAT scores for the Chemistry 012 section in 1994 relative to the Chemistry 017 scores.

 

(11)  Finally, several of the collaborative learning studies cited earlier took place at universities with a high percentage of resident students.  Over 80% of the class in this study were commuters, many of whom work 12 hours or more per week at outside jobs.  Beaver Campus students formed teams based primarily on availability of meeting times and tended, therefore, to include more cultural, ethnic, and perhaps intellectual homogeneity than would be found in a residential campus.  The relatively low median for Math SAT scores is especially indicative that intellectual homogeneity was the rule rather than the exception.

 

Both the instructor and intern thank Dr. Larry Spence and the staff of the Penn State's Schreyer Institute for Innovation in Learning for their support and encouragement.  If the project spawned some failures these were "intelligent failures" which can lead to design changes and future improvements.  Both the instructor and intern spent many hours in training, planning, meeting with the CQI team, designing exercises, and evaluating progress.  In this respect, implementing collaborative learning was very time-intensive.  To have some of the negative student comments state there was "lack of organization" was somewhat discouraging and suggests that the project attempted too much, too soon, with students who, to some extent, lacked the maturity to adjust and commitment to their studies that the instructor expected.  Another frequently mentioned negative comment on the end of semester open-ended SRTE instrument was "the instructor expects us to learn on our own".  Yes, he did, and for some students, pherhaps the course succeded.

 

LITERATURE CITED

1.  Birk, J. P.; Kurtz, M. J. J. Chem. Educ., 1996, 73, 615.

2.  Dougherty, R. C.; Bowen, C. W.; Berger, T.; Rees, W.; Mellon, E. K.; Pullam, E. J. Chem. Educ., 1995, 72, 793.

3.  Drake, B. D.; Garcia, M.A.; Wingard, D. A.; Smith, R. L. J. Chem. Educ., 1994, 71, 592.

4.  Dinan, P. J.; Frydrychowski, V. A. J. Chem. Educ., 1995, 72, 429.

5.  Wright, J. C. J. Chem. Educ., 1996, 73, 827.

6.  Felder, R. M. J. Chem. Educ., 1996, 73, 1996.

7.  Kogut, L. S. J. Chem. Educ., 1997, 74, 720.

8.  Harwood, W. S. J. Chem. Educ., 1996, 73, 229.