top of page
This site was designed with the
.com
website builder. Create your website today.Start Now

WEEK FIVE: Thursday May 25, 2017

Evidence-Based research Panel

1:00 PM - 2:30 PM

PANELISTS I Open the e-Handout

 

Rachelle Crosbie-Watson, Ph.D.

Professor and Vice-Chair, Department of Integrative Biology and Physiology, UCLA

Professor, Department of Neurology David Geffen School of Medicine, UCLA

​

Research Area / Specialization

Molecular Basis of Muscle Function and Muscular Dystrophy

 

Relevant Course(s)

Undergraduate general chemistry (CHEM 001; 200-500 students) and inorganic chemistry (CHEM 150A; 15-80 students).

​

Teaching or learning problem/challenge that made you rethink your learning model or approach or try something new

  • PS121 “Disease Mechanisms and Therapies” is an upper division undergraduate elective that is focused on creating meaningful learning experiences for undergraduate STEM majors using L. Dee Fink’s Taxonomy of Significant Learning.

  • PS121 is a fully online course with synchronous and asynchronous activities that brings together biological science undergraduates from across each of the nine University of California campuses.

  • Online learning communities are formed through small discussion sections and community projects that promote team-based work. Our preliminary data support that PS121 is a meaningful, transformative experience for students in which they are driven to learn deeply by focusing on a single complex problem to develop critical thinking skills, while being stimulated to think beyond themselves.

  • In PS121, undergraduates learn to integrate concepts from many disciplines to address real world challenges and to engage with the community. PS121 is a highly structured course that employs scientific teaching practices to benefit all students.

  • PS121 fulfills the Diversity Requirement at UCLA for its consideration of disability and socioeconomic factors that can affect access to healthcare.

​

How you addressed that challenge (if applicable, what technology, learning strategy, etc. you integrated to do so)

  • We were motivated to create a scalable course that could be offered across the entire University of California system to address issues of increasing student enrollment in the absence of increasing faculty recruitment.  

  • For these reasons, we sought to transform a popular face-to-face upper division elective into an online venue.  

  • Fully online course models have not yet been fully embraced by students and faculty as there is a belief that students are unable to connect with faculty in the online format, that there is poor instruction in online courses, and that online courses diminishes the educational experiences, which may result in learner frustration and isolation (Brown, 1996; Hara & Kling, 2000; Piccoli, Ahmad, & Ives, 2001).  

​

How students responded, any challenges, and lessons learned

  • We sought to investigate how students perceived different online teaching styles and formats. Our education team creating eight different videos that delivered the same content, but varied in the presentation style.  

  • The presentation styles that were tested include: Classic Classroom, Weatherman, Demo, Learning Glass, Pen Tablet, Interview, Talking Head, and Slides On/Off. The videos were shown to a classroom of over 175 undergraduates in the physiological sciences core, which is the final course that is required for completion before taking upper division electives.

  • Students were provided with a written survey that included the question “Do you think that this video is effective for learning?” and a box for open-ended comments.  

  • Students ranked each video on a scale of 1 to 5 (5 being the highest).  We created a new coding scheme to evaluate the open comments.  

  • The codes were assigned to categories that included

    1. stage of video lesson preparation (pre-production, lesson design, filming, post-production editing)

    2. role in video lesson (director and/or instructor)

    3. content (instructor, production, user experience, screen, and pedagogy)

​

If you conducted research, was there any evidence-based impact?

  • Students ranked the Learning Glass as the most effective video style and the Interview was the lowest scoring style.  

  • The Learning Glass received highly effective rankings in all three categories.  

  • Decisions made by the instructor and director were reported to be important to the success of the Learning Glass.  Students had a positive user experience and felt that the Learning Glass promoted effective learning.  

  • Positive comments on the Learning Glass included that the video was engaging and that they felt connected to the instructor since the instructor was making “eye contact” and speaking to them directly.  

  • The Weatherman and the Pen Tablet were also highly ranked by students.  

  • Students reported that the instructor’s blocking the slides in both of these video styles decreased their overall ranking of these styles.  

  • The critique of screen/content visibility was frequently cited by students and is an issue that can be addressed with appropriate pre-production planning.

  • Other comments from students that affected their scoring of each video including the quality of the video production, the use of gestures (distracting or helpful), and the speaking pace of the instructor.

​

Media

 

Teaching or learning problem/challenge that made you rethink your learning model or approach or try something new

  • In the face of mounting evidence revealing active learning approaches result in improved student learning outcomes compared to traditional passive lecturing, there is a growing need to change the way instructors teach large introductory science courses.

  • However, a large proportion of STEM faculty continues to use traditional instructor-centered lectures in their classrooms.

  • In an effort to create a low barrier approach for the implementation of active learning pedagogies in introductory science courses, flipped classroom modules for large enrollment general chemistry course sequence have been created.

 

How you addressed that challenge (if applicable, what technology, learning strategy, etc. you integrated to do so)

  • Student response systems called clickers (Example 1) and problem-based case studies (Example 2) have been used to increase student engagement, and how flipped classroom modules have integrated these case studies as collaborative group problem solving activities in 250–500 seat lecture halls.

 

How students responded, any challenges, and lessons learned

  • Students generally provide positive feedback about the flipped classroom modules, though some students periodically indicate they feel they would rather have the instructor teach them in lecture as opposed to doing some of the learning independently prior to lecture. It is important to inform the students why we are using this type of instructional scaffold and how it has been found to help students be more successful.

 

If you conducted research, was there any evidence-based impact?

  • Preliminary evaluation efforts found the flipped classroom modules provided convenient access to learning materials that increased the use of active learning in lecture and resulted in a significant improvement in the course grade point average (GPA) compared to a non-flipped class. These results suggest this approach to implementing a flipped classroom can act as a model for integrating active learning into large enrollment introductory chemistry courses that yields successful outcomes.

 

Media

​

​

​

​

​

Jack Eichler, Ph.D.

Associate Professor of Teaching, Department of Chemistry

​

Research Area

Undergraduate general chemistry and inorganic chemistry, chemistry education research, and research aimed at synthesizing metallotherapeutic cancer drugs.

​

Relevant Course(s) 

Undergraduate general chemistry (CHEM 001; 200-500 students) and inorganic chemistry (CHEM 150A; 15-80 students).

Teaching or learning problem/challenge that made you rethink your learning model or approach or try something new

  • Used clickers extensively, but wanted to add a little more “pep” and to change it up from time to time by using Kahoot!

 

How you addressed that challenge (if applicable, what technology, learning strategy, etc. you integrated to do so)

  • I used Kahoot! sessions about once every other week (6 total for the course), with each session comprised of six questions. We also used Kahoot! on the last day of class for a review session (20 questions). In addition to trying Kahoot! out, we wanted to determine if student performance on Kahoot! was predictive of their exam performance and student attitudes towards Kahoot!.

​

How students responded, any challenges, and lessons learned

  • Students emphatically enjoyed using Kahoot! (as evidenced by their reactions in class and their ratings in an end of course survey). Greater than 80% of students said that it was both fun and an effective way to learn and that we should play about once a week.

  • Some challenges were:

    • How long to set the question timer for (10 or 20 seconds). For a class of 100-150 students, 20 seconds is about right. For smaller classes I would probably use 10 seconds.

    • Image resolution. Sometimes the images are a bit fuzzy and/or small, so I suggest doing a practice run ahead of time to make sure images look OK.

    • Possible burn-out. One published article shows a burn-out effect from playing Kahoot! too often, so it should probably be played once a week or less (definitely not every day).

​

If you conducted research, was there any evidence-based impact?

  • Intro: Student response systems (often called “clickers”) are typically used in college science courses as a form of active learning to increase engagement and promote learning. Recent modifications to student response systems include “gamification” where game-like elements (such as competitions, earning points for speed, winning medals, etc) are added with the goal of further increasing participation, engagement, and excitement in the classroom. However, little is known about the relative impacts of traditional student response systems compared to those of gamified systems when both are used in the same course. The goal of this study was to compare student performance with and perceptions of a traditional student response system (iClicker) and a gamified version (Kahoot! that were both used in the same course.

  • Hypothesis: Our hypotheses were that student performance from iClicker would be more predictive of exam performance than Kahoot!, but that students would perceive Kahoot! as being more fun and engaging than iClicker.

  • Methods: Students from two sections of a large enrollment human anatomy course (n = 254) answered in-class iClicker questions for participation credit and voluntarily answered in-class Kahoot! questions for no credit. Students also completed end-of-course surveys rating their perceptions of both systems. Multiple linear regression models were used to analyze exam performance as a function of student performance on iClicker questions and Kahoot! questions (while controlling for student demographics).

  • Results: The models showed that both iClicker performance and Kahoot! performance were significantly positively correlated with exam performance with nearly identical effect sizes. Additionally, student survey data showed that over 80% of students rated both systems as being both fun and effective ways to learn. However, while >95% of students reported that while iClickers should be used in every day of class, approximately two-thirds of students reported that Kahoot! should only be played once a week.

  • Conclusion: These results suggest that student performance with traditional and gamified student response systems may be similarly correlated with exam performance but that gamified systems may need to be used more sparingly than traditional systems in order to maintain high levels of engagement.  

 

Media

​

​

Justin Shaffer, Ph.D.

Assistant Teaching Professor, Cell and Developmental Biology

​

Research Area

Assessing the efficacy of high structure teaching practices in a variety of college science classes.  High structure teaching involves pre-class assignments, in-class active learning, weekly review assignments, and multiple cumulative exams.  More specifically, he is studying the impacts of in-class active learning exercises and out-of-class assignments on student performance and attitudes in a variety of contexts.

​

Course(s): Bio Sci D170, Applied Human Anatomy, ~130 students

​

Teaching or learning problem/challenge that made you rethink your learning model or approach or try something new

  • At UCSB there are two levels of general chemistry regular (2000 students) and honors (50 students).  This results in the vast majority of students taking regular general chemistry. This is a required class for most science majors and is the first science class that students take as well as a prerequisite for most other science classes.  Students come into the class with a wides range of different chemistry backgrounds from no Chemistry in high school to two years of high school Chemistry (1 year of regular chemistry and 1 year of AP chemistry).  This wide discrepancy in backgrounds makes this class challenging to teach.

 

How you addressed that challenge (if applicable, what technology, learning strategy, etc. you integrated to do so)

  • In order to help students prepare to take Chem 1A as well as to try and avoid  busy work UCSB started assigning a pre Chem 1A assignment via ALEKS.  ALEKS is an adaptive learning homework system in which students are given adaptive knowledge check which determines what students know (~ 20 questions).  The knowledge checks are used to  generate an unique homework assignment for each student that only contains the material that students do not know.  In addition, after completing 25 topics, in homework mode, students are given an additional knowledge check to verify their learning is on track.  These knowledge checks can either remove topics from students homework because the students already knows the material or add topics to the homework because the students forgot the material.  The pre Chem 1A assignment has 130 topics (~77% math and ~33% chemistry) on average the first knowledge check shows that student know 59% of the topics.

​

How students responded, any challenges, and lessons learned

  • ALEKS is used the entire general chemistry series.  Most student respond positively to ALEKS especially the pre Chem 1A assignment. They feel that the assignment gives them the opportunity to review their chemistry before they are too busy with other classes. In addition it also allows student to realize that they should take advantage of university services such as Campus Learning Assistance Services (CLAS) early on in the quarter.

​

If you conducted research, was there any evidence-based impact?

  • Research on ALEKS at UCSB found thee findings:

  • Finding 1: For every 1% increase on masted topics on the initial assessment, the probability of a student succeeding* in Chem 1a increased by 0.5%

  • Finding 2: Completing the summer assignment increased the probability of a student succeeding in Chem 1A by ~30%

  • Finding 3: Students that score in the lowest quartile (when taking the initial assessment) who complete the summer assignment have an average grade 0.45 high than those that do not.

  • *Succeeding = Earning a grade of B- or better.

​

Media ​

​

​

Darby Feldwinn, Ph.D.

Lecturer with Security of Employment, Department of Chemistry & Biochemistry, University of California, Santa Barbara

​

Research Area

Darby co-founded SciTrek. SciTrek is a 2nd-8th grade outreach program that carries out long-term inquiry investigations with students. In addition it provides professional development for teachers to increase their comfort with leading inquiry based science in the classroom.  Darby is the program director for the elementary (2nd -5th grade) portion of SciTrek.  As program director she writes all of SciTrek’s curriculum, runs professional development workshops for teachers and volunteers, runs SciTrek modules in local classroom, and oversees the research being carried out on the program. 

​

Course(s)

Chem 1A (first of three quarters of chemistry for science majors).  At UCSB Chem 1A is taught in classes of 350 students.  Typically in the Fall we run 6 sections resulting in ~2000 students taking Chem 1A.  

​

Jack Eichler
Rachelle Crosbie-Watson
Justin Shaffer
Darby Feldwinn

Teaching or learning problem/challenge that made you rethink your learning model or approach or try something new

  • Designing an online class presented many new challenges that we hadn’t considered before. The asynchronous timing of our lectures was particularly challenging.

  • Our concern was that engaging and retaining the student’s attention would be difficult. To address this, we created shorter lectures with more illustrations to engage students’ attention. In addition, we had to design a style that was effective and interactive, because learner isolation was a big problem when creating online courses.

  • Additionally, we included annotations in the lecture material in order to give students an opportunity to assess their learning. By considering the student’s perspective while creating lecture materials, we were able to create an effective online course.

 

How you addressed that challenge (if applicable, what technology, learning strategy, etc. you integrated to do so)

  • We reduced lecture video lengths, provided clear visuals, and produced high quality recordings. And in addition, all course assignments focused on maximizing impact and emphasizing critical thinking.

  • Our lessons focused on applying the principles of our  course on science and research broadly. The hope was to inspire thinking outside of the box and reading between the lines more than rote memorization.

 

How students responded, any challenges, and lessons learned

  • Students responded overwhelmingly positively to the course. Many reported that they felt transformed and felt that their career goals were strengthened as a result of the course. Students were able to explore the human side of disease, which resonated strongly with their own perspectives of disability and inclusion.

 

If you conducted research, was there any evidence-based impact?

  • Yes, we found that there was a significant positive impact that the human side of disease had on motivating students and improving their connection to the material.

 

Media

​

​

​

Ronny Choe

Academic Coordinator; Head TA, Integrative Biology and Physiology, UCLA

​

Research Area

Neurobiology of learning and memory; Education

​

Relevant Course(s) 

Physiological Sciences 121 (PHYSCI 121; 180 students)

Choe
bottom of page