Dr. Hoy showed that thinking well is a universal skill that applies across disciplines, but added a perspective and structure that could lead to liberating students’ intelligence in our classes in line with Dr. Bowen’s arguments. First, he suggested that we must teach as if students matter. Second, we must ask students to be thoughtful rather than declarative by teaching them to stop and think first and then write about their ideas. To illustrate this, he asked the audience at his presentation to read a page and a half of text written by George Steiner about how to read text. First he asked each member of the audience to identify what s/he thought was the most meaningful sentence to him or her and underline it. He then asked several members of the audience to read the sentence they underlined. Then he asked each person to identify the sentence s/he believed was most important to the writer. Again, he asked several members of the audience to read that sentence aloud. Finally, he asked all to identify the sentence in the text that most accurately captured the meaning of the entire piece, and again asked some of the members to read the sentence aloud. No one read the same sentence twice in any of the three contexts.

Dr. Hoy made the point that we all filter text through our “ego eye,” our personal experiences and history leading to individual differences in interpretation, and that in the piece we read there was no single declarative sentence that explained its meaning. Nonetheless, there was a sense of meaning “hanging in the room” for all to be identified by being inferentially astute and capturing the sense of mystery floating up from the piece. The audience members (or students in a class) captured this sense of meaning and mystery about the reading in uniquely individual ways.

From a metacognitive point of view, Dr. Hoy suggested that creative and productive thinking follows an inductive process observed through different “eyes.” Most typically we observe facts through our “ego eye,” the perspective colored by our past experiences. The “ego eye” may not serve us well though, because it is likely to dismiss important facts and observations as irrelevant when they are not, as it is colored by our past experiences. Rather, he suggested, we should open our “slumbering eye” (call it anima, soul, imagination, subconscious perception or revelation) and dare to take risks by viewing evidence from this different perspective, thus increasing the probability of conceiving new ideas.

When observing with the “slumbering eye” conclusions can never be certain, because of the interplay of intuition, analysis, imagination, induction, and dialectical thinking. It allows us to converse with images and carry on subconscious sensing of something significant without knowing yet what that is. As the process continues and gains strength with additional information and thought, at some point ideas gain levels of consciousness that can be expressed in words.

Dr. Hoy starts this discovery process by asking students to deeply examine minute particulars (facts, evidence) and start writing about them in a first person story-telling format to have them thinking within and about this universe of details. Through personal story telling he encourages students to become careful readers and the careful reader, regardless of discipline or the type of text, engages in the complex business of dialoging with complex text, as opposed to cursory examinations often conducted by students. Careful readers understand that conceiving an idea is not a simple process, but an arduous energy-intensive personal effort dependent on commitment.

In the dialog with the text through our “slumbering eye” we conceive of fuzzy ideas that “float up” from the reading; emanate from between the lines. These fuzzy ideas then require translation and substantiation before we can produce a thoughtfully written product, whatever that product may be. The first step in learning the creative process therefore is for students to realize that they need to first collect evidence. The evidence that is gathered then must be carefully observed, followed by inductive reasoning and inferences from which ill-formed, fuzzy ideas arise that consolidate and lead to a thoughtful product.

This process, while admittedly risky because it may be outside accepted norm and the mainstream, can take us in new directions as it liberates the creative mind. Dr. Hoy suggested allowing students to tell stories about the “evidence” they read, saw, or heard, because story telling is what humans are born to do. He suggested asking students to write a personal story about a paper they read rather than a declarative summary of the facts, because it may help them open their “slumbering eye.” There are other options such as letting them write a letter to the author, the smartest person they know or even a fellow student to explain their thoughts about the reading. This exercise allows the student to express even fuzzy ideas they are not certain about, but writing in this context motivates them to move to clarity of thought. This can be done in any discipline on any topic. The story or letter then becomes the source of discussions with peers in the classroom that can lead to further insights and clarity.

Dr. Hoy also illustrated the process using a short film clip. The images shown in the film were treated as the “facts” or “evidence.” By asking the audience members what details they observed in the film he made them become more aware. Having generated a list of factual observations they could point to, allowed them to create an inventory of events they had seen. This inventory enabled them to move to inferences and create ideas and concepts not directly observable in the film. A prompt to help students do the same with a video clip could be “I observed that….., therefore …..”.

This list of observations (facts) also served to reduce the impact of the “ego eye” to some extent, because objective observations are not filtered through previous experiences. They are seen by all observers and can be verified with little disagreement. This objective observation exercise prevents the “ego eye” from relegating irrelevant everything that does not confirm previous experiences, and closing the mind to seeing new perspectives with the “slumbering eye.” According to Dr. Hoy, the greatest gift teachers can give students is helping them see with their “slumbering eye,” because that is how innovative and creative ideas arise. It works by liberating language not formerly known and seeing landscapes of the mind not formerly experienced. Allowing students to proceed with this exploration is challenging, because the process may lead them to places others may not agree with or perceive as conceptual mine fields not in keeping with the norms of the discipline. If students are encouraged to explore these uncharted paths, they need to be given the freedom to do so within the boundaries of the course but without stifling innovation and creativity.

Dr. Hoy commented that these various exercises (e.g. watching images without teacher commentary, writing letters to smartest person, telling a story about observations) during the early stages of the process provide the freedom students need to observe with their “slumbering eye” and unleash their innate creativity of thought. Students need to learn how to think creatively and not simply write declarative statements that often are copies of what already has been written. Rather than having students “complete the forms” that exist in every discipline and stifle innovative thinking, allow them to spend time in this creative space. He reemphasized that we cannot know if students can write clearly unless we allow them to write stories they care about in a personal narrative about passages they read, images they saw, reports they heard. The selected source material ensures they write within the constraints of the course.

Dr. Hoy emphasized that “nothing comes from nothing.” Asking students to regurgitate existing information does not lead to creativity and innovation; habitual thinking blocks the development of the mind. To prevent habitual or reproductive thinking, students must write in the first person using “familiar essays” that are close to their heart. In these essays they should be allowed to use relevant experiences of any kind in a self-organized sequence that is meaningful to them and provides evidence for new ideas. The process of conception serves to make sense of an idea. This process is unique to each and every student. When teachers seek conformity with personal perceptions they will receive the same product from all students and little creative and innovative thinking. Consequently, teachers’ filters, their “ego eyes,” may constrain students’ creative capabilities and innovative thoughts. Asking students to write about “fuzzy ideas” within the constraints of the class will help them gain clarity and allows to teacher to determine whether they thoroughly understand important concepts.

Students’ thought processes are guided by the facts observed and because of that do occur within the context of the assignment before them, but the aim is to help them realize that they can think independently rather than be limited to regurgitate existing material. Unleashing students’ creativity is not for creativity’s sake: it serves to expand the mind. Dr. Hoy emphasized that this is by no means an easy process and it requires massive work. Examples of previous writings produced by students can be very helpful in illustrating final outcomes that need to be achieved in a particular class. Thoughtfully creating a collection of examples that represent work by students at the beginning of the semester and at the end helps design the progressions and exercises students need to complete to successfully navigate the challenges associated with creative thinking and writing clearly. Simply requiring that students write a paper on a topic will not help them learn to think and write clearly in any discipline. Providing examples of work by other students may help them realize that their work must achieve higher levels of thought, creativity, and innovation than is commonly expected.

The pedagogical strategy to help student reach higher can be relatively simple:

1. Students engage with several sources of information such as papers or sections of relevant text, observed images or film, podcast or video segments, lab demonstrations, etc. This is the pertinent evidence (content) selected by the teacher and the fundamental elements that set the context for the exercise.
2. They write a personal story about each objective element the read, saw or listened to, drawing out the ideas that form in their minds. Again, each piece of evidence should be carefully selected to draw out students’ thoughts through their writing.
3. Then they combine their writings into a synthesized version of the ideas that flowed forth from the various sources they examined and wrote about.
4. Following the individual process, time must be allocated in class so students can read each other’s work and do something with it. For example, the readers could be asked to identify in the writing:
   1. What they believe is the most important sentence in the writing.
   2. What they believe the author thinks is the most important sentence.
   3. Which sentence best captures the essential meaning of the writing.
5. Finally, they engage in discussions with their peers and the teacher about their ideas. Great things can happen at this stage because of the serious preparatory work.

These are thinking exercises that serve to make the fuzzy ideas that emanate from the evidence more concrete. The teacher can guide the process by carefully selecting and ordering the presentation of the evidence. As such, all the evidence constitutes an intellectual inductive puzzle that each student is allowed to solve through their own eyes within the context of the course and the constraints of the assignment.

Finally, students should be asked to collect all their work in a learning portfolio that documents their process and the products of their work. The learning portfolio becomes a vehicle for metacognitive reflection that allows students to analyze their learning process and gain greater insight into the workings of their mind. The learning portfolio will also help the teacher examine the learning process to gain a better understanding of how students develop their thinking. Analyses of the learning portfolio can then be used to revise the course.

Harry Meeuwsen

Director, Center for Excellence in Teaching and Learning

Dr. Jose Bowen, in his dynamic keynote presentation, argued that residential education (going to school on an actual campus) is under threat from numerous forces ranging from private online educational institutions, state budget cuts, and the cost conscious consumer. He argued that if residential higher education institutions want to justify their higher costs, they need to ensure that the education students receive in its brick and mortar facilities is worth their money.

Students still seem to prefer coming to a campus for the rich experiences and to take classes from professors in person, but that does not mean that they will continue to do so if the residential institutions don’t deliver a high quality product. According to Dr. Bowen, the true benefit residential campuses offer is the person-to-person interactions between faculty members, the experts in their disciplines, and the students. To keep these interactions of the highest possible value, every minute in class needs to be carefully planned and fully exploited through teaching strategies that are difficult if not impossible to deliver online.

To make every minute valuable, Dr. Bowen suggested that one-way lectures using PowerPoint presentations to disseminate information should be eliminated as much as possible, especially when that information is likely freely available on the internet (for a quick sample of what is available for free, take a look at the online Kahn Academy; http://www.khanacademy.org/). He argued that podcasts such as those at Kahn Academy have great benefits over using classroom lectures to distributed information and explain concepts. Students can stop and repeat parts any time if they missed something and archive the podcast for later viewing whenever necessary! He suggested we should leverage that capability.

An appropriate podcast or other internet-based resource can be posted so students can examine it prior to class. They may then be asked to complete a task based on what they learned from the online resource such as a brief pre-class assessment quiz on the content of the podcast, solving a problem related to the concepts, etc. If the assignment is due one hour before class, the teacher can use if for “just in time teaching.” The results of the assignment can be used to provide feedback in class and correct misunderstandings, or to decide whether students need more practice exercises in class.

Of course one could chose to create custom podcasts, but Dr. Bowen recommended spending some time examining internet material before going through the effort of creating custom podcasts. Even if existing material is not 100% perfect, students may still be able to learn what they need from it. Dr. Bowen suggested that if acceptable presentations or information can’t be found on the internet, and faculty members believe their explanation of the material is critical to help students learn better, creating a lecture podcast would be better than lecturing in class. Class time can then be used for more complex problems, deeper discussions, and debates over the content from the podcasts. Information is cheap, easy to come by, and merely a vehicle to help students achieve higher order thinking skills such as the capability to apply, analyze, create, and evaluate (see Bloom’s taxonomy). These are the type of activities they should conduct during the class sessions, while necessary background information is obtained outside the session.

Students should behave similarly in class as researchers behave in the lab. Researchers use information as a resource to develop new hypotheses that will be tested. That capability, to use information effectively to develop and test hypotheses, however simple, is what all students should learn in class. The capability to apply, analyze, synthesize, and evaluate does not develop unless students are asked to actively engage with the material, think about it, manipulate it, elaborate on it, distinguish it from other information, discuss it with others and transfer it to new situations and problems. In other words, they do something substantive with the information.

To make this possible, students need to read and study required materials in advance of the class session and be prepared to engage in demanding problem solving exercises, discussions, debates, team work, reviews, etc, that demand they put their best thinking forward. Careful, integrated course design and progressive disclosure of complexity through scaffolding of exercises will help even novice students learn how to apply, analyze, synthesize, and evaluate information in any discipline. Faculty members with their expertise can provide students value beyond online courses by superimposing their insights, wisdom, intelligence, and problem solving skills on live interaction and training in the classroom.

The direct impact of faculty members’ expertise lies in the real-time formulation of meaningful questions and explanations, engaging discussions, and the modification of problems sets that help students explore new concepts and gain understanding quicker. Consequently, designing purposeful practice opportunities and appropriate challenges and exercises that provide students with authentic exploratory experiences in the discipline during class sessions is the cornerstone of residential educational value. Fink’s (2003) integrated course design strategy provides an excellent model to develop courses that are clear, coherent and efficiently focused on optimal learning; following his model ensures that there are no disconnects between learning goals, assessments, and learning activities that could be disruptive to the learning process and thus unproductive.

A challenging, well-integrated course creates practice time in the classroom focused on achieving performance goals and high quality outcomes. The class sessions are structured like practice sessions for any activity, whether it is football, chess, learning to play the piano, or painting. Students come to a “training session” and should leave class exhausted, because they worked so hard. Developing mental fitness and strength requires well designed learning experiences, extensive practice and repetition, feedback, guidance, collaboration with peers, discussions with faculty members, encouragement and reward. Class sessions need to provide students with significant intellectual challenges and carefully designed learning activities that take their knowledge and skill level to new heights. That growth is exciting to watch and makes teaching and learning rewarding.

To help optimize time spend in class and achieve excellence in teaching, Ambrose, et al (2011) published a book entitled “How learning Works: 7 Research-based principles for smart teaching.” In each of 7 chapters the authors present challenging faculty-student interactions related to teaching and learning that are common occurrences. In each chapter they delve into the research literature related to each issue, and present suggestions on how teachers might prevent or address these problems.

While I strayed a bit from Dr. Bowen’s focus, my thanks to him for providing a new perspective, some concrete examples, and further food for thought about teaching and learning.

Harry Meeuwsen

Director, Center for Excellence in Teaching and Learning

To the learning objective purists, multiple-choice exams amount to heresy. To the rest of us, however, they can serve a very useful purpose, if they are done correctly.  In a recent CETaL Fellows’ meeting, I brought up the issue of using multiple choice exams and providing appropriate feedback to students. Is a raw score on an exam appropriate feedback for our students? Whether you agree or disagree with this type of feedback the issue remains that the majority of instructors choose to simply provide a number as the only feedback to their students. This blog entry is a summary of our discussion and some of my own thoughts about this particular issue.

If we were to discuss controversial topics in the area of learning assessment, the use of objective-type questions (a.k.a., multiple-choice-question exams) would be at the top of the list. A simple search on Google Scholar using the terms “multiple choice exams” and “learning” results in almost 54,000 hits with papers, discussions, syllabi, and other materials. A cursory examination of the titles and abstracts suggests that the topic is one that has stirred enough debate over the years that scholars on both sides of the issue have felt strongly enough to conduct all sorts of studies and report their findings in academic and non-academic circles. Yet, the issue still remains: are objective-type exams useful in assessing learning and how can the students benefit from them?

In this blog entry, I will argue that if properly designed and executed, objective exams serve an important purpose in our pursuit to assess learning and provide useful feedback to both our students and accountability partners. First, let’s tackle the issue of developing appropriate assessment tools. Needless to say, these tools should be designed with a purpose in mind. Normally, this purpose is spelled out in the learning objectives that we set out to accomplish in our courses. Using Bloom’s taxonomy of educational objectives, these may vary from assessing basic knowledge (the student’s ability to memorize and recall facts), comprehension (the student’s ability to interpret and extrapolate facts and ideas), application (the student’s ability to take what the know and apply it in various contexts), analysis (the students’ ability to break down new information and assess its potential uses, advantages, disadvantages, etc.), synthesis (the student’s ability to put together various pieces of information to form an opinion about a new situation), and evaluation (the student’s ability to assess the worth of others’ work and ideas).

Individual exams do not necessarily need to meet all types of learning objectives as suggested by Bloom. However, each question on a multiple choice exam must address one or more of these objectives or, better yet, the objectives that we outlined in our syllabi.  In other words, questions included in an objective-type exam must assess whether students are indeed meeting any and all objectives that we set out to examine in a particular exam or course. One other important issue here has to do with classroom activities; what we do in the classroom (in this case, I consider a classroom to be the context within which course activities occur, not a room in a building), must be designed in such a way to help students meet the overall learning objectives. So, how do we design multiple choice exams that meet our course learning objectives?

Viewing an exam as a measurement instrument allows us to develop an exam that is both valid and reliable. Validity in this case would imply that we are measuring what we are supposed to be measuring. That is, the learning objectives set forth in our syllabus for the particular portion of the course that the exam covers. Reliability, on the other hand, can be assessed in a number of ways such as multiple questions addressing the same learning objective as opposed to just one question.  According to some CETaL Fellows, designing a valid and reliable multiple choice exam takes a lot of effort.  Simply using the test bank provided by the textbook publisher may substantially reduce the effort, but it does not mean that the resulting exam is valid and reliable because of its origin. Therefore, if one chooses to use the test bank, it is best to treat the supplied questions as a starting point for creating your own questions. The same care we devote to research instruments needs to be devoted to developing appropriate questions and exams for our courses. A good practice would be to develop a table, which shows all learning objectives under scrutiny and the corresponding questions that address each of those objectives.  Consequently, by incorporating student performance on the various sets of questions (objectives) we can readily surmise whether our students are meeting those objectives or not. These data can also serve as evidence of learning for our accountability partners.  Needless to say designing good questions requires skill and experience that comes with practice and learning by following good examples. Another good practice is to look at the questions that a lot of students answer incorrectly to see if indeed the students just simply do not know the right answer or the question is just bad. Asking students to provide their input about individual questions is not a sign of weakness on the instructor’s part, but an indication of the instructor’s quest to improve the exam.

Perhaps, the most important issue with regard to multiple choice exams does not have to do with the questions themselves but the feedback provided to the students. Providing a numerical score or percentage means very little to the instructor and to the students learning-wise. It shifts the focus from learning and accountability to grading. In turn, that may actually explain some of the research purporting a significant correlation between grades and student evaluations of teaching. In other words, we motivate our students to evaluate us on the wrong parameters. So, the student evaluation instruments might be reliable (I consistently receive high evaluations), but they are not valid (they are measuring whether my students like the grades I give them as opposed to what they learn in my class). Then, what is the best way to provide feedback to our students on multiple choice exams?

We can start the feedback process by sharing the learning objective information with our students. We can keep it simple: I included these sets of questions to determine whether you will be able to apply the material (be specific) that we examined in class. If you have not performed well (define what that means), then you must improve your performance by following these strategies (provide these). In small class sections, it is possible to provide individual feedback to students. In larger sections, the feedback needs to be provided as a summary, normally posted on Blackboard. In addition to this, it is often very useful to identify the questions that most students missed (for this, look at the summary report provided by the Scantron machine), revisit the material or issues associated with this material and ask the students to take a follow-up quiz.  Again, the point is not the improvement in grades, but for the students to learn. Repetition, follow-up, and follow through encourage learning. Simple scores do nothing but lead to aggravation (or temporary satisfaction for those who received a good score), but do nothing for learning purposes.

As we think about assessment, in general, it is always a good practice to start with a plan by outlining the learning objectives that we intend to address in a particular exam. Then, we design exam items or questions that would capture the students’ performance on each of the objectives. Importantly, the learning process does not end after the students complete the exam. Some would argue that returning the exam to students and discussing the exam in class is, perhaps, one of the most important parts of the assessment process. We must always close the loop.

Dr. John Hadjimarcou

Marketing and Management

The University of Texas at El Paso

We saw a much deeper engagement with our assignments, and skills that were easily transferable to the other disciplines our students would eventually enter.

If you would like to take this step from the traditional to the technological, and enhance your course assignments with a multi-modal/multi-discursive approach, there are a few steps we would recommend for the process:

Step 1: Student Release Form

It is important to have students sign a waiver allowing you to publish or use their work after it’s completed. Student samples will be invaluable to you for the next time this is taught, or if you end up publishing the final product in some way.

Step 2: Decide on the purpose and learning goals you want to accomplish.

Perhaps it is:

A. To inform on a topic/subject/concept relevant to the class
B. To persuade/advocate for an action, change, and/or social purpose
C. To apply research in a meaningful way
D. To learn a new technology/communication skill
E. To demonstrate understanding of a process, task, problem (for Mathematics, for
example, students can teach or show how to solve a specific problem or concept)
F. To reflect on what has been learned in a semester.

There can obviously be overlap with these, but you want to determine a primary learning goal—it will help you in the construction of a rubric later.

Step 3: Decide whether or not you want it to be collaborative.

These types of projects and assignments don’t have to be done in groups—I’ve seen terrific products created by one person and by groups. I’ve also seen not-so-good products be done by one person and by groups. Doesn’t really matter—its all about the level of commitment and ownership of the project that they have.

However, if you decide to do this collaboratively, I would recommend no more than 3 per group. Also, teach clear parameters about collaboration (conflict-resolution activities) and working in groups (group management, scheduling). And always make it clear that a student can be “fired” from a group (have a clear process for that too).

Step 4: Create assignment sheet with clear objectives, skills and learning goals
identified and how they relate to the course.

Students need to be able to see that you are doing to this to further class learning objectives and not for novelty. Content first—technology second.

Step 5: Create smaller, scaffolding activities and assignments that will lead to
the overall completion of the product.

Can be in-class small group work, research, technology training—all meant to complete a specific component of the project. This emphasizes that this is a process, and will encourage time-management and make this all less overwhelming.

Step 6: Create a rubric that assesses these objectives and learning goals

You will see that no rubric is perfect, and will change as you work through the assessment process over time, but it will help re-emphasize the goals of the assignment and help you to measure/assess the content. I recommend giving the rubric to students at the start of the assignment—so they know how they are going to be assessed, and that this isn’t a subjective grading process. I would also recommend going over it mid-process, to re-emphasize the goals again.

Step 7: Set up support network/resources for you and students

A. Students will need to have help with the content, so you are the best resource for
this.

B. Some available software for multimedia projects:
IMovie and GarageBand on MACs

MovieMaker and Photostory

IMovie and MovieMaker will allow students to incorporate video that they’ve taken, or downloaded from YouTube. The parameters for use should be no more than 15-20 seconds at a time. You want them to be selective about what they use, and to take the role of support to maintain the integrity of their voice in the project and to prevent overuse.

C. Students will need to have access to technology and technology support. You can set up trainings in ATLAS for the software, and you can also reserve lab time in the evenings to allow your community of learners to work in their groups, but also with the rest of their classmates.

D. The Student Technology Center in the Library is now offering workshops on technology, including the software mentioned here.

E. I recommend use of the UGLC building and the labs over the library and LACIT. The technical support is better, and they are very well aware of projects like this.

F. The University Writing Center can help students with scripts and organizing the content of their communication.

G. You are also more than welcome to contact me (stvarela@utep.edu) for technology questions, assignment and rubric questions—anything to do with this type of pedagogical approach and assignment.

Step 8: Documentation and Fair Use

Any outside research they refer to or directly integrate should be cited in a credits sequence at the end. A Fair Use Statement is also recommended to cover the use of things like Google Images or YouTube.

Step 9: Value and Accountability

Need to show students this is a worthwhile learning experience, and that you value the work that will come out of it. You might think about:

A. Class presentation-where you watch their final products in class, have them introduce it and answer questions from the audience afterwards. Can be part of the final grade.

B. Community collaboration/application-you may have already set up a community connection for your class (Center for Civic Engagement/Service Learning, non-profit organizations, community partners) or simply personally have connection to a group/organization that this project can be connected with.

C. A showcase of some sort (public viewing, create a You-Tube Channel/Vimeo site, a website, a place on Blackboard).

The point is that this is such a visual result and a commitment on behalf of yourself and the students, so this needs to be honored in some way besides a grade.

Publication of these projects sometimes creates more value and accountability than a grade ever could!
In First-Year Composition, we created the FYC Film Festival and a website to showcase this work. In addition, Instructional Support Services created a virtual theater to show our student’s projects in Second Life.

Check out our student’s work @ http://filmfestival.uglc.utep.edu

Step 10: Student Release Form—just in case you forgot!

Some final advice: the more engaged and excited about this approach that YOU are, the more your students will be!

Mr. Steve Varela

Department of English/CETaL Fellow

The University of Texas at El Paso

The Sun Conference theme “From black board to Blackboard and beyond,” captures the paradigm shift in teaching and learning due to the disruptive technologies that are changing the way in which we interact with content, each other, and the students. Teaching tools have moved from the black board and teachers’ presentation skills, to include a great variety of stationary and mobile electronic devices that are changing the way we teach and learn. We find ourselves in a maelstrom of gadgets and apps, without clear ideas what really enriches teaching and learning. The conference aims to shed some new light on these issues. Join us and our featured speakers for an exploration of new frontiers beyond our black boards.