How am I feeling?

I haven’t been blogging near as much as I had hoped through this, my latest foray into online learning.  This week I have been tasked with sending an email to my instructor to relay how I am feeling about the course thus far and, as such, I now have some extrinsic motivation to actually take some time, think about what I am doing and feeling and put it down into words.

The fact is, I’m not feeling too good about my learning in this course, Evaluation of Learning.  Sure, the content is interesting, but I’m struggling to stay engaged.  The facilitation of this online course has been good.  Certainly, I have taken other online courses where I felt that the instructor was mailing it in, so to speak, half-heartedly monitoring discussion and providing feedback, in between ordering more mai-tais at the tiki-bar.  This is not the case here. The facilitation has been timely and balanced, not too leading but encouraging enough.  And, as I said in a previous post, the feedback on my initial assignment was thoughtful and helpful.  So, what is the reason for my general malaise?

Well, Fenwick and Parsons, as well as a host of other adult ed scholars, talk about grounding adult education in the lived experience of the learner.  Adult learners need to understand concepts in the real world.  They need to draw parallels with their own experience; they need to touch, see, hear, and feel the context.  They need to try on learning like a new pair of shoes or glasses.  Put them on, strut around a bit, wear them to work and play and see how they feel; how do they integrate into and change how they experience the world in which they live.  As I’ve tried on the learning in this course, it hasn’t integrated well into my world.  It hasn’t changed my experience that much. 

Part of the blame lies in the course design; I am increasingly convinced that it was designed for the adult educator working in a post-secondary-type institution.  That ain’t me.  Even the course text, “a resource for educators and trainers” lives up to it’s billing.  It’s a resource, no different than that handy little cordless screwdriver that I pack with me on any fix-it project that I have to do around the house.  Heck, it even has toolboxes in the back.  It’s not really a challenging read.  It doesn’t present theories or ideas that piss me off or make me cheer.  It’s just…there. 

But another part of the blame lies with me.  I’m at a bit of a crossroads in my work right now.  I’m not directly involved in adult ed, as I’ve said in past posts.  As an analyst, there’s about three degrees of separation between me and evaluating learning.  At best, I’m evaluating results that may be attributed to learning in a round-about way.  There is some possibility that in the near future, evaluating learning will take on more importance due to significant technological change within my organization, but it’s not entirely certain that I will be involved in the teaching or the evaluating.  But I’m also at a crossroads in my career path.  I’m not finding much joy in being an analyst and, quite frankly, much of my scholastic work hasn’t really been grounded in what I do.  And if I am really being truthful here, what I aspire to do isn’t really clear so I’m not finding much joy in the possibility of a future career direction either.  It’s difficult to ground what I learn in the meaning of the work that I do when that work doesn’t provide a lot of meaning for me.  And it’s even more difficult to ground my learning in some future professional vision that’s at best fuzzy and at worse so far beyond the horizon that it really has no definition at all.  Kind of like trying to build the foundation of a house in the clouds. 

So it is with some trepidation that I approached the assignments in this course.  How can I examine a particular issue in evaluating learning when I don’t really evaluate learning as part of my professional life?  I can’t even start from the premise of, “When I graduate and get a job teaching, I think that this issue will be important to me….”.  I’m not so sure that I want to teach, in the traditional sense of the word.  But I do know that learning is something that I enjoy.  And teaching in the broader sense is something that I enjoy and believe will be an essential component of any work that I do in the future.  Whether it pays for the food on the table or not, teaching nourishes me, as does learning.  So it is from this premise that I approach the assignments in this course.

Feedback from my first assignment - refining my research question

I received my first assignment back from my instructor.  For those of you who are interested in such things as marks, I’ll tell you that I did well and that’s as much as I’ll say.  Because, really, that shouldn’t be the point.  Okay, I admit it.  It’s the first thing I look at when I get the rubric back.  Anyway, what is really important is the feedback that I received from my instructor.  You’ll recall from a previous post that this assignment was to be a sort of staging exercise where I was tasked to devise a research question that would be the focus for the following two assignments that would, in turn, constitute the evaluation of my performance in the course (combined with a mark for my participation).  So, my instructor’s feedback and guidance was of particular importance as it would be helpful to me throughout the rest of my assigned work.  And her feedback did not disappoint.  I have taken one other course with this instructor, a face-to-face core course, and her most recent feedback was concise, to the point and thought provoking – just as I remembered from my previous experience with her and hoped it would be in this latest course.  “The question is a good start”, she wrote, “but I am wondering if it is too broad as it addresses both the construction of knowledge and the development of a learning community.  If the question is too broad, it will take more time to research than what the course allows.  Can you revise the question to be more specific?”

I gave some considerable thought to her feedback and this is where I arrived.  What I am really interested in is the assessment of a learner's contribution to the learning community in an online environment and the challenges that this represents without the nuance that is present in a face-to-face classroom (e.g. tone of voice, body language, etc.).  The construction of knowledge seems to be easier to assess in the online classroom because there is a written transcript of the discussion.  In fact, some of my early research suggests that there are electronic tools that have been and continue to be developed to measure the level to which individual learners contribute to knowledge construction.  My thinking then, is that the contribution to the development of a learning community (what some have referred to as a social presence) online is really that much more difficult because of the medium.  The medium actually focuses all of the discussion toward knowledge construction at the detriment to the building of a social presence for the learning group.  I think that my research will tell me that half of the battle is to set up the asynchronous discussion in such a way as to allow time for the discussion to be solely about developing a learning community and I think that this will be interesting as well as useful to me professionally.  So, in an effort to provide more focus to my research here is the revised question that I've come up with:

How can we evaluate online asynchronous discussions to effectively assess a learner's contribution to the development of a learning community? 

My first assignment: Evaluating Online Asynchronous Discussion

Since the introduction of the Internet, all manner of organizations and institutions have sought to exploit it as a means to deliver education, training and other learning events. Online learning is fast becoming a dominate way that learners engage with course content, teachers and each other.  Fenwick and Parsons (2009) describe the phenomenon this way: “The move to online delivery is not so much away from traditional higher education but towards using new technologies” (p. 173).  One such technology is computer mediated communication within a virtual classroom, representing a shift away from the traditional bricks-and-mortar environment.  It also represents a significant shift away from the traditional means of constructing knowledge through face-to-face social interaction and discussion, mediated solely by teachers and learners.  This paper will begin to explore how social interaction occurs within the virtual classroom and more specifically, the unique issues that asynchronous discussion presents to the teacher and learner with regards to evaluation.  I will relate my experience, professionally and scholastically, with online discussion, explaining my interest in the subject and the impetus for further research.  Lastly, this paper will conclude with a research question that will serve as a signpost, guiding my upcoming literature review of evaluation of online asynchronous discussion.

The Issue

Online interaction and discussion can be synchronous, meaning it can occur all at once where learners ‘meet’ online at a specified time such as in a chat room, or it can be asynchronous, occurring over time, where learners check into an online discussion forum, posting their own comments and reading and responding to others whenever they see fit.  In the short history of computer mediated learning, asynchronous discussion forums have proven to be the method of choice for learner interaction.  The reasons for this are rooted in the benefits that it provides to the learner and the teacher.  Learners find an asynchronous approach attractive because of the flexibility that it provides; they can interact with the ‘class’ when they want and are better able to balance their participation with other commitments such as family and work.   Teachers find benefit in the evaluation of asynchronous discussion; there is a script of the discussion, in the form of threads that can be reviewed and analyzed.  Learners and teachers both find benefit in the purposeful structure that asynchronous discussion can provide to the construction of knowledge; discussion posts reflect significant thought, often bringing in further resources (e.g. videos, hyperlinks to articles, podcasts) and responses are typically coherent with and build upon themes introduced in earlier posts.

However, asynchronous discussion also presents challenges to learners and teachers.  First, it assumes a level of technical literacy and capacity.  Teachers and learners need to be able to navigate the learning management system and utilize its tools effectively and efficiently.  Second, it assumes that learners can express themselves as well via the written word as they can verbally.  Lastly, non-verbal cues such as body language and tone of voice are missing in the online classroom.  The lack of non-verbal cues combined with the purposeful and arguably clinical construction of knowledge contributes to a lack of dynamism in the asynchronous discussion.  Teachers become moderators rather than facilitators and learners become collaborators rather than peers.  This potential lack of social presence (Aragon, 2003) makes it difficult to wholly evaluate the effectiveness of asynchronous discussion in terms of learner development.  Consequently, it is difficult to wholly assess a learner’s contribution to the learning community via the asynchronous discussion forum. 

Personal Significance of the Issue

As an adult learner, I returned to post-secondary education later in life.  I began working towards a degree in adult education by attending face-to-face classes with a small cohort of classmates.  Over a period of two and a half years, I attended class once each week with this same group of learners.  Much of the learning that occurred took place in discussions that occurred both within and outside of the classrooms.  The discussions weren’t always purposeful, guided or facilitated; they were often tangential, off-topic, and quite often purely social.  When I returned to my studies some years later, I elected to pursue my remaining course requirements via online learning.  Many of my co-learners in these new classrooms were new to me but not to each other.  While they appeared faceless to me, they didn’t appear faceless to each other.  I have found the virtual classroom significantly more isolating in comparison to the face-to-face classrooms that were my experience earlier in my scholastic pursuits.  I have contributed much of this feeling to the nature of the online environment and, in particular, asynchronous discussion.  That said, I have been curious of level of cohesiveness that I have witnessed between some of my co-learners.  And, as a student of adult education, I have wondered how this cohesiveness can be fostered, developed and evaluated in the online environment. 

Professionally, I have worked in the social services for nearly two decades.  Much of the work that I have done, helping people to understand, cope with, and facilitate change, I have found akin to teaching and learning.  In my present role, I am greatly involved in ushering in some significant changes to how we provide service to those in need.  The changes are sweeping, the organization is quite large, and the supports to facilitate the learning are minimal.  As such, we are looking at new ways to deliver training and online, asynchronous discussion is a method that we are considering. 

It is from these two perspectives, as a student and as a professional, that the issue of evaluating online asynchronous discussion piqued my curiosity. And, it is from these two perspectives that I will conduct my research.

The Research Question

Nisbet (2004) argues that “although [the asynchronous discussion group] is only one type of e-encounter, it is arguably one of the most important” and “as part of a wider project examining facilitation of online discussion group interaction, the dilemma of how to measure group interaction” is significant (p. 122).  Further, Hughes, Ventura and Dando (2007) assert that “with no means of assessing the emotional state or responses from students there is a lack of means by which to develop online facilitation effectiveness” (p. 17).  So, while discussion is one of the most significant ways to construct knowledge and develop a learning community, it is difficult to evaluate the learner in both of these paradigms within the online asynchronous discussion.  Consequently, the research question that will guide my upcoming literature review is as follows:  How can we evaluate online asynchronous discussions to effectively assess the construction of knowledge and the development of a learning community? 

Conclusion

Increasingly, we are living in an asynchronous world.  Communication is text-based, presented so as to be read, considered, and responded to at some point in time in the future.  In a fast-paced world that demands flexibility and just-in-time solutions, it appears to make economic and logistic sense.  In the realm of online learning, this type of communication is championed by scholars for its effect on eliminating bias in evaluation.  But an opposing perspective sees discussion devoid of social presence, and as a consequence, evaluation that is less than whole.  Reconciling these two perspectives is the challenge facing the evaluation of online asynchronous discussion and will be the focus of my future research. 

References

Fenwick, T., & Parsons, J. (2009). The art of evaluation: A resource for educators and trainers. Toronto, ON: Thompson Educational Publishing, Inc.

Aragon, S. R. (2003). Creating social presence in online environments. New Directions for Adult and Continuing Education, (100). 57-68.

Nisbet, D. (2004). Measuring the quantity and quality of online discussion group interaction.. Vol. 1. 122-139.

Hughes, M., Ventura, S. & Dando, M. (2007). Assessing social presence in online discussion groups: A replication study. Innovations in Education and Teaching International. Vol. 44 (1). 17-29. 

WWPS (What would Paulo say)?

This past week, we discussed developing criteria for evaluation – those standards by which we measure achievement of learning.  We talked about criterion-, normative- and self-referenced standards of achievement.  And we talked about indicators, too.  Those specific behaviours that tell you learners are meeting the standards of achievement.  Blah, blah, blah. Booooooring.  I know.  I’m yawning as well.  Your standard goals and objectives stuff.  But as it happens, the subject matter this week was fortuitous, for a couple of reasons.  First, I had to hand in my first assignment on Saturday, so if there was a week where I could half-ass it (in terms of my attention to the session content), this was it.  Second, the issue of measurement became a focal point of my research while completing my assignment, and, through that research, I was able to view the session content under a bit of a refreshed lens. 

Here’s what we’ve been tasked to do: describe an issue/problem from our own experience in evaluating adult learning either as a facilitator or student and explain the reason the issue is important.  Not a big deal, but here was the rub.  This assignment would form the basis for the other two assignments in the course: a literature review and a lesson plan to teach our peers about the issue and our research findings. 

So, what issue/problem did I select?  Well, since I’m not currently teaching professionally, I came at it from a learner’s perspective and chose the problem of assessing online asynchronous discussion.  This idea has been bouncing around in my brain for a while now since I started down this road of online learning. 

Prior to starting online studies, with the exception of a brief traditional distance ed course, my experience with adult ed was entirely face-to-face.  In fact, since returning to school in my mid-twenties, the classes that I took were exclusively small and comprised of tight-knit groups that stayed together over a long period of time over many different courses.  This experience has influenced my belief that knowledge is most often created through critically reflective discussion and debate including all of the nuances (read non-verbal) that such discussion entails.  So, when my participation in such discussions were evaluated in the past, it was my assumption and expectation that those nuances were taken into account as well as all of those intangibles that are the hallmarks of good group work.  That belief was further cemented when I read Paulo Freire’s Liberation Pedagogy model.

Put simply, learning happens when we are transformed from a state of being unaware (magical conciousness) to being critically aware (critical conciousness).  When we become aware of a problem, are able to define and discuss it with a common language and act on solutions, then we have liberated our minds, bodies and spirits.  We have truly learned.  And the linchpin in all of this is that group of people and their interactions.  Not just what we say together but how we say it and it what context.  All of those nuanced interactions that take place in and out of the classroom, sometimes heated, sometimes ugly, sometimes light, sometimes hilarious, but always thought-provoking, always transformative.  So, it is these moments that have shaped my evaluation of good discussion.  And, it is from this perspective that I find the evaluation of online, asynchronous discussion troubling and worth more research.   

My experience with online discussion has left me somewhat wanting.  While it is lauded for its flexibility and its ability to aid in the construction of knowledge, I have found it to be stilted, stop-and-go, somewhat canned, and certainly lacking the dynamism that I experienced in the face-to-face environment.  How can you evaluate a learner’s ability to provoke if you can’t see their incredulous eyebrow lift before they challenge an idea or hear the tone of their voice?  How can you assess the leadership of a learner who uses the tenor of their voice and a deliberate, measured vocal pace to bring together a number of opposing views and develop a common understanding? These are just a few of the intangibles, just a few of the nuances that I’m sure Paulo saw in the groups that he was involved in when he was establishing his theory. 

I had to conclude my assignment with a research question.  And, while I crafted something that would “serve as a signpost, guiding my literature review”, as I conducted a cursory scan of the literature and drafted and redrafted this initial assignment, what I really kept asking myself was, “What would Paulo say?”