e-Literate

Well, I should have learned by now not to promise in advance how many parts a series will have and when I will complete them. It seems that, as soon as I declare publicly what my blogging plans are, Brother Fate smacks me upside the head with his big, fat overcommitment stick.

But apparently I’m a slow learner, because here comes another statement I’m bound to regret. If you work on competencies and rubrics for ANGEL, Blackboard, Moodle, Sakai, or any other system and are willing to share information, I’d like to talk to you. My goal would be to write not a review but an overview of the basic principles underlying your system design, much as I have done with this series. Many thanks to Kenneth Chapman and the D2L team for being so open. I think this kind of dialog really helps us move the state of the art forward.

At any rate, this is very late and somewhat abbreviated, but here, at last, is Part III of the series.

In the last two parts, I talked about D2L’s notions of learning objectives and competencies. The piece that completes the picture is assessments. Recall that an assessment is a way of measuring whether a student has met a learning objective which, in turn, rolls up to one or more competencies. You can, of course, have more than one assessment that measures the same thing. The most obvious form of assessment is a test and, indeed, D2L allows instructors to link tests to learning objectives by setting a threshold passing score to meet the objective. Of course, there’s a lot that you can’t assess with a standard test format. So, for everything else, D2L supports rubrics.
For those who are unfamiliar with the term, rubrics are basically grid-form job aids that help make more clear and objective assessments that have some element of subjectivity in them. For example, rubrics can be used to evaluate history essays or mathematical modeling of complex real-world problems. Think of a table where the vertical axis represents scores (for example, a scale of 1 to 5) and the horizontal axis represents dimensions by which a student will be evaluated (for example, grammaticality or strength of supporting evidence). The cells in the middle of the grid provide definitions of what it would mean to score a given level for a given dimension of evaluation. Rubrics help student have a clearer sense of how they will be evaluated. They also help teachers ensure that they are evaluating each student’s assignment consistently. D2L allows you to attach rubrics to assignments and then roll these rubrics up (as assessments) into learning objectives, creating a very flexible system. Like learning objectives and competencies, rubrics can be shared with different faculty or even different departments. This creates possibilities for a kind of bottom-up approach to norming the curriculum by sharing assessment definitions, in a similar way that sharing learning objects can promote consistent quality.

So those are the high-level details of the system that I intend to cover in this series. With what remains of this post, I’d like to very briefly touch on some areas where I’d like to see more cross-platform collaboration.

The first is in the area of interoperability. There exists an IMS standard for exchanging competency information called Reusable Definition of Competency or Educational Objective (RDCEO). D2L’s system supports it, and that’s fine. But until we have some sort of competency profile exchange in which groups (e.g., university departments, disciplinary groups like the Modern Language Association, etc.) can share and rate profiles, I think this is of limited value. At the last Sakai conference, Connexions Executive Director Joel Thierstein introduced a notion he called “lenses,” which are ways for organizations to give stamps of approval to contributed content in the Connexions repository. Thierstein believes that lenses will enable the kind of vetting that university administrations tend to be more comfortable with, allowing a system of accreditation that’s more likely to be adopted for, say, tenure and promotion purposes to grow up around a more bottom-up democratic repository without interfering with it. It would be interesting to have this sort of thing for competencies, as well as the underlying repository to support it.

Perhaps more importantly, I’d like to see a lightweight standard that allows for the attachment of assessments, including rubrics, to any student activity in any application. There’s some foundation work in this general direction being done right now under the auspices of IMS Enterpise Services v2 (which I hope to be able to write about in more detail Real Soon Now). Whether or not assessments are (or should be) standardized, we should at least have a standard that lets us attach the ability to assess student activities in a wide range of tools across many platforms.

And finally, I’d like to see work done to semantically tie assessments, competencies and learning objectives to learning objects. Ideally, a project like Connexions or MERLOT would build out a repository like the one I mentioned above and explicitly map learning objects to assessments, learning objectives, and competencies. At the very least, though, we should have some sort of standard for making such a mapping.

There’s much, much more to say here, but given my backlog of blogging and non-blogging obligations, I’m going to call it a series here. Rest assured, though, that I will be returning to this topic.