During the period April to July, 2006, the University of the West Indies Distance Education Centre (UWIDEC) delivered its second offering of the online course, Local E-governance in the Caribbean. The course is a joint venture between UWIDEC and UNESCO, with UNESCO being responsible for the overall concept and for initial course design and development. The UWIDEC version was developed from the original UNESCO version, which was produced in 2004. Its aim is to assist public sector professionals to assume leadership roles in an evolving public sector in which increased emphasis is being placed on good governance and on the use of electronic tools for the delivery of government services.

COURSE RATIONALE

The expectation underpinning the delivery of this course was that participants would not only acquire and treat with new knowledge in the area of e-governance and e-government, but that they would also develop new perspectives about their own roles as professionals with responsibility for changing systems and processes within their various organizations and, equally important, for occupying a leadership role for effecting the change. Fundamental to all this is the re-evaluation of the self as a professional in an organization that is being transformed and an appraisal of the skills required to participate in, and contribute to that transformation.

In offering this course, therefore, UWIDEC sought to provide the opportunity for participants to build and/or strengthen skills consistent with their evolving role in the work environment.

It is against this background that this paper seeks to evaluate this second offering. The evaluation is based on data collected from different aspects of course delivery as well as from a questionnaire administered to students at the end of the delivery period.

THE STUDENT COHORT

Thirty students, comprising twenty women and ten men, from seven English speaking Caribbean countries registered for the course. Thirteen were from Trinidad and Tobago; eight from Jamaica, three from St. Vincent, three from Barbados and one each from Belize, the Bahamas and the British Virgin Islands.  Thirteen participants had a Master’s degree, ten others recorded a Bachelor’s as their highest academic level, five had diplomas, but not degrees, and one did not submit his academic record. 

About half of the participants (16) were employed by governments. Others were associated with governments in other ways; they either worked in related research or consultancy, or for organizations that are supported by, or support, government initiatives, or they were involved with non-governmental organizations.

Posts held varied, but the majority had training and employment related to information technology (14). Five others were employed in senior civil service posts that were not IT-related, five were in secretarial/clerical positions, three were involved with teaching or training, two were in research posts and one was a lawyer. On the whole the student cohort comprised mainly persons in middle management positions, but also included those who may be categorized as administrative support staff.

THE COURSE

Course Design

The 11-week course was based on UWIDEC’s blended learning/asynchronous delivery mode. In that regard, it comprised two components, namely self-study text-based instructional materials provided in a downloadable electronic format and a web-based learning management system that provided the environment for the active, interactive aspects of the course. These core components were complemented by a comprehensive annotated bibliography. The course structure was based on seven weekly units of study materials, one week of mid-course review and a two-week seminar.

The mid-course review was built in to provide opportunity for review and reflection after the first six units and to allow time for the completion of the second assignment. A two-week seminar followed the mid- course review week. The seminar represented an attempt at engaging students in interactive, informal learning.  Students were expected to explore two discussion topics in depth through two discussion forums, drawing on their own experience and independent research.  This activity also included two wikis, the intended use of which will be explained later.

The online component comprised both informational an active/interactive components.

Course Content

The topics covered in the seven study units were as follows:

• Core features of the Information Society
• Governance and e-governance
• E-governance: analyzing the concept
• E-government at work
• E-democracy
• Local e-government
• Formulating legislation for e-governance

There were also twenty-three unit-related units.

It should be noted that while the course acknowledged that the concepts of good governance and e-governance also apply to systems and structures in civil society and the private sector, the focus was the application to government.

Course delivery

 Units of work were released in the online LMS on a weekly basis. In addition to studying the course content and readings, students were required to participate in weekly discussion forums based on topics drawn from the content of the respective units. The course coordinator and tutor shared the responsibility for moderating these discussions.

As indicated earlier, one week was designated for course review and this was followed by the two-week seminar. Students were also required to complete three compulsory assignments, the first of which was done as a group project. Ninety percent of the final course grade was allocated to these three assignments while the other ten percent was allocated to participation in the interactive aspects of the course.

FINDINGS

Selected results

The course had a 77% retention rate. As far as overall assessment was concerned, five students earned a distinction (70% and over); twelve earned B+ (60–69%); six earned B (50–59%); seven failed to complete the course.

Seventeen out of the thirty students completed the questionnaire. On the whole, they were reasonably satisfied with the delivery of the course. For example, 16 either agreed or strongly agreed with the statement that the online assignments/activities were clearly related to the course objectives; the same number rated the statement that the online assignments/activities encouraged critical thinking in the same way. In response to the statement “the online teaching staff was well prepared”, four responded ‘strongly agree’, nine responded ‘agree’ and four, ‘partially agree’. To the statement ‘The responses of the teaching staff to questions and assignments were timely’, one student gave it a ‘strongly agree’ rating, eight responded ‘agree’ five, ‘partially agree’ and three ‘disagree’. It is evident that the spread of the responses for the last two items are somewhat different from that for the first two.

An examination of the online activity reports of the students showed that more than 25 students accessed the course overview, the student chat room and the annotated bibliography. There was little interest in information provided on Internet research and avoiding plagiarism, the UWI online databases or on feedback on the third assignment;  less than 15 students accessed these respective elements. There was also a decline in students’ access of the course materials as the course progressed. This included both the units themselves and the related readings. This decline was evident even for students who completed the course successfully.

In terms of the weekly discussion forums, 13 students participated in the first and sixteen in the second. Apparently 21 of the 30 students had the intention of participating in the seminar, this being the number of participants who accessed the information on how the seminar weeks were organized and what was expected. 

In general, students appeared to have enjoyed the course and to think that they had benefited from it. Following are some responses to the open-ended survey question, “What were the most enriching and memorable parts of the course”?

• For me the most enriching activity was the group assignment which necessitated the consolidation of each group member’s contribution into a final document.
• The readings were well chosen and enlightening.
• Being able to easily interact with the Caribbean participants and sharing ideas.

Notwithstanding the above, students also had views about what they liked least about the course, for example,

• The group assignment. While it was a learning experience, I thought it was rather unbecoming of an online course given the fact that group work is difficult enough with face-to-face interaction…
• I found the course to be distant and lacking feeling. Maybe I am too accustomed to the traditional classroom learning. I found the interaction to be very cold except for the group assignment interaction. The course was too text focus. Posting course materials to the site each week without voice and video was not enough for me ….

Selected issues

Following are some areas of interest:

Response to amount of reading

In the post-course evaluation, many students expressed the view that there was too much reading to be done in too little time. Based on the course structure, students were required to read the teaching material that comprised the units themselves and the supporting resources that were directly linked to them.  Supplementary resources included both text material posted on the course website, as well as links to other websites. Also provided was a very comprehensive annotated bibliography, which was structured according to the themes of the course, with whole text articles being provided for many of the materials referenced.

One cannot therefore disagree with the observation made about the heavy reading load.

That notwithstanding, it should also be noted that, given that these professionals will be practicing in an real-world environment that includes the information-rich environment of the world-wide web, the requirement for managing large amounts of information goes beyond their study of this course and would be a feature of their every day work experience. Competency in evaluating, selecting, synthesizing and applying information, all with a view of generating new knowledge in the shortest possible timeframe, are skills that would be required not only for this course but also in their job situations.

In this regard, a comment made by one student during the stipulated review week (Week 7) is worth noting:

I believe I took up too much time on reviewing Units 1 to 6 but in doing so, I came across material that I did not have a chance to read through thoroughly before… I (now) find that the course so far is extremely interesting and informative. It is an important subject for our region.

This comment suggests that, building in the review week was an important dimension of overall delivery..

Participation in group discussions

For each unit, students were required to participate in group discussions, based on a topic posted by the Course Coordinator. The topic was selected from the course material to generate discussions in which students would explore, expand on and/or critique the issue on which the topic was based.

While the group discussion is a typical feature of all online courses, it was considered critical for this course, given the fact that these professionals would be required to participate in work-related exchanges that would have important implications for the organization. An increasing number of these exchanges are taking place online and often with individuals whom they may not have previously met face-to-face. It is therefore important to build skills in constructing and articulating perspectives and in ensuring that arguments presented are grounded in reliable and valid information.

A reasonably satisfactory proportion of students actually posted contributions. This was probably due to the fact that a portion of the final grade (10%) was allocated for this exercise.  The level of language competence for communicating in this environment was found to be generally appropriate. Of interest is the ratio of contributions posted to hits made in the forums. Taking the forum for Units 3 and 4 as an example, the number of postings ranged from one to five for persons who actually contributed, while the number of times they accessed the forum averaged 34 times per person, with four persons accessing it less than 10 times and one, 122 times. While it would have been more beneficial if the ratio of postings to hits could have been closer, it would appear that students place a significant amount of value on learning from their peers.

When the actual content of the postings are examined closely, two issues emerge. First, notwithstanding an intention to engage the students in analysis based on reflection, students’ contributions tended to be more biased towards the descriptive and prescriptive and less towards the analytical and evaluative. For example, in response to the topic on e-government, one student gave a fairly lengthy account of how ICTs were being used in one country in support of information sharing within a particular sector. Another, in response to a seminar topic, outlined how the introduction of a taxpayer registration number (TRN) had enhanced the government’s records-keeping and revenue collection services. While such postings may be worthwhile in terms of adding to the storehouse of Caribbean-relevant information, they make little contribution to the building of the quality of knowledge required for addressing the actual problems within which such information may be applied.

There were those postings that focused on describing problems, but these rarely analysed the problems in order to propose: one student, for example, commented about a national policy:

 Caribbean nations could adopt the best methodologies from other countries, but should develop their own computer policies…. For instance, in 2003, (named country) developed its national computer strategy, yet the performance, in terms of implementation, is not as it should be. The challenges include insufficient human capital to execute the strategy ….. digital divide issues …..

Such a posting appears not to go beyond a surface observation that things are not working well. Even when the posting appears to be dealing with the factors that gave rise to the problem, it simply repeats the standard areas of concern that are often stated in several contexts. Thus, while they may be appropriate, their capacity to assist in addressing the situation at hand is limited since they have not been contextualized in any meaningful way.

In contrast to the above, there were also those instances that demonstrated good skill in using the discussion topic as a stimulus for examining and analysing practice and, on that basis, for making proposals for the use of ICTs in a manner that can enhance governance

processes. Drawing on  an example of citizen use of the cellular phone to gather evidence to record an incident of police brutality, one student went on to show how ICTs can serve the important function of empowering the citizenry and of ‘quickening’ the political process  to be more transparent and accountable…” In that regard, the student was able to take anecdotal information beyond the level of a specific incident, to offer a suggestion that can have implications far beyond the incident described.

This ability for integrating information from various sources and to move into the realm of problem solution based on analysis, approaches the quality of engagement that one would have hoped to see in the discussion forum. In fact, one of the students themselves commented as follows about this aspect

Not much stimulating discussion and sharing of ideas for change, seemed more like a portal for information than engaging discussion.

Given the importance of this dimension in a course like this, it is evident that UWIDEC must review its own strategies for facilitating the use of the discussion forum, both in terms of initial instructions as well as in terms of the type of interventions made during the exchanges. With regard to the latter, it would appear that a conscious effort needs to be made to systematically utilize questioning and other strategies that have the best chance of drawing out an analytical and problem solving behaviours on the part of all students.

Access to course content

One important observation about students’ use of the course materials provided is that the number accessing the units posted on a weekly basis declined towards the end of the study period. Thus, the last units were accessed by fewer students than the first. At the same time, one notes that in terms of the final results, of the 25 students who completed the course, one earned a distinction, 12 earned a B+, and six earned a B grade. The question that comes to mind is whether students were able to successfully complete the course without actually engaging with the materials intended to form the basis of their study.

While the non-access may appear to be a case of lack of engagement with the material, one should also explore other possible factors. For example, it is likely that the task of downloading and printing was the responsibility of individuals who had more adequate resources to do so, and that others would simply get a photocopied version from their peers. It is also likely that additional readings were very adequate, and were dealing with the issues, hence there may have been a feeling that there was no need to spend time studying the units themselves. Whatever the reason, there is need to investigate more closely this decline in access to the later units, when the lack of access did not impact adversely on students’ ability to complete the course successfully.

Course assessment

The first of three equally weighted assignments was a group assignment. Students were grouped into five groups with six students in each group.  Students were given a choice of three topics and were asked to complete a 1500-word essay in three weeks. For group work, students were provided with a chat room, group forums, in which each group’s discussions were private, and group members also communicated by email.

This exercise may be seen as a moderate success in online collaboration, which in itself is an important skill that must be developed for the implementation of e-governance. However, the Course Coordinator’s comments highlighted some weaknesses in the academic approach to the task.

While there was some evidence of independent research, the writing tended to be descriptive and lacking in critical analysis. Sometimes statements were made without the support of authoritative sources or empirical data, or opinions were given as facts.  Key concepts were not defined properly with a number of concepts value-laden and used incorrectly.

The message here, and throughout the course, that critical analysis is required to evaluate all information that is being disseminated, taking into consideration supporting evidence and the possibility of political, social or economic bias, is particularly important for those who are expected to be the change agents in the field of governance.

With regard to the students’ response to the collaborative exercise of handling a group assignment there were mixed messages. As in offline situations, some students felt that not all members of their group were pulling their weight, that the problems of collaborating, such as allocating tasks and receiving input, were exacerbated online, and that three weeks was not long enough to complete the assignment, although all groups submitted on the due date. However, in the course evaluation survey there was some positive feedback. One student went so far as to say that this was “the most enriching activity” of the course, and another felt that it led to particularly interesting exchanges of ideas and stimulated online discussion.

Use of Wiki

Having covered core concepts in six weekly units, and following a course review week, weeks eight and nine were designated as a seminar. The expectation was that these two weeks would encourage  learning among peers based on collaboration between participants.

Two open-edited wikis were provided as an exercise in online collaboration. Participants were asked to list the sources of information that they identified in formulating their responses to the two discussion topics in the wikis. In an attempt to build the content of each wiki into an annotated bibliography, contributors were also asked to write a brief comment about each source.

All of the 21 students who participated in the seminar explored the first wiki, and 15 explored the second.  However, only one student actually edited the first, and two the second. Instead students supplied references within  their forum contributions. Consequently, the intention of having a collaborative activity that would yield a product that was jointly developed was not realized.

RECOMMENDATIONS

In light of the above the following are recommended:

• Clearer articulation of identified learning skills as intended outcomes of course.
• More detailed guidelines about learning skills to be developed and/or enhanced.
• More specific guidelines that students should pay attention to guidelines provided.
• Provide opportunity for students to reflect on their work-related experiences as a forerunner to engaging with theoretical aspects of the course.
• Consider the inclusion of a ‘live’ synchronous component.
• Increase period for course delivery.

*This paper was co-authored with Dr. Pamela Collins, who also served as  website manager for the delivery of the course.