INTRODUCTION

Computer programming is a difficult and challenging subject area which places a heavy cognitive load on students (Garner, 2000). After two years of learning programming, most novice programmers are still struggling to be proficient (AECT, 2001; Moursound, 2002). From an examination of current research in this field, one reason why computer programming instruction is challenging, lies in a lack of understanding about good instructional approaches in this direction (Tholander, Karlgren & Ramberg, 1998).

CABLE, the pedagogical model researched in this study is an attempt to devise a viable instructional model based around the construct of apprenticeship. Cognitive apprenticeship is a model of instruction that involves the effective communication of domain knowledge in such a way that the students become aware of the thought processes involved in knowledge construction within that domain (Brill, Kim and Galloway, 2001). Cognitive apprenticeship is directed at teaching processes that experts use to handle complex tasks, and is characterized by a number of teaching methods (Moursound, 2002). These include modelling, scaffolding, coaching, articulation, reflection and exploration (Jarvela, 1995). Another instructional strategy that is gaining prominence as an effective teaching method and is integrated into the learning environment being trialed is collaborative learning.

Our starting assumption is that the computer can be used as a tool assisting in both cognitive apprenticeship and collaborative learning. An important element of the CABLE learning environment is its use of computer mediated communication techniques for implementing aspects of cognitive apprenticeship such as scaffolding, coaching, feedback and modelling (Levin & Waugh, 1998). Computer mediated communication techniques used in implementing tele-apprenticeship in the current study include email, online notes, interactive tests and a bulletin board.

This paper presents the findings of a study which is one of a series of trials which evaluated the effectiveness of the online implementation of CABLE, in the teaching of Java programming at the National University of Samoa. Since the CABLE procedures were developmental in nature, some improvements to the procedures took place between the phases.

PARTICIPANTS

The average age of the participants in both phases was 19 years. Participants were allocated to either the CABLE treatment or were taught in accord with the traditional university model of teaching (i.e., non-CABLE methods).

There were 53 participants (32 CABLE, 21 traditional) for Phase 1 from the University Preparatory Computer Studies class, but by the time the study was completed, the numbers had dropped to 30 students (22 females, 8 males) in the CABLE treatment and 16 students (7 females, 9 males) in the traditional treatment.

The participants for Phase 2 were students enrolled within their first year of studies at the National University of Samoa from the Foundation (64%) and non-Foundation (36%) programs. Initially 78 students (38 CABLE, 40 non-CABLE) consented to participation in the study. At the end of the study, complete sets of data were available from sixty seven students who completed the questionnaire (30 CABLE, 37 non-CABLE).

INSTRUMENTATION

Instruments used to collect data for this study consisted of (a) separate questionnaires for collecting attitudinal data for CABLE and the traditional groups, (b) post-test scores, and (c) student interviews. The questionnaires for both CABLE and traditional groups had in common the first 11 questions and the two unstructured questions but the questionnaire for the CABLE group had additional questions (Question 12 to 17) which evaluated the effectiveness of online learning and collaborative learning. The Likert items for all these questions were constructed with a 4 point scale with responses of, (1) strongly disagree, (2) disagree, (3) agree, and (4) strongly agree.

Questions 1 to 11, evaluated student attitudes towards computers, the effectiveness of feedback, effect on self-confidence, students' love of learning and motivation for learning. The items for this scale are listed below:

Question 12 to 15 evaluated student perceptions on the effectiveness of e-mail feedback, the effectiveness of online notes, FAQ's, and interactive testing. These items are listed below:

Question 16 and Question 17, evaluated student perceptions on the effectiveness of collaborative learning. These items are presented below:

The two open ended questions in the questionnaire probed problems encountered by students in their learning environment and also their evaluation of the effectiveness of the mode of learning. The two items appear below:

PROCEDURES

The use of websites for hosting online notes, discussion forum, and e-mail feedback were the online components used as part of the implementation of CABLE. There was considerable overlap in the instructional approaches in the two treatments. Students in both the CABLE and traditional groups were given the same set of notes and exercises on JAVA programming, and also used the same Java environment, JBuilder. Both groups were exposed to elements of the cognitive apprenticeship based approach such as feedback and coaching. In both groups, the teacher modelled computer programming theory and JAVA programming concepts using worked examples and real-life examples.

One main difference between CABLE and traditional approaches, was in the provision of feedback to students. In the traditional approach, feedback was student initiated. Feedback was provided by lecturers when requested by students. Within the CABLE approach, feedback was more structured and was provided by an online system where the lecturer provided individualised feedback via e-mails. On a weekly basis, the students were expected to send to the lecturer, an e-mail which answered several questions

The first question required them to describe activities or topics covered during the week. The second and third question required them to identify problematic areas, and post specific queries. The last question required the student to reflect upon the usefulness of what they had learnt. From student feedback, the lecturer was able to compile some frequently asked questions (FAQs) and their solutions on the class web-site. Students were also encouraged to e-mail as often as possible, whenever they encountered problems in class.

A second main differentiating factor between the CABLE and the traditional approaches is the cultivation of metacognition. Students were encouraged to reflect on their progress and also articulate their thinking processes in the form of “think-alouds”.

The third main differentiating factor between the CABLE and traditional approaches is the incorporation of elements of collaborative learning. For example, students collaborated in such programming tasks as UML modelling, and also in completing class programming assignments or projects.

One of the main distinguishing features of cognitive apprenticeship is modelling. Within CABLE, the teacher provided modelling in various ways: (a) by demonstrating object-oriented programming concepts and skills, and (b) by using certain problem-solving heuristics to demonstrate how to model JAVA applications.

Another component of the CABLE approach is coaching. This was implemented in several ways: (a) by the lecturer giving expert coaching in class, (b) by means of expert help via e-mail; (c) by peer coaching from other students as they collaborate in certain programming activities, and (d) by means of interactive online tests. Students could test their level of knowledge and skills by taking these tests and by clicking on a button, the test would be graded and instant feedback of their test score would be returned.

Coaching and modelling was also facilitated by the use of the debugger facility of JBuilder, the integrated developer environment for creating Java programs. The debugger allowed students to “step through” any JAVA program and visualise the sequence of execution of the JAVA program.

A salient feature of the CABLE approach is contextualisation of abstract tasks. JAVA programming was taught within the context of the systems life cycle so that students could see the steps of the process as integrated within a larger context but at the same time focus on the individual details. Situated learning was also facilitated by means of collaboration in pairs for programming activities, as in real life, programming is usually carried out by a team of developers.

Another important feature of cognitive apprenticeship is visualisation. In CABLE, this was achieved by the lecturer articulating his thought processes and by posting notes and sample solutions on the class website. Within the CABLE approach, the lecturer provided scaffolding by providing the most appropriate teaching strategy to facilitate support for the student.This help or guidance was gradually withdrawn (fading), when students demonstrated, that they could now step through the process with confidence. The learners would then be given more complex tasks.

After six weeks of exposure to the treatments, all students completed the final test paper, and a questionnaire intended to tap into their evaluations of their course experience.

Evaluation of the effectiveness of online learning was based on the Triple P (Perceptions, Processes and Products) Framework as proposed by Ryba, Selby and Mentis (2001). The Triple P Framework evaluated online communities by: (a) documenting the views and experiences of students (Perceptions), (b) analysing the content of interaction patterns of electronic contributions (Processes), and (c) verifying the products that result from the online learning process (Products).

Student perceptions were collected from (a) Questions 12 to 17 of the CABLE questionnaire, (b) student interviews, and (c) log of student e-mails.

The products of online learning for this study are the e-mail messages and FAQs. Based on the Triple P Framework, contents of these products were categorized using Poole's (2000) method of coding students' participation and then analysed using Salomon's 5 stage model, to evaluate the stage of growth, of the online community. According to Salomon (2000), there are 5 stages in the development of an online learning community. Stage one is when community members develop the motivation to access and use the web environment proficiently. Stage two is the establishment of online identities and the initiative to socialise with others online. Stage three is characterised by participants initiating the process of assisting and providing mutual support in information exchange. Stage four is characterised by course related group discussions; increased collaboration amongst members of the online community as they collaborate in online work. Finally, stage five, is characterised by the achievement of personal goals and an ability to reflect on the learning process. Analysis of the processes of online learning is linked to Poole's (2000) work of coding students' participation in an online course in which messages are categorised as one or more of the following: (a)Technical messages relating to the website and managing on-line learning (Stage one of Salomon's model - access & motivation); (b) Social - messages that are non-academic in nature (Stage two of Salomon's model - socialisation); and (c) Coursework - information related to the course content and academic work (Stages three to five of Salomon's model - information exchange, knowledge construction and development)

RESULTS PHASE 1

For the purpose of statistical analyses, an aggregate variable OLS (Online Learning Score) was created to measure positive affect or liking for online learning. This was computed by summing the responses for Questions 12 to 15. (OLS could take values between 4 and 16). The four items were then entered into a factor analysis using a Principal Components procedure which indicated that a single factor resolution was possible. SPSS Reliability analysis was then used to check on scale properties, to reveal that the internal reliability coefficient alpha, skewness and kurtosis were all within acceptable range. For further investigation, the two treatment groups were split into two groups on the basis of the prior test scores. The median value of the prior test scores in each phase was used, to generate a classification of high ability and low ability students

CABLE group participants, evidenced high levels of liking for the online environment, as indicated by the values of the OLS aggregate (mean of 12.75, SD of 1.98). The OLS scores ranged from 7 to 16 with 84.4 % of the respondents exhibiting scores above the natural midpoint of 10. Comparison of high-ability (mean of 12.93, SD of 2.5) and low-ability (mean of 12.61, SD of 1.6) groups indicated no significant differences on the levels of OLS, F (1.30) = .196, ns. The majority of the students interviewed, agreed on the usefulness and effectiveness of e-mail as a means of feedback. The main source of frustration was technical problems preventing effective access to e-mail. These included hardware failure, and inability to access the Internet and the intranet. Most of the students regarded e-mail as very helpful as it gave useful and immediate feedback. All of the students interviewed liked online notes, giving the main reason for liking, as the ease of access, and also, so that they could access it at any time. Again, the main complaints were technical, such as computers breaking down and inability to access the Internet. All of the students interviewed liked the idea of sample solutions online as they said they could: (a) access it any time, (b) useful for revision, and (c) were useful for doing test corrections.

Evaluation using the Triple P Framework

Evaluation using the Triple P Framework showed that the online community in Phase 1 had progressed to stage three of the five-stage model where students were involved in information exchange using e-mail and the discussion forum (88% of the content were course related in nature, none of the messages were social in nature and 12% of the content was technical and related mainly to problems of access and computer failure. The volume of messages had increased by 50% by the end of the project).

In terms of processes, the students were not only proficient in using the online environment, but were also proficient in using the online environment for receiving coaching, feedback and scaffolding (88%). In fact one could also claim that students were also using the e-mail facility for online discussion forums (7%) and for a few of them, the ability to use it for reflection on their work (5%).

In terms of products, student participation in e-mail included technical messages relating to the website or managing of the helpdesk and questions related to course work.

RESULTS PHASE 2

Participants in the CABLE group evidenced high levels of liking for the online environment, as indicated by the values of the OLS aggregate (mean 12.5, SD = 1.9). The actual scores ranged from 8 to 16 with 72% of the scores lying above the natural midpoint of 10. Comparison using ability status as criteria, reported no significant differences between high-ability (mean = 12.4, SD = 1.8) and low-ability groups (mean = 11.6, SD = 1.9) on the levels of OLS, F (1, 25) = 1.37, ns. All of the students interviewed liked the use of e-mail as a means for feedback, quoting reasons such as: (a) ease of access, (b) advantageous for shy students, and (c) being able to ask questions freely. The main problems were the reliability of the technology such as inability to access the website, and also the lack of knowledge and skills to use the website and e-mail facility. All of the students interviewed agreed on the effectiveness of online notes giving as the main reason the ease of access. Again, the main problem was the reliability of the technology. Except for one student who had not attempted this feature, all of the students indicated the usefulness of the sample tests giving as the main reason its usefulness in revision for tests.

Evaluation using the Triple P Framework

The results of the evaluation of the effectiveness of the online learning environment using the Triple P Framework were very similar to those in Phase 1. Results showed that the online community in Phase 2 had progressed to stage three of the five stage model where students were involved in information exchange using e-mail and the discussion forum (80% of the content were course related in nature, 4% of the messages were social in nature and 16% of the content were technical and related mainly to problems of access and computer failure. The volume of the messages had increased by 56 % by the end of Phase2).

In terms of processes, the students were not only proficient in using the online environment, but were also proficient in using the online environment for receiving coaching, feedback and scaffolding. Students were also using the e-mail facility for online discussion forums and for a few of them, the ability to use it for reflection on their work.

In terms of products, student participation in e-mail included technical messages relating to the website or managing of the helpdesk and questions related to course work.

CONCLUSION

In summary, the results of the two phases seem to indicate the effectiveness of the online learning environment as evident from the inspection of the OLS aggregate, student interview responses, an examination of the perceptions of students, an analysis of the content of student e-mail messages and the processes students engaged and were proficient in. However, the results also suggested the need for a stable technical infrastructure and also training in the use of the online environment as factors needed for effective implementation of the online environment. To conclude, the present study is but an initial step in the development of the pedagogical framework for teaching programming at the National University of Samoa. Further research and more development work needs to be carried out in order to ensure further improvements to the CABLE environment.

BIBLIOGRAPHY

AECT 2001. The handbook of Research for Educational Communications and Technology. In URL: http://www.aect.org/intranet/publications/edtech/24/24-05.html [accessed June 12th 2003]

Brill, J., Kim, B., & Galloway, C. (2001). Cognitive apprenticeships as an instructional model. In M. Orey (Ed.), Emerging perspectives on learning, teaching, and technology. Retrieved April 5^th, 2004, from http://itstudio.coe.uga.edu/ebook/CognitiveApprenticeship.htm.

Garner, S (2000). Cognitive Load Reduction in Problem Solving Domains, Edith Cowan University.

Jarvela, S. (1995). The Cognitive Apprenticeship Model in a Technologically Rich Learning Environment: Interpreting the Learning Interaction. Learning and Instruction 5(3), 237-259.

Levin, J., & Waugh, M. (1998). Teaching teleapprenticeships: Electronic network-based educational frameworks for improving teacher education. Journal of Interactive Learning Environments, 6(1-2), 39-58.

Moursound, D.G. (1996, 2002). Increasing your expertise as a problem solver: Some roles of computers. Eugene, OR: ISTE. Copyright (C) David Moursund 2002.

Tholander J., Karlgren K., Ramberg R. (1998), Cognitive Apprenticeship in Training for Conceptual Modeling in URL [http://www.dsv.su.se/~klas/Publications/webnet98.pdf] accessed July 15^th 2003

Poole, D. (2000). “Student participation in a discussion-oriented online course: A case study.” Journal of Research on Computing in Education, vol 22 no. 2, pp162-177.

Ryba. K, Selby, L., & Mentis, M. (2001). Analysing the effectiveness of on-line learning found in communities.[Electronic version]. Retrieved September 25, 2003, from

http://www.ecu.edu.au/conferences/herdsa/main/papers/nonref/pdf/KenRyba.pdf

Salomon, G. (2000). E-Moderating: The key to teaching and learning online. Kogan Page, London.