Digitally Teaching Digital Skills: Lessons Drawn from a Small Private Online Course SPOC on ‘Modelling and Simulation in Archaeology’ at Leiden University

2.1. Massive Open Online Courses (MOOCs)

The most common and most widely known format of online courses is a MOOC – Massive Open Online Course (Hoy 2014; McAuley et al. 2010). There is a growing interest in their potential as an educational platform providing training to the wider audience and it is also increasingly recognised as a tool for keeping the general public up to date with the latest developments in academic and professional research (Pickering et al. 2017). Although there is not one accepted definition of what constitutes a MOOC, the following characteristics are often invoked (McAuley et al. 2010):

• open to unlimited participation: the participation rates are expected to run in the thousands;
• free of charge: often the content is released under one of the open creative common licenses; however, the courses usually have a beginning and an end and the content is not freely available outside of this period;
• digitally delivered: MOOCs are a form of distance learning based on the Internet (as opposed to, for example, correspondence courses);
• educational: consisting of a curriculum, learning objectives, and some forms of knowledge testing and generally concluded with a certificate (not always free of charge);
• student-led learning process: although the courses may have a structure, for example, with content released in weekly instalments, it is the students ‘who self-organize their participation according to learning goals, prior knowledge and skills, and common interests.’ (McAuley et al. 2010: 4).

An example of an archaeological MOOC is ‘Osteoarchaeology: The Truth in our Bones’ offered by Leiden University (https://www.coursera.org/learn/truthinourbones-osteoarchaeology-archaeology, verified 19 March 2019). This course is a result of Leiden University’s strategy to explore and develop innovative teaching formats. Provided through the popular MOOC platform – Coursera – it aims to show anyone what human skeletal remains can teach about life in the past. It is structured around five weeks requiring four to six hours of study per week and consistently receives high rates from its students. In the first nine months after its launch in September 2016, the course counted ca 10,000 participants, out of which ca. 6,400 completed certain lessons and ca. 500 the entire course.¹

The main strength of MOOCs is their scalability, meaning that an increase in the number of participants does not linearly translate into a higher amount of work that the teachers have to invest into the course. This means that the educational content can be provided to a very high number of participants at a diminishing cost because after the high initial labour investment the course ‘teaches itself’.² On the other hand, this limits the types of student-teacher interactions and personalisation of the content to particular needs and interests of students (auto-grading for instance, see Fox et al. 2014). For example, individual feedback would be impossible to deliver. Equally, the types of exercises used during MOOCs do not enable students to develop their own initiative or follow their particular interests. For example, a common format in classroom teaching, a short essay, is not a viable type of assessment because the amount of work needed to grade it would vastly surpass the resources dedicated to the course. Thus, more ‘creative’ forms of knowledge acquisition and retention have to be excluded from MOOCs as too costly. Finally, due to the open format of a MOOC the retention rates are generally low, oscillating in the low tens at best (Alraimi, Zo, & Ciganek 2015). Nevertheless, the absolute number of students completing the most popular MOOCs often exceeds anything that a classroom module can hope to accomplish (see the Osteoarchaeology MOOC example above).

MOOCs give access to high quality content to people who would not otherwise be able to access it – e.g., because of their location or financial status (Pickering et al. 2017). Also, because of the flexible schedule courses can be taken by members of the public whose commitments (e.g., a full-time job) would otherwise prevent them from extending their skill set. Last, but not least, participating in a MOOC allows you to join a community of similarly interested individuals, often in large numbers.

At the time of writing, Coursera lists more than 2000 courses, of which only a handful are related to archaeology. Other platforms such as FutureLearn, Udacity, and EdX list other archaeological courses but their number is equally limited. This stands in stark contrast to the hundreds of courses focusing on practical, usually computational or mathematical skills such as Coding in Python, Statistics, Web Design, or Linear Algebra. Although anecdotal, this indicates that the format lends itself well to teaching practical skills, in particular, computational techniques.

In contrast, archaeological MOOCs are used predominantly as an outreach tool with the aim to disseminate local expert knowledge to the general public (Alcock et al. 2016). Thus, they target wide audiences beyond the home institution and are not integrated into the curricula of regular degree programmes (Emanuel 2015). To the best of the authors’ knowledge, none of the institutions which developed archaeological MOOCs allow their own students to gain credit from participation in these courses. This highlights the general perception of MOOCs as tools for providing entry level education rather than carefully tailored and specialised knowledge traditionally delivered in classroom teaching (Emanuel & Lamb 2015).

2.2. Small Private Online Courses (SPOCs)

Small Private Online Courses (SPOCs) are less common than their more famous counterpart, MOOCs. They serve a different purpose while combining traits of both MOOCs and more traditional teaching formats (Fox et al. 2014) (see Table 1). In general, in a SPOC course students aim to master online materials provided in a way similar to MOOCs. However, the SPOC teachers are always present and engaging the students, facilitating learning activities, assessing student progress, and providing feedback at the individual level (Jing Ping 2016). Thus, the main difference between SPOCs and MOOCs is the level of interaction between the teachers and students and the ability to individualise the learning experience (see Table 1).

Thus, just like traditional on-campus courses and unlike MOOCs, SPOCs are often fully integrated into the curricula of specific degree programmes. Participants have to meet admission requirements (including prior knowledge and experience), need to enrol, and receive credits for their participation. Access to course material is restricted to enrolled students only and student numbers are limited and comparable to traditional classroom courses.

Just like MOOCs, SPOCs are fully online. All course materials are provided via a web platform, and all assignments and exams are submitted and graded online. Interaction between students and teachers, as well as among students, happens online via forums, video conferences, etc. Actual face-to-face contact is limited.

Also, similarly to MOOCs, students self-organise their learning process by deciding on the time, place and the mode of studying. This freedom to shape one’s participation is to some extend curbed by the course structure since the course content is usually released in regular instalments. Also, because of the lack of control over the students’ participation their progress if often closely monitored. For example, it is common to use frequent tests and assignments as means of controlling that each student is following the pace of the course.

Finally, in most SPOCs the interactions between the teachers and the students and within the student group occur more frequently than in MOOCs or general courses. In particular, the students can seek individual support and regularly receive individualised feedback – something that would not be possible in a MOOC. As a result, the students can also be given more freedom to shape the direction of their learning, for example by choosing their own case study or research question.

3.1. Rationale

The SPOC on ‘Modelling and Simulation in Archaeology’ was run in the 2016/2017 academic year as part of the MSc programme ‘Digital Archaeology’ at Leiden University. Its main purpose was to provide in-depth, practical training in one of the subfields of Digital Archaeology, modelling and simulation (Niazi & Temkin 2017). In traditional teaching, this course would be taught over seven weeks, with a two-hour classroom lesson each week. While this timeframe had to be followed to ensure a smooth integration into the general educational schedule, the SPOC format allowed a great deal of flexibility in how the workload of students and instructors was distributed.

The main challenge of the course was to teach students a highly technical skill – computer programming in the context of modelling and simulation – against an archaeological background. In this it differed from more traditional archaeological courses concerned with transmitting knowledge and understanding rather than a practical ability. Thus, the more traditional approach of lectures, accompanied by reading and essay writing was unlikely to achieve our objectives. Existing on-line modelling and programming courses were not of the level and context that we envisioned for our students.

The nature of the educational content of the course made us choose the SPOC format to accommodate the specificity of the learning process, in particular:

• The practical aspect of the skill we wanted to transmit means that students have to ‘try it out for themselves’. Computer programming is not learnt by listening to others lecturing about it but by writing the code itself, and the level of knowledge is directly proportional to the amount of coding one has done. Therefore, the student-teacher interaction is mostly limited to guiding the students’ way through their individual learning process rather than providing them with large amounts of information.
• Because the time involved in computer programming is 10% writing code and 90% trying to figure out why it does not work, a key skill necessary to master it is the ability to deal with problems as they arise. Knowing where and how to look for solutions as well as developing the resilience to temporary setbacks were two crucial elements of the learning process that could only be gained from experience and had to be acquired by each student individually.
• In most cases there is no ‘one’ answer to a problem in computer programming, so an individual approach is necessary to promote creativity and to develop agility in the students – two key features of good computer programmers. In fact, the students often surprised us with solutions that we did not anticipate.
• Many useful sources on modelling and simulation are available online, such as forums, blogs, tutorials, libraries, papers etc. Using an online format for teaching allowed us to seamlessly integrate these valuable materials.

On top of that, there were a number of practical considerations that made the SPOC format attractive for us, in particular:

• The SPOC allowed us to offer the course not just to Leiden University students, but also to selected external participants.
• One of the teachers was based in another country during the course, and could only participate in the course thanks to its online format.
• The SPOC was used to practice with the format and gain better understanding of the requirements of this type of teaching. This goal was supported by Leiden University and its Faculty of Archaeology which are currently exploring a range of new teaching formats to improve the quality of teaching for both students and teachers and to mitigate pressure on limited resources such as classrooms.
• Due to its modular structure, the materials developed for the course (videos, manuals, assignments) can be reused in other contexts, e.g., short workshops.
• Finally, the process of course development was supported by the Leiden Online Learning Lab who provided expert advice throughout the process as well as facilities and services such as filming the video clips.³

In sum, the decision to choose the SPOC format for the course was predominantly driven by considerations related to requirements of the content we needed to deliver, but practical aspects also played a role.

3.2. Design and structure

The SPOC is a very specific teaching format and as such required stepping out of the traditional educational mindset (Jing Ping 2016). Furthermore, because of the relative novelty of this type of teaching, it is not possible to reach out to what traditionally would be the sources of support and advice, namely the expertise of our colleagues and our own educational experiences. This is where the Leiden Online Learning Lab stepped in. The design process was facilitated by their staff who gave valuable pedagogical and technical advice.

The basic design of the course was jointly elaborated during a two-day workshop at the Online Learning Lab in early 2016. The structure of the course was established using the framework of objectives (Granger 1964). That is, a list of essential, preferred, and desirable learning outcomes was prepared by the course leaders and used to design the course elements. As a result, each video, reading assignment, e-tivity (explained below), etc. was judged on its ability to contribute to the defined learning objectives. This simplified the process and provided a very strong framework for the course. It also enabled the teachers to think through the students’ learning process and ensure that they have the information necessary to complete the assignments and understand the upcoming content, such that each element builds on previous elements.

In practical terms the basic structure of the course was initially defined using post-it notes and then transferred to a spreadsheet in Google Docs, which then served as a collaborative working environment for the teachers to develop the actual course materials. The course and course load have been designed based on the seven-week structure of general education at the Leiden Faculty of Archaeology (Figure 1).

One of the key notions is that students should always get a chance to test their understanding of the lesson’s contents or get a feedback on a newly acquired skill prior to being tested on it in a graded assignment. Thus, it is not recommended to make the students read a paper or watch a lecture and then test them on their ability to apply this knowledge in a new context (e.g., code a simulation). Instead, it is much more effective in terms of educational outcomes to give them first a small not-graded assignment providing a safe space to practice their skill and understanding.

The second important notion was the recognition of the importance of active methods of acquiring knowledge. Thus, although the majority of the content was transmitted to students via videos, reading assignments, and self-guided tutorials, all of these elements were followed by ‘e-tivities’. These are small tasks designed to test the students’ understanding and make them actively engage with the content provided. For example, although a number of reading assignments were given to the students to illustrate a given core concept in more depth or through a case study, it was the e-tivities in the form of simple, multiple-choice questions that ensured that the main points were highlighted and that the students had to spend some time actively thinking about them.

Finally, it has been noted many times that the best teachers are one’s peers (Bernard et al. 2009; O’Donnell & O’Kelly 1994; multiple authors in O’Donnell & King 1999). Thus, the course made ample use of online forums. Certain course elements required students to interact with each other, e.g., in order to comment on other students’ assignments. This turned out to be a double-edged sword as often the first solutions to coding assignments posted on the forum were copied or only slightly altered by students who were solving the tasks later. Thus, in a number of cases where we expected a diverse range of solutions, the students converged on one or two instead. On the other hand, errors made in the earlier posts and corrected by students or teachers did not repeat in later posts. Of course, students were informed and required to sign a statement about the ethics of digital communication and online learning making them aware of such issues as plagiarism or taking an unfair advantage such as the services of cheat-for-hire companies.

3.3. Learning goals and focus

The major learning goal of this course was the application of modelling and simulation techniques in an archaeological context (Lake 2014). Thus, the course, while teaching programming skills, was constructed with a strong focus on archaeology, in particular in these three aspects:

1. All examples presented to the students in lectures, exercises and manuals addressed archaeologically relevant issues; all of the required and most of the optional readings were selected from archaeological literature.
2. Midway through the course the students were asked to program a simulation given a simple description including flowcharts. Specifically, students had to write an essay how this model can be used in an archaeological context and provide possible model extensions. This requires the students to apply their archaeological knowledge while thinking about questions and assumptions about the past. The purpose of the case study was introduced with this section:

   Purpose: While excavating a small Roman village, archaeologists have found many pottery sherds. So far they have identified five different source locations based on materials and styles. They have however been unable to find the actual kilns/potteries. You are asked to create a model to identify the most likely locations where the pots have been produced to guide future excavation efforts. This must be implemented as an agent-based model and simulations must be run in NetLogo.

3. In the final assignment each student had to come up with an archaeologically relevant research question and then formally design and implement a simulation to address this issue. Analysis, validation and subsequent interpretation of the simulation results was done in relation to the archaeological background. Due to the amount of work required to complete the assignment the work was split into several parts and students were allotted three weeks to complete the different elements. Focal point was the research question. We asked to define them as follows:
   

   Task: Formulate the archaeological research question that you want to solve using modelling and simulation tools. Search and list at least two papers that provide the archaeological background for your research question. From Lake (2014), choose the most relevant modelling paper. Relevant papers align with your research question or use the method you want to use. Post your research question on the forum in max two sentences + references.

Students were asked to actively search for archaeologically relevant case studies applying modelling techniques from model repositories and the literature. During the exercises and especially in the last assignment students had to identify and translate implicit, conceptual models derived from archaeological research into formal coded models and thus reflect on the construction of narratives in archaeology in general.

3.4. Participation

The course was delivered between the 27th of March and the 19th of May, 2017. Students required to do a retake were offered an additional three-week extension period. Thirty-four students enrolled for the online course, among them seven in the MSc track Digital Archaeology for whom the course was compulsory. The remainder came from other master’s programmes, in particular from Osteoarchaeology (Figure 2).

Twenty-nine students completed the course by submitting the final assignment, including retakes. Out of these, twenty-six students completed the course with a grade higher than or equal to 5.5 that was required to pass the course. These final results (Figure 3) suggest that the majority of students have met the learning objectives set out by the instructors, that is they acquired the skills necessary to build a scientifically sound simulation of an archaeological case study.

These results, although obviously limited by the small sample size, show the general ability of archaeology students to gain high-level technical skills. Considering that only seven students came from the Msc in Digital Archaeology indicating their strong prime interest in computer applications, the fact that most of the class completed the course successfully is highly encouraging.

4.1. Time considerations

The course was taught over seven weeks framed by a soft launch (week 0) and an evaluation (week 8) (Figure 1). Each weekly lesson consisted of a short video lecture and different exercises such as reading assignments, coding-tutorials and e-tivities. Finally, most weeks were wrapped up with a graded assignment. Thus, while students were able to manage their time themselves within each week, it was not possible to do the whole course in a much shorter or longer period than seven weeks.

The fact that the course content was highly individualised, e.g., students could choose which model to work on, meant that the most time-consuming task was giving feedback to students on their assignments. This feature of SPOCs is most radically different from classroom and MOOC teaching – the majority of time is dedicated to assisting individual students rather than developing teaching content for the class as a whole. Each student required a different amount of interaction with the teachers, and although all of them received regular feedback on their tasks and assignments some had to be further supported as they struggled with the material.

As is the case with any university course, preparation of the course content took significant amounts of time, in particular due to the interactive nature of most of the course elements. For example, each reading assignment was accompanied by a multiple-choice quiz, which required more time and effort to prepare than simply asking the students to read a given paper and discuss it in class. Similarly, the tutorials guiding students through the various aspects of coding had to be carefully crafted to avoid confusion since the students had limited means of contacting the teachers to ask for clarification as they would normally do during a class-based course.

Another time-consuming task when preparing the course was the scripting, recording, and editing of the videos. Scripts and visual aids needed to convey the desired message while having the appropriate style for the video format, considering that students do not have the opportunity to ask questions during or immediately after the lecture. Recording of the videos had to be repeated multiple times before spoken text and visual aids were free of errors and appropriate in style and matched seamlessly. A seven-minute video required up to two hours of filming. This material was then cut, edited and converted into proper format by the Online Learning Lab.

In addition to giving individual feedback on assignments or occasional general questions, time was spent on addressing the whole class or individual students via the online forums and during a weekly online meeting. Forums were installed for each week, and students were encouraged and sometimes obliged to respond to the questions of other students. Once a week, a general online meeting for the whole class was organised using the Adobe Connect conferencing software (http://www.adobe.com/products/adobeconnect.html, accessed 19 March 2019). Teachers served as hosts and moderators, while students were admitted as regular participants. Moderating was facilitated by options such a virtually raising a hand, muting/unmuting microphones etc. Only the first meeting was obligatory, to allow the teachers to introduce the course format and content, and to allow general questions. The meetings were scheduled towards the end of the week in order to allow students to work with the materials for that week and then put forward any questions they would have.

4.2. Online logistics

Leiden University selected the commercial platform NEO LMS (Learning Management System) for its online courses. It was selected because if offers a complete online environment for students, allows non-Leiden students to participate, and connects to services that include Turnitin and paying services. Teachers and students need to create an account on that platform before they enrol for courses via access codes. Providing standard formats and templates for common course elements such as lessons, announcements, assignments, forums, etc., NEO LMS allows teachers to create custom, online courses according to their own requirements and specifications, upload own content, receive and grade assignments, and generally administer the course in a single environment.

Most course materials (texts, assignments) were developed in Google Docs allowing review, version control, and parallel development. Unfortunately, it was not possible to import these formatted text documents into the NEO LMS system, such that each element had to be copied manually. The course’s contents could be edited at all times, even when the course had already started.

The online system made it easy to track students’ progress and to see which students did not submit the weekly assignments on time and to gather data regarding the outcomes of each task (Figure 4). In some types of assignments, for example multiple choice quizzes, even more detailed analytics are available. These analytics give the teachers instant feedback on their effectiveness and enable them to gain insight into the students’ level of understanding and react accordingly during the course. It is also easy to adapt the content or modify the assignments for future instalments of the course as particularly problematic elements are highlighted.

4.3. Students’ experience

The main challenge was, without much doubt, the topic of the course. The course had to introduce the students to a new scientific technique, that is: provide them with a good understanding of the scientific method, while giving them practical skills in a highly technical domain. An additional hurdle was the novelty of the epistemology of simulation and the unfamiliarity of computer programming. These skills significantly differ from the mainstream archaeological practice, therefore limiting the ability to build upon students’ existing knowledge. Finally, although teaching archaeology students to code was the main objective of the course, an equally important consideration was to ensure that they learn how to be independent in their coding endeavours. This meant that the course was mostly providing tools to learn rather than the knowledge itself.

Very quickly it became apparent that a big challenge was to change the students’ attitude towards the learning process, in particular:

• students tended to think that if they did all the activities the knowledge and skill would somehow be attained even if they put no effort in actively trying to understand the information presented to them;
• computer programming can only be mastered through the investment of significant amounts of time, especially for novice programmers learning a new language/tool. The on-line materials (videos, assignments) only required two to three hours while the remaining hours (a total of 20 hour per week) had to be spent developing code. The fact that the allotted 20 hours were actually needed came as a surprise to most students;
• the attitude ‘I did not solve the problem, but I worked on it for many hours, thus I deserve a good grade’ was common and students found it surprising when we repeatedly stressed that the goal is not a high level of effort but a working solution;
• a number of participants tried to blame everything but themselves – the tutorials, the teachers, or even the programming language (‘There must be something wrong with Netlogo!’) – for their failure to solve a task.

Readers engaging in computational courses will probably find these points very familiar. In general, they become apparent in fields where success is easily measurable (e.g., the code is either working or not), and where it is skill rather than knowledge that is being assessed. Here the online environment and the fact that the students had to organise their learning and take responsibility for it significantly helped with overcoming the mentality challenge. The fact that students were confronted with multiple tasks per week and had to solve them largely unaided made them increasingly self-reliant and independent. The effort they would have to put into contacting the teachers, explaining the problem, and waiting for the response meant that in many cases they found it easier to just look for the solution themselves. This self-reliance and perseverance were two key aspects of computer programming that we wanted to instil in our participants.

Initially regarded as a core element of the course, the Thursday online sessions were met with mixed success and enthusiasm. Many students were hesitant about participating in online meetings, especially via video. Technical constraints further limited the effectiveness of the meetings. For example, in order to avoid constant background noise, most attendants had to mute their microphones during most of the session. This quickly led us to use the text-based chat in parallel to the video conference. In the end, most interaction happened via the chat function. While not intended, this had the advantage of recording the chat and posting it on the forum afterwards as a transcript of the online session.

We also noticed early on that students often met in self-organised, study groups in order to work on the course together. These offline meetings supported community building and helped to solve many problems, as we learnt from the students. We encouraged this since community building is part of the general educational process.

The distribution of the final results (Figure 3) shows that the learning objectives (understanding simulation methods, coding skills) were well within the reach of the vast majority of students and that in a surprisingly (to us) large number of cases the final outcome was a high grade. This is an important case study as the course topic is very far removed from traditional archaeological education, making it one of the most challenging topics to master.

However, the students predominantly fell into two groups: although the majority did very well, there was also a group of students who almost entirely failed to gain the skill (five students did not participate in the final assignment and are not included in Figure 3). Also, moderate/medium results were rare. This is not entirely surprising, again, due to the nature of the subject being taught (Robins et al. 2003). One cannot learn how to code ‘only a little bit’, if the goal is to build a working archaeological simulation – the necessary skill is an ‘all or nothing’. Equally, it cannot be acquired in a short time frame. Thus, students who did not follow a strategy of regular studying sessions evenly distributed throughout the seven weeks of the course preferring instead to invest their time and effort elsewhere and only really apply themselves to this course in the last weeks were bound to fail.

This highlights the need for even more intense monitoring of students’ progress to ensure that at the end ‘no one is left behind’. This in turn enforces a specific teaching strategy–teaching by testing. Because of the course’s time limits large parts of the teaching process had to be shifted from such tasks as reading or watching educational clips to tasks involving students testing their skills and gaining insight in the process. Pushing this strategy even further could ensure a decrease in the number of students who fail to acquire the necessary skills to pass the course.

The students were asked to fill in a feedback form upon completion of the course. This is standard procedure at Leiden University; however, this was the first time the students were asked to complete the form online. In sum, students found the course challenging yet rewarding. Their complaints about a high level of workload were consistent throughout the feedback (96% deemed it high or very high) as was their assurance that they ‘learnt a lot’ (92% of answers). Equally, 84% of students agreed that the assignments stimulated analysing and understanding of the course materials. However, only 60% agreed with the statement ‘The website materials formed an effective support for the course’ (with 32% neither agreeing, nor disagreeing), and only 52% considered the course material adequate. Given that the majority of students successfully completed the course, the online materials were clearly adequate.⁴ Thus the lukewarm attitude towards them shows that the characteristics of the online format did not meet with much enthusiasm, although the fact that this was the first iteration of the course and therefore also the ‘test run’ for the online content has to be taken into account.