Think of the learning outcomes first, then the activity, then the assessment, THEN the technology!

This is module five of a Learning To Teach Online MOOC with Coursera, (all content featured has been curated by the University of South Wales for Coursera) you can read about module 1 here and about module 2 here, and about module 3 here and about module 4 here.

Module Learning Outcomes

By engaging with the content, and completing the activities and related discussion in this module, you should be able to:

1. Identify assessment strategies that would be suitable for your own course design
2. Relate assessment strategies to specific technologies, understanding their benefits and limitations

Overview Video

This module will explore technologies useful for assessment strategies (these might be the same as those technologies used for activities) and should be closely related to learning outcomes. Technologies that can assess if and how far learners have met learning outcomes will be explored.

Setting a Learning Goal

Image available from here

For this module (chosen from a list):

Gain confidence with tools and methods that I can use in my practice

For the whole MOOC (chosen from a list):

I want to gain knowledge about blended/online learning and use this to re-design my course.

Key Concepts Videos

Image available from here

Online Assessment Strategies(Video 1)

Assessment shouldn’t just happen at the end of the process- what you decide to assess should be closely aligned with measurable learning outcomes. It helps teachers to know what the students have learnt and it helps students to know where they need to go next.

Formative and summative assessment

Think of the learning outcomes first, then the activity, then the assessment, THEN the technology! The activity and the assessment can be closely aligned.

Benefits of online assessment:

• Convenient submission, assessment and feedback can take place online.
• If you set assessments up so that the computer marks it then students can do this at their own pace and when they need it- and it reduces your marking load.
• Detection or plagiarism software can be incredibly helpful.
• Online gradebooks from submissions shared with learners (their grades and feedback only) can help them manage their own learning.

Three steps to help students engage with online technologies for assessment purposes:

• Ensure learners have time to practice with the tool
• Trial a task to ensure the instructions are clear (have a TA or colleague review your task)
• Provide instructions of what to do if the technology fails- think worst case scenario plan Bs

Formative online assessment in medicine (Video 2)

Large cohorts make individual feedback difficult but it’s crucial that they get some so learning technologies can help that.

Diagrams have been developed where students can place stars on specific parts- immediate feedback lets students know whether they’re correct- after three failed attempts, feedback is received about where the stars should have been dragged and the student is sent to a video that will address their areas for development.

A student suggests how online concept mapping has helped her to understand how different things link together- especially when she hasn’t understood much about a topic.It gives her a place to start.

Quizzes with a range of questions- true/false, multiple choice, image based questions etc. they submit their work and then get detailed feedback to their responses automatically. They then get a % for each of the components within the assessment so they know where to do more study before their final exams.

Online testable concept maps have been used to allow students to see the big picture. This works as a flow diagram with stages and concepts linked by arrows in all directions- the student is given one with parts missing and they need to drag and drop the concepts into the right place. They get a completed map for feedback at the end so they can see the errors they’ve made.

Another student says he values all of the online components for consolidating what he’s learnt and he uses it for exam revision too. Another one says that being online is no-confrontational, she doesn’t fear asking the questions about what she doesn’t know as she doesn’t have to and the feedback has been really helpful in the areas she needed to work on. Another student said that the freedom to choose WHEN to study has been beneficial.

The main challenge is to predict students’ common misconceptions, create assessments to address these and provide feedback that’s detailed enough to address these remediations.

It’s crucial that anything that is created is well thought-out beforehand, connected to the curriculum and effectively evaluated afterwards and is continually improved.

Comprehensive analytics have been important so that teaching becomes responsive to the areas of learning most students need to work on.

The university made an app available for students to use wherever they were and in the workplace too- contained key things they needed to know/check.

Using Audio Feedback (Visual Arts) (Video 3)

Used in a pure online course.

Writing feedback was taking too long- it was taking 30-40 minutes per student.

By picking up my iPod and just speaking that feedback, worked really well.

Once he’s looked at the work on screen, he used Audacity, presses record and begins giving feedback. It can be paused and then started again so that more feedback can be added.

He downloads it as an MP3 file as it works on any device and won’t require learners to download additional pieces of software. They’re also small files so it’s great for delivering it- he places it in their portfolio area so they can listen to it live immediately. They could also download it so they can keep it elsewhere- on their desktop/iPod.

Originally, the teacher did it this way for themselves but the students found it such an incredible connection to a teacher they had never met.

I don’t plan the feedback first before recording. I MAY write down one or two words as he looks at the work online but then he just talks.

Teachers don’t need a lot of technology expertise to be able to do this but there are great benefits for you and your students.

It’s great in a visual arts context to use phone/ipod so that he can wander around a space, view work and record as he goes.

It doesn’t matter at all if the teacher pauses or stutters etc- it makes it human. What’s important is that it helps the students learn.

The teacher felt self-conscious to begin with but then it becomes a natural conversation with the student and it’s much easier and more enjoyable.

It’s important that the feedback is shared PRIVATELY with the student. They can then share it with their peers if they would like to- it’s the students’ choice.

Written threads on a message board are useful too for a record of learning and progress then can look back on but he uses audio feedback for the end of unit assessments.

Student generated online assessments (Video 4)

Medicles was developed- an independent website where students can undertake tutorials created by other students.

Bitesize assessment in medicine- things you could do in 15/20 minutes

You could test yourself or apply knowledge gained elsewhere

For instance- a labelled structure that would be labelled by typing

Students can test their knowledge, know what they need to develop and then their scores are stored as a progress chart

We opened it to everyone as we knew we couldn’t keep the site up-to-date on our own

The website is tech free- you fill out web forms to submit a resource and then the e-learning resource is created from the input you’ve made.

The students who started the site review submissions to give feedback about the quality and clarity of the tutorials. Other students are used to help with this review too- it remains a peer-to-peer site.

Doctors and medical students alike are really keen to get involved.

Using ePortfolios as a reflective tool (Video 5)

Having the reflections online means that more of a dialogue can take place throughout the year.

Semester 1- Pebble Pad shared just with the tutor

Semester 2- a template was created that mirrored the original paper template and this was linked to the requirements of the course. This was also connected to graduation and professional formation so students could see how it linked to future steps.

A student’s feedback- Advice and ideas were received from the tutor and also their peers.

We looked for activities where the reflective practice was naturally occurring- they supported learners with the technology aspects and then the reflective aspects.

The netiquette of the space is described as ‘thinking with your fingers’, which means that it avoids very formal, theoretical approaches from the lecturers on the PGCE programme- which can put the students off contributing.

Pebble pals and blogging buddies- trainee teachers can use this extra responsibility towards their teaching hours.

One student says that she is really looking forward to using the blog when she’s a teacher too as she’s really valued the reflection and the feedback that accompanies it.

Supporting PDF for themes raised in all of the above videos

Activities

Image available from here

5.1- Assessment of self-confidence

In a list of technology resources, I had not heard of the following:
Respondus, and Smart Sparrow

When teaching large classes, how can online assessments help teachers? (tick all that apply)

• Online assessments could eliminate the need for paper
• A flipped classroom strategy could be employed, where students complete online assessments before class, and teachers can spot those who need more help
• Marks could be more easily benchmarked across all teachers teaching the same course using an online grading system
• Cheating could be reduced by using randomised question banks that reduce the predictability of the questions
• Elements of the assessments such as multiple choice questions could be marked by the computer, saving the teacher time that could be used to give more in depth feedback and support to students who perform poorly
• Online assessments do not need to be collected or transferred from office to home and then from home to office
• Online assessments can be automatically submitted and students do not need to travel to the campus to submit the assignments

When using an online exam strategy, what kind of issues could arise? (tick all that apply)

• Computer marked online exam questions may limit students to close ended questions and might not help students demonstrate a depth of understanding in different aspects of solving complex mathematical problems
• Students could cheat by having someone else undertake online exams or activities for them
• A student could be interrupted by unreliable technology while taking the exam. This could delay or prevent them from completing it, especially if the exam is timed

What are the factors that stop you from conducting assessments online? (tick all that apply)

I responded that whilst I do run some online assessments, I don’t always know the best tool for the job and it can sometimes be time-consuming (at first).

In the table below, indicate whether you have undertaken any of the assessment activities listed using an online technology (either as a learner or a teacher). Indicate whether you think the online assessment related to the activity was effective or not. (tick all that apply)

• Extended Writing
• Presentations
• Online quizzes
• Essays or other written work

The above are all deemed to be effective. There was one that did not make it here and that’s ‘group work’. It has been the way I’ve set it up in the past I think but group dynamics are perhaps even more challenging in online spaces.

Some recommended resources for me to explore:

What is the difference between formative and summative assessment?
Core principles of effective assessment
Assessing Learning in Online Education: The Role of Technology in Improving Student Outcomes
On-line assessment
Educational design as transdisciplinary partnership: supporting assessment design for online
The Benefits of Using Online Assessments
Online Formative Feedback Assessments: Benefits for Learning
Discussion Board
Transforming Assessment
Assessment 2.0

Assessment 5.2- Determining effective online assessment strategies

What strategies might you employ to ensure that you have provided enough technical guidance for your students if you have asked them to use an online technology for an assessment task? (tick all that apply)

• Provide very clear instructions for every step of undertaking the assessment using the online tool, and follow them exactly yourself before giving the instructions to students
• Provide a link to the tool’s homepage where students can search for support themselves (this was deemed incorrect but the feedback was that it could be high stress for the student if this was all that was provided- I would provide this alongside other options)
• Allow students to practice using the tool on a non-assessable practice task well before the first assessment is due so that they can become familiar with it
• Have a backup plan that will prevent students being disadvantaged in case the technology fails when they are trying to complete the assessment task

Which of the following would you consider to be formative assessment? (tick all that apply)

• A written submission of intent where students outline their chosen topic and argument of an essay they wish to write
• An informal discussion about an assignment between a teacher and student (this was deemed incorrect but I disagree- it’s a great way of knowing where the students’ misconceptions lie and it doesn’t need to be formalised- this may just be my view though!)
• An assessable submission of a student’s sketches and research in progress for a design project
• An online quiz that is part of a flipped classroom strategy

Scenario: You are working for a company that trains emergency response teams for dangerous situations such as disaster relief, hostage situations, and search and rescue operations. The company runs a series of certified online courses which are supplemented by a number of intensive face-to-face workshops. You have been asked to measure the trainees’ achievement of the learning outcomes of the training program, using assessment tasks that will be conducted online after the final face-to-face workshop.

You may wish to consult these useful resources about designing assessment and choosing relevant technologies to help you complete this question:

For each learning outcome and activity listed below, choose the most suitable assessment strategy and technology that you think would best help the trainees demonstrate their achievement of the outcome in an online assessment from the ones provided.

Learning Outcome #1:
Demonstrate the most effective negotiation responses to be employed in a variety of situations during a hostage scenario, to bring it to a successful conclusion.

Activity #1:
Read and discuss case studies of successful hostage negotiations

Assessment Strategy: Select from the following one type of assessment that best measures the trainees’ achievement of the learning outcome

I chose role play because they can try out the scenario for themselves

Online Technology: Select from the following one technology that you think will best facilitate your chosen assessment

I chose adaptive quiz because they can select from multiple choice responses for each scenario to indicate how they would respond to each.

Learning Outcome #2:
Demonstrate successful application of approved emergency rescue procedures for a person trapped within a building that is on fire.

Activity #2:
Undergo online practical training exercises during workshops

Assessment Strategy: Select from the following one type of assessment that best measures the trainees’ achievement of the learning outcome

I chose rescue someone from a burning building (role-play) because this is most likely to assess their practical skill levels.

Online Technology: Select from the following one technology that you think will best facilitate your chosen assessment

Here, I chose 3D simulation as this would allow learners to do this practical activity using technology.

These are my recommendations based on my responses (I don’t know whether they were correct or not- no feedback this time!)

Supporting Online Students: A Guide to Planning, Implementation, and Evaluation of Services
Selecting Assessment Technologies
Peerstudio
Online assessment (looks comprehensive)
Web tools to support inquiry-based learning
Using Technologies to Support Assessment
Web-based peer assessment: feedback for students with various thinking-styles
Examples of Formative Assessment

Activity 5.3- Strategy

Q. Do you think that an online assessment strategy might be relevant to your own teaching? If so how might you use online assessment in your own teaching depending on the sector in which you are teaching [e.g. K-12, ESL, Higher Education or other) and what would be the benefits?

A. Online assessment will prove vital as my classes move to having 30 mins of their allocated learning time based online from September. I have been trialing tools this year and will need to put them into place from September. Videos and quizzes embedded somewhere they can be tracked (EdPuzzle or EduCanon) will be vital. I can use videos I have made previously, some I source elsewhere and some I create such as those related to my commentary on sample essay responses.

I have used video feedback before and this will certainly be something I continue.

I have used forms before and these will provide a bulk of my assessment as I can use Flubaroo to self-mark some of them and for all questions- I will have a record of responses. I will consider how I can tailor the feedback that is sent to students when they get it incorrect so that it’s as detailed and helpful as possible.

Q. If you were to adopt an online assessment strategy in your teaching, do you have access to suitable online technology to facilitate it? How might you find out?

A. I am quite up-to-date with technologies available to me. I wish to continue with online feedback and have used tools before which I am unsure will still be available on the college network so I need to do some testing and then perhaps some research to ensure I have a solution that can be used easily by us all. Conveniently, I share an office with the learning technologies manager so he can definitely help!

Q. Consider an assessment task you are familiar with. Are there opportunities for student feedback during the assessment task or only at the end? Are the assessment methods effective? Why, why not?

A. With all assessment methods I use, students can generally just provide feedback to me at the end. Some may email/G+ me if they have issues with it or questions about how to use it but quite often, I’ll know something has been ineffective due to loq numbers of students completing the task.

In the case of EdPuzzle- I believe it to be effective as it provides immediate feedback in many cases (and not long afterwards- when I have added comments in other cases). Issues have arisen with learners being unable to open it inside college and forgetting their login details/code to join the class. This year, I have checked the site is fully accessible inside college and I will be asking all learners to use their College Google account logins for all tools so that there’s just one set of details to remember. I will ensure the code is shared everywhere!

Q. If your answer was no to the previous question, consider at what stage and how your students’ learning could be assessed. Are there opportunities to incorporate self and peer-assessment into the assessment task you’ve just written about?

A. I realise I have misunderstood the question above. The online assessments named above do not provide any opportunity for assessment during. I could incorporate some confidence ratings in EdPuzzles or Google forms as a method of self-assessment throughout. It would still be difficult to incorporate elements of peer assessment into those though…

When I use Google docs for independent writing/text analysis or the students write on their blogs or even when they create infographics/presentations/posters, these online assessments integrate self and peer assessment well.

Discussion Forums

Image available from here

In a discussion about how to conduct online assessment in maths- wikis and blogs were suggested for students to show their working out and I offered Go Formative, a tool Matt Miller shared at our recent #ReadTL15 Teaching & Learning conference where learners essentially respond using an online whiteboard.

https://goformative.com/

In another discussion about the uses of e-portfolios, I posted the following comment:

Hi everyone,
What tools would you recommend for the creation of E-Portfolios (aside from Google sites or a blog site)? Could something like Educlipper work?
https://educlipper.net/main.html

I look forward to the responses!

In a thread about how to provide comments on a video submission sent by a student, I posted the following:

Hi Jelske,
Using a free tool like EdPuzzle could work perhaps?
This is a tool you can use to add questions and comments to videos- it draws from places like YouTube, Khan Academy and Vimeo but you can upload your own videos too.
You upload/locate the video the student has shared with you and then you can click through to add what you need to- comments for feedback or perhaps questions to check understanding. You cna then publish the video publicly or privately and then share the link with the student.

I haven’t used it for feedback before- just as an online assessment but it’s occuring to me that it could work quite well!
https://edpuzzle.com/

Also, if the student shares their video via YouTube- either privately with you or unlisted then you could use the comments box at the bottom to add your feedback and mark etc.?

Hopefully this helps and I might use EdPuzzle in the same way now!