It is a lovely rainy day today in St.Petersburg, which is perfect when you actually need to do some work (my room windows face west, so it can be really hot and uncomfortable in the evening if a summer day was hot and sunny). So WordCamp Europe contributor day which I mentioned in my previous post actually helped us realize an issue with the course organization (so the lesson here is: do beta testing as early as possible! It’s always helpful to get an opinion of someone from outside the group which is working on the project). I have written a detailed post on this in the official blog of the community/outreach team for the WordPress open source project for us to be able to understand how should we proceed. You’re welcome to check it out! 🙂
Time goes by really fast and I’m writing this sitting in the Vienna Schwechat airport waiting for my plane as WordCamp Europe is already over. Those days were filled with awesome people, events and things, but per best instructional design practices 🙂 I’ll focus on the top three things I really enjoyed about WCEU.
1. WordCamp Spirit
As a newcomer I was smitten by such a chill, low-key, welcoming environment. Even though that wasn’t quite a regular WordCamp, but the biggest in Europe (check out a panoramic image WCEU shot at Q&A with Matt Mullenweg – whoa, that’s a lot of people!), it still felt cozy and open for everyone, regardless of the experience level. And all the automatticians (employees of Automattic, the company behind wordpress.com) were really super-nice and invited me tag along despite the fact that I was overwhelmed and stricken with introversion most of the time 🙂
2. Contributor day
After two days of listening to the speakers it felt really good to move to action! Not only it was amazing to see five hundred people willing to contribute to WordPress and get to know what are other teams doing, but we also had a chance to test the WordCamp & meetup organizer training courses which I was working on last month. And there was a lot of useful feedback which we’ll use to improve. You can take a look at the document which summarizes what we found out here.
Three of the most important findings are:
- We need to agree on the way content is going to be synced between organizer’s handbooks and the training site.
- The navigation of the course is to be improved with some custom fixes like removing the links we don’t want to be clicked, forcing the links to open in different window and generally making sure learner understands where & why to click.
- The timing estimations should be added so that people would understand how long will them take to go through the training – especially given the fact that WordCamp Organizer course will be a long one.
WordCamp Europe 2016 really was the kind of conference that was open to everyone, regardless of experience with WordPress, with a diverse range of topics. This was really helpful to me to get a better perspective in what is going on in the community and learn from others’ experience. I have selected three talks I was most impressed with – and you can watch them as well at WordPress.tv:
- My friend the Impostor Syndrome by Sonja Leix. A lot was said already about the impostor syndrome and techniques one could employ to cope with it, but this honest, brave and deeply moving talk is still really worth watching.
- WordPress: the early years by Mike Little. This one is a co-founder’s view of the early years of WordPress which is not-surprisingly very interesting.
- Code is Poetry: A Musician’s Tale by Helen Hou-Sandí. The shameful thing is that I haven’t actually seen this one as I was in another track but EVERYONE says it was awesome and features a real piano so you can’t miss this – and I will definitely watch it.
To sum up, I think the biggest “take away” for a myself as a first time WordCamp attendee is about how big and generous the WordPress community is. And that even one with not much expertise in coding can become a part of this community and start contributing. What’s required is really just some persistence, a tiny bit of bravery and willingness to learn and explore. And I do have all of that 🙂
The last week I was working with a newly created WordPress Meetup Organizer handbook. The training as it is now can be found here. As you can see, the pages were moved to the testing site and small quizzes were added. You are very welcome to try it out, and if you have any feedback regarding the way it works please feel free to let me know.
Some changes we will implement later are:
- Replace the links to the handbook materials with the links to the equivalent training site materials – after it will be decided where exactly will this site be hosted,
- Possibly expand the introduction and conclusion a bit.
Another alteration of the initial plan I didn’t mention last time is that we decided to have small quizzes after each lesson as opposed to larger quizzes at the end of each module, that is composed out of several lessons. The reasoning for that is that Sensei is designed in the way that if you want the “composed” quiz, all of the questions have to be linked to the module’s last lesson – which can become a disadvantage if you decide to move one of the lessons to another module or delete one of the lessons — the questions appended to this lessons will get orphaned and end up impeding the learning experience. On the other hand, having all questions based on the lesson’s content linked to exactly this lesson enables a more flexible, chunking and reuse-oriented structure.
Next week I’ll be working on questions for the WordPress deputy handbook-based training. And I will also get to go to WordCamp Europe, which I’m so excited about! The whole trip is actually thanks to Outreachy Travel Allowance: as part of Outreachy internship, an intern has a $500 travel allowance, which can be used to travel or gain admittance to a relevant conference or meeting. And I’m really looking forward to seeing community members in person and witnessing all the WordCamp glory and bonanza in real life, not just imagining it while reading the WordCamp organization-related materials. So please expect a full report about this trip next week 🙂 And now I have a question to other interns who are reading this blog 🙂 – what conferences/events related to your internship do you plan to attend? Will it be soon? Will you get to travel?
The last week we continued with WordCamp Organizer Handbook-based questions: the second half of the question pool was composed, reviewed and moved to the testing site we’re working on for now.
This actually raised a couple of questions on the quiz settings:
- What should be the passmark percentage for the quizzes included in the course?
- Should a learner be able to see which questions she answered incorrectly?
- Should a learner be given any further specific feedback based on the option she chose?
For question #1 it was decided the passing rate for all of the quizzes would be 100% because it’s really important that deputies/WordCamp & meetup organizers have mastery over program materials. That might force the learner to read the lesson not once, but twice – but that way we will be sure the most important points were all covered and hopefully understood.
If some of the questions from the set are answered incorrectly, we will ask a learner to retake the quiz. Given that, I thought we should really make sure that it is clear for the learner in this case which of the questions are answered incorrectly (then she can read the lesson again paying attention to the details related to this question). I recently had to pass a quiz myself where only the percentage of correct answers were shown, which made it hard to understand what was correct or not. It even involved some combinatorics. 🙂 The plugin we use, Sensei, unfortunately, does not allow retaking quizzes and displaying the questions answered incorrectly at the same time, so we had to add some custom functionality here. At the moment, the answer notes is pretty rough and displays overlapping text, but that’s something that we are going to remedy once we have all the content in place.
Regarding question #3, we toyed with the idea of having specific feedback to the questions options: i.e. when a learner selects a correct option there is something like “This is correct. <A rephrasing of the correct answer>”, and if not “No, this is not correct. <Explanation why not>”. The advantage of having feedback is that learner would get to understand better why they are wrong immediately, and not feel confused. There are also disadvantages: they won’t be likely to go and reread the text to try to understand why they are wrong, which may limit the understanding. And it actually turned out that with Sensei it’s only possible at the moment to have one feedback item shown no matter what the chosen option is. That lead us to leave feedback-related plans for now.
All in all, I think it’s a really nice compromise between keeping the main idea of what was originally planned and adjusting it to the way the e-learning plugin we use actually can do things.
Next week I’ll be working on the new content added to the Meetup organizer handbook. The flow will be similar to the way the other two were/are being handled:
- Create an outline based on the way materials will be organized
- Move the handbook content to Sensei
- Start creating questions
Which of the following was my main activity during the week 2 of Outreachy program?
A. Composing the outlines for the trainings for WordCamp Organizers and WordPress Deputies
B. Creating quiz questions for WordCamp Organizers training
C. Chilling at the beach and eating strawberries
And the correct answer is B! (You could probably tell. And we don’t even have a beach where I live.)
You may remember in my last post I mentioned I like writing questions for quizzes quite a lot. Here are some of the rules I use when composing them. But first, let’s mention the anatomy of the question.
When does it make sense to book an unusual venue for your WordCamp, such as a public aquarium? (question stem)
A. You want your WordCamp’s “underwater” theme to be truly memorable for the participants. (distractor)
B. Your connections through the meetup allow you to get the venue for free. (correct answer)
C. No other WordCamp has been held in an aquarium – you’ll make history! (distractor)
D. It is the only one that is available for the date you have in mind. (distractor)
Starting with the rules relating to the question stem:
- Focus the questions solely on the material covered in the course. The goal here is to help the learners retain key material and assess how well they master it, not to make them feel stupid or trick them.
- Try to keep the wording clean and simple. It’s annoying to have to read the question several times only to understand what’s being asked.
- Follow the learning objective with your questions. It’s important to ask that people remember the exact answer only for the questions they absolutely need to know it according to the goal we have in the corresponding lesson. Otherwise, it’s better when they are encouraged to think.
Now, let’s discuss writing distractors:
- Try to keep the options about the same length, or at least do not let the correct answer to be the only “long” option
- Avoid ‘all of the above’ and ‘none of the above’. That can be really confusing, especially if the system you use will shuffle the options. If a question that has multiple correct answers is required, a multiple-response question is a better option.
- The distractors must be plausible. If a learner can choose the correct answer right away just because none of the other options make any sense, that will not help the learning process much.
Some of these rules are harder to follow than others, but it’s important to try 😺
Next week I’ll be finalizing quiz questions for WordCamp Organizers training and transferring them to the testing site we’re working on for now.
So the first week of Outreachy went by and now it’s time to talk about my project in a bit more detail. The goal of the first month of my internship is to create three training solutions based on the existing handbooks: one for community deputies, one for WordCamp organizers, and one for meetup organizers. Basically, this is going to be the same material as the handbooks are currently featuring, but reorganized and with quizzes inserted. The idea behind these changes is to increase the efficiency of the material by:
- Enhancing retention by making the learners practice effortful retrieval of the material they just read when answering the quiz questions
- Accenting the points we consider being the most important by asking questions based on them
- Providing the learners with an opportunity to self-check their comprehension of the material
And I’d like to get into a bit more detail on why the task I do is important for WordPress Community and why we think that creating courses based on content which already exists and is used by the learners is something worth doing in this case. The number of people getting involved within the WordCamps, Meetups and community deputy program grows every year, and WordPress needs to make sure they understand correctly what is expected of them, how to make the effort they are contributing as efficient as possible and how to ask according to the guidelines to minimize the disruptions and possible issues or miscommunications. The learning materials reorganization I will be doing is meant to improve the way the new volunteers are being trained. My mentor, Andrea Middleton, goes into a bit more detail on the problem in her blog post in Make WordPress Community blog post.
Given that, the first step was to analyze the material and the first deliverable was composing outlines. Organizing the materials and creating a course backbone out of them is a crucial step in a training delivery – we have to make sure course content is organised logically, the chunks of information are easily digestible and a learner is able to get a solid comprehension of the subject matter when linearly going through the material as suggested. So I was reading and re-reading the handbooks for a while, and after that attempting various ways to organize them in a way that would make the most sense for someone who will get to see the course for the first time.
The second part of the week I was ready to move the materials to the e-learning solution we are using – the WordPress plugin Sensei, while shaping it according to the outlines. With Sensei I’ve been able to figure out how to do most of the things I needed without the manual, so it probably says quite a lot about its usability 🙂 and with the issues I got stuck on, I got help from Andrea and Hugh Lashbrooke who actually has been a lead developer of Sensei for quite a while! Hehe lucky me. However, there were also some caveats. I have planned the outline to be three levels (Module -> Unit -> Lesson) but then it turned out Sensei only supports 2 levels. Which is definitely something I should have checked beforehand. Mea culpa. However, Hugh suggested a great workaround we agreed to use – “Course Category” will be used as the top level of the course entity, and what I planned to be “modules” will be “courses” in terms of Sensei terminology. Course categories work similarly to the standard blog categories, but for courses, and I’m really happy with the way this feature saved the day.
Next steps will be writing questions – and this is something I’m actually looking forward to! I really like figuring out how to ask a question on the learning objective we want to reach, and how to create distractors for the questions that would be plausible but wrong. Creating multiple choice questions is actually quite an art.
Overall, I’m really enjoying my experience so far. It’s nice to be learning something, and it’s absolutely awesome to have support from my mentor when I’m feeling a bit stuck or unsure how to proceed. So stay tuned, and I’ll be back next week, telling you more about how my journey will be unfolding.
I’m starting this blog to share my experiences being one of 41 interns accepted for May – August 2016 round of Outreachy – which I’m terribly excited and grateful for. Outreachy “helps people from groups underrepresented in free and open source software get involved”, so I’m looking forward to:
- Communicating with other women, trans and gender-nonconforming people, and people of color working in FOSS.
- Making an actual contribution to the open-source software world which we all benefit so much from.
- Share my experience!
I will be working for WordPress enhancing training and education material – basically creating and further improving the WordPress and WordCamps tutorials, workshops and online courses. It was really nice to get chosen for this project, as it actually highlights that open-source is not only about coding, but there’s also supporting work to be done such as graphic design, documentation, community management, instructional design, etc
I will be sure to share my tips based on the personal experience of applying to the program and stories of what happens next in this blog so stay tuned 🙂