Late thoughts on the Open Ed 2010 conference

Last two months have been so busy that I haven’t written anything on this blog. There is one blog post that I planned to write already weeks ago. It’s a summary of the Open Education 2010 conference that took place in Barcelona. Several participants have already shared their thoughts about the conference: Ismael Peña-López, Molly Kleinman, John Robertson, Brian Lamb, Scott Leslie, Pieter Kleymeer, Paul Stacey and Vic Jenkins. There is also a wiki page and a Cloudworks site about the conference. In my post I want to point out some of the presentations.

One of the keynote speakers Wim Westera talked about the next steps in OER strategies (presentation). On of the important issues that must be solved is the financing model. Westera argued that micro pricing model where people pay for views or downloads doesn’t work in education. Instead of that we should look at the cross-funding. He proposed 8 cross-funding schemes (see slide 30). In my opinion the most promising models are the freemium model and the rock concert model. We can see the freemium model working quite well in Flickr where the pro account costs $24.95 for a year. Some other services like SlideShare have failed to provide a reasonable price for ad-free accounts ($19 monthly). I wonder what could be additional pro services in LeMill that would attract active authors to pay a small fee that helps us to cover the hosting costs (detailed statistics about the use of their resources, management of exercise results, …).

Another keynote presenter Erik Duval was talking about removing friction in open education. His presentation made me think about large numbers. GLOBE has reached more 1 million learning objects. This is a great result, but what is behind that number? How many of these resources have a Creative Commons license that enables educators to improve and remix the resource? Are these resources somehow connected to the authors? In many repositories the resources are not published by the actual authors. I think that in case of user generated content it is important to maintain these connections. This is something that the OER community must work on. LeMill has 24 000 resources but unfortunately we have some technical problems with our OAI-PMH interface and these are not available through GLOBE yet. If we get this fixed then LeMill resources would make 2% of all OER’s available. Erik showed also an interesting comparison data from dataTEL initiative (see slide 50). We have to make LeMill data also available for dataTEL community.

From the conference presentations I really enjoyed Rory McGreal’s talk about approaches to OER course development (article). He listed 13 suggestions for developing OER course content. I find his suggestions very relevant for Estonian context. In recent years we have had several large project where lecturers develop online course modules. However, the quality of these modules varies a lot and often there are technical limitations for reuse and re-editing (source files not available, etc).

A lot of interesting things are going on in University of British Columbia. Brian Lamb, Scott Leslie and Novak Rogic presented how they use open platforms in UBC. One wiki installation is used for the whole university and integrated with university login system. This has allowed them to reach critical mass of people who collaborate online. They have even hired a wiki gardener to manage the site. Instead of developing their own platforms they rely on major open source platforms (MediaWiki and WordPress) and add/develop the needed plugins.

Another wiki-related presentation from UBC took place in Mozilla Drumbeat Festival. Jon Beasley-Murray talked about a course project where his students were writing Wikipedia articles. Three of the articles were finally featured on the Wikipedia main page. I should start improving the Estonian Wikipedia in my courses.

Brian Panulla and Megan Kohler presented an ontological model for exchange of assessment rubrics (article, presentation). I have experienced it in my own teaching that coming up with a detailed assessment criteria is more difficult than creating/finding/adapting the learning content. Many educators would benefit from a sharing platform of educational rubrics. Actually in Estonia we have a platform for creating and sharing rubrics for primary and secondary education. Estonian teachers have created more than 250 rubrics that are published under Creative Commons BY-NC-SA license.

Nathan Yergler presented a new search engine prototype that is developed by Creative Commons (article, slidecast). DiscoverEd search engine provides only open educational resources as results. Currently the results come from OER Commons, Connexions, Open Courseware Consortium and several other repositories. The number of indexed content cannot be compared with GLOBE but the results should include only OER’s. I hope that the user experience of this search engine will be in a same level with other Creative Commons tools when it will come out of the prototype phase.

Two interesting projects come also from the Open University UK. Anna De Liddo presented qualitative data analysis tool Cohere (article, presentation). Cohere is a web-based tool and it is specially designed to annotate and analyze web data (blog and forum posts, etc.). In the paper they analyzed one course that took place in P2PU. Cohere had very nice network and cluster visualizations of memos and participants. This work is very interesting for me because we have developed simple course visualizations in EduFeedr. This should be also interesting for my Finnish colleagues who are planning to do a visual knowledge building tool for the next version of FLE.

Another really promising project from OU UK was presented by Simon Buckingham Shum (article, presentation). SocialLearn is a learning community that is designed by the principles of Web 2.0 and social software. Every member has a dashboard with gadgets and a user profile. It is possible to form communities, make connections and follow other people in SocialLearn. The site will be launched later this year.

All together I am really happy that I had a chance to participate the Open Ed conference. There were quite a lot of interesting presentations and probably I missed some good presentations in parallel sessions. Unfortunately there were only five participants from Eastern Europe. If I look at the use of LeMill I think that open educational resources have a big potential in these countries. It would be good if the Open Ed conference would take place in Europe every second year.

Proceedings of the conference are available in UOC repository and can be downloaded also as a complete PDF book.

Posted in Conferences | Comments Off on Late thoughts on the Open Ed 2010 conference

EduFeedr presentations at Open Ed 2010 and MUPPLE10

I have presented EduFeedr in two conferences recently. The first presentation took place at the MUPPLE10 workshop that was part of the EC-TEL 2010 conference. This presentation and article was focused more on the technical implementation of EduFeedr. The paper “Aggregating Student Blogs with EduFeedr. Lessons Learned from First Tryouts” (PDF) was published in CEUR-WS.

The second presentation was at the Open Ed 2010 conference. This was a more general presentation of EduFeedr and its use on open online courses. I will embed here the slides from this presentation. The paper will be hopefully also online soon.

Both presentations received feedback from the audience, especially the presentation at the Open Ed conference. I will summarize here answers to some of the questions.

To what extent are we going to extend EduFeedr? We have agreed on two limitations that won’t allow EduFeedr to grow into a bloatware LMS. At first only the facilitators have user accounts in EduFeedr. This means that we cannot authenticate the participants after they have enrolled to the course. Still, there will be some activities that they are able to do anonymously. The second limitation is that EduFeedr works without any plugins on the course blog or participant blogs. Most of the learners use hosted blogs that do not allow plugins. This eliminates more advanced assignment workflows. My colleagues are also working on a little bit similar project where they have developed a WordPress plugin LePress for giving and submitting assignments.

Why havent we used PubSubHuubbub or FeedBack? As far as I know, both technologies require plugins on blogs. Therefore we cannot use these because of the decided limitations.

Why do we follow main comments feed instead of separate comment feeds for each blog post that belongs to the course? This is a good question. By following separate feeds we would not aggregate comments that are not related with the course. The downside of this approach is the increasing number of feeds that we have to follow. In a typical course every participant will make 10…15 blog posts. A possible solution is to aggregate comments for older blog posts less often.

How well does EduFeedr scale for large courses? Our current courses have a moderate number of participants (max: 32, total: 152). We have also tested with data from a course with 70 blogs. I would say that current user interface solutions (for example progress diagram) would become a limiting factor sooner than the performance of the system. However, not all people who sign up for massive open online courses are using a blog. For example on PLENK 2010 there are about 1200 participants and 200+ blogs.

Is it possible to install your own instance of EduFeedr? Yes, of course. You can download the source code from the development site. However, the installation procedure is not really straightforward. In addition to EduFeedr you have to set up EduSuckr web service that is actually aggregating the blogs. We would be very happy, if you would use EduFeedr on Then we have a better overview of ongoing courses and can identify problems more easily.

EduFeedr could aggregate information about open courses from other EduFeedr instances. This is a great idea! Thanks! However, I’m afraid that it will take time until we get that far.

Why we don’t host EduFeedr development site on some open community such as GitHub? This is a good suggestion. Currently we are using our own Trac and SVN that are connected with user accounts in our department. Therefore it is not easy to involve other interested people to the project. I will discuss this with our developers.

Why are we planning to evaluate the usability with think aloud testing? I do not have a comprehensive experience in think aloud testing but in few cases this method has worked well for us. We give a rather open task for the test users and use Silverback to record the sessions.

We should try to differentiate between the usability and usefulness when doing the evaluation. This is a good comment. I will take it into account.

On the development side it has been quiet in recent weeks. We released a bug fix release two weeks ago and now the developers are busy with another project. I hope to release EduFeedr 0.6 before the Christmas. This release should have a new method for connecting blog posts with the assignments and comments with the authors. Also we are planning to have participant profiles that contain all the blog posts and comments written by the person.

Posted in Conferences, EduFeedr, Presentations | 2 Comments

Presentation in the doctoral seminar

Last week I had a presentation in the Media Lab doctoral seminar. I have been busy with one project deliverable for the last few days and managed to upload the presentation slides (with a few minor edits) just now.

I received quite useful feedback for my work in the seminar. I had a feeling that my plan about combining the articles/projects together as the “open education ecosystem” made sense for most of the people.

Some suggestions:

  • “Designing the Open Education Ecosystem” is too ambitious title. I have designed only some tools to the ecosystem. Also, digital ecosystems have a certain degree of self-organization and cannot be completely designed. Therefore “Desining for the Open Education Ecosystem” would be more suitable title.
  • The 6th article should be written as the introduction chapter of the dissertation. Five articles would be enough for the dissertation. Later the introduction chapter could be also summarized as an article.
  • The main research question “What are the pedagogical, technical and social considerations for designing educational tools for the open education ecosystem?” is too wide. However, finding the balance between the pedagogical, technical and social constraints seems to be the challenge that I have had in each project.
  • I received a good hint that I can use “layers” in my digital ecosystem terminology. Etienne Wenger at al have written a book “Digital Habitats” where they have some ecosystem diagrams with layers. My colleagues in Tallinn have been critical about using layers, because it is not part of the natural ecosystems. I am not sure how far I should go with using the natural ecosystem analogy.
  • I should look at the ‘practice-led research’ method to strengthen the design methodology part.
  • People are missing from my layer images. This is something that must be there when the the three layers are merged into one diagram.
  • Some people were not sure if “social” is the best name for the third layer. At the moment most of the items in that layer are connected to openness. There was a joke to call it the “leftover layer”.

My colleage Teemu Leinonen has proposed a research-based design methodology (only presentation, full article is not available online) where software prototype is seen as hypothesis. In the beginning of the design process the hypothesis is quite fuzzy and it will become more clear during the stages of the design process. For example in LeMill project our hypothesis was that “an online service with learning resources that can be edited and improved by others with tools for social networking and matching of interests among the participating teachers could enhance sharing and collaboration around learning materials“. I have an hypothesis for each case but the main hypothesis can be formulated when my understanding about the open education ecosystem is more clear.

Later we discussed with Teemu about the target audience of my dissertation. I think that the main target will be the open education community. I have learned a lot from the design community but my dissertation will not contribute back so much to the design community. My second target group would be the technology-enhanced learning community, especially now when they are starting to see the importance of design and openness.

Posted in Doctoral research, Presentations | Comments Off on Presentation in the doctoral seminar

Update on my doctoral research

This Thursday I will have a presentation in the Aalto University Media Lab doctoral seminar. It is more than two years since my last presentation in the doctoral seminar and my ideas have developed further. Instead of focusing only on LeMill project and collaborative authoring of open educational resources I have taken a wider approach and look at the open education as an ecosystem. The current working title of my dissertation is…

Designing the Open Education Ecosystem

I am planning to do an article based dissertation that contains six papers. Four papers will focus on the projects and concepts that I have developed during my doctoral research. Last two papers will focus on the idea of learning environment as a digital ecosystem. The projects discussed in the dissertation have been developed together with my colleagues from the Learning Environments research group at the Aalto University Media Lab and Centre for Educational Technology at the Tallinn University. However, I have had a leading role in the design process.

Article 1: Progressive Inquiry Learning Object Templates

In most cases digital learning resources are used for individual learning (reading, looking, playing, quizzes) or by teachers in their class-room or online teaching (presentations). In PILOT project we are arguing that learning resources should be designed and presented in a special way in order to promote truly social constructivist learning. The project is based on the concept of progressive inquiry learning object templates (PILOT’s). These learning objects support progressive inquiry knowledge building process in computer and database supported Knowledge Building environments, found for instance in Fle3 virtual learning environment. Design research methods such as participatory design and scenario-based design are used in the project to generate distributable and reusable PILOT’s.

PILOT is the first concept that I developed when I started my doctoral studies. This was still before the rise of Web 2.0 and at time I was not thinking about learning environments as digital ecosystems. However, now I see PILOT learning resources and Fle3 learning environment as part of the ecosystem.

An article about the concept is published as a journal paper:

Põldoja, H., Leinonen, T., Väljataga, T., Ellonen, A., Priha, M. (2006). Progressive Inquiry Learning Object Templates (PILOT). International Journal on E-Learning. 5 (1), 103-111. Chesapeake, VA: AACE. [Online]

Article 2: LeMill

My most important project is the open educational resources authoring community LeMill, which has over 14 500 members and over 13 000 reusable learning resources, all created by the community members. The design of LeMill has tackled numerous challenges that hinder the authoring and sharing of educational resources by communities of teachers. The information architecture of LeMill scaffolds authors toward collaboration and sharing. The licensing scheme encourages reusing and remixing of educational content. LeMill is also a multicultural and multilingual community with teachers from more than 60 countries and learning resources in more than 40 languages.

Ecosystem idea is strongly present in the design of LeMill. Learning resources can be created by reusing and remixing content from other Web 2.0 environments. PILOT concept presented in article 1 is developed as one learning resource template in LeMill environment.

In LeMill project I was responsible for the conceptual design and interaction design. An article about our design solutions behind LeMill is published as a journal paper:

Leinonen, T., Purma, J., Põldoja, H., Toikkanen, T. (2010). Information Architecture and Design Solutions Scaffolding Authoring of Open Educational Resources. IEEE Transactions on Learning Technologies. 3 (2), 116-128. IEEE Computer Society. [Online]

Article 3: EduFeedr

In order to promote LeMill environment and authoring of open educational resources we have organized several open online courses for teachers. In these courses every participant creates their personal learning environment. Typically learners use a blog to reflect on the course assignments and various Web 2.0 environments to create content and to communicate with co-learners. One of the challenges in this kind of courses is to follow all the learning activities that take place in blogs, wikis and various Web 2.0 environments.

EduFeedr is an educationally enhanced online feed reader that is specifically designed for following open online courses that take place in blogs. Currently I have published workshop papers about the conceptual design (PDF) and technical implementation (PDF) of EduFeedr. The public beta of EduFeedr was launced in September 2010 and currently we are evaluating EduFeedr in several courses.

For the dissertation I am planning to write a paper that contains also the results from the evaluation of EduFeedr.

Article 4: LeContract

Another issue in open online courses is related to learner motivation. A lot of people who start open courses do not complete the course. Setting up personal learning objectives and strategy to reach the goals could keep learners on track. One of the ways to achieve this is to follow a personal learning contract procedure. In our current courses the students are using generic tools such as blogs to write learning contracts.

We propose that a special tool would scaffold and support the personal learning contract procedure. LeContract provides structural templates that define important parts of the learning contract. During the learning project the contracts can be reviewed and the achievement of individual learning objectives can be evaluated. Learning contracts created in LeContract could be embedded to learner blogs.

LeContract is currently in the phase of conceptual design. This work has been presented as a conference paper:

Põldoja, H., Väljataga, T. (2010). Externalization of a PLE: Conceptual Design of LeContract. In: The PLE 2010 Conference Proceedings. [PDF]

More information about LeContract concept can be found in the LeContract blog.

Article 5

The fifth article will analyzes multiple cases where teachers use LeMill together with other Web 2.0 tools to create and remix open educational resources. In this paper I’m planning to introduce the concept of digital ecosystems and describe various patterns that occur in the learning resource collections created by the teachers. The digital ecosystems approach enables teachers to become designers who create mashups with various online tools.

Article 6: The Open Education Ecosystem

The final paper will present a theoretical framework of the open education ecosystem. In that paper I am planning to draw connections between ecosystems in biology, open systems and digital ecosystems. In the seminar I will explain my current understanding about the three layers of the open education ecosystem and how the tools developed in my research fit into this ecosystem.

In my dissertation I will argue that the design of learning environments would benefit from the digital ecosystems approach. In this kind of open and personal learning environment every teacher and learner takes the role of a designer.

(Sorry about hyperlinks instead of proper references. This is a blog post.)

Posted in Doctoral research | Comments Off on Update on my doctoral research

EduFeedr – experience from first two weeks

It is now two weeks since we launced EduFeedr public beta site at During that time my colleagues have started four courses in EduFeedr. First two are master level courses, the third one is for first year BA students and the last one is an inservice teacher training:

The basic functionality of setting up a course and enrolling to the course is working fine. The facilitator must copy the blogroll code manually to the text widget in the course blog. In my courses I have updated the blogroll once a day if there are some new participants.

Since the participants do not have user accounts in EduFeedr they cannot change their details after enrolling to the course. Our reason behind this decision was to keep the system simple by having less users with accounts. We assumed that the minimal data that is asked during the enrollment (name, e-mail, blog address) does not change during the course. However, we have noticed that in few cases people have enrolled to the course twice to change their blog address. In that case the facilitator should remove one of the entries.

First weeks of testing have pointed out one important limitation. Currently it is not possible to aggregate blog posts only from a certain category (WordPress) or label (Blogger). Our first blog-based courses were inservice teacher trainings. For most of the participants in these courses created a new blog. However, with master students it is common that they will use one blog as their learning diary and have a separate category for each course. We hope to remove this limitation in version 0.6. So far the facilitator can hide posts from other courses manually.

Course feed page is quite useful for following all the blogs and discussions. However, for large courses it is not enough to display only 10 recent blog posts and comments. The facilitator should have an option to specify the number of blog posts and comments that are displayed. In the current version all pingbacks are also displayed in the comments column. In some of the courses there are actually more pingbacks than real comments, because we ask our participants to link to the weekly assignment in the course blog when they are posting their response. The facilitator can hide these pingbacks manually.

It is still to early to demonstrate good examples of progress chart and social network diagram, but the testing has indicated a few defects and limitations with these two visualizations.

This coming week we will be presenting EduFeedr in the MUPPLE workshop at the EC-TEL 2010 conference. After that we are planning to release version 0.5.1 with some fixes to small defects and user interface design issues. The next major release 0.6 requires more work and we are not sure when it will be ready. If everything goes fine we might release it before the Open Education Conference.

Posted in EduFeedr | Comments Off on EduFeedr – experience from first two weeks