BadgeRank and BadgeScore: OpenBadges value

CC Reconocimiento – CompartirIgual (by-sa)



Author: Jordi Moretón Galí

Idea and ReviewFlavio Escribano

Spanish Version / Versión en Castellano Download

Recently, and following the kickoff of BadgeCulture project, we asked ourselves: ¿what’s an OpenBadge worth? That stroke us as a relevant question, since an OpenBadge is usually linked to educational content, and this is a field being greatly disrupted by new learning ways and models (MOOCs, i.e.), making societies to consider what is the value of all those credentials out there that allow the certification of knowledge, skills, experiences, etc. For example, if I’m looking for a software developer, how do I value, initially, a candidate that has a major degree in Computer Engineering from the University of Barcelona versus another that has some digital certificates (OpenBadges, i.e.) of one or two MOOC in Computer Science from Stanford University taken via Coursera? Ultimately, our intention here is to start a classic question but in a new playfield (digital) and vehicle (OpenBadge).

BadgeRank y BadgeScore. Fuente: elaboración propia
BadgeRank y BadgeScore by Flavio Escribano []
With this post we want to turn public this question and our initial inquiries about how to calculate the value of an OpenBadge and the resource (usually educational) linked to it, in a time when we see the necessity to get an “Score” that allows us to, first, navigate through the large volume of available OpenBadges and the linked resources, and second, the get recommendations of new OpenBadges through that same “Score” factoring in all the already achieved credentials in our profile. We think, too, the metadata model under OpenBadges will need to be modified in order to support this characteristic, and the visual anatomy of it should be able to show that “Score”.

We detect a two-fold problem:

  1. The need for an indicator of the intrinsic value of an OpenBadge, which reflects the value, visibility or absolute importance of it, and therefore the course, or another source of said OpenBadge. This we call BadgeRank.
  2. The need for an adaptive value of an OpenBadge in relation to the user profile. This we call BadgeScore and reflect the relevance of this OpenBadge in relation to the user need. Take for example two typical users, the eternal student and the employer:
    1. The student has difficulty in guiding their learning in a context in which not only matters the formal education, but also non-formal and informal. This context raises a large volume and variety of learning resources and many learning centers, physical or virtual. The difficulty of choosing both the learning itself and the center where it’s taught, and the few tools available to the user are based on assessments of their social environment, studies focusing on universities, personal preferences, etc.
    2. The difficulty that the employer has in assessing/valuing the CV of candidates. If I’m looking for a java developer, ¿how much is worth a major degree in computer engineering versus a java MOOC at Udacity? Which of those candidates is the right one according to their credentials?

The BadgeRank is the reference value that will be used to calculate the BadgeScore, since this one will adapt to the user query and profile, and will allow for ranking the search results. Both BadgeRank and BadgeScore will be linked to OpenBadge.

Use Case:

Year 2020. A Mathematics major by University of Barcelona, with a CV composed by the major OpenBadge and other ones for each subject, wants to specialize in algorithm programming for financial markets. The system will take into account his major, the programming subject he already took in there, the specific university, that he doesn’t have much knowledge about financial markets (he doesn’t have any OpenBadge certifying it) and that he wants to specialize in a very specific field that requires both complex algorithm development and knowledge about financial markets behavior. The search system understands all those parameters and lists the results ranked by BadgeScore (it could also show the BadgeRank, to get across that the result is personalized according to the BadgeScore and the user profile):

  1. Algorithms for financial markets for mathematicians course at Univ. of Barcelona – BadgeRank:6,5, BadgeScore:1000
  2. Financial markets for scientists course at Univ. of Barcelona – BadgeRank:6, BadgeScore:800
  3. Finance and Economics major degree at Harvard – BadgeRank:9, BadgeScore:400
  4. Software development course at Harvard – BadgeRank:8, BadgeScore:250

So, the system recommends taking a course that, even if it doesn’t have much visibility or intrinsic value (BadgeRank), to the user profile is more relevant (BadgeScore) than other options.

Our proposal aims to identify the necessities to generate a synthetic ranking made of learning resources and it’s OpenBadges that allows any type of user, to search a concept, i.e. tropical medicine, and lists the results ordered by relevance of the learning resource in relation to the searched concept and the user profile, what we named BadgeScore, and implement this as a visible one, unique and differentiated, in the OpenBadge visual structure and metadata model.

We think we need to explore concepts from different areas to understand the synergies to our proposal:

Badge Pathways.
Badge Pathways. Via:

We have detected computational ways for the generation of rankings from data graphs, from information content calculations, from semantic relations, from the aggregation and analysis of social networks, etc. In general, we think the philosophy we are thinking about is similar to those of webpage rankings, like PageRank. There are many papers that describe how to get rankings in such a fashion, like this, this or this.

Page Rank de Google. Fuente:
Page Rank de Google. Fuente:

We believe OpenBadge, once interlinked by pathways, will generate a network of resources that will make possible to calculate rankings and establish new ways of assessment (BadgeRank) and help decision supporting in the search of new learning ways, search candidates for open positions, etc. according to querying user profiles (BadgeScore).

We open up the discussion, first, about the value of our idea, and if there is, we open also to the community the research line about scoring and OpenBadges.

2 thoughts on “BadgeRank and BadgeScore: OpenBadges value

  1. Hola, una pregunta:
    No me queda claro cuáles son los elementos que influirían en el badgerank de un badge…¿el prestigio de la entidad que lo emite? pero entonces el problema es que no hay métodos objetivos para medir ese prestigio, no?
    Gracias, un saludo.

    1. Muchas gracias por tu comentario Mercè,
      como bien observas en el artículo no se especifica demasiado, quizá precisamente porque es ahí donde radica la complejidad de la propuesta. Sugerimos un sistema parecido al que usa Google para el ranking de las webs que indexa, pero obviamente en el caso de las instituciones, valoraciones de usuario, peso en el ecosistema, prestigio del profesorado, etc. Están implicados procesos de digitalización, inclusión y cruces de bases de datos desde una infinidad de fuentes. Hay plataformas como Linkedin, Academia e incluso Facebook que ya contemplan sistemas de grafos con ranking para instituciones e individuos, por lo que sería una posibilidad procesar dicha información (en caso de que quisieran colaborar o abrir esos datos). Por otro lado está la algoritmia que se debe diseñar con el fin de distribuir los pesos de una forma coherente entre los datos que deben asociarse a un Badge en concreto.

      Pensamos que por eso necesitamos generar un equipo multidisciplinarm y así lo hemos sugerido en:!topic/openbadges/l09Cm93IXxk

      Espero que hayamos respondido a tu pregunta.

      Saludos y gracias por tu aportación.

Comments are closed.