Tuesday, March 25, 2008

Measuring Social Capital

Our interview with John Gardner (long mp3 file) as part of these case studies captured an interesting question from John regarding Social Capital -- he was interested in thinking about how to measure it (assessment being a driver to guide action).

Theron and I've been thinking about the question about the question.

Traditionally university extension has focused on getting knowledge from the university to the periphery, and we would have measured our capital by the success of that (mostly broadcast) model.

Web 2.0 has us thinking about how to also get the knowledge of the periphery in to the university, and measuring capital by two-way dialog.

I think John pointed to this in his talk (last December) to Crops and Soils Dept when he was exploring the implications of centrally produced ideas like Roundup Ready vs the understandings of local conditions (both growing conditions and human ecology).

Increasing sustainability seems to require increasing ability to adapt to niches, and that is likely to involve moving knowledge of successes from the periphery to other niches on other parts of the periphery. The university could play a role, but not likely a broadcast role (more like narrowcast) in this.

This does not answer the question of how to measure social capital, but does suggest that the measurement won't be based on Web 1.0 ideas

Wednesday, March 19, 2008

Learning Portfolio Strategy: Be Public

This is one of the "portfolio patterns" (borrowing from the "Pattern Language" of Christopher Alexander) we have distilled from our case study of users creating learning portfolios.

A strategy that users of a learning portfolio adopt is to be public on the Internet. The goal is to create and join public communities, and by interlinking, raise the “Google Rank” of the problem and the problem solvers.

Information Scarcity vs Information Abundance

In a conversation (long .mp3 file) with Dennis Haarsager, Interim CEO of National Public Radio, he described the Internet as "'anti-scarcity', it’s about information abundance. The way to obtain value is not in controlling a scarce resource, the value is to be had in the ability to extract value from the mass of information, by organizing it, filtering it, 'chunking' it" What he called an 'information [organization] theory of value.'

The opposite perspective is the value of scarcity. “While some industry pundits have proclaimed print-on-demand to be the future of publishing, there will always be a positional advantage to the conventional book. It says somebody thought enough of this writing to run off a whole batch. In sum … the Web will never destroy older media because their technical difficulties and risks help create glamor and interest. At the same time, however, the Web does nibble at their base…” Edward Tenner The Prestigious Inconvenience of Print, The Chronicle Review, B7-B8, March 9 2007. One of the values Tenner claims is that print content undergoes more review and scrutiny in the production process. If production cost, risk, and technical difficulties enhance the prestige of older communication media, one of the problems the traditional book still faces is lag times.

The role of the public showcase in the learning portfolio

In our interview, Haarsager argued for the public lectures he gives on his chosen problem. The lecture is a showcase portfolio of Haarsager’s current, best thinking. The medium is mostly broadcast, but he feels it allows him to reach new audiences, and to get kinds of feedback about his ideas that he does not get in comments on his blog.

Tamez is also creating showcase “mini-portfolios” in the form of printed fliers and media interviews. These productions may have some of the risk-related prestige that Tenner ascribes to printed books, while at the same time having the new audience-reaching and immediacy values that Haarsager associates with his lectures. In her learning portfolio, these mini-portfolios document where Tamez’ thinking was at points in her learning trajectory.

The learner who works in public, in order to gain any value from working in public, must participate in collaborative efforts to extract (or make) value from the information richness of the Internet. This kind of strategy has been called Learning 2.0 by Stephen Downes, who has created this diagram to describe differences between groups and networks as organizational strategies for learning (original sketch midway down this post).

Group (Learning 1.0)Network (Learning 2.0)
Groups require unityNetworks require diversity
Groups require coherenceNetworks require autonomy
Groups require privacy or segregationNetworks require openness
Groups require focus of voiceNetworks require interaction
Downes also suggests that this working in public is not necessarily "more" work, rather its an attitude to push the work you are doing for various more private audiences into a public arena. Tamez demonstrates a variant of this in her portfolio, where she adapted a strategy of cc:ing her portfolio when writing email.

Another value of learning in public is the assessment that comes from dialog in community. This assessment takes two kinds of forms: assessment of the idea, testing its merits, and assessment of the individual, in the sense of reputation and social capital. These ideas about assessment will be addressed in a later section of this analysis.

Link and be linked; Tag and “Digg” Making value among plenty

“But Google and its ilk notwithstanding, the sheer volume of information, its global origins, and especially the dynamic, real-time nature of information today is simply overwhelming our traditional, centralized institutions of information screening and management – whether research libraries, book and journal publishers, or newspapers and other news media.” Peter J.M. Nicholson, “The Intellectual in the Infosphere” The Chronicle Review, B6-B7 March 9, 2007.

Saving all one’s work in a portfolio creates the same problem of information saturation on a more personal scale. One challenge is to “rise above” the information to see the ideas, and from the ideas, rise to action.

In his video “The machine is (us)ing us”, David Wesch points to a consequence of the networking ideas Downes is suggesting. Wesch explores the implications of linking from one document to another. Linking provides a kind of metadata for the thing linked. It says that the two web pages share some relationship between them, but it does not say what that relationship is (e.g., citation, example, counter example, refutation).

Linking is one of the traits Google uses to determine page rank. As a result, a Google search is one means to perform a “rise above” on a large collection of related material. What the Google rank does is call attention to an item in the collection of documents. The weakness is that it points to a specific item, it does not present a gestalt across many items that could surface emergent themes.

Tagging is a mechanism for adding metadata to your own materials (e.g., blog posts or wiki pages) to organize them. It is also a mechanism for adding metadata to other people’s content, for your personal organizational purposes. Patterns emerge from group tagging of the same thing and related things. Tag Clouds are visual representations of the frequency of appearance of tags among a collection of documents, and serve as a way to see emergent patterns. The Flickr photo sharing site has a page of tag clouds, including one for this week and last 24 hours. As I write "ArthurCClark" is among the tags of the week, Clark having passed away recently.

Chirag Metha gives an example of extending the tag idea to an analysis of all the words in a text to show emergent themes. His site has an “aging tag cloud” analysis of US presidential documents. The engine has the usual tag cloud text graphical analysis, but Metha extends the tag cloud idea by trying “to figure out how long ago a given word hit its peak usage and brighten[ing] the recently used words while fading away words haven't been used in a while.” The result is a trend analysis over time of words used in American politics.

Other tools for rising above can be seen in Gapminder, which is a demonstration the Trendalyzer engine’s rendering of UN demographic data over time. By sweeping across time in the visual rendering of the data, one can find both big trends (the 3rd world is getting richer and having fewer children per woman) and local events like the genocide in Rwanda. These observations in masses of data can lead to hypothesis generation and exploration.

While the idea of “rising above” the data in a portfolio is an appealing one, among the cases we’ve examined so far, we do not see evidence of use of any these strategies to help the portfolio author guide their reflection or future action. The values of working in public seem to be coming from collaboration on a problem and assessment arising in that collaboration.

(Updated with new pp just below table on 7/15/08)

Friday, March 14, 2008

Goal for a Learning Portfolio: Solve a problem

This is one of the "portfolio patterns" (borrowing from the "Pattern Language" of Christopher Alexander) we have distilled from our case study of users creating learning portfolios

The goal for, and perhaps a defining characteristic of, a learning portfolio is to be a workspace in which to solve a problem. That is the feature that makes it like a Personal Learning Environment (PLE) and unlike the common showcase portfolio.

In an article in the Chronicle appearing March 7, 2008, James Barker gives an example of this type of workspace as a physical place, the architecture studio:

“In my view, the architecture design studio is the best learning experience ever invented to produce the kind of deep, engaged learning and creative graduates that are so needed today. Small groups of students work with a master teacher on a semester-long or yearlong team project to design solutions to a specific problem or to meet a particular need…

“For example, our students in planning and design have helped communities throughout our state preserve historic buildings, revitalize dying town centers, and plan new parks, bikeways, and green space. For every project, they interview the key people involved; gather statistics on demographics and traffic patterns; collect previous plans, deeds, and plats; photograph the site from every conceivable angle; and put all of those data on a computer.

“Eventually they brainstorm ideas, discuss them, refine them, and present them to their teachers and clients in a process that we, in architecture, call a "design charette." Then, and only then, are the best ideas sifted through the filter of what is possible…

“In the process of doing such public-service projects, our students learn about research, communication, interpersonal relationships, culture, politics, municipal government, creativity (its power and its limits), and teamwork."
The studio space in which this work is done becomes a walk-in portfolio, with sketches, photos, models displayed on every surface and open for comment by all passers-by.

Here are some examples of learning portfolio questions

A Learning Portfolio tracks growth over time

In order to be an aid to the learner solving a problem, the portfolio needs to track the artifacts of the work, and needs to facilitate and encourage the learner to look at their growth over time. As we will see, this same property of tracking growth will play a role in assessing the learning outcomes.

Carol Dweck writes in The Secret to Raising Smart Kids: "Many people assume that superior intelligence or ability is a key to success. But more than three decades of research shows that an overemphasis on intellect or talent—and the implication that such traits are innate and fixed—leaves people vulnerable to failure, fearful of challenges and unmotivated to learn.

Teaching people to have a 'growth mind-set,' which encourages a focus on effort rather than on intelligence or talent, produces high achievers in school and in life."

So what implications, if any, would these concepts have on the design of learning opportunities. For starters, what if the interface to view growth focused more on learning over time instead of grades on high stakes tests? We have been looking at tools like Gapminder (Trendalyzer) and Microsoft's Photosynth for inspirations of tools could allow learners to see patterns hidden within the data of their learning record. We are imagining a learning tool that would provide the student a way to view dynamic representations of their approach/efforts/ learning over time instead of a series of grades in a drop box.

Learning inside/ Outside the University
Self-directed vs Teacher Directed Learning
In Imagine there's no courses, Jeff Cobb points at George Seimens' World without courses (voice over slides) and each explore the question of how organizations and institutions measure and derive value from unstructured, informal learning activities. Examining learning portfolios has us looking an these issues and the ways informal learners (e.g. Hotz) create learning communities and tackle and solve complex problems and how they derive rewards from the learning gained.

Stephen Downes' writing on the 'ideal open online course' concludes that it would 'not look like a course at all, just the assessment.' He postulates, "were students given the opportunity to attempt the assessment, without the requirement that they sit through lectures or otherwise proprietary forms of learning, then they would create their own learning resources."

And Clay Burrell echos some of Downes' ideas when he writes about learning writing in a a blog (aka learning portfolio) situated in community and in context, "students would write self-directed blogs. No homework assignments allowed in terms of subject matter, though standards of style and conventions would be set... assessment would be based on readership, comments, subscriptions, visitor stats, Technorati authority ranking... self-assessment (italics added) and other non-authoritarian, teacher-gives-grades assessment styles." Here Burrell is rephrasing Downes "in a non-reductive system [of assessment] accomplishment in a discipline is recognized."

It is this latter behavior that we are seeing in some of the learning portfolios we are studying, coaches within the institution helping students learning outside of it but the measure of accomplishment is recognition by and among communities.

And this thinking leads us to rethink the relationship of the institution and its students to its alumni. For example, graduate student Dana Desoto met with Theron Desrosier and Jayme Jacobson recently about an alumni web site for the WSU School of Communications. This led to a good brainstorm about the goals of the project, the value for the university and for the alumni, and the assumptions we have about the relationship of the Alumni to the university.

The idea they developed was a collaborative learning portfolio that connects alumni and students with common interests and promotes the flow of intellectual capital between the Communications School and the professional community.

Value to Alumni:
  1. Alumni use comm. students as a economical source of innovation.
    1. Alumni propose projects for teams of students (build a web 2.0 marketing strategy for my company)
    2. Alumni who are looking for new employees get a more authentic picture of skills and abilities.

  2. Alumni are valued as more than a deep pocket.
Value to Students:
  1. Comm. students use alumni as a source of authentic activities, advice, connection to the profession, and feedback.
    1. Comm students build eportfolios around real projects as evidence for learning and/or hiring.
Value for Communications School:
  1. Comm School uses this symbiosis as a source of feedback on the alignment and relevance of curriculum, learning outcomes, and activity design.
  2. The professional community is a partner in the continual improvement of the program.
Who is the Learner/Who is the Audience?
Blurring the boundaries of the university by facilitating students working on authentic problems situated in communities outside the university, and assessing their work, not with reductive tests, but with the level of recognition and accomplishment the student achieves does something else. It blurs the line between the learner and the audience.

We are seeing this blurring in the portfolios we have studied. We are beginning to talk about learning communities where members play differing roles in supporting the learning growth of the whole. George Hotz honors this learning community when he credits his collaborators even as the national press is focused on him.

Case Studies of Electronic Portfolios for Learning

This is the first in a series of posts describing some work funded by Microsoft. We are posting in this format to invite reader comment and trackback. The work described below is an example of a learning portfolio, and this post is our problem statement.

Nils Peterson, Theron DesRosier, Jayme Jacobson, Gary Brown


We have written about students’ changing technology proclivities and the changing landscape for Learning Management Systems (LMS) in this Microsoft white paper for EDUCAUSE 2007, in JOLT, Innovate, this blog, and in this interview). This document begins a case study of learners who use electronic portfolios to advance their learning. It does not explore uses of electronic portfolios as “showcases” of best work. The latter uses are facilitated by ePortfolio tools in several of the common LMS products and in several widely used Student Information Systems whose common trait is to facilitate institutional assessment, not learning.

The kinds of uses of ePortfolios we are examining are closely aligned with Personal Learning Environments (PLE). What we are finding in the cases that follow are users implementing what is suggested in Scott Wilson’s Future VLE diagram; an ad hoc, assemblage of Web 2.0 components (the term "Worldware" applies to the components). (Scott refers to a “VLE” (virtual learning environment) which might be either a personal or institutional learning environment. For our purposes here, read Scott as proposing a PLE.)

One of the questions we are exploring in this work is the potential of Microsoft SharePoint 2007 MySite Subsites (WSS) to serve as the central building block in Wilson’s Future VLE, a hub for the learner, and potentially a collaboration and/or presentation space for the learner or learner and segment of the community.

In this document, we prefer to retain the term “portfolio,” rather than PLE, for these activities because we want to connect to a body of literature on portfolio practices, including the commonly offered mantra: collect, select, reflect, connect and project (into the world). We draw a sharp distinction between the learning portfolio discussed here and the “showcase” or summative portfolio, especially when the creation of the portfolio is at the request of a third party for summative assessment purposes.

We also prefer the portfolio language to that of PLE because we value the learner consciously leaving a ‘learning trace’ as they work on a problem in the space, and we see the capturing and sharing of that trace is an important part of documenting learning. A recent employer poll supports this bias for richer documentation of learner skills. Some of our interest in this work began by documenting the learning trace that is evident in Hotz’ blog of his collaboration to unlock the iPhone.

In addition to Hotz, we have been examining electronic learning portfolios created by students and professionals at Washington State University, conducting interviews of them, which were captured with audio recordings, white board diagrams and/or both.

Several themes arise from studying these cases:
  • Learning portfolios have a goal, or problem to solve;
  • They adopt strategies that are public;
  • They are implemented in multiple tools and spaces where collaborators are already present, or can be expected to congregate;
  • They understand Social Capital, and the portfolio practitioner seeks to develop and leverage it;
  • A key use of Social Capital and a reason to work in public is to develop an assessment community that can provide feedback and insights;
  • The portfolio, especially its repository strategies attempt to facilitate reflection and synthesis to move the learner (and community) from information to idea to action;
  • Users of Learning Portfolios work in multiple modes, including the arts, to convey their synthesis and call to action.

Other posts in this series can be found in this blog, under the tag Learning Portfolio.

Framing the Problem

This blog will attempt to emulate what we think we've learned from George Hotz, how to be a node in a learning community working on a problem. Our statements of the problem(s) we are working on are tagged here. We view this space as one element in our Learning Portfolio, and will link to other portions of our portfolio among systems we host and world systems we have adopted. From time to time we anticipate writing reflections on our use of this space and on our changing understanding of portfolios for learning.

Our organization, CTLT, is committed to the advancement of authentic learning—learning that takes place in and beyond the classroom; that encourages the exchange of knowledge across disciplinary, institutional, and national boundaries; and that recognizes the need for participation in the global dialogue.

The problems that we are exploring include:
Learning portfolios (ePortfolios)
Assessment Communities and community assessment
Identification of (and learning about) global competancies

We invite your comments and trackbacks to connect our work to your thinking and to a community of like-minded explorers.