Planet Sakai

September 18, 2018

Dr. Chuck

Dr. Chuck Coursera Office Hours in India in October and December

This Fall, I will be coming to India twice to give talks.  I will be attending these conferences and giving talks:

PyCon India October 6-7, Hyderabad

IIT Techfest, December 14-16, Mumbai

I will also be visiting Microsoft in Bangalore on the December trip to give a talk.

So I would like to arrange office hours in Hyderabad, Bangalore, and Mumbai as part of these trips.   In order to help me make arrangements (reserving venues, etc), if you might be able to attend the office hours, please fill out the Google form:

Once I make arrangements with exact dates and times I will send another announcement to the class.

I a very excited as this will be the first time I have ever been to India :)   So I hope to have a chance to meet as many of you as I can on these trips.’

by Charles Severance at September 18, 2018 06:50 PM

Progress on LTI 1.3 and LTI Advantage in Sakai and Tsugi

I have been making good progress towards LTI Advantage in Sakai – the following have initial implementations and are in Master:

– LTI 1.3 Core

– LTI 1.3 Basic Outcomes

– LTI 1.3 Lineitem

– LTI 1.3 Names and Roles (with a Groups extension available)

The specs are still somewhat in flux so as we clean up bits here and there the code will evolve but the major work is in place.  

Like I have mentioned before,  I have a nice set of utility routines in to handle OAuth 2.0, Access Tokens, Java Web Tokens, and PKI message signing.  The speed at which I can roll out new web services using the LTI 1.3 security model is pretty amazing.   I adapted the old Sakai memberships extension to become LTI 1.3 Names and Roles with the OAuth 2.0 security model in less than two days because of the nice OAuth 2.0 infrastructure I have built.

As Sakai evolves, so does Tsugi.  I use Tsugi to test all the Sakai LTI 1.3 implementations.  I have built an LTI 1.3 test harness in Tsugi that exercises the LTI 1.3 APIs:

I will get this into production on one of my Tsugi servers so when I write LTI 1.3 QA documentation we will have a ready made server to work with.

You can track activity this by watching the JIRAs connected to this JIRA:

Up next for LTI Advantage:

– LTI 1.3 ContentItem – Should be < 1 week

– LTI 1.3 Assignments and Grades – ???

The Assignments and Grades is the most intricate spec so I will leave its implementation as my last task.

by Charles Severance at September 18, 2018 02:21 PM

September 16, 2018

Michael Feldstein

Can Pearson Sell Efficacy?

Almost five years ago, when Pearson announced that the company would reorganize itself around efficacy, I was impressed by the degree to which the company was going "all in" on a very much unproven strategy. It was incredibly bold. I wrote,

In all my years of covering the ed tech industry, I have never seen a company be so explicit and detailed about their strategy as Pearson is being now with their efficacy publications. Yes, there is plenty of marketing speak here. But there is also quite a bit about what they are actually doing as a company internally—details about pilots and quality reviews and hiring processes and M&A criteria. These are the gears that make a company go. The changes that Pearson is making in these areas are the best clues we can possibly have as to what the company really means when they say that they want efficacy to be at the core of their business going forward. And they have published this information for all the world to see.

These now-public details suggest a hugely ambitious change effort within the company. Phil and I have consulted for a few textbook publishers, including Pearson, and I worked for Cengage for a year and a half. We have a pretty good idea of the magnitude of the change management challenges these companies face right now and the strategies that various publishers are bringing to bear in an effort to meet them. I can say with absolute conviction that what Pearson has announced is no half-hearted attempt or PR window dressing, and I can say with equal conviction that what they are attempting will be enormously difficult to pull off. They are not screwing around. Whatever happens going forward, Pearson is likely to be a business school case study for the ages.

Pearson bet the farm on efficacy. As far as I can tell, they are still betting the farm on efficacy. If anything, they have doubled down in the five years since I wrote those words.

There are a number of reasons why this bet is remarkable, not least of which is that we still don't know if a curricular materials company can be successful in the long run by promoting efficacy as a primary value proposition. In addition to all the hard work that Pearson needs to do to credibly claim that their products support some sensible definition of efficacy, they also have to tackle the equally hard challenge of convincing faculty that the company's vision of efficacy, as delivered in their products, is a good reason to pick their product instead of the (many) alternatives. They have an enormous customer communications challenge. That was one of the main points of my original 7,000-word post on the company's strategy.

Which is why I find it mystifying to see an article in Forbes which seems almost designed to distract from or even directly undermine the efficacy message that the company has been carefully honing over the last half a decade. The article, "How 174 Year Old Pearson Is Developing The Netflix Of Education" is a longish interview with Albert Hitchcock, Pearson's Chief Operating Officer and Chief Technology Officer. The jarring title refers back to Mr. Hitchcock's previous use of the analogy in an interview entitled 'Pearson aims to become the 'Netflix' of education.' In responding to that first piece two years ago, I wrote,

Given his profile on LinkedIn, Mr. Hitchcock appears to be new to education except for whatever memories he has of his own days as a student. Let me offer a couple of suggestions on how to get along in education for him and the many vendor employees who are in a similar situation:

  1. Never, ever, say that you want your company to be the Uber of education, the Airbnb of education, the Pokemon GO of education, or the [insert name of tech darling] of education unless you really enjoy being a recluse (or you are secretly a double agent for your employer's direct competitor).
  2. If you absolutely must say something like the above, then do not say you are the Netflix of education. Honestly, Netflix isn't even great at being the Netflix of movies. The last time they recommended a movie that I actually wanted to watch was...uh...never.

There is a recurring cultural fantasy that "solving" the education "problem" consists of creating a customized playlist of little content bits. So really, more like the Spotify of education, if you want to play that game. This idea enrages educators because it trivializes what they do. Nobody who has taught believes that proper sequencing of content chunks is the hard part. (For a fully fleshed out prior example—or a "worked example" in teaching parlance—of how the sorts of comments Mr. Hitchcock have made typically play out in educational corporate branding over time, see my post-mortem on Udacity's pivot away from higher education.)

In the more recent Forbes piece, the interviewer picks up on the Netflix analogy and asks Mr. Hitchcock about it. Not only does Mr. Hitchcock choose not to disavow the analogy; he expands on it by adding Spotify and Amazon. I'll parse his exact language later in this post in the interest of fairness, but my point from the previous post stands. Regardless of the intention behind the analogies, they are toxic and should be avoided at all costs. This is doubly true for Pearson because, as I will explain in the next section of this post, they also run directly counter to the more credible, more refined articulation of the efficacy strategy that the company is promoting in 2018.

The fact that the analogies did make it into an interview with a mainstream publication like Forbes raises some larger questions. Is Pearson less committed to efficacy than I have believed them to be? Is there poor alignment in the company around what efficacy means? Are they just really bad at messaging? Did Mr. Hitchcock's intent somehow get represented unfairly by a few poorly chosen words that were then taken out of context by an editor?

To find out the answers to these questions, I spoke with Pearson's President of Global Product Tim Bozik, SVP of Efficacy and Research Kate Edwards, and CEO John Fallon. Here's the short version of what I have concluded, based on those interviews and other evidence:

  1. I am mostly still convinced that Pearson is fully committed to efficacy as a primary value proposition for all of their products and services going forward.
  2. I do believe that the Forbes interviewer was bad, as were some of the editorial choices made by the publication. (Particularly the headline.)
  3.  Even granting the benefit of the doubt on the previous point, Mr. Hitchcock made a number of very bad choices that no executive in his position should make and that cannot be explained away by blaming the editor or the interviewer.
  4. Mr. Bozik and Ms. Edwards seemed fully aligned on what Pearson means by efficacy, how to talk about it, and how central it is to the company. Within the rules that they were bound to follow in an on-the-record interview through formal PR channels, they did the best job they could to articulate the most attractive and compelling version of Pearson's efficacy strategy while diplomatically establishing some distance from Mr. Hitchcock's questionable analogies.
  5. Mr. Fallon chose a different path. He went to some lengths to justify or explain away the analogies. In doing so, he re-opened questions that Ms. Edwards and Mr. Bozik had all but closed for me regarding whether Pearson has the sensitivity and message discipline they will need to earn their customers' trust regarding their commitment to efficacy.

For the long version, read on.

But before you do, you should be aware of our conflicts of interest, which I'm going to describe in more than the usual detail. Pearson is a current sponsor of the Empirical Educator Project. In addition, they have periodically engaged us in consulting projects over the years (although we do not have any current engagements with them.) Some of these projects have involved their efficacy work, either directly or indirectly. We received prior permission from the company to blog publicly about the first such engagement. Most recently, we were hired to review the company's public  efficacy reports, provide the kind of feedback that we might publish in a blog post, and offer suggestions for improving the work. You can judge for yourself whether these engagements make us more or less trustworthy in our assessments.

Netflix and Spotify analogies are fundamentally incompatible with Pearson's view of efficacy

In order to understand why these analogies are particularly bad for Pearson's efficacy effort, it's important to understand how the company's position on efficacy has evolved. In that original post five years ago, I wrote,

Let's think some more about the analogy to efficacy in health care. Suppose Pfizer declared that they were going to define the standards by which efficacy in medicine would be measured. They would conduct internal research, cross-reference it with external research, come up with a rating system for the research, and define what it means for medicines to be effective. They would then apply those standards to their own medicines. And, after all is said and done, they would share their system with physicians and university researchers in the hopes that the medical community might be reassured about the quality of Pfizer's products and maybe even contribute some ideas to the framework around the edges. How confident would we be that what Pfizer delivers would consistently be in the objective best interest of improving health? This is not entirely hypothetical; much of the drug research that happens today is sponsored by drug companies. Unsurprisingly, this state of affairs is viewed by many as deeply problematic, to say the least. It certainly doesn't help the brand value of Pfizer. But at least much of that medical research is conducted by physicians and academic researchers and is subject to the scientific peer review process. Pearson is creating their framework largely on their own, selectively inviting in external participants here and there.

I get why they had to do this. The company is bleeding money, it will take some time to stop the flow of blood, and they couldn't wait to build consensus before they tied the tourniquet. But it is not going to get them where they want to go. While there are obvious concerns about ethics and about whether such a company-driven approach is fundamentally compatible with progress on complex questions such as defining what an education is good for and how we know when we have achieved these ends, I want to focus on the business aspects of the problem. I want to focus on why continuing down this path is bad for Pearson. Or rather, why driving hard toward becoming facilitators rather than owners of efficacy research is good for Pearson.

I could give a number of examples, but one should hopefully suffice. In preparing to write this post, I asked Annie Cellini, Pearson's Senior Vice President of Marketing and Strategy, whether Pearson intends to share the completed rubrics for their products with customers and prospects. This was her reply:

Though we don't plan to share product efficacy scoring as part of our sales and marketing materials per se, where a product has a strong research and evidence base, we will communicate that to customers. It's also worth saying that the most important output of an efficacy review isn't a rubric score. We believe that much of a review's value comes from the conversations that it prompts teams to have, which focus on the path forward, and on how to improve the product or service from a learner perspective. A poor score does not mean the product doesn’t work well. It often means that teams are not collecting the type of data needed in order to get a sufficiently robust view of the product’s efficacy, or they may not have a sufficiently practical plan to continuously enhance the product based on data. Their improvement plan will encourage them to start gathering new information, to start working in new ways, and to make sure that their customers are aligned with the outcomes they plan to achieve and understand their role in the product's path to efficacy.

This is a perfectly sensible and responsible reply if you believe that the main value of the Efficacy Framework to customers is in the data that results from the work a product team does after an efficacy review. But remember, the magic of the rubric is in the norming conversations. Annie's reply suggests that Pearson understands this in terms of the Pearson-internal processes but not yet in terms of their relationships with their customers. If Pearson were to say to faculty, "Here's what we think we know about the efficacy of this product, here's what we don't know yet, and here is how we are thinking about the question," they might get a number of responses. Maybe they would get, "Oh, well here's how I know that it's effective with my class." Or "The reason that you don't have a good answer on effectiveness yet is that your rubric doesn't provide a way to capture the educational value that your product delivers for my students." Or "I don't use this product because it has direct educational effectiveness. It frees me up from some grunt work so that I can conduct activities with the class that have educational impact." Most of all, if you're John Fallon, you really want faculty to say to their sales reps, "Huh. I never thought about the product in quite those terms, and it makes me think a little differently about how I might use it going forward. What can you tell me about the effectiveness of this other product that I'm thinking about using, at least as Pearson sees it?" And you really want your sales reps to run back to the product teams, hair on fire, saying "Quick! Tell me everything you know about the effectiveness of this product!"

Pearson won't get that conversation by just publishing end results of their internal analysis when they have them, which means that they have a high risk of failing to align their products with the needs and desires of their market if they think about the relationship between their framework and their customers in that way. I don't think Pearson fully gets that yet. While the authors of The Incomplete Guide frequently invoke terms like "community" and "leaders" in the document, they generally seem to mean the community and leaders within Pearson. The company's efforts to reach out to the academic community for feedback and participation are generally framed as an extension of their efforts rather than the very heart of them. And yet, Pearson's brightest possible future is not as a company that designs educationally effective products, but as one that facilitates conversation and research about efficacy within the broader academic community (and in so doing is able to design products that their customers agree are effective for important educational goals as determined by meaningful measures).

The Netflix, Spotify and, to a lesser extent, Amazon analogies all speak directly to the question of whether Pearson intends to define efficacy for educators or with educators. Netflix famously just deleted—not just removed, but deleted—all user reviews and switched from a five-star review system to a simple "thumbs-up/thumbs-down." The subhead on the Vanity Fair article I linked to in the previous sentence is "A step closer to a wholly 'because you watched' world." The implication is that Netflix trusts the algorithm more than the humans to evaluate quality. Equally famously, Apple CEO Tim Cook has highlighted the company's decision to use human curators of music, in contrast to Spotify's decision to rely completely on its algorithms:

We worry about the humanity being drained out of music, about it becoming a bits-and-bytes kind of world instead of the art and craft.

Whenever Pearson makes any mention of Netflix or Spotify—or Amazon, which is known for its recommendation engine as well—the company risks appearing to come down on the side of the algorithm over the human. Educators worry about the humanity being drained out of teaching, about it becoming a bits-and-bytes kind of world instead of the art and craft. In 2018, is that where Pearson has come down on their definition of efficacy?

Not as far as I can tell.

Pearson, in fact, recently made a high-profile hire away from Intel of artificial intelligence expert Milena Marinova. Here's how she described her work in an interview with The Bookseller:

Right now, I am working on developing human-centric AI – this means making the learning experience better for students and teachers; enabling lifelong learning through more accessible and affordable products; and building better products and solutions using new technology.

What does "human-centric AI" mean in this context? Mr. Bozik commented on this directly in our interview. He described it as an effort to help teachers and students get a better view into their own learning. He said the company is placing a strong emphasis on early intervention and formative assessment, and providing "feedback and insights." He also said,

Pearson strives for respect for both the ambition and the humility of [its efficacy strategy]. The ambition is that efficacy means outcomes. Full stop. It's the potential to help people live better lives. That is our purpose. The humility part is that it’s hard. We don’t underestimate the difficulty of it. It starts with having an empathy for teachers and learners, rather than just teaching and learning.

For Ms. Edwards' perspective, while I could quote her from my interview, I think you'll get a better sense of her from her lightning talk at the Empirical Educator Project summit in February:

Here's the part of her talk that jumps out as relevant to the question at hand:

How do we maximize the uniquely human attributes that educators—faculty—bring to the table, and combine them with all the productivity enhancements that come from things like data science, AI, and the improvements we're seeing as the result of technology. All with the emphasis on helping students achieve outcomes that really matter to them.

Ms. Edwards then goes on to talk about the resources that Pearson either has already contributed or will contribute under a Creative Commons license. These include a set of rubrics for evaluating curricular materials (or course designs) based on a set of academically accepted and empirically verified learning science principles and a set of tools for instructors that want to conduct action research. It's these contributions, these public actions, that speak the most persuasively to Phil and me when we evaluate a company's intent. (For more on Ms. Edwards' views and her characterization of Pearson's efficacy work, you can read her recent posts on LinkedIn.)

Taken together with Pearson's other work, such as their first publicly released efficacy reports on their products, the message conveyed by the actual work that I have seen from the company is clear: They aspire to define efficacy with educators and students. As I wrote earlier in this post, I have had opportunities to view this work from inside and out over the past five years. I honestly believe that the people working on the efficacy and product teams—Ms. Edwards' and Mr. Bozik's teams, respectively—buy into that goal and are actively working toward it.

Pearson's intent doesn't matter if nobody believes them

The problem is that intent of the people working at the company, level of commitment to that intent by the CEO and the small circle of people that make decisions within a company, and customer perception of intent are three different things. When I published my post the first time Mr. Hitchcock was quoted talking about Netflix, some of the people who worked on product design and efficacy at Pearson were quite angry with me. They felt that I had misrepresented Pearson's position, because—and this is key—they also felt that Mr. Hitchcock's comments were not representative of Pearson's position. I replied that I wasn't sure there was any way to determine what Pearson's position is. Mr. Hitchcock is quite senior. He reports directly to the CEO. I could ask other executives for their own responses, but they wouldn't be any more authoritative than Mr. Hitchcock's. Unless I could speak directly to Mr. Fallon—which I didn't believe would be possible at the time—I had no way of definitively determining which beliefs the elusive entity known as "Pearson" holds.

What I did know is how academia would receive the phrase "the Netflix of Education." I knew this because Mr. Hitchcock was not the first person to use it, as was acknowledged by a Pearson employee in a 2016 EdSurge story entitled "Why We Don't Need a 'Netflix for Education.'" Its toxicity was widely known inside Pearson, at least within the product and efficacy groups.

Last month, five years into the massive effort to align the company around efficacy, the analogy shows up again, expanded upon, by the same senior executive. And this time, it's not published in a specialty IT outlet. It's in Forbes. Interviews at that level do not happen at big companies without being vetted. If you're a senior executive at a company like Pearson, you go through formal media training, and you are often accompanied on interviews by a PR professional. These tend to be usually serious, orchestrated affairs. There are rules. How was it possible that this interview was approved, went through the proper channels, and went off the rails? Do Pearson executives see it as being as far off-message as I do? Is there conflict or confusion about the efficacy strategy at the highest levels of the organization that isn't obvious to the casual observer?

Let's take a look at what Mr. Hitchcock actually said and then turn to the executive on-the-record reactions to it.

Mr. Hitchcock's comments were bad and cannot be explained away by bad editing

Here's the part of the article where Netflix came up:

High: You described the vision of delivering Pearson's education, content, and services through a single platform as creating the "Netflix of Education." Could you talk about this long-term vision?

Hitchcock: The intention of the message was to have the viewpoint that we needed to move to a platform type of model where we have all our products, services, and capabilities that we deliver to our customers in a single ecosystem. A great deal has been written around this model at Pearson, and that is especially relevant as our company grew through acquisitions. Ours is a diverse business that is over 170 years old and has had many different types of companies under the umbrella of Pearson for many years. Recently, education has become the primary focus, but that complexity that was acquired over those decades was inherent within the company. The way we serve our customers had been across many different brands and many different types of digital products. We sell millions of books and material on the digital side. We had to figure out how to transform that to be a consistent, high-quality branding experience and one that is comparable to the best customer experiences out there.

Silicon Valley companies create the benchmark for the digital experience by being platform businesses. Our vision is to leverage the opportunity to transform along similar lines in terms of having a single platform globally that could deliver all our educational content and courseware. Furthermore, this would allow us to move into a more personalized experience that delivers high-quality education outcomes. It would be game-changing for not only Pearson, but for the entire industry if we could create that single platform, similar to Netflix, Spotify, and Amazon. [Emphasis added.] This platform would be highly scalable, global in nature, high-quality, and a platform that could deliver all our experiences around the world to millions of learners. We are currently working to deliver high-quality courses to students that are proven to help them learn and progress their lives through education and ultimately to their professional lives.

In the interest of fairness, I'll enumerate the extenuating circumstances:

  • The interviewer, not Mr. Hitchcock, brought up Netflix. And a Forbes editor, not Mr. Hitchcock, chose to feature Netflix in the headline.
  • Mr. Hitchcock seems to be making a point about the need to support a seamless, end-to-end, born-digital customer experience for students. Further, he is absolutely correct that Pearson, like all of the larger textbook publishers, grew through acquisition and have had a patchwork of many different systems that were never designed to work together. These are entirely valid and important issues for Pearson's CTO and COO to discuss.
  • He does not take the last, most toxic step (this time) of describing Pearson as the "Netflix of education." In fact, he seems to make a modest effort to narrow the scope of the analogy in response to the question: "The intention of the message was to have the viewpoint...."

Is it possible that Mr. Hitchcock meant something innocuous and unobjectionable by the analogy? Yes, it is. Is it possible that he honestly tried, in his own way, to communicate the narrowness of his intent with the analogy? Absolutely. Is this a relatively minor and understandable slip that would have been fine if it had just been worded a little more clearly?

Absolutely not.

I reiterate: There are no circumstances under which any EdTech executive should ever make an analogy between their company or offering and Netflix or Spotify. Period. Full Stop. It is not a mistake that anyone in Mr. Hitchcock's position should ever make. In fact, if the analogy is brought up by an interviewer, as it was in this case, the first words out of the executive's response should be "To be clear, Pearson does not aspire to be the Netflix of education." That goes doubly for Mr. Hitchcock because he actually said that once before and should have taken the opportunity to correct the record, and trebly because the analogy can easily be interpreted as being in direct contradiction to the company's bet-the-farm strategy, around which it has taken them five years to arrive at their current calibration in message and in action.

Nor can this easily be chocked up to bad editing. In fact, I question whether there was any significant editing whatsoever. This isn't one of those tightly written articles in the reporter's own voice with a one- or two-sentence quote sprinkled in here or there. This is an interview, with short questions followed by long answers that do not appear to be heavily edited for clarity or brevity.

Even worse, there is a pattern to Mr. Hitchcock's answers which leave the distinct impression that he, the CTO and COO, is directly responsible for or taking credit for the company's strategy on teaching and learning. He deftly pairs "I believe" statements with "Pearson has done" statements. It's very easy to read these statements as Mr. Hitchcock making a causal connection between his beliefs and Pearson's strategy decisions, which is an impression that is only reinforced by the sycophantic fawning of the interviewer.

Here's an example:

High: How is the process of experimentation around new technologies done? Specifically, as new technologies emerge, how do you encourage your team to begin to experiment and to decipher more of the specifics of how that might be leveraged?

Hitchcock: [...] Overall, I believe it is a combination of aspects. I believe it is about us looking at other ideas that other companies are implementing and seeing where we can bring these concepts together to deliver with the lasting initiatives that we are trying to put in place at Pearson. Then, as we create the single platform, it is about how we start thinking about future R&D and looking forward towards a five-year horizon. It is about seeing the ideas that are coming that we can then bolt into the platform to take us to the next level. More specifically, with AI, we recently hired a senior leader from Intel to help us to bring the entire AI area to focus in terms of what it is going to do for our business. We are building AI and machine learning capabilities and technology in other areas of our company to look at how we transform all aspects of our business using AI and machine learning.

As I mentioned earlier, it is true that Pearson has hired a senior AI leader from intel. She reports to Mr. Bozik, not Mr. Hitchcock. As President of Global Product, Mr. Bozik's part of the company makes product design decisions. Mr. Hitchcock's group is in charge of the software development required to build the product. Both men report directly to Mr. Fallon. Ms. Edwards reports to Mr. Bozik. I don't know what role Mr. Hitchcock may or may not have played in the new hire, but she definitely does not work for him. That is not at all clear from the interview.

Here's another example:

High: With new technology and innovation, you underscored how the pace of change is faster than ever. Additionally, fostering learning agility for individuals, enterprises, and the people who work within companies has become an important topic especially given the fact that a skill that one has today may be rendered obsolete quickly. How do you work to stay agile and ensure that your organization is doing the same thing?

Hitchcock: We recently released a report with the Oxford Martin School on the future skills and learning that will be required in the workplace. One of the things we focused on is the aspect of employability and equipping students with the skills that are necessary for future employment and progression within employment. One of the areas I believe is a great opportunity for Pearson is to increasingly serve education, not just through school and formal education, but with employers to transform their workforce. When I connect to our suppliers in the technology industry or my fellow CIOs, one thing that is evident is that nearly every company is faced with the challenge of re-skilling their workforce. This must be done to cope with the demands of the digital revolution that we are going through and the impact of AI, robotics, and other emerging technologies. I believe Pearson is exceptionally well positioned to assist companies in transforming the skills they need to acquire and develop.

I started out as an engineer, and I became incorporated and then chartered through the Institute of Engineering and Technology. An engineer is no different than a surgeon or an accountant in the sense that the profession demands ongoing development certifications. I believe people want to continue the learning part. Because of this, the opportunity is there for us to become the delivery of lifelong learning for both individuals, education institutions, and government to achieve that. In terms of how we do that internally, at Pearson, there is a big focus on internal employee development.

We started the Technology Academy shortly after I joined to try and take the technology workforce to the next level in terms of their abilities to equip them with the skills needed for our own journey around the digital change. That is something we will continue to work on and sponsor. We work very closely with institutions and universities to do that as well as our own people. It is a fascinating journey and an incredibly important one for society over the next few years. As society changes, we need to evolve to support the future digital requirements.

One could be forgiven for assuming, based on this answer, that Pearson's focus on workplace skills and continuing education was Mr. Hitchcock's idea, which he arrived at on his own based on his personal biographical experience. But again, Mr. Bozik and Ms. Edwards have far more authority to set this kind of product strategy than Mr. Hitchcock does.

The net effect of Mr. Hitchcock's answers is a real problem for Pearson's brand. What is a typical academic's nightmare vision of Pearson's worst possible future? A company whose ideas of education come from its CTO; a man whose previous job was being CIO for a phone company, who thinks that Spotify and Netflix are positive examples of the company that Pearson wants to become and isn't afraid to say so, repeatedly, in public, despite previous criticism for having done so.

Also? No mention of efficacy in the article. Anywhere. Not once. Not a word. Not a whisper. In an wide-ranging, open-ended interview about Pearson's future.

Pearson's senior executives tried their best to clean up the mess

As I mentioned earlier, it's hard to figure out Pearson's official position on...well...anything. But Pearson's official position, or what the organization collectively "thinks," is a critical question in this case. Given that the interview of somebody who directly reports to the CEO, and who repeated (and arguably expanded upon) an analogy for which the company had taken flak, the question is whether the root of the problem is a failure of corporate communication or a failure of alignment of senior management. This is a high-stakes question. And it's one that can only be answered definitively by Pearson's most senior executives.

Given that, I wanted to get as close to a definitive answer as possible. So I did something that we don't typically do at e-Literate. I went through proper PR channels. There are a number of reasons we don't like gathering information this way. One is that the people we are talking to are chaperoned and constrained to hew as closely as possible to the company line. When we approach people informally, they are freer to give us information off the record that provides further validation or nuance to their on-the-record comments. We also generally try to cultivate an approach to our analysis that doesn't depend on executives continuing to grant us access because we believe that doing so minimizes one kind of pressure on our objectivity.

But for this particular interview, the constraints of the formal channel were helpful. I could already make pretty good inferences about what Ms. Edwards and Mr. Bozik likely think as individuals. I have gotten to know Ms. Edwards through the Empirical Educator Project and, while I haven't had much direct exposure to Mr. Bozik, I know quite a bit about the work that has come out of his shop. Further, neither Ms. Edwards nor Mr. Bozik were responsible for—or could be individually responsible for—Mr. Hitchcock's comments. As his peers, they were not in a position to make statements that were more definitive than his.

By going through the formal PR channel, I could speak not just to them but through them to the aggregate entity known as "Pearson." I wanted Mr. Bozik and Ms. Edwards to have the kind of pre-interview huddle orchestrated by PR that occurs when these media requests are made, and I wanted them to be constrained by whatever had been decided in that huddle. By doing so, they could provide answers that are official in a way that they wouldn't be through less formal channels. Their job in this kind of an interview is to give their best, most honest version of the official company line.

(By the way, I didn't ask specifically for these two executives. Part of the experiment was to see who Pearson decided to send. They did not decide to send Mr. Hitchcock. I don't know if that was purely due to scheduling conflicts or if there were other considerations.)

Let's be clear: 90% of my interview with Mr. Bozik and Ms. Edwards was kabuki theater. It was highly ritualized and scripted, with moments of drama that were entirely predictable. That was by design and in no way their fault. They were constrained by the script. I even gave them my basic line of questioning in advance so that the "company" could formulate "its" answers in advance. My questions consisted of four parts:

  1. Describe Pearson's current view of what "efficacy" means, preferably covering specific examples of efficacy work
  2. Clarify Mr. Hitchcock’s role in the company specifically as it pertains to decision-making around the efficacy initiative and its integration with product design
  3. Review Mr. Hitchcock's comments in Forbes, in the context of the first two parts of our discussion
  4. Describe Pearson's official position on whether and in what sense it aspires to be the Netflix, Spotify, or Amazon of education

The answers to the first two sets of questions were entirely predictable. I wanted to make sure that the official characterizations in the interview were consistent with what I thought I already knew. They were. Mr. Bozik and Ms. Edwards did their job. They faithfully described what I had come to understand as the company's official positions on those questions. The third and fourth sets of questions were the test. In the third, Mr. Bozik did his best to provide the most charitable honest reading of Mr. Hitchcock's comments that he could. Again, he did his job.

When I asked him the fourth question, using more or less the same words as above, he gave the the only correct answer: "Let's be clear: Pearson does not aspire to be the Netflix or Spotify of education." Those were the very first words out of his mouth. Yet again, he did his job.

At that point, I went off-script. I asked him how, given that statement, Mr. Hitchcock could use that analogy for a second time. In the entire ninety-minute interview, this was the only moment when Mr. Bozik seemed to struggle for words. As he should have. Because there is no good answer to that question.

I had put him in an impossible situation. He was bound by the rules of the kabuki play we were in to give answers that support the company line (and his colleague). He didn't have the option to say, "Off the record, it was a dumb thing to say." He had only two options he could take without breaking the rules of engagement: rationalize that which I believe he knew was wrong, or flail around for the best honest answer he could give.

Mr. Bozik chose to flail. I think better of him for it.

But their CEO blew it

At the end of the interview, I found out that Pearson was going to grant my request to speak directly to the company's CEO. I honestly didn't expect that request to be met. Pearson is at least several times as large as the next largest ed tech company that we cover. By some measures, they are an order of magnitude larger. So I was surprised and gratified that Mr. Fallon was willing to make time for the conversation.

And it was an important step for the specific purpose at hand. Mr. Fallon is the only person at Pearson who is empowered to fully repudiate statements by the man who reports directly to him. (This isn't true at all companies, but it tends to be true at companies as large as Pearson.) Once again, I telegraphed my line of questioning in advance, which was essentially an abbreviated version of the same approach I had used with the previous interview.

To my surprise, Mr. Fallon jumped ahead to address Mr. Hitchcock's comments directly. And to more or less defend them. He argued that a key constituency for Pearson is what he called the "Spotify generation," or "Generation Z," to which he attributed the following characteristics:

  • They would rather rent or subscribe than own
  • They have a higher expectation of the use of video
  • They probably expect to be able to engage with courses through shorter and more frequent modules
  • They have higher expectations of user experience

The first characteristic is arguable. Within the world of curricular materials, students are strongly price-sensitive. The primary mechanism that the industry has offered students to get a lower price is rental/subscription. Students have tended to rent or subscribe to curricular materials. Does this mean that they prefer rental or subscription, or just that they are picking the least expensive option? The publishers have a financial interest in promoting rental and subscription, but the evidence of a Spotify-style preference among students seems limited.

The other three characteristics are probably true and definitely not new. All three are trends that the sector has known about and anticipated for literally decades. Before there was the Spotify or Netflix of education, there was the iTunes playlist of education. And before that, there were re-usable learning objects. All of these points are sufficiently simple, accessible, and well understood already that an analogy to Spotify or Netflix adds nothing useful.

Mr. Fallon was intent on getting me to admit that there is a reasonable interpretation of the analogy. Which I did. But his defense missed the point entirely. Sure, in principle, with some thought, one can find a non-offensive explanation for an analogy to Spotify or Netflix. But why would you bother to make such an analogy in the first place? And for heaven's sake, why would you do so in public when you know that your customers are likely to find it offensive by default?

When I pressed, Mr. Fallon said, "You have never heard or read me say that Pearson wants to be the Netflix or Spotify of education."

That is a carefully parsed sentence.

In fairness, Mr. Fallon then went on to demonstrate to me that he knows exactly what the company's official and nuanced 2018 version of efficacy is. He can cite it chapter and verse. He can refer to prior public statements that align with that vision, such as his recommendation of World Without Mind, a book about the dangers of romanticizing the power of artificial intelligence. He can speak at length and in detail about the challenges of the lack of agreed-upon frameworks for student data privacy in educational research.

Which is not surprising. Mr. Fallon is a smart guy. Furthermore, he was the one who bet Pearson's future on efficacy. It's probably not an exaggeration to say that he bet his career on it as well. It would be surprising if he didn't know this stuff inside and out. I'm not sure why he didn't just say the analogies were bad and do not represent Pearson's position. Maybe he was just trying to convince me that this was a non-story and not worth writing about. Had he simply led with the points about the challenges of doing efficacy right and refrained from vigorously defending the indefensible, he might have succeeded. Instead, after all of the work involved in making the interviews happen, and all of Pearson's executive time and PR time spent dealing with me, we're right back where we started.

I don't know Pearson's official position on the analogy to Netflix or Spotify because Mr. Fallon—the only person at the company who is fully empowered to definitively lay the question to rest—chose to parse words and defend a bad interview rather than just saying, "Yeah, we don't think about our goals that way and shouldn't say things like that." In a few sentences, Mr. Fallon could have closed the door on the one aspect of the Forbes article that I feel compelled to write about. But he didn't.

To be clear, the aspect in question isn't that Mr. Hitchcock said something dumb in public. And it's not that Mr. Fallon was unwilling to admit on the record that Mr. Hitchcock said something dumb. It's that Pearson, as represented by the behavior of its CEO, does not appear to fully recognize, or at least fully acknowledge, how delicate and existential a messaging challenge they've created by betting their future on efficacy.

Pearson still communicates like a roll-up, and it can't afford to do so anymore

So here's the real question: Why? Why has Pearson made these unforced errors and then, having made them, compounded them?

Once again, I can't know the answer for certain. There often are some organizational politics behind stories like this one. That sort of thing is largely outside of our purview at e-Literate. But the company's behavior may also be symptomatic of a more systemic problem. As I said earlier, Mr. Hitchcock was absolutely correct in saying that Pearson had a huge technology mess resulting from the fact that it is a "roll-up"—a big company that was created by buying up many small companies. He is also correct in saying that Pearson cannot compete unless they fix that mess by creating one seamless platform. (This is exactly where it is tempting to make analogies to giant internet companies and exactly where EdTech executives should resist that temptation.)

Pearson has had an analogous problem with their organizational culture (as have all the major textbook publishers). It used to be that every editor had his or her own fiefdom, run relatively independently from the others. This doesn't work when you all have to deliver content via a common and complex technology platform and when you have to adhere to common standards of learning design. The company has been working on changing that culture for a while, has made some progress, and probably isn't finished yet.

But there's a third leg to this stool which appears to be the one that Pearson is falling over: message discipline (which is icky marketing-speak for "stick to talking about the things you think are important to communicate and don't go off on tangents about things that are less important"). Even just a few years ago, it would have been impossible to describe the company's official positions or priorities on much because they were a conglomerate. What priorities do textbook editors share with the publishers of the Financial Times and the people who run Penguin Books? There has always been a certain amount of laissez faire chaos in Pearson's communications, for understandable reasons. But the company has been selling off all the parts of the business that don't fit under the common theme of educational impact. I don't think it would be an exaggeration to say that Pearson aspires to sell only one thing, and that thing is efficacy.

And yet, they don't communicate that way. They still communicate like a company with lots of products and lots of messages for lots of different audiences. They have not made this part of their transformation and, based on what what I have seen during the course of writing this story, I'm not confident that they even know that they need to make this change. I have no reason to believe this is the fault of Pearson's marketing and communications people. It's much more likely that the problem is the prioritization that is being given to them, or not being given to them, from the top.

Nobody yet knows if a large education vendor can succeed by making efficacy its main product. But I can say one thing with confidence: They won't succeed if they don't sell it. And right now, they are not selling it. They say some things about efficacy, but they say some things about a lot of things, like Netflix, and Spotify, and Microsoft Hololens, and jobs of the future, and probably lots of other stuff that Gmail mercifully filters out for me. If they want their monumental bet to have a real chance of paying off, then they need to be talking about efficacy. Only efficacy. Always efficacy. Everything else needs to be subordinated to a very specific, meticulously crafted, always-being-tuned message about efficacy. Anything that outright clashes or distracts from that message should be mercilessly cut. Any missteps should be immediately and definitively corrected. Saying something stupid in public is a recoverable sin for a company. Failing to relentlessly focus on communicating the complex and somewhat controversial value proposition, upon which you have bet the entire future of your company, to an academic audience that is not terribly inclined to listen to or trust your messages in the first place? That may not be a recoverable mistake.

The post Can Pearson Sell Efficacy? appeared first on e-Literate.

by Michael Feldstein at September 16, 2018 09:05 PM

September 13, 2018

Michael Feldstein

Extension Engine and OPM Market Transparency

A couple of weeks ago, our response to Open SUNY's Request for Information (RFI) on Online Program Management services (OPMs), we published an abridged version of our response on our company website and wrote a post about our observations here on e-Literate. Since, then, ExtensionEngine has followed suit. Here's the introduction to the version that they published on their site:

The online program management (OPM) landscape is a confusing one, the result of rapid evolution and an ever-greater assortment of businesses keen on winning their share of what has become a very lucrative market. We do not envy the task of any institution of higher learning seeking to upgrade their online learning program, and even less one considering the launch of their first program.

A few days ago, MindWires — a strategic consultancy and advisory firm — took an unusual step, one that begins to make the task a little easier by increasing transparency among all of these diverse providers: they published their response to a request for information (RFI) from the State University of New York (SUNY).

The action reflects — and begins to correct — concerns expressed by MindWires’ Michael Feldstein in a series of articles regarding the complicated 2018 OPM market. Greater transparency, as modeled by this move, may be exactly what the marketplace needs to make sense of itself.

In fact, the SUNY RFI itself has made a contribution to greater clarity. This RFI was more than a request for information on what various providers could do for them; rather, it asked the question: What do we need to know before we move forward? Asking for this guidance before generating a request for proposal, or RFP (which will come later), was a brilliant and insightful move on SUNY’s part.

The SUNY RFI listed 15 information-gathering objectives. As we formulated our own response, we noticed that we were, in effect, creating a road map through the wilderness of the 2018 OPM market, one that could be valuable to many institutions of higher learning that are thinking of creating or upgrading an online learning program. Some sections are more pertinent to multicampus systems like SUNY’s, but much of the road map applies to any institution, regardless of size.

For those who are not familiar with ExtensionEngine, we are a professional services organization that designs, builds, launches, and markets custom learning experiences — an integrated, holistic experience designed for learners, pedagogy, vision, and brand. We are a fee-for-service partner — no revenue sharing — and are paid by the hour to provide a full suite of services to help our clients to create successful online learning.

So, in the spirit of transparency inspired by MindWires’ publication of their response, we are following suit to share portions our response to SUNY’s RFI. Below is a substantial excerpt from the road map through the OPM landscapewe created for SUNY, shared with their permission.

This is great stuff. In mature product categories like the LMS, this kind of sharing can do more harm than good, because schools tend to just copy and paste requirements without giving a lot of thought about or investigation into which requirements are right for their particular context. But in the still-maturing OPM and broader Digital Enablement Solutions product categories, this kind of sharing helps to surface the differences in needs and contexts that lead to different kinds of optimal solutions. Publishing both the requests and, at least, abridged versions of the responses is very helpful in advancing the conversation. As ExtensionEngine's Scott Moore noted in the blog post, such responses can "creat[e] a road map through the wilderness of the...OPM market."

More transparency in this space would be extremely helpful at this time. We're giving some thought into different ways to accomplish that aim.

Stay tuned.

The post Extension Engine and OPM Market Transparency appeared first on e-Literate.

by Michael Feldstein at September 13, 2018 04:40 PM

Blackboard Learn Ultra in 2018: Is it ready and does it matter?

One of our longest-running series of posts is on the prospects and status of Blackboard’s Learn Ultra, the user experience redesign and move to the cloud of the world’s second-most-used (behind Moodle) and highest-revenue-producing academic LMS. It is fashionable to claim the LMS is dead or passé, yet this product category remains the centerpiece of educational technology at colleges, universities, and K-12 districts. To understand Learn Ultra is to understand much of the overall LMS market.

Looking at the timeline of e-Literate coverage, the broad story emerges:

  • 2012: Blackboard acquires Moodlerooms, and one benefit was the see of a cloud-based platform strategy.
  • 2014: Blackboard prematurely announced Learn Ultra (new user experience) and Learn SaaS (move to the cloud).
  • 2015: Learn Ultra is already a year late (more realistically, the BbWorld14 announcement was wildly unrealistic), but the University of Phoenix selects Learn Ultra as its next-generation learning platform.
  • 2016: Blackboard switches CEOs, admits that Learn Ultra is still not ready, and struggles with a major messaging problem around the transition.
  • 2017: No coverage - maybe we got tired of lack of not having customers to talk to.
  • 2018: Blackboard makes Learn Ultra the core of its message at its users conference BbWorld18.

(Note: It might be easier to view this as a full page timeline instead of the embed within this post.)

What we have been asked multiple times, by institutions, by investors, by other ed tech companies: is Learn Ultra ready, and does it matter? The unsurprising answer to both questions is a qualified and somewhat confusing it depends. To describe a little further, Learn Ultra's Base Navigation is ready, but the Course View is not ready for meaningful adoption; furthermore, Learn Ultra is important to Blackboard’s future, but we think SaaS is more important to its present.

Is It Ready?

To ask this question requires an understanding of terminology, as we first described in this post. Blackboard1 is pushing the metric that there are 61 or 62 Learn Ultra customers “in production” or "using Ultra", yet we have found very few that use, or even plan to use, Learn Ultra as their primary, institution-wide LMS. What gives? What became quite clear at this year's BbWorld 18 users conference is that when Blackboard says in production, what they mean is that the LMS administrator has enabled the Ultra navigation, which uses the new Ultra user experience framework as the landing page / dashboard with activity feed that users see before entering a specific course. The company calls this Base Navigation, but at this point every course can be configured to be in the Original Experience or the Ultra Experience. Thus, enabling the possibility of running a course in Ultra counts as in production (note that Blackboard now lists 91 clients on Ultra).

Slide from BbWorld18

Once a school has enabled Learn Ultra Base Navigation, they could choose to move exclusively to Ultra (e.g. the University of Phoenix, Northwest Florida State College, and a few others), or they could choose to keep all courses in Original (e.g. Northeastern State University), or they could choose to have some courses in Ultra and some in Original (used by the majority of schools investigating Ultra). This last mode is known as Dual Course mode, and even Blackboard executives seemed surprised to find out that the vast majority of schools putting Ultra in production are in fact running in Dual Course. For many of these schools, there are no definitive plans to move exclusively to Ultra.

Upon re-reading this description, I believe that I should give more credit than just describing a landing page and base navigation. The cross-course functionality is and has been a long-term goal of Ultra, as we described as early as 2016.

Creating a brand for a set of design goals is inherently fraught. Let's look at two examples of how it makes communication of Blackboard's strategy tricky for them. First, there's mobile. Blackboard came out last year with a mobile app called Bb Student. It provides students with that activity stream view across courses and, of course, it's mobile-first. (In fact, it's mobile-only at the moment.) Furthermore, the company has made the product available for both traditional 9.x customers (which at this point is pretty much everybody) and their SaaS customers. People inside the company feel like they should be getting more credit for delivering on two major design goals (mobile first and stream-based activity views) as well as for delivering it to customers on the 9.x platform (which was more significant of a technical achievement than is immediately obvious).

This is not a matter of Blackboard moving the goal posts, per se, and it is probably more accurate to say that Ultra cross-course functionality enabled by Base Navigation is ready and showing some benefits.

Learn Ultra Base Navigation Brochure

The challenge is that this move is not sufficient to make a material change in Blackboard's company prospects. At best, Learn Ultra Base Navigation without usage of Course View will slightly slow down the rate of customers defecting for another LMS. For Learn Ultra to matter and to make Learn newly competitive, they need customers to also use Course View as the primary choice at their institution, and that usage by-and-large is not ready outside of a handful of schools.As an example from BbWorld18, Belmont University presented their experience moving to Learn SaaS (the important issue in the present, and a predecessor for  adopting Ultra)  and to "the Ultra Experience". But after we asked a question about faculty adoption of the Ultra Course Experience, the administrator clarified that there are no courses running Ultra - all they have done is enable the Ultra base navigation, and they do not expect to do any course migrations for at least another year. This full adoption of Learn Ultra might become important in the future, but it is not driving decisions today.

Does It Matter?

Given that we’ve been asking for three entire years whether Learn Ultra is ready, one obvious follow-up question is whether it matters any longer. I do think the question matters as Blackboard is pinning their corporate turnaround on Learn Ultra as the core piece, and this message was heavily promoted at BbWorld 18.

This messaging makes some sense in that we consider it unlikely that Blackboard can gain significant numbers of new clients (those moving from another LMS to Blackboard Learn, beyond a dozen or fewer schools) without Learn Ultra. Learn Original Experience has too much baggage and is too dated to compete with Canvas or Brightspace by D2L, at least in North America. The company's new Learn LMS clients are largely the University of Phoenix and ANGEL contract conversions.

To be fair, the exceptions include several schools in North Dakota (migrating from Moodle) as well as Northwest Florida State College (migrating from D2L). But even with NW Florida State, they based their decision on Learn Ultra.

Based on interviews with clients arranged by Blackboard, and based on our own connections at BbWorld, what we consistently heard during dozens of interviews and from listening to panel discussions was that Learn Ultra Course View makes sense primarily for programs or schools that have not been on Blackboard Learn before. Bb Learn clients seem to have too many expectations of needing the same functionality they had before, pushing Ultra to be largely feature-compatible with Original and thus losing some of its simplicity in the process. When the Illinois Institute of Technology migrated from Learn self-hosting to SaaS, which enabled them to explore Ultra, they chose one program and a handful of faculty that were willing to jump into Learn Ultra, but for almost all others they are sticking with the Original Course View for the time being. The soonest they would expect moving primarily to Learn Ultra at the Course View would bet 3-5 years from now. You can hear the same dynamic in a recent Rod's Pulse Podcast (shared under CC-BY-NC-SA license and also available at Inside Higher Ed), with Rod Murray interviewing Rob McCunney about their school (University of the Sciences) and its migration to SaaS and Ultra. It is a fascinating first-hand description of their experiences. Please note that they use use the terms Traditional and Original interchangeably.

R Murray: We turned on the SaaS in January, but we really kept the Original Experience until July. In mid-July we flipped the switch to turn on the Ultra Experience. Now again for those of you are not as familiar with the way Blackboard works, that wasn't changing the course format. They were still traditional courses, but you know all those tabs and modules disappeared, and we ended up with a new base navigation in Blackboard, which they called the Ultra Experience. So that was a major change, and we do have some summer sessions, but we felt this was the least painful way to turn it on. We didn't want to wait till August just before our fall students came back.

In terms of turning on the Ultra Experience, what were some of the major issues that you saw, that you were concerned with?

R McCunney: Besides the fact that we lost the tabs and modules, so we kind of rolled out OneCampus as Rod said, I think one of the major things that that I noticed wasn't really even on the admin side, it was just getting people used to something that was completely different as soon as they logged in to that institution page. Where's my stuff, where is my modules? It just looks completely different than what they're used to, and there's some stuff missing, and we replaced it. We put stuff in other areas, and we communicated that, but that was probably the biggest hurdle. Just what is this, what is this Blackboard Ultra that you're changing me to? And at that point, in July we only changed basically the institution page. Your average user, once they get into their course they didn't notice anything different, but that initial freak out of here I don't know where my form is. We told them a dozen times where it is, but it's somewhere else. That was probably one of the major hurdles, and I didn't anticipate that as being a big as an issue as it was. I thought there was going to be more nuts and bolts issues, which there were very few of those for the most part.

R Murray: Right now of course we don't have that many students here in the summer, so the real test will be in another week or so when students come back and really start. We all start kicking the tires, even on the traditional courses within the Ultra Experience.


R Murray: The next big change that we have to live through has to do with converting courses to the Ultra course view. Now here at our university we certainly didn't do it en masse, we went to play with it for at least this term and maybe next. But there are some courses that we decided made a lot of sense to convert to the Ultra course view. Those schools that those courses that are brand new, especially online courses that are brand new, it made sense to develop them directly in the [Ultra] course view.

Schools are trying out the Ultra Experience in terms of the landing page and cross-course functionality, but by-and-large they are very cautious jumping into the Ultra Course View where most of the functionality resides.

Based on this situation, we believe that the migration to Learn SaaS might be a better indicator - at least in the short run - than Ultra adoption of whether a school plans to stick with Blackboard. When a school moves to Learn SaaS, they all tend to sign contract extensions for 1 - 3 years or at least internally plan no LMS migrations for more than 1 - 3 years. And the migration to Learn SaaS does not suffer from the vague terminology issues - a school either uses Learn deployed on SaaS (through AWS) or they don’t.

Learn SaaS progress slide

383 clients on Learn SaaS as of BbWorld 18 is good progress and easy to understand. This issue is what likely matters more to Blackboard clients today and for the next few years, but in the long run the company needs Learn Ultra to be accepted - including at the course level - in order to become more competitive and pick up new clients.

In the end, Learn Ultra is partially ready and does matter, more so in the future, but the Learn SaaS migration matters much more today. This answer is a real improvement over the situation a year ago and even from the beginning of this year, but it is still a far cry from a simple yes and yes answer that Blackboard would like to have.

Update: Fixed name of Rob McCunney

The post Blackboard Learn Ultra in 2018: Is it ready and does it matter? appeared first on e-Literate.

by Phil Hill at September 13, 2018 03:08 AM

September 11, 2018

Dr. Chuck

Two Abstracts for the Sakai Virtual Conference – November 7, 2018

These are two abstracts I wrote for the Sakai Virtual Conference ( on November 7, 2018. This is a great conference – it has a good number of tracks and is virtual so it can be attended by people around the world at very low cost.

OAuth 2.0, Public / Private Keys, and LTI 1.3, Oh My!

This session will cover the basic technologies that underlie IMS Learning Tools Interoperability (LTI) version 1.3 that makes use of OAuth 2.0, Public/Private Keys, and Java Web Tokens.  This session is about the technologies and there will be another session that will cover how LTI 1.3 is built into and works within Sakai 19.

Looking at LTI 1.3 in Sakai-19

This session will cover the end-user and administrator user interface for the support for LTI 1.3 in Sakai-19.   This session will not cover the implementation details and underlying technologies used to implement LTI 1.3 – that will be a separate session.

by Charles Severance at September 11, 2018 10:47 PM

Adam Marshall

Turnitin Maintenance: Sept 22 4pm – 12:30am (Sept 23)

A message from Turnitin.

To clarify, during the maintenance period (Sept 22 4pm – 12:30am (Sept 23))

  • it will not be possible to create or submit assignments via the Turnitin website
  • it will not be possible to create a new Turnitin-enabled assignment in WebLearn (although you can create a regular assignment and then enable Turnitin checking at a later date)
  • all submissions to Turnitin-enabled assignments made via WebLearn will be queued and checked once the maintenance has run its course

Dear Customers,

You may have noticed that some Turnitin customers recently experienced intermittent service interruptions or degradations. We apologize if you were one of the customers affected, and we are happy to report that our services are currently stable.

To prevent further issues of this nature, we will be performing some proactive emergency maintenance that will temporarily disrupt our services including:

  • Turnitin and TurnitinUK (including all integrations)
  • SimCheckOEM (TCA API)
  • Feedback Studio for iOS
  • iThenticate (including iThenticate via API)
  • WriteCheck

When will services be unavailable?

PDT Sept 22 8am – 4:30pm

BST Sept 22 4pm – 12:30am (Sept 23)

AEST Sept 23 1am – 9:30am

Is your time zone not listed? Click here to find out when you’ll be affected.

How will you be affected?

Users will be unable to submit and grade during this maintenance window, so please ensure that any submission deadlines are adjusted to fall outside of the window.

Thanks for your understanding and continued support of Turnitin.

Stay up to date with Turnitin system status by following @TurnitinStatus on Twitter.

Follow @TurnitinStatus



by Adam Marshall at September 11, 2018 09:16 AM

September 07, 2018

Adam Marshall

Improvements to WebLearn in version 11-ox12

WebLearn was upgraded to version 11-ox12 at the end of August: this was a small release.


  • Lessons, the correct list of content providers is now listed on the More Tools > External Tool Import’ page – this is the page where IMS Content Item ‘objects’ are linked to or embedded
  • Lessons – embedded H5P objects now include a “resize script” which prevents scroll bars from appearing on the H5P item
  • The spreadsheet export from Gradebook now contains a user’s SSO (or email address for external users) in addition to their full name.
  • Resources can now export an IMS Common Cartridge (as well as a ZIP file) – this is useful for moving material to Canvas – more work in this area is planned

There is a separate blog post about using the brilliant H5P.

by Adam Marshall at September 07, 2018 01:14 PM

September 03, 2018

Apereo Foundation

Apereo Foundation RFP for CPA Firms

Apereo Foundation RFP for CPA Firms

The Apereo Foundation is requesting proposals from CPA firms that have extensive experience in providing nonprofit accounting, as well as 990 review and preparation services for our organization. Proposals due to us by Noon on September 28 for consideration.

by Michelle Hall at September 03, 2018 06:33 PM

August 31, 2018

Adam Marshall

Free Apereo OAE service to close

We recently received this from the people at *Research about the *Unity platform. (*Unity is another name for the Apereo Open Academic Environment)

Dear Colleague

*Unity will close down on 10 September 2018. After that you won’t be able to use it any more, so if you have any material you want to keep hold of, you need to download it before then.

We’ve taken this decision because, although *Unity was popular with many users, it wasn’t adopted by university IT departments. Without their support, we can’t make a success of it.

Our goal was to create an open platform for cooperation between academics everywhere with shared governance, so that academia could reclaim space that has been colonised by big technology players. This idea is explained here: But, with some notable exceptions, it proved impossible to get university IT departments to buy into this vision.

An FAQ for dealing with practical issues has been posted at and you can also ask questions to Please note that if we receive large numbers of queries it may take some time to reply.

Thank you for using *Unity and good luck with your studies.

With best wishes,

Martin Atkinson
Technology Director

by Adam Marshall at August 31, 2018 03:19 PM

August 20, 2018

Apereo Foundation

2018 Apereo Webinars

2018 Apereo Webinars

Learn something new about Apereo projects by joining the community at these four webinars held in September, October and December 2018. 

by Michelle Hall at August 20, 2018 05:12 PM

2018 Apereo Webinars

2018 Apereo Webinars

Learn something new about Apereo projects by joining the community at these four webinars held in September, October and December 2018. 

by Michelle Hall at August 20, 2018 05:12 PM

August 15, 2018

Sakai Project

Now Open! Call for Proposals for the Sakai Virtual Conference 2018

Sakai Project Logo

We are actively seeking presenters who are knowledgeable about teaching with Sakai. You don’t need to be a technical expert to share your experiences! Submit your proposal today! The deadline for submissions is September 21st, 2018.

Save the Date: The Sakai Virtual Conference will take place entirely online on Wednesday, November 7th.

by MHall at August 15, 2018 06:58 PM

August 13, 2018

Sakai Project

Sakai Community Survey - Number of Users at Your Institution

We would like your help in tallying up the total number of Sakai users worldwide.

by MHall at August 13, 2018 04:33 PM

August 01, 2018

Sakai Project

Sakai 12.3 maintenance is released!

Dear Community,

I'm pleased to announce on behalf of the worldwide community that Sakai 12.3 is released and available for downloading [1]!

Sakai 12.3 has 65 improvements [2] including:

  • 27 fixes in Assignments
  • 7 fixes in Tests & Quizzes (Samigo)

Other areas improved include:

by MHall at August 01, 2018 05:05 PM

July 04, 2018


F2F Course Site Content Import

If you’re tasked with teaching an upcoming course that you’ve taught in the past with the University – there’s no need to rebuild everything from scratch – unless you want to.

Faculty teaching face to face (F2F) courses can benefit from the course content import process in Site Info. This process allows you to pull in all your assignments, syllabus, gradebook, handouts and other files associated with the course – as used in a previous offering of the course.

To do this, you need to be an instructor in both course sites (the former and the upcoming). Go to the upcoming course site, and select Site Info>Import from Site:


Next, select the kind of import you wish to perform. I typically suggest using the replacement option “I would like to replace my data”. On the next screen select which course you’d like to pull content in FROM.  Be careful here making sure you select the SOURCE of the content you’ll import. Next click Continue.

On the next screen select the tools/areas of content you wish to import. Keep in mind it’s always a good idea to import the Resources, because files referred to in Assignments, Quizzes, Lessons or Announcements could refer to those files, and in order for those links to work properly the corresponding resources must be likewise imported.

Finally complete the import process and watch for the email to be sent to you – notifying you of the import process being completed. You can find out more information about the process here.

Want to watch the whole process in real time? Take a gander here:

by Dave E. at July 04, 2018 06:56 PM

June 11, 2018

Apereo OAE

Strategic re-positioning: OAE in the world of NGDLE

The experience of the Open Academic Environment Project (OAE) forms a significant practical contribution to the emerging vision of the ‘Next Generation Digital Learning Environment’, or NGDLE. Specifically, OAE contributes core collaboration tools and services that can be used in the context of a class, of a formal or informal group outside a class, and indeed of such a group outside an institution. This set of tools and services leverages academic infrastructure, such as Access Management Federations, or widely used commercial infrastructure for authentication, open APIs for popular third-party software (e.g. video conference) and open standards such as LTI and xAPI.

Beyond the LMS/VLE

OAE is widely used by staff in French higher education in the context of research and other inter-institutional collaboration. The project is now examining future directions which bring OAE closer to students – and to learning. This is driven by a groundswell among learners. There is strong anecdotal evidence that students in France are chafing at the constraints of the LMS/VLE. They are beginning to use social media – not necessarily with adequate data or other safeguards – to overcome the perceived limitations of the LMS/VLE. The core functionality of OAE – people forming groups to collaborate around content – provides a means of circumventing the LMS’s limitations without selling one’s soul – or one’s data – to the social media giants. OAE embodies key capabilities supporting social and unstructured learning, and indeed could be adapted and configured as a ‘student owned environment’: a safe space for sharing and discussion of ideas leading to organic group activities. The desires and requirements of students have not featured strongly in NGDLE conversations to this point: The OAE project, beginning with work in France, will explore student discontent with the LMS, and seek to work together with LMS solution providers and software communities to provide a richer and more engaging experience for learners.

Integration points and data flows

OAE has three principal objectives in this area:

  1. OAE has a basic (uncertified) implementation of the IMSGlobal Learning Tools Interoperability specification. This will be enriched to further effect integration with the LMS/VLE where it is required. OAE will not assume such integration is required without evidence. It will not drive such integration on the basis of technical feasibility, but by needs expressed by learners and educators.
  2. Driven by the significant growth of usage of the Karuta ePortfolio software in France, OAE will explore how student-selected evidence of competency can easily be provided for Karuta, and what other connections might be required or desirable between the two systems.
  3. Given the growth of interest in learning analytics in France and globally, OAE will become an exemplary emitter of learning analytics data and will act wherever possible to analyse each new or old feature from a designed analytics perspective. Learning analytics data will flow from learning designs embedded in OAE, not simply be the accidental output that constitutes a technical log file.

OAE is continuing to develop and transform its sustainability model. The change is essentially from a model based primarily on financially-based contributions to that of a mixed mode community-based model, where financial contributions are encouraged alongside individual, institutional and organisational volunteered contributions of code, documentation and other non-code artefacts. There are two preconditions for accomplishing this. The first, which applies specifically to code, is clearing a layer of technical debt in order to more easily encourage and facilitate contributions around modern software frameworks and tools. OAE is committed to paying down this debt and encouraging contributions from developers outside the project.

The second is both more complex and more straightforward; straightforward to describe, but complex to realise. Put simply, answers to questions around wasteful duplication of resources in deploying software in education have fallen out of balance with reality. The pendulum has swung from “local” through “cloud first” to “cloud only”. Innovation around learning, which by its very nature often begins locally, is often stifled by the industrial-style massification of ‘the hosted LMS’ which emphasises conformity with a single model. As a result of this strategy, institutions have switched from software development and maintenance to contract management. In many cases, this means that they have tended to swap creative, problem-solving capability for an administrative capability. It is almost as though e-learning has entered a “Fordist” phase, with only the green shoots of LTI enabled niche applications and individual institutional initiatives providing hope of a rather more postmodern – and flexible - future.

OAE retains its desire and ambition to provide a scalable solution that remains “cloud ready”. The project believes, however, that the future is federated. Patchworks of juridical and legal frameworks across national and regional boundaries alone – particularly around privacy - should drive a reconsideration of “cloud only” as a strategy for institutions with global appetites. Institutions with such appetites – and there are few now which do not have them – will distribute, federate and firewall systems to work around legislative roadblocks, bumps in the road, and brick walls. OAE will, then, begin to consider and work on inter-host federation of content and other services. This will, of necessity, begin small. It will, however, remain the principled grit in the strategic oyster. As more partners join the project, OAE will start designing a federation architectural layer that will lay the foundation to a scenario where OAE instances dynamically exchange data among themselves in a seamless and efficient way according to a variety of use cases.

ID 22-MAY-18 Amended 23-MAY-18

June 11, 2018 12:00 PM

May 01, 2018


Will Sakai look different following the upgrade?

While there are some improvements to accessibility and some on-going tweaks to improve color contrast issues, the upgrade to Sakai will not affect the overall appearance that much.  For mobile users – the difference in course navigation will be much-improved.

Desktop/Laptop view:

Sakai 11
Sakai - Pre Upgrade Desktop View

Following Upgrade:
Sakai - Post Upgrade Desktop View

Mobile view (Sakai 11/Post-Upgrade):
Sakai - Pre Upgrade Mobile View  Sakai - Post Upgrade Mobile View

More detail will be distributed in the coming weeks and those following the upgrade.

by Dave E. at May 01, 2018 07:53 PM

Gradebook Calculation Anomoly

In what appears to be a gradebook calculation anomaly, be sure items are categorized appropriately even if you course is only using categories for organization – otherwise final course grade calculations may be inaccurate – as the following video explains.


To address categorization of an item, check the Gradebook>Settings>Categories and Weighting to insure you’ve setup the gradebook correctly (specific to each course).  Next insure all items which have bearing on the overall grade are INCLUDED in the course grade calculation – making sure they DO NOT have a calculator with a slash through it AND that they are not in an uncategorized category:


by Dave E. at May 01, 2018 06:23 PM

April 01, 2018

Aaron Zeckoski