You clicked on a link, so you read this, right? (Or, some thoughts on learning analytics.)

Most recently, I sat through a series of webinars about learning analytics that was sponsored by EDUCAUSE as well as the Bill and Melinda Gates Foundation, and IBM, too. At my institution, there is some interest in learning analytics, but administrators are still feeling this out and trying to determine what the purpose of learning analytics will be.

My initial observations of some the issues are how LA, which appears to have the purpose of modeling an individual's performance and then taking some action based on that model, handles complex statistical procedures. Not the most exciting topic for a blog entry, so I'll be brief.

1. What is the process for model selection in an LA model that encompasses a variety of variables? One question that is important to address is the process of model selection. That is, when researchers develop a prediction model, we use a procedure to select the variables that become part of the model. This procedure can be an automated procedure in a software package (such as stepwise model selection) or a deliberate procedure of testing one variable at a time (by hand) to determine the first variable in the model, the second variable, and so on until you have the final model. In LA, how does this happen? Is one model applied to the data regardless of the quality of fit? Is it transparent?

2. We must be careful to avoid saying that prediction models predict an individual's exact performance. There is always uncertainty. Statistics--especially inferential statistics--is a science that embraces error and uncertainty. That is, a prediction model always has some degree of error--and even a good model predicts the average response of some outcome variable. If I say that my prediction model indicates that this student should receive the GPA of 2.9, then I'm assuming that every student is going to have the average outcome. Some will get higher GPAs, some will get lower. If we do LA, we need to make a commitment to helping our colleagues understand the nature of error.

3. In LA from automated systems like LMS's can we be sure of the data we're analyzing? It is tempting to assume that a page view in a content system means that students read that page. Perhaps they did. As researchers, we may need to develop research methods that enable us to learn about the quality of the interaction in a system like an LMS. This can be done with some kind of instrument that asks students more about how they interacted with some electronic resource. Surely, some systems are catching up to this by enabling more interactivity (such as e-textbooks that allow highlighting, note-taking, and sharing of questions with instructors), but we should be skeptical of each variable we consider in a LA procedure.

E-text Unconference (Day 1)

I graduated from the University of Illinois, where this unconference is being held, about 17 years ago. I jogged through campus this morning and saw that most of the businesses that were here are gone--except for Espresso Royale, Lando hair salon, and the textbook stores. I even worked at one of these textbook stores that would clear amazing amounts of money during textbook rush.

So it's interesting to be at a conference at which there are people who are trying to develop e-textbook platforms and others who want to know where the business of textbooks is headed.

During day 1 of the conference, there was a short presentation by a speaker (a consultant) who talked about the business model for publishers. He argued that the more e-textbooks contain personalized content, the more they will see a business value. It was not clear why. Would publishers want to control the kinds of annotations that students add to e-textbooks or would they use that data to market products? (The speaker referred to this as "differentiated content.") The speaker acknowledged research (from Daytona State College) that showed that students were not enthusiastic about using e-textbooks again after using them for a semester. (One finding of this research was that students preferred renting paper textbooks.)

We voted on which 7 topics to talk about in small groups. Winners were student receptivity, technology platforms, large e-text initiatives--and I forgot the other four. Accessibility was not one of them, and one person in the crowd spoke up and asked where that could be discussed.

At the student receptivity discussion, we talked about a variety of ways that schools have tried to measure students' acceptance (or avoidance) of e-textbooks. There are many librarians here--one says that it is logical for them to work with e-textbooks because libraries have existing relationships with publishers. One large school has conducted a big pilot with an e-textbook provider and found that the majority of students did not want to use an e-textbook again. Also, he said that most of the students (and instructors) did not take advantage of the annotation tools in the e-reader (and there was no relationship between e-text usage and students' grades). They are doing the pilot project again in Fall 2012 because they want to try the system again now that it has been improved. Lots of discussion about how students like paper textbooks, like sharing them with classmates, like selling them back. One finding was that students who did not like the e-textbook felt that it was too distracting to use an e-textbook on their laptop, where they were tempted to facebook, surf the web, chat, etc. There is concern that one e-reader provider would like institutions to use their assessment instruments. Some schools have run into trouble trying to ask their own questions.

So the people involved with companies who develop e-readers have had mixed results...some very enthusiastic people are here, working on their own e-textbook initiative at UIUC. Self publishing of textbooks is big at UIUC. At the table where they talked about initiatives to publish e-textbooks locally, it was all UIUC people. And they did not all know each other. Some UIUC faculty, who have written their e-textbooks, are here to discuss their processes. One chemistry professor has made an online app for chemical equations, which is integrated with his online e-textbook and is working well for him. UIUC seems exceptional in this area of self publishing.

The UIUC undergraduate writing program has developed an e-textbook that they sell to their students. It is an HTML 5 e-textbook with annotation tools, and is accessible. They wanted to build an accessible environment because the companies that make e-readers have often not paid enough attention to accessibility and just put "pictures of textbooks" in their systems that do not meet accessibility standards. To purchase the writing e-textbook (used by all students in undergraduate rhetoric), students go to the textbook store and purchase a scratch-off card with a code that they use to access the e-textbook. They must buy this because the entire course "happens" in the e-textbook--my understanding is that without the e-textbook students cannot complete assignments. They need their own. They can download an epub version that they can use offline, but they won't have access to their annotations. Essentially, there is one e-textbook, and students' annotations live on a database. Each time they log-on, their annotations are pulled down to the e-text they are viewing. (When the developers were building this, they asked a professor if she would want to receive an e-mail each time someone created a highlight. She said no way.)

There isn't much research about this initiative. The head of undergraduate writing program won a national award for this innovation in instruction. They have collected a bunch of surveys that they are still analyzing.

More thoughts to come...

Taking Time to Reflect on Learning in MITx

Last week, I decided to return to practice problems in week 1 and complete some of the problems that I did not have time to finish. (Each week I do as many practice problems as I can until working on the homework is all I have time for.) I was happy to see that I could answer these problems (which were about node analysis and voltage divider circuits, two relatively basic concepts) without too much difficulty. I am learning, I thought to myself. I need to spend this time going over what I have learned and making the abilities my own, so to speak. In MITx I am always pushing ahead as diligently as I can and not spending time reflecting on my learning.

The designers have been changing small things in the course. It's interesting to see the changes, such as making the current text in the transcript bold when it is spoken in the video. The word "SPEED" now appears next to the buttons for changing the speed of the videos. Other students make suggestions for changing the interface or adding an interface for mobile users. Not me--I'm usually coming from behind…

Which reminds me: three cheers for Wolfram Alpha! When I needed to compute derivatives for the incremental analysis, I only need to type the equation into the computational engine. (My calculator is not good enough for this.) Students were suggesting a variety of tools--such as a math program you can download from Microsoft.

Still, you know you're in deeper waters when you can't solve the problem on your calculator that was sufficient for any other class you've taken. I'm uneasy that I'm avoiding brushing up my calculus--who am I kidding?--whole new coats of math are needed here! I was able to get a decent grade on the homework, but left the last problem untouched because I could only get through half of the lectures for the topic. I am starting to feel the effects of not having the prereqs. I wonder if MITx will have to enforce a certain prior level of knowledge before letting people take a course. Surely I'm not the only one with these problems...but for the moment this may not be a main concern.

The first exam was announced: it will be in the last week of April, and we have 24 hours to complete it once we begin. I've already marked the calendar and told my wife--who, by the way, did a bunch of my chores to give me time to complete MITx work.

Completing Labs in MITx: Thank Heaven for Trial and Error

Well, I solved Lab 3 with help from my wife and a hint from the discussion forum, and some late night persistence. In this problem we needed to build a series of digital gates that are connected to three voltage sources. Professor says an Intel chip has a billion gates, but our lab involved three, so we start small. Really small.

In Circuits and Electronics, we must complete a weekly lab, an assignment in which we must build a circuit that satisfies certain parameters. Part of the assignment involves getting the appropriate configuration of resistors, wires, and other elements, and another part of the assignment means getting the right values for these elements. It’s a rather safe way to do a lab in basic electrical engineering; I know the only shock I’ll receive is how many times I get the answer wrong.

A lot of my problems with this lab were due to my own ignorance about how to use the circuit sandbox. First, I did not know that you could cross wires in a diagram, which, it turned out, was necessary for the solution to the problem. Of course, crossing wires is acceptable because in an actual circuit, a wire can go over or under a different wire. In my crude diagram, you can see that the red voltage source (the circle) needs to be connected to the red gate (the diamond), so the wire must go over the other gate. I finally found a discussion posting that mentioned that you can cross wires, but another person said that this is in the directions in how to use the sandbox, so I’m to blame for not reading all the directions….One weakness I have is that I am spending more time trying to understand equations than trying to understand the common-sense rules for putting these things together.

But a funny thing is that I was able to get the correct answer by using trial and error to determine the resistance values for these gates. This seems rather unscientific to me, but one hour before the deadline makes me happy for any advantage I may get. And yet, wouldn’t a situation in an actual lab be much much different? For example, at one point, I put the resistance value of a MOSFet so high that I shorted part of the circuit. Now, in the sandbox, who cares, right? But surely this kind of guessing could be dangerous in a real circuit.

One thing I don’t like about the circuit sandbox is that I use “backspace” to delete elements after I’ve selected them. If I hit the button more than once, I go back in the web browser and lose my circuit.

Where's the show-your-work-for-partial-credit button in MITx?

I stayed up late on Sunday trying to complete the second lab, which--how do I explain i?--involved building a circuit with a set of resistors that produced a certain maximum and minimum output voltage. At one point during the day, I had posted a question in the forum about a concept involved in the problem (the concept of superposition) and some nice people directed me the textbook to read more about it.

But I also came across an answer that included a link to another posting that, I assumed, discussed the concept in more depth. Well, it included a possible answer to the lab. This is another difficulty I have with MITx: avoiding posts that include explicit answers while I am looking for guidance on concepts. I can't unsee those answers, unfortunately.

The Honor Code forbids the posting of answers to graded assignments, but people still post these things. Sometimes people post equations that include every step for solving a problem and only leave out the answer. Really, as a participant in this class, I must keep my own honor code and be judicious when I consult the discussion forum for help with concepts. I phrase questions to make it clear that I just want help understanding a concept.

So my dilemma was whether to use the information I saw--which I would not have understood on my own--or go back to trying the lab by myself. After my poor wife stayed up trying to help me, I decided to give up and take the 0. They drop the two lowest lab scores anyway.

Still, this got me thinking about how MITx does not allow partial credit. I had tried out several different possible answers, and was able to get the correct minimum output voltage. Shouldn't that count for something? The circuit is either correct or incorrect. Perhaps this is a feature of large, free online courses: there aren't the teachers who can scan through thousands of answers to see if the answer deserves partial credit. But I can't help but wonder why it is helpful for students to get a perfect score on a lab or no points at all. Could the labs be broken into several steps in which students could receive credit for completing part of the system correctly?

Today, I looked up the answer to the lab and then posted a question in the discussion forum about the circuit diagram. In ten minutes, I had received two answers from people, and I think I understand my mistake now. And this is what I wanted all the time.

Week 2 in MITx: Feedback in the System

So homework 2 has proved to be quite a bit more difficult. There is no easy question like there was in the first homework. Now I must apply the Thevenin Method to solve a circuit with input and output voltages. Never did I think turning on the light could be so complicated (it actually is probably a simple circuit). Today, my iron broke and I briefly thought about trying to take it apart to find the problem. But I'll probably just get to Target for a new one.

What is the best kind of feedback that you can receive in a massive online course? Lately, I get the red "x" a ton as, there's no denying it, I have a harder time in the class. (Reminder: your answers are assessed immediately by the system: green check means yay!, red x means grr.) The red X propels me to keep studying…I try to think of it as x marks the spot you need to study more.

Feedback has been studied extensively--a significant meta-analysis of feedback was published in 2007 that indicated that cues had the highest effect size. Simple praise had a very small effect size. And feedback about a correct answer had a greater effect size than feedback about incorrect answer--this is particularly difficult feedback to get in a large online course in which I sometimes don't fully understand the answers I'm getting.

The research says to seek out cues, and I could use more cues in my work in MITx. I seem to be able to find them by consulting the discussion forum at just the right time, with a posting that does not give away the entire answer.

More thoughts about peer tutoring in MITx. I posted my first question (about how to identify parallel resistors) and shortly received several responses. In the system, I was able to choose the correct answer--and I was gratified to see that I could choose more than one correct answer. This system of allowing people to accept answers helps to close discussion threads that might continue even after they are useful to people. Being able to accept more than one answer shows that MITx values multiple perspectives and different ways of explaining a concept.

MITx Homework 1 and Lab 1 are finished. Whew.

his weekend I finished both the first homework and the first lab, and I received perfect scores on both. This was not without a lot of work, and several late nights while I scoured the textbook, scribbled circuits and formulas all over sheets of paper, and checked the discussion forum for hints on ways to solve the more difficult problems. Just an hour from the deadline, I am starting to see posts from people who want others to post answers in the discussion forum. In response, the staff or other users with enough karma points are closing those threads or reminding people that homework answers cannot be posted in the discussion forum.

Just a reminder: in MITx, the homeworks are online. When you solve a problem, you type the answer into a box, press the "check" button, and--if you're correct--you see a green checkmark beside the field. Some problems are so hard that I do small dances whenever I can get that green checkmark to show up. Most of the time I get the red X, which means, "Try again." The problems have several parts, so you need to get that first answer correct so that it can be plugged into another equation and then your answers fall like dominoes (well, like slowly falling dominoes).

I am grateful, most of all, that the course (so far) has not required any math that is more complicated than algebra. If anything, the problems are hard because you develop a lot of independent equations for different variables (voltage, resistance, current) that you can solve for by a process of substitution. The homework took me a long time--at least 5-10 hours over several days, I think. A lot of my learning seems to happen in the constant process of trial and error--coming up with equations, solving for values, trying out the result.

I have learned that one weakness in the system is that I have gotten several answers by just changing the sign of the value I entered in the box. For this reason, I am still pretty weak on identifying whether currents associated with voltage sources are positive or negative. I have changed x to -x and gone from wrong to right, but I still don't know why I was wrong the first time.

I find that I do get many of the equations or heuristics for solving circuits from the lectures or reading, but the discussion forums contain nuggets of information that I need for solving questions. I look for posts that contain a possible way of thinking about a problem without actually solving the problem. That is hard because part of me just wants the answer.

I am holding my breath that I can continue to succeed despite not having the prereq--I fear that Coulomb's law and electric fields are stalking me in the magnetosphere. But the assumptions of the lumped matter discipline have created a "playground," as the professor calls it, that make things simple (though not easy).

When I checked my grade, I saw that I have earned 3% of the final grade. That 3% is going to be polished and put in a case.

Creating My Own ZPD?

How hard it is to not just look up answers! In the practice problems, there are links to the discussion forum where people put up the answers to people's questions. Today I was working on a problem involving Kirchhoff's laws, and I was unsure about the sign that should be applied to a specific voltage for different elements in the circuit. I thought I'd go to the forum and read about signs of voltages, but the page I landed on had the answer front and center. I tried to look away--was that the answer I saw? It was. I put it in and it helped me think about the rest of the problem--for which I needed to keep myself from looking at the rest of the answers.

I did see that one of the T.A.'s posted a response asking people not to post answers in the discussion forums. And by the way, there are some really smart people posting answers; I wonder how many people taking this class have already passed a similar course at another institution.

After seeing the answer I returned to the problem, and began trying out answers. Eventually, I went back to the textbook and read about how to assign the correct sign to a voltage when I am examining elements in a loop. This gave me the key I needed to answer the rest of the questions about how to solve for different elements in the circuit.

I know that some struggling is part of the learning--necessary for keeping that development moving along, to crudely paraphrase Vygotsky. But in MITx, I know that the answers are not far away. It's as easy for me to find the answer as it is to check e-mail. With some fortitude, I was able to figure out the rest of the answers. This makes me think about the zone of proximal development and how this zone is defined in activity when my "more knowledgeable peer" is the online textbook, or a post in a discussion forum. The particular difficulty I have is determining exactly what I don't know, or (in other words) my potential level of development and the right activities for developing a conception that is closer to my potential level. In MITx, I am painfully aware of my actual level of development and am grateful for the tips

How is my understanding of circuits changing--I am learning more about resistance and the different kind of elements within a lumped circuit. It really is simplified by Kirchoff's Laws that turn circuits into junior high math problems, provided you can figure out where to put the sign on currents and voltages, which I am learning to do, though very slowly.

Into the First Unit

It's amazing how far I can get by understanding basic relationships in equations. Such as, I have no idea what a linear resistor, but when I'm asked to identify a graph of current over voltage, I know that an independent current will only have one value on the y axis. I find that this kind of deduction is getting me by quite well so far. So far the equations I'm using are linear expressions, thankfully. One smart thing that MITx does: they demarcate the video lectures according to where there is a question that I'm supposed to spend time thinking about. For example, the professor has an empty graph and asks me what I think a relationship will look like. The video ends and I'm supposed to start the next one when I'm ready to go. Below each exercise is a link to a discussion forum where I may post questions or read answers to other questions. Please note: open these links in a new tab. These discussion forums are really helping me understand the concepts behind power entering a circuit. Did I also mention that I'm very aware of my point values, how many points I do or don't have? When the professor was recording his lecture, his dog was barking. I can't say why that makes me feel better about this class: if he has a dog, he probably tolerates people that make the same mistake over and over.

My progress on MITx: Lab Tutorial

Today I worked on the lab tutorial in which you practice using a tool that lets you build and test circuits. It's a point-and-click interface in which you drag and drop things like voltage sources and resistors. The cool thing is that the system tests the circuit and displays the values of resistance. I don't really know much about circuits, but I was able to build the one based on the copy that the system provided. Currently, my conception of a circuit is that it is a device that supplies electrical current to a simple or complex machine, but I may be very wrong about that. I found the total resistance and the current flowing from the positive to negative.

The next few problems in the lab tutorial seem harder, and I will likely take a while to answer them. I have noticed that there is a lot of help in the discussion forums, if I can find the right answer to my question. I have gotten better at using the labels to find answers. (Although sometimes someone posts something weird, such as a question about why the Apple store is down. The moderators shut that one down pretty quick.)

On a side note, I recently received the badge of "Teacher" because I answered someone's question about the function of the check button. This seems more like a "Teaching Assistant" function, but I find myself pleased to have received this acknowledgement. Also, it is nice to help someone else who is working on the course.

While I'm doing this, I keep asking myself what I'm learning, and so far I think I'm learning more about how to operate the interface. Of course, this is very important.