Section 1.1 The Runestone Philosophy
The primary reason that Runestone was created was to be an interactive eBook server that would motivate students to actually read and experiment with the content provided by the author. This idea was born in my head in the fall of 2010 as I was suffering from a serious case of writers block while trying to work on a second edition of a paper textbook. Why should textbooks be static? Especially computer science textbooks!? From this frustration I stopped working on the update and started trying to create an interactive computer science textbook. My primary motivation was to make every example “runnable” and “editable” so that students could read code, run code, and then modify that code to see how it behaved differently with their changes.
Having come to Luther College from the
GroupLens research group and an internet startup company, I also knew the value of collecting data. So I also set out to make sure that every action a student took within the textbook was captured. I had no idea what we were going do with the data, but my intuition was that we could learn valuable lessons about
how students were learning if we captured and analyzed enough data. Collect the data first and then figure out how to make use of it later.
Everything else you see about Runestone Academy today came later, and I’ll cover more of that evolution in
Section 1.5. But the takeway is that for us the student experience is always foremost in our minds. Everything that you as an instructor can do with Runestone follows from that philosophy of collect the data first and then see what we can do with it.
I had never heard of the term
Learning Engineering until the summer of 2021, but that is really what Runestone is and has always been about.
Wikipedia defines Learning engineering as follows Learning Engineering is the systematic application of evidence-based principles and methods from educational technology and the learning sciences to create engaging and effective learning experiences, support the difficulties and challenges of learners as they learn, and come to better understand learners and learning. It emphasizes the use of a human-centered design approach in conjunction with analyses of rich data sets to iteratively develop and improve those designs to address specific learning needs, opportunities, and problems, often with the help of technology.
Today (August 2022) if you asked me I would say that Runestone is a
LEAP. A learning Engineering and Analytics Portal. Many of the innovative features in Runestone have come from first class pedagogical research. All of Runestone has been evolved over the years by paying attention to the data we collect and the feedback we get from instructors and students. All of this has been possible because we realized early on that most instructors do not want to run and manage their own servers, and that we could provide a valuable service by providing a portal to free, high quality interactive textbooks.
The diagram below illustrates what we are trying to create. Which is a virtuous circle whereby students learn more effectively, teachers can learn to teach more effectively, and authors can learn to write better textbooks.
All of this is to say that although Runestone supports many of the functions of an LMS, Runestone was never designed to be an LMS. Runestone is much more, and in order for you as an instructor to get the most out of Runestone you may need to adjust your thinking and expectations. Hopefully this guide will help you do that.
You have attempted
of
activities on this page.
Section 1.5 A Brief History of Runestone
The Runestone project began in on my sabbatical in 2011. I’ve already mentioned that I had serious writers block because I wanted computer science textbooks to be interactive. The first stage in fixing that problem was finding a way to run Python in the browser. This led me to the
Skulpt Project Scottie Graham had implemented Python 2.6-ish by re-implementing CPython in Javascript. While the core was there Skulpt was missing many pieces, such as a turtle graphics module. Since I loved using turtle graphics as a way for students to get fun, immediate feedback on their code I set about trying to add turtle graphics to skulpt. Since I knew very little about Javascript that sucked up a couple of months of my Sabbatical, but was a lot more fun than updating a textbook. With turtle graphics in hand I knew I had hit on something!
The next problem was that I knew that as an author I didn’t want to have to write a 100 lines of custom javascript for every example! I needed a way, ideally a really simple way of just surrounding some python code with a wrapper and letting a tool build the html and javascript needed to create a widget on the webpage that ran the Python. This led me to
Sphinx Sphinx is a wonderful writing tool, widely used in the Python community. It also has a rich collection of extensions, and a syntax that is just what I was looking for! With sphinx I could write something like:
.. activecode:: some_id
print("hello world")
Sphinx would turn that into html that looked like:
<div class="runestone explainer ac_section ">
<div data-component="activecode" id=over_ac_example1 data-question_label="1.1.1">
<div id=over_ac_example1_question class="ac_question col-md-12">
</div>
<textarea data-lang="python" id="over_ac_example1_editor"
data-timelimit=25000 data-coach="true" data-codelens="true"
data-audio=''
data-wasm=/_static
style="visibility: hidden;">
print("My first program adds a list of numbers")
myList = [2, 4, 6, 8, 10]
total = 0
for num in myList:
total = total + num
print(total)
</textarea>
</div>
</div>
</pre>
Stir in some javascript and you have a working runnable example! That looks like this:
With Sphinx I had a way to write large documents in a very simple markup language called
restructuredText and a way to extend the markup with interactive textbook elements, which at that time was still only running python. But it gave me a huge sense of optimism that my goal of having an interactive textbook was achievable.
The next problem was to translate our textbook called
Python Programming in Context from LaTeX to restructuredText and converting the examples to be runnable. But Jones and Bartlett publishers had different ideas. I had signed away the digital rights to the book in our original contract and Jones and Bartlett were not willing to let me convert the book. They were going to sells PDFs along with some kind of digital code to a code practice system online. I was really depressed, I loved that book and its approach. I hated the publisher (and still do) for their greed and short sightedness. I didn’t want to start over and write yet another intro to CS book from scratch. So I started looking for something else to start with. This led me to
How to Think like a Computer Scientist by
Allen Downey and
Jeffrey Elkner. This book at a GNU Free Documentation License, and was alread in restructuredText!! So I remixed that book and added interactive examples which led to the interactive edition of that book.
The publisher of My other book,
Problem Solving with Algorithms and Data Structures using Python was
Franklin Beedle. I’ll never forget the conversation I had with Jim Leisy, our publisher, at SIGCSE. I had just given him a demo of a couple chapters of the interactive How to Think book and asked him if we could do the same thing with our data structures book. He said, “I’ve always wanted to change the world, and this looks like my chance, lets do it!”
In the Summer of 2011 I was at the ACM Awards ceremony in San Jose, I was there as part of the team accepting the
Software System award. At the banquet I started a conversation with Mark Guzdial and Barb Ericson who were accepting the Karl Karlstrom outstanding educator award. I told them about Runestone and that conversation started a pivotal partnership that has lasted to this day. Their reaction was this is wonderful and the kind of research tool we have been looking for. Now can you add multiple choice questions? fill in the blank? And many many more ideas including Parsons problems, which were the subject of Barb’s PhD research.
The next several years were super busy, I managed to find money to get Luther students to work with me on Runestone during the summer months. In the summer of 2014 we had the first Runestone Hackathon at the University of Michigan. This workshop was pivotal because it was the beginning of the instructor interface. I had always resisted adding too many instructor facing features to Runestone, I did not want Runestone to become another crappy LMS!! But Paul Resnick and Barb Ericson convinced me that adding features to make the lives of instructors better was necessary, and that we could do things that LMS could never dream of doing to help teachers understand their students better. We did a lot of whiteboard designing that year and worked on it until it all really came together in the summer of 2016 when we had the second hackathon at Luther College.
By the late Fall of 2017 I was developing a real desire to focus all of my efforts on Runestone. But I wasn’t sure how to make that happen until Luther announced an early retirement incentive. Anyone with 15 years of service to the college was eligible and could get a years salary and stay on the college health plan for five years. This was a dream come true, I just met the 15 year criteria and the years salary would give me time to figure out how to make Runestone sustainble in a way that could pay me some kind of salary.
Things got even more interesting in the Spring of 2018 when I was offered a consulting gig with Google. They wanted to use Runestone as a platform for their applied computing program. In a nutshell the program’s goals were to introduce computer science to colleges with underserved student populations that had not offered computer science before. I basically got to work on Runestone and coach professors from the participating colleges in how to use Runestone and how to teach introductory CS. The program expanded into a second course to introduce Data Science to those colleges, which resulted in the How to Think like a Data Scientist book. My gig at Google was supposed to last 5 months, but it ended up lasting two years.
Sometime in the early2019 I received an email from Oscar Levin that read as follows
On behalf of the workshop organizers, Jim Fowler, Mitch Keller, Matthew
Leingang, and Oscar Levin, we would like to invite you to participate
in the workshop
"Interactive assessments in open source textbooks"
from December 9 to December 13, 2019.
I didn’t know anything about
AIM, and I had never met Oscar but the workshop sounded interesting. So I said yes. When I arrived at the first day of the workshop and listened to the introductory talk, I knew that I had found a group of people I wanted to work with more. Many of the people in this workshop were authors of open source math textbooks written in
PreTeXt. A project started by Rob Beezer at about the same time I had started Runestone.
It was early in the afternoon of the first day of the workshop when I found myself in a room with David Farmer, and after I had given a quick demonstration of Runestone and its capabilities he said something like: “This is great, but I don’t know why you would want to keep developing this Runestone Markup language when you could just use PreTeXt and focus your efforts on other more valuable work.” It was one of those direct statements that hit me hard, should I be offended? embarrased? honored? It took me a couple of days and getting to know both David and Rob better to realize the wisdom of that statement. But by the end of the workshop we were able to host a version of Rob’s book A First Course in Linear Algebra on a Runestone Server. We also realized that our goals were very well aligned and that we could accomplish more by working together than we could by continuing to work separately.
It took an entire global pandemic for Rob and I to really integrate the Runestone Components into PreTeXt. And it took me a Summer to cook up a conversion process that turned restructuredText into PreTeXt. But that explains why this instructor guide looks a bit different than the Runestone books you may be used to seeing. I converted the instructor guide to PreTeXt and have been working on adding it and making it more useful. What you see in this book is a little glimpse of the future. Whats more is that students really benefit from this partnership. Books in PreTeXt are far more accessible than anything we have done with restructuredText and they are far more mobile ready as well.
Building a system like Runestone is a lot of work, and I am amazed at what we have accomplished with very little funding. The future looks very exciting Runestone Academy has incorporated and is operating as a non-profit. We have applied for and are waiting approval of the formal 501(c)(3) tax status with the IRS. I am hopeful that I will be able to announce some significant support from an NSF program designed to help open source projects like Runestone and PreTeXt become sustainable long term. I think I did a pretty good job of outlining our vision in
Section 1.1. We have an exciting and necessary mission and if you are inspired to help us out in any way we would welcome your efforts!
You have attempted
of
activities on this page.
Section 1.6 Frequently Asked Questions
Subsection 1.6.1 Why Can’t I access the Instructor Page?
Subsection 1.6.2 How do I become an instructor?
I seem to have a student account, how can I get/create an instructor account?
I created an instructor account but…
First lets clear up a couple of misconcptions. There is no way for me to know, just by your email address that you are an instructor. You could be anyone. So, there is no such thing as an instructor account. Accounts can have various roles including: student, instructor, author, editor, and administrator.
You get the instructor role by creating a course. This gives you the privileges associated with being an instructor but ONLY for that course and any other course(s) you create. You are not an instructor for any of the open courses or anyone else’s course. There is a really nice video to help you get started in the
Instructors Guide 1 Please check it out as it will answer many other questions and help you make the most of Runestone.
Subsection 1.6.3 How do I learn more about the instructor interface
Subsection 1.6.4 Help! I cannot log in. Every time I try to log in, I have to reset my password!
In 99.99% of the cases, this is because you are confusing your email address with the username you were forced to enter when you register. So, go through the password reset process and get yourself logged in again. Then look in the upper right hand corner of the page and pull down the user menu (the one that looks like a person.) Your username will be displayed at the top of that menu. We use username instead of email address to protect the privacy of our users.
Subsection 1.6.5 I’m not getting the password reset email, can you reset my password?
Technically, I can. But I won’t. Runestone handles over 100 password resets a day automatically. It can handle yours as well. There are two things you should check.
Check your SPAM folder for the password reset email. and/or add runestone to your contacts list to avoid that in the future.
Talk to your IT folks and make sure they are not blocking Runestone. We’re a small group, but we did everything right with our email setup to make the military academies happy, so I think your folks ought to be happy too. However some schools disallow any external email that is not whit-elisted. Please ask them to white-list runestoneinteractive@gmail.com as well as info@runestone.academy
If you are a student, talk to your instructor. Your instructor can reset your password in the case where the email system is failing us.
Subsection 1.6.6 I can’t log in to my Coursera course, what do I do?
Sorry, but we do not have control over Coursera and especially the courses from UMSI. Those courses are designed for you to access through Canvas. If that is not working please contact your coursera instructor.
Subsection 1.6.7 How do I add students to my course?
See this post for a detailed description:
Privacy and Runestone 4 You invite them, either in class or with an email. Tell them to register at runestone.academy and then give them the name of the course you created. During the registration process they type in the name you used, and they are registered. I usually just do this the first day of class to make sure everyone gets registered and then give them a little tour of the book and all of the things they can do.
In the future, we may look into importing a class from some LMS system like Moodle and others.
Subsection 1.6.8 What do those green checkmarks and orange circles mean?
A green checkmark for a page means that you have completed that page, either you have satisfied yourself that you are done OR you have attempted all of the interactive activities on that page. It does NOT mean you got everything correct. Clicking the button bottom of the page WILL NOT give you full credit on a reading assignment, you still need to do the work.
Historically this was ONLY for your own personal benefit to help you keep track of the things you had read and the things you were currently reading. It has evolved and with that evolution comes some confusion. Hopefully this clears it up.
Subsection 1.6.9 How come there is no Green Checkmark for the Exercises pages?
Some exercises pages have over 300 problems. Do you really think we expect you to do ALL of them? We expect your instructor to use some discretion when assigning you homework. You will not find green checkmarks for Exercises or for table of contents pages.
Sorry if that freaks you out, but that is just how we do it.
Subsection 1.6.10 Where do I find answers to all the homework questions?
In my years of teaching computer science I was always learning something new and trying to stay ahead of the students. I understand that for many of you this might be a first time teaching a CS course, and that for many you are overworked and underpaid. I hear you, and I feel your pain.
But just as I tell my students, you can’t really learn this stuff just by reading the answer, you have to do it.
We have many books on Runestone Academy written by many authors. Whether or not authors provide an instructor guide or answers to all the questions is up to them. So there is no blanket answer to the question.
For myself there are several reasons:
Writing an instructor guide that contains all of the answers to the questions essentially doubles the work of writing a book, and even more so maintaining the book. In all the textbooks I’ve used over the years for CS I’ve only ever had one that had a solution guide and that was largely for all of the math in a networking text.
For 99% of the questions there is no one right answer. Very likely if I give you an answer there is a better one out there, and I really don’t want to get into arguments about what is better. Especially since a lot of it comes down to style.
Doing the work is the best way to prepare to teach it. You are smarter, wiser, and a better problem solver than your students. Its actually good for them to see how you approach solving the problem rather consulting some answer key.
From a practical perspective it would be very hard for me to decide who gets the materials. How do I know you are an instructor? I can’t possibly take the time to go check out every user that claims to be an instructor to see if they are legit. Right now the only benefit of being an instructor is that you get to see and grade the work of your students. If you are a student pretending to be an instructor this gives you no advantage.
As I think about how best to invest the hours of my day that I have to work on this, I have very little energy or excitement for writing an answer key. So I choose to spend my time elsewhere. If others were to do so, and want to publish it somehow, I would be very happy to accept a PR!
I wold much rather put my energy into developing good unit tests, so that you know when you are right and when you are not there yet without me giving you solution.
With all of that, behind me, I’m still thinking about it, and trying to find a way to do it where its NOT a huge amount of extra work, and I can keep things in sync. I’ve thought about charging for the instructor guide as one way to both disuade casual cheaters and to help fund Runestone development. Stay tuned…
Subsection 1.6.11 Can I build my own course and host it here?
Yes, we are currently hosting many courses for many different institutions around the world. In fact in 2013 one large institution had 800 students using one of the books.
The best approach is to use our system to build your own textbook. This gives you several advantages as an instructor including:
a simple grading interface for homework problems at the end of each chapter
some simple reports on your students activities within the textbook.
at a glance information about the multiple choice and fill in the blank questions embedded in the text.
Subsection 1.6.12 How do I build my own course?
Subsection 1.6.13 Is this site reliable enough to use in class?
Yes. We host this on a very reliable service and we monitor our traffic constantly. We use a content distribution network for increased scalability and reliability. In the Fall of 2020 we serve over 50,000 students a day with no downtime. Runestone is also used for a couple of courses on coursera which have hundreds of thousands of students, and by many large Universities.
Subsection 1.6.14 Why did you change domain names?
The old domain name, “interactivepython.org” was becoming quite inacurate. Although we started with Python we have a lot of users that are using the AP CS Review book (Java) and my Java for Python programmers book. There is also at least one statistics textbook written in runestone as well. I saw the domain name “runestone.academy” and thought that would be cool.
Subsection 1.6.15 I want to reuse my course from last year, what should I do?
You should create a new course and then from the instructor page in the new course copy the assignments from your old course. The assignment dates will be adjusted based on the class start date.
The instructor interface allows you to copy all your assignments from a past course into your current course, so re-using an old assignment is pretty simple. It even re-calculates a due date for your based on the delta between your old term start date and when you had the assignment due that term.
Subsection 1.6.16 How do I update my course to get the latest bug fixes?
Nothing. Bug fixes are automatically deployed weekly.
Subsection 1.6.17 I was just experimenting and I want to delete my course
Once you are done experimenting please delete your course from the instructors interface. If this does not happen I may have to invent a way to go back and remove courses that were clearly created as an experiment but never really used by students.
Subsection 1.6.18 What if I want to add a new section or chapter?
That would be awesome. This whole book is open source. You can grab a copy of the source on
github 7 . The source for thinkcspy and pythonds is in the source folder and there is a subfolder for each chapter. If you want to make a whole new chapter then create a folder and follow the conventions of one of the other chapters. There is full documentation for the markup language at
docs.runestoneinteractive.org 8 . When you are finished make a pull request and we’ll review your material and incorporate them into the book.
Subsection 1.6.19 What if I want to add my own exercises?
You can add your own custom exercises by editing the assignments.rst file. This file is meant for the descriptive or question text for a programming exercise. Adding an exercise to this file does not automatically add it to the grading interface for your course, you still need to do that from the instructors interface. See this
blog post 9 for more information.
New exercises are always welcome and we would love to expand the number of exercises. The simplest way is to go to the
github issues 10 page and file a new issue. In the description simply include the text for the exercise and which chapter you think it should go in. We’ll take it from there. After we’ve added the exercise you can rebuild your book and it will be there.
Subsection 1.6.20 My students are completing work but it is not showing up. Help!
There are many answers to this, as the question itself is very vague. Where is it not showing up? What is not showing up? The first thing to say is that Runestone goes to great lengths to save everything! It is extremely rare for an action to go unrecorded. With that said here are things to check.
Make sure your students are logged in to YOUR course and not the open course that you based yours on. If your student is logged in to your course they will see the name of your course in the upper left corner of the browser window.
Make sure they are paying attention to the progress bar. If the progress bar is gray they are not logged in! If it is green they are. The progress bar will show how many of their activities have been recorded for that page.
Check the assignment page and click on the self grade button. This is yet another way that students can verify their work has been saved.
For students work to show up in the gradebook you will need to use the grading interface to run the autograder which tabulates the scores for the assignment and saves them in the database. You also need to release the grades in order for the students to see them on the student progress page. If you haven’t released them then they will not see their progress.
Make sure your assignment deadlines are set up correctly or that you have unchecked the box to only count activities before the deadline.
From the student progress page you can click on the name of any student and see the raw log details. You can also click on the overview reports link from that page and download a CSV file that you can import into your favorite spreadsheet. This will show you counts of activities for each student, the timestamp of the first time a student interacts with anything in runestone and more.
Subsection 1.6.21 How do I change courses or register for an additional course?
Under the 👤 menu you will find the Change Course option. click on that and then at the bottom of the page click on the Enroll in a Course button. From there you can just enter the new course in the Course Name field.
Subsection 1.6.22 What course name should I use when I register?
If you are enrolled in a course at a high school, college or university your instructor should give you the name. If they didn’t you need to ask them. If you are learning on your own, The registration page gives you the full list as does our library page. But here are the legal, open, courses that anyone can sign up for. Since you are here you didn’t see that so here is the list:
apcsareview
cpp4python
cppds
csawesome
fopp
httlads
java4python
JS4Python
learnwebgl2
overview
pythonds
pythoned
StudentCSP
TeacherCSP
thinkcspy
webfundamentals
Subsection 1.6.23 What version of Python does your book use?
Ok, this is a question that has the potential to start nasty religious wars. The technical answer is that this book uses a version of Python called
Skulpt 11 . It is entirely written in Javascript so that it runs right in the browser. We think this is very cool. Now some people get all crazy about whether they should teach Python 3 or Python 2. The truth is that for CS1 and CS2 it really does not matter. Skulpt can do print with or without parenthesis, and / can do true division or integer division and lets face it for CS1 that is really all that matters. Sure, there are differences, but are you really going to start out by teaching your students about dict_keys and how they are different from a list. If so, I think you are cruel and you should teach your students APL. If you want to slant your teaching toward Python 3, you can do that with this book. If you want to lean towards 2, you can do that too. When you build your course there is a configuration parameter that lets you choose Python3, this forces you to use parenthesis when you print, and it makes python / default to true division, and // to integer division.
Subsection 1.6.24 I think there is a bug in your book what should I do?
Subsection 1.6.25 I have a question that is not covered here!
Subsection 1.6.26 My Internet is slow, I want to use Runestone offline
This might require a little extra learning on your part but here is what you need to do. If you can’t get this to work on your own ask your teacher for some help.
Install Python
Use pip to install runestone (pip install runestone)
Edit the pavement.py file for your book and set dynamic_pages = False and useRunestoneServices=False
In the folder created for your book run runestone build –all
6. Run the command runestone serve and then in your browser go to
http://localhost:8000 16 This will let you access your book from a webserver running on your own computer, It will save your work locally, as long as you use the same browser. But it will NOT share or sync any information with Runestone.academy
Subsection 1.6.27 Why can’t I just get a PDF?
Because what we are focused on is building great interactive textbooks, pdfs are not interactive. Update!! Our partnership with the PreTeXt team allows us to build books in many formats including PDF. If your book has been converted there is a good chance we can get you a PDF.
Subsection 1.6.28 How does Runestone handle timezones?
Very well thank you! Every student interaction is logged to the runestone database using UTC time. When you create an assignment with a deadline store the time in the timezone of the instructor making the assignment as the deadline. Given that we can fairly judge whether a student has completed their work on time. Trust me we have figured this out over the years and learned a lot about time zones. Did you know there are places in the world where there are half or quarter hour differences between two timezones??!!
I realize that students will often complain that they had their work done on time but the darn Runestone would not give them credit. Usually they are wrong. But you have several ways of checking out their claim including downloading the log for your course or looking at the Student Activity report and checking their first and last interactions with the problem. The software has proven itself to be correct way more often than the students and we use one of the most reliable databases in the world (Postgresql) to keep track of student activities!
You have attempted
of
activities on this page.
