One of the bugs still in the Waterloo Course Planner is the handling of prerequisite sentences that end with “* students only.”. The fix I made was rather simple. Though converting existing test cases to unit tests did not help because none of the older grammar rules were changed and therefore the tests were not broken, it did help in the development of new grammar rules. Python unit tests does a nice job of pinpointing the exact place where the expected results differ from running the code.

“* students only.” appearing in a string results in the tokenizer printing out a list of invalid tokens. There are two ways of ignoring strings in PLY:

1.  t_ignore_ in the tokenizer
2. change t_ignore_ to a token and add a new rule that returns an empty string in the parser

I used the second method this time, since the semicolon preceding “* students only.” would also need to be ignored. Semicolons are treated as a signal for an “and” clause otherwise. The new rule looks like the following:

First up on the list is the bad news: the old share bar is permanently gone as Google has shut it down. That’s right! Google has shut down a competitor’s service after buying it. At least the meebo team is honest and left us a message:

Meebo was acquired by Google on June 4, 2012.

Our team is now working just a few miles down the road with the Google+ team, where we continue to focus on creating delightful experiences for our users, developers, and publishers.

Meebo’s products are no longer available, but you can find all the latest and greatest things that we’re working on at https://developers.google.com/+/.

Thanks,
The Meebo Team

Elsewhere on the internet, you can find news reports about the shutdown. That gives credence to the “Google is Evil” motto.

On the course explorer side, there has recently been a change to Amazon’s Ecommerce API, breaking code for getting book titles and URLs that I wrote several years ago. Basically, most of the responses returned changed from single item ones to

to multiple item ones

As with many of these bugs, they are easy to fix once cause of the problem is found. If you know XML, an exercise for you is to spot the difference between the top one and the bottom one.

For a long time, I’ve been accustomed to hitting the Caps Lock key when entering course codes in the URL box when using the Waterloo Course Planner. It’s the quickest way to navigate to a course. However, whenever I didn’t capitalize course codes, no course is shown. This is because the database/ORM layer requires cases to match. So now I’ve implemented lower case course codes so that ‘/calendar/engl/101a’ redirects to  ‘/calendar/ENGL/101A’ .

After contacting the author of the NLP parsing site, I got the Wikipedia linking service back running again for course descriptions. These links were helpful at the start for getting ahead of the official course calendar in search rankings.

As usual, I received no notice about server migrations which caused the site to go down in a few intervals while I worked on and off last summer, connecting to the cloud infrastructure. Now these mined keywords have become a reliable starting point for YouTube video searches, which can be seen on many of the pages, CS 486, for example. At any rate, one word phrases are too short as they won’t return relevant videos and are ignored. However, in some instances, as in CS 486, “planning” and “uncertainty” are keywords in the domain. One solution for this problem from my artificial intelligence class is to use a Bayesian classifier to categorize the Wikipedia page. I have applied this technique to external course links, with mixed results. Perhaps the Bayesian classifier is not trained enough or the categories are not partitioned into specific subjects. On the other hand, it works well for categories it knows, showing an MIT course titled “Techniques in Artificial Intelligence” for CS 486.

Just working on this project yesterday, I discovered the Facebook login was no longer working. I noticed from my other projects Facebook changed their API and the library needed to be upgraded.  Back in 2009, the best library available was socialauth. Now I’m using social_auth in my newer projects. There’s more documentation at first glance. Although problems showed up from time to time, upgrading always fixed them. So I would recommend social_auth.

It turns out I could use an independent login module, already a part of the shared codebase for 2 other projects, at a cost. The cost is to add the login button to keep the logic flow the same. Later, I used a dependent observable to automatically save after the authentication state changes. Simply pass the load or save function as an argument to the login decorator (Python term, same concept in JavaScript):

The save function begins by checking if the user is logged in:

afterLogin is called in the observable:

After a hectic few minutes of upgrading the site with about 3 different kinds of server errors and many email debug messages, I finished upgrading the database and the settings file.

Maybe next time I will use if DEBUG statements in my settings file so it can be copied straight to server. But the main reason I opted not to was because of the facebook, twitter, and google keys which had to be set differently. On the other hand, those can be in a conditional debug statement, too.
Overall, it was a thrill to work with the project again, bring back the gifts from more recent projects, and see the brilliantly written code.

Compatibility with the old API means a library can be upgraded without affecting the codebase. I recently upgraded these libraries used in coursetree:

jQuery UI <1.8.14 -> 1.8.17

Raphaël 1.5 -> 2.0.2

In the case of  Raphaël, the size was reduced from 151 kB to 86 kB. However, the JIT and Knockout libraries did not maintain compatibility, because I was an early adopter of the canvas and MVC libraries. Regardless, there is a speed improvement for new visitors because I combined many small JavaScript files into larger ones.

When implementing the scrolling menu for Chrome and Firefox, I prototyped the menu using fiddlesalad’s LESS CSS editor. Then, I copied the compiled CSS and started coding with a straight forward procedural approach.

Later, I noticed that the performance can be improved by caching jQuery results. Also, many lines of code were repetitive. Before moving onto the more challenging part of the menu design, scrolling to the current location, I made it object oriented using plain JavaScript.

Writing classes in JavaScript brings unexpected quirks, such as this being set to an HTML Element. However, using an additional helper, bindMethod, this problem is avoided. I used the scrollTop method to scroll to the top of the menu:

Most bugs in the system are now edge cases, while others are produced by aging.

• External links to online courses from the OpenCourseWare initiative (mostly MIT) are once again available due to the search service being down for several months
• Twitter social auth back to working after upgrading the package
• Fixed bug where the splash had a z-index of 1, blocking interaction with the page. This only happened when loading or saving and the splash wasn’t played. Due to the splash being disabled for development, this bug could only be found in production.
• Added exception catching when student doesn’t exist in model

Working with JavaScript closures in an asynchronous context (callbacks) was a memorable experience when I first wrote the code for course links. Now I had a chance to look at it again after restoring the functionality during a recent bugfix.

A look under the hood of the fourth nested AJAX call:

• postCORS is a function that provides cross browser cross origin POST requests. XDomain request is used for IE, while the usual jQuery.post is used for othe browsers.
• because each callback needs to have an associated site dictionary from the previous callback, they need to have their own scope. Otherwise, a global variable would be overwritten each time through the loop in the parent closure.
• g simply returns whatever is passed to it. This is one way of creating an independent scope. A related way is to call new on a function that is a constructor.

The goal of coursetree 2.0 is to leverage the current cloud infrastructure to deliver semantic applications that help users find the information they are looking for.

Features currently planned:

1. Course search that understands what the user wants
2. Filtering of irrelevant links
3. Pattern based degree data mining

Draft implementation strategy:

1. Let Google search index Wikipedia and video links
2. A Bayesian classifier will be used to categorize link content into subjects
3. Template induction and template scraping

Features under consideration:

1. Adaptable prerequisite semantic analysis
2. Fully automated template learning and template extraction
3. Relevant course links/suggested courses

Tenative ideas:

1. Genetic algorithm for grammar rule generation with fitness score assigned according to the total number of parse errors
2. Use hashing algorithms to detect similarity in sections of a page, feed similar sections using wrapper induction to generate template
3. Build map of courses using anti-requisites and display nearest neighbors

Unsuccessful incubation features:

1. Using genetic algorithms to generate templates for wrapper induction
2. Switch to parse trees extract noun phrases for Wikipedia link candidates
3. YQL for video link scraping

Lessons learned and salvaged:

1. Don’t use genetic programming methods where scores cannot be assigned to each individual “program”, as many of the templates were simply fails with zero scores
2. Although in some cases successful (with a comma separated list of noun phrases), in other cases single words were marked as noun phrases in the parse tree instead of a more desirable longer phrase
3. Due to frequent changes in video sites, nested JavaScript callbacks with closures to glue previews to links made the code a target to be recycled