Fiddle Salad has accumulated many features, but where are they? In keeping with one of its principles, features are hidden until the situation calls for them. One example is the JavaScript errors that are shown on hover over the results window. Recently, I’ve decided showing an unobtrusive tips window is as good as hiding features to reduce UI clutter. So, let’s start with 4 lines of Knockout code.

I start by setting startup to a storage value using store.js. It uses HTML5 local storage with cookies for fallback. Since I want to show the tips panel by default and hideTipsOnStartup would not be set, startup is set to the opposite hideTipsOnStartup. JavaScript’s ! and CoffeeScript’s not casts an expression to a boolean, as required by Knockout’s checked binding.
Next, I add a manual subscription to the observable so that its value is stored each time the checkbox is checked or unchecked.
Finally, the checkbox is bound to tips.startup where tips is a TipsPanel instance.

While wrapping up the Fiddle Salad project and doing cross browser testing, I found that Firefox wouldn’t run my project, at least on a local host.

This was one of the reasons I couldn’t get a stable release all in one shot. I initially thought it was caused by the IP address or the port number, but others report it’s just a problem with not having a domain. So one way is to add a domain mydomain.com in the Windows host file in C:\Windows\System32\drivers\etc.

The changes are applied immediately after saving, and I’m able to run the site locally in Firefox.

Another way is to use one of the Worker polyfills that bypasses the Firefox security checks. I suggest fakeworker, but in my case I would need to rewrite some code to be compatible with the old API to use it. Of course, this was only one of the many problems I found on the uploaded version, so I had to prioritize which problems to fix first. It’s always better to go for the efficiency gains at the start and visible results at the end. So I made a Django media sync javascript debug processor for future debug purposes on the production site.

Creating clean, readable code is a primary imperative when working with large systems. Today, I have factored the knockout view model from a single object for both projects into an inheritance based hierarchy. Now in the middle of it, I’m replacing a programmingLanguage variable with a language class that also handles the language for the style sheet (LESS, SCSS) and document (Jade, ECO).

Then I paused and thought it didn’t seem right. I remember Enums were more terse in C. So I decided to us an Enum class.

Now I like that much better. It’s like automatic numbering, except I don’t have to read it.

Just working on this project yesterday, I discovered the Facebook login was no longer working. I noticed from my other projects Facebook changed their API and the library needed to be upgraded.  Back in 2009, the best library available was socialauth. Now I’m using social_auth in my newer projects. There’s more documentation at first glance. Although problems showed up from time to time, upgrading always fixed them. So I would recommend social_auth.

It turns out I could use an independent login module, already a part of the shared codebase for 2 other projects, at a cost. The cost is to add the login button to keep the logic flow the same. Later, I used a dependent observable to automatically save after the authentication state changes. Simply pass the load or save function as an argument to the login decorator (Python term, same concept in JavaScript):

The save function begins by checking if the user is logged in:

afterLogin is called in the observable:

After a hectic few minutes of upgrading the site with about 3 different kinds of server errors and many email debug messages, I finished upgrading the database and the settings file.

Maybe next time I will use if DEBUG statements in my settings file so it can be copied straight to server. But the main reason I opted not to was because of the facebook, twitter, and google keys which had to be set differently. On the other hand, those can be in a conditional debug statement, too.
Overall, it was a thrill to work with the project again, bring back the gifts from more recent projects, and see the brilliantly written code.

When implementing the scrolling menu for Chrome and Firefox, I prototyped the menu using fiddlesalad’s LESS CSS editor. Then, I copied the compiled CSS and started coding with a straight forward procedural approach.

Later, I noticed that the performance can be improved by caching jQuery results. Also, many lines of code were repetitive. Before moving onto the more challenging part of the menu design, scrolling to the current location, I made it object oriented using plain JavaScript.

Writing classes in JavaScript brings unexpected quirks, such as this being set to an HTML Element. However, using an additional helper, bindMethod, this problem is avoided. I used the scrollTop method to scroll to the top of the menu:

Working with JavaScript closures in an asynchronous context (callbacks) was a memorable experience when I first wrote the code for course links. Now I had a chance to look at it again after restoring the functionality during a recent bugfix.

A look under the hood of the fourth nested AJAX call:

• postCORS is a function that provides cross browser cross origin POST requests. XDomain request is used for IE, while the usual jQuery.post is used for othe browsers.
• because each callback needs to have an associated site dictionary from the previous callback, they need to have their own scope. Otherwise, a global variable would be overwritten each time through the loop in the parent closure.
• g simply returns whatever is passed to it. This is one way of creating an independent scope. A related way is to call new on a function that is a constructor.

In modern browsers, g-zip compression is a standard feature. The typical compression ratio for a plain text file is 30%, reducing the download time of web content by 70% and making it load 2-3 times faster. In spite of the speed up, g-zip is an old algorithm based on LZ77. Since then, newer algorithms have been invented, with LZMA being the standard. On Linux, LZMA typically produces files half the size compared to g-zip. This tutorial will show you how to use an LZMA compressed file produced by the standard lzma command on Unix machines directly in a client side web application. The rest of the post assumes you have the JavaScript libraries for LZMA and binary AJAX set up.

First, Make a Compressed File

Next,  Read Binary Data

This is a pretty simple step, except the while loop counter may be unintuitive. getUint8 increments the file pointer, though it wasn’t documented in the API specification. I spent an hour or so comparing the output in hex. One of the problem was that

is the same as

in little Endian. You can try it in the decompressor, just replace the bytes in the hello world lzma on compression level 3. However, I figured out the problem as soon as I compared view.length and int_arr.length. They were multiples of 2! That always has significance in computing, in this case it meant I was reading every other byte. After correcting the while loop, I moved onto decoding the binary.

Third, Enjoy the Decoding

Yes, this is a rather boring thing to do while waiting, but do enjoy it.

Benefits

Using LZMA compression rather than g-zip, I was able to reduce a g-zipped file to 2/3 of its size, reducing the download time by 33%. The LZMA decompression algorithm could be improved to use an array to store results, joining them at the end, rather than appending to a string. It is not recommended to use this method unless you have large files. The libraries themselves take up about 50kb with g-zip. Furthermore, it is unsuitable for downloads where files are sent directly to the user, without being used by the application, since the user would have the decompression utilities.

The Scenario

Suppose a javascript function is needed to parse text and execute another function for certain lines of text. Each line of the text is ended with a line break. If there is a keyword ‘get’, then the word following it needs to be checked if it is in the list of files on the server, and get it, using a GET request. So ‘get this’ would get the file ‘this’ from the server.

Object Oriented Approach

First, I tried doing this using the traditional way, with array indexes of letters.

I stopped right there the next line is supposed to be a for each loop. That satisfied the requirements for using a functional library called underscore.js.

Mixed Approach

Not my favorite one, as it makes the painting look like a collage, and I think it made an error in the code harder to spot. See if you can spot it:

Aha! The space should actually be newline.

Functional Approach
This time, I had to modify the code to use a single word from the word array, the last one.
The main line that violated the functional approach was

along with the array accesses, which should be replaced with first and rest. It turns out there is a purely functional way:

The custom executeGet function has been replaced by jQuery’s $.get.

Conclusions
Which approach is better? Using the OO approach, a local index variable would be needed to be created for the for loop, along with other locals, which adds irrelevant lines to the code. On the other hand, each line on its own makes sense. However, functional code cannot be understood except as a whole, because there are no variables. Statelessness has its own beauty in programming. It’s similar to running a quantum computer, not caring how bits disappear or time warp, but only for the result of the computation.

The goal of coursetree 2.0 is to leverage the current cloud infrastructure to deliver semantic applications that help users find the information they are looking for.

Features currently planned:

1. Course search that understands what the user wants
2. Filtering of irrelevant links
3. Pattern based degree data mining

Draft implementation strategy:

1. Let Google search index Wikipedia and video links
2. A Bayesian classifier will be used to categorize link content into subjects
3. Template induction and template scraping

Features under consideration:

1. Adaptable prerequisite semantic analysis
2. Fully automated template learning and template extraction
3. Relevant course links/suggested courses

Tenative ideas:

1. Genetic algorithm for grammar rule generation with fitness score assigned according to the total number of parse errors
2. Use hashing algorithms to detect similarity in sections of a page, feed similar sections using wrapper induction to generate template
3. Build map of courses using anti-requisites and display nearest neighbors

Unsuccessful incubation features:

1. Using genetic algorithms to generate templates for wrapper induction
2. Switch to parse trees extract noun phrases for Wikipedia link candidates
3. YQL for video link scraping

Lessons learned and salvaged:

1. Don’t use genetic programming methods where scores cannot be assigned to each individual “program”, as many of the templates were simply fails with zero scores
2. Although in some cases successful (with a comma separated list of noun phrases), in other cases single words were marked as noun phrases in the parse tree instead of a more desirable longer phrase
3. Due to frequent changes in video sites, nested JavaScript callbacks with closures to glue previews to links made the code a target to be recycled

Many web frameworks have URLs in the format of domain/category/page, but what if there’s a need for that outside of the framework, like in client-side scripting? Two lines of code is enough for getting those URL parameters:

One special feature of JavaScript regular expressions is (?…) is a group that does not show in the result array.

If you wanted to map the result to different actions: