Sabayon Upgrade Mystery Solved

Dec 20 2012

This was about one and half years ago on another machine that I no longer use, but the insight just came to me. Many things just lined up and fell into place instantly. I had Sabayon installed on my Inspiron 1520 quad-booting with XP, 7, and Elive. Running the system update before installing LLVM Clang for comping Python to JavaScript, the system got broken. I ran a system update to upgrade all packages because I had been using 7.  Unfortunately, the it broke the package manager!

This should never happen, and I wasn’t able to find any threads on Google. So I gave up on Sabayon and got a desktop to run Xubuntu in a VM. I thought the package manager was too dumb, stuck doing an upgrade of itself.

As all Linux users know, once the package manager is broken, the system is bonked. Installing, upgrading, or removing any software must go through the package manager. The upgrade finished half-way, I was left with a console without X or a graphical window manager. Fortunately, I had many other OS’s to use on the machine. They’re still there today, and today I just read the Sabayon wiki on mixing portage and entropy.

…when mixing Portage and Entropy, never use Portage to update Portage. Sabayon uses a version of Portage that is hard masked in Portage. This means that you will actually be DOWNGRADING, not upgrading portage …

So the summer before this one, I simply downgraded the package manager, which caused it to stop working.

 

 

No responses yet

Course Planner Lower Case Course Names

Dec 19 2012

For a long time, I’ve been accustomed to hitting the Caps Lock key when entering course codes in the URL box when using the Waterloo Course Planner. It’s the quickest way to navigate to a course. However, whenever I didn’t capitalize course codes, no course is shown. This is because the database/ORM layer requires cases to match. So now I’ve implemented lower case course codes so that ‘/calendar/engl/101a’ redirects to  ‘/calendar/ENGL/101A’ .

No responses yet

Knockout or Grails with jQuery

Sep 26 2012

Knowing when to use Knockout or not is an art. Plain jQuery works fine for keeping track of unchecked users that were checked before.

var existingUsers;
function setExistingUsers () {
    existingUsers = [];
    $('#userList input[type="checkbox"]').each(function () {
        if($(this).attr('checked')) existingUsers.push($(this).val())
    });
}

$('form').submit(function(){
    var unselectedUsers = [];
    $('#userList input[type="checkbox"]:not(:checked)').each(function () {
        unselectedUsers.push($(this).val())
    });
    $('[name="_deletedUsers"]').val(JSON.stringify(_.intersect(existingUsers, unselectedUsers)));
});
$('#allUsers').click(function(){
    if ($(this).is(":checked")) {
        $('#userList input[type="checkbox"]').prop('checked', true);
    } else {
        $('#userList input[type="checkbox"]').prop('checked', false);
    }
})

jQuery is used here for the “All” checkbox to select all users and to find the deleted users when submitting the form. Another part of the page displays the selected users for ordering with a sortable interface, so that users can be assigned priorities. Whenever a user is checked, he is added to the bottom of list. Whenever he is unchecked, he is removed from the sortable interface. With two views on the same data that need to be maintained, code without a proper MVC structure gets messy as the application grows. I even had to change Underscore‘s default “<% %>” templating to avoid conflicts with GSP.

var existingUsers;
_.templateSettings = {
  interpolate : /\{\{(.+?)\}\}/g
};
var escalationUserTemplate =  _.template("<span>{{ name }}</span>");
function setExistingUsers () {
    existingUsers = [];
    $('#userList input[type="checkbox"]').each(function () {
        if($(this).attr('checked')) existingUsers.push($(this).val())
    });
    $('#sortable').html(_.map(existingUsers, function(user) {
        escalationUserTemplate({name: user})
    }).join(''));
    $('#userList input[type="checkbox"]').click(function () {
        setExistingUsers()
    });
}

$('form').submit(function(){
    var unselectedUsers = [];
    $('#userList input[type="checkbox"]:not(:checked)').each(function () {
        unselectedUsers.push($(this).val())
    });
    $('[name="_deletedUsers"]').val(JSON.stringify(_.intersect(existingUsers, unselectedUsers)));
});
$('#allUsers').click(function(){
    if ($(this).is(":checked")) {
        $('#userList input[type="checkbox"]').prop('checked', true);
    } else {
        $('#userList input[type="checkbox"]').prop('checked', false);
    }
});

Setting the HTML for the other view each time a checkbox state changes is a lot less efficient than using Knockout. It even comes with a templating system that uses the HTML DOM, which is much better than having it in a string. Knockout’s MVC structure facilitated with managing users. I wrote a model to handle user filtering, checking/unchecking all users, and for showing different sections of the view depending on a radio box group.

User = (data) ->
  @name = ko.observable(data.name)
  @selected = ko.observable(_.include(existingUsers, data.id))
  @id = data.id
  @departments = data.departments
  @

ViewModel = ->
  @filter = ko.observable('')
  @selectedDepartment = ko.observable(pagingGroup)
  @users = ko.observableArray([])
  @departments = ko.observableArray([])
  @pagingGroupType = ko.observable('broadcast')

  @filteredUsers = ko.computed( =>
    filter = @filter().toLowerCase()
    department = @selectedDepartment()
    unless filter or department
      @users()
    else
      usersByName = @users()
      if filter
        usersByName = ko.utils.arrayFilter usersByName, (item) ->
          ko.utils.stringStartsWith item.name().toLowerCase(), filter
      if department
        usersByName = ko.utils.arrayFilter usersByName, (item) ->
          _.include(item.departments, department)
      usersByName
  )

  @selectedUsers = ko.computed( =>
    department = @selectedDepartment()
    _.chain(@users())
      .filter((user) =>
        user.selected()
      )
      .filter((user) =>
        if department
          _.include(user.departments, department)
        else
          true
      )
      .value()
  )

  @selectAllFilteredUsers = =>
    _.each(@filteredUsers(), (user) ->
      user.selected(true)
    )

  @deselectAllFilteredUsers = =>
    _.each(@filteredUsers(), (user) ->
      user.selected(false)
    )

  mappedUsers = $.map(accountUsers, (item) ->
    new User(item)
  )
  @users mappedUsers
  allDepartments = _.union.apply({}, _(mappedUsers).map( (user) ->
    user.departments
  ))

  @departments allDepartments
  @

window.viewModel = new ViewModel()

ko.applyBindings(viewModel)

The views for user selection and ordering was also very straightforward. I used jQuery UI’s sortable lists for ordering users.

<div class="select-block flexcroll" data-bind="foreach: filteredUsers">
    <div class="select-box" id="userList">
        <label><input type="checkbox" data-bind="value: id, checked: selected, attr: { index: $index }" /><span data-bind="text: name"></span></label>
    </div>
</div>
...
<div id="sortable" class="connectedSortable select-block flexcroll selectors" data-bind="foreach: selectedUsers">
    <span data-bind="text: name"></span>
</div>

The best part is the old code for finding deleted users is still relevant.

$('form').submit(function(){
    var unselectedUsers = [];
    $('#userList input[type="checkbox"]:not(:checked)').each(function () {
        unselectedUsers.push(parseInt($(this).val()))
    });
    $('#userList input[type="checkbox"]').each(function(index){
        $(this).attr('name', 'members[' + index + '].user.id'); // Grails automatically saves one to many relationships
    })
    $('[name="_deletedUsers"]').val(JSON.stringify(_.intersect(existingUsers, unselectedUsers)));
});
$('#allUsers').click(function(){
    if ($(this).is(":checked")) {
        viewModel.selectAllFilteredUsers()
    } else {
        viewModel.deselectAllFilteredUsers()
    }
})

Now the AJAX call for users in a department has been replaced with front-end filtering, which also allows for searching by names using the filter observable.

No responses yet

Convert Grails 2 View to Use Knockout JS

Sep 20 2012

When the basic functionality of a web application has been developed, it’s time to add interactive features. Filtering a list of users is one task that is better done on the client side than on the server, to provide faster response. First, the Groovy variables need to be converted to JSON to be rendered in the view.

def newMessage = {
  def user = User.get(session.user.id)
  def account = CustomerAccount.get(session.account.id)
  [messageInstance: new Message(params), usersForAllDepartments: userService.getUsersForAllDepartments(user)]
}
def newMessage = {

  ...
  [messageInstance: new Message(params), usersForAllDepartments: userService.getUsersForAllDepartments(user).collect{[id: it.id, name: it.name]} as JSON]
}

Next, change the looping constructs and logic to use Knockout:

<div class="select-box"><label><span class="mark m-red">${user.name}</span></label></div>
<!-- ko foreach: recipients -->
<div class="select-box"><label><input type="checkbox" name="recipientUserList[]" data-bind="value: id, checked: selected" /></label></div>
<!-- /ko -->

Simply replace the tags with Knockout’s containerless control flow tags and bind to data values. Knockout is more terse, as the user for each iteration doesn’t need to be referenced. Render the JSON list of users on the page before the main part of the JavaScript code, which I like to keep in a separate deferred (CoffeeScript) file.

<script type="text/javascript">// <![CDATA[
usersForAllDepartments = ${usersForAllDepartments};
// ]]></script>

Finally, apply Knockout’s bindings to it with the user list loaded.

function Recipient(data) {
  this.name = ko.observable(data.name);
  this.selected = ko.observable(false);
  this.id = data.id;
}
function MessageViewModel() {
  var self = this;
  self.recipients = ko.observableArray([]);

  // Load initial state
  var mappedRecipients = $.map(usersForAllDepartments, function(item) { return new Recipient(item) });
  self.recipients(mappedRecipients);
}
ko.applyBindings(new MessageViewModel());

The rendered HTML looks exactly the same as before (except for data-bind).

<div class="select-block flexcroll">
<!-- ko foreach: recipients -->
<div class="select-box"><label><input type="checkbox" name="recipientUserList[]" value="5" data-bind="value: id, checked: selected" /><span class="mark m-red" data-bind="text: name">Bobby Tester</span></label></div>
<div class="select-box"><label><input type="checkbox" name="recipientUserList[]" value="3" data-bind="value: id, checked: selected" /><span class="mark m-red" data-bind="text: name">Manfred Moser</span></label></div>
<div class="select-box"><label><input type="checkbox" name="recipientUserList[]" value="2" data-bind="value: id, checked: selected" /><span class="mark m-red" data-bind="text: name">Sys Admin</span></label></div>
<!-- /ko -->
</div>

One response so far

Course Planner Wikipedia Links

Sep 09 2012

After contacting the author of the NLP parsing site, I got the Wikipedia linking service back running again for course descriptions. These links were helpful at the start for getting ahead of the official course calendar in search rankings.

As usual, I received no notice about server migrations which caused the site to go down in a few intervals while I worked on and off last summer, connecting to the cloud infrastructure. Now these mined keywords have become a reliable starting point for YouTube video searches, which can be seen on many of the pages, CS 486, for example. At any rate, one word phrases are too short as they won’t return relevant videos and are ignored. However, in some instances, as in CS 486, “planning” and “uncertainty” are keywords in the domain. One solution for this problem from my artificial intelligence class is to use a Bayesian classifier to categorize the Wikipedia page. I have applied this technique to external course links, with mixed results. Perhaps the Bayesian classifier is not trained enough or the categories are not partitioned into specific subjects. On the other hand, it works well for categories it knows, showing an MIT course titled “Techniques in Artificial Intelligence” for CS 486.

No responses yet

1985: Beginnings of the Digital Age

May 12 2012

According to a mathematical interpretation of the measurements of the Great Pyramid, late 1985 marks a significant time of influences for our current age. Taking a look at the publicly contributed list of events on Wikipedia for 1985,  a number of items form the fabric of our society:

  • First release of Windows
  • Free Software Foundation founded
  • Richard Stallman writes the GNU manifesto
  • NeXT, which much of Mac OS X is based on, is founded by Steve Jobs
  • Nintendo is released with Mario Brothers bundled
  • First Calvin and Hobes comic

Another source summarizes:

 As the spread of aids increases Governments round the world start screening Blood donations for AIDS. On the technology front the first .com is registered and the first version of Windows is released Ver 1.0 . Terrorists continue to perform acts of terrorism including the hijack of TWA Flight 847 and the Italian Cruise Liner “Achille Lauro “. Famine in Ethiopia is shown more on TV News in July and Live Aid concerts around the world raise many millions to help the starving in Africa and the pop industry in US joins together to sing “We Are The World”.

The major trends of a global awareness and people working together to solve problems are apparent.

No responses yet

Planning For the Next Version of Fiddle Salad: How I Nearly Jumped to My Next Project

Mar 31 2012

The work done on Fiddle Salad this month would not have been possible without last month’s planning. Furthermore, Fiddle Salad would not have been my idea if I did not invest time in building Python Fiddle. Python Fiddle was really the end product of 9 years of dreams of running a high performance computer and the result of my experience using Gentoo Linux. So I bought a computer to build Python Fiddle, which also turned out to be necessary to run the latest IDE and development tools to build Fiddle Salad.  When I started working with the Python interpreter in JavaScript,  it was horrendously slow. It took about 20 seconds to load and took up almost 1GB of memory. Any text editor except Vim without syntax highlighting was quick enough to edit the 12MB source code file.

Fiddle Salad is an evolution of both the original idea and code base that belonged to Python Fiddle. Now it is really Fiddle Salad that’s driving the development of Python Fiddle, because they share much of the code base. 

So this is the third major milestone, which I almost gave up on before I embarked on it. Before I started work on this milestone, actually a day or two before I planned, I suddenly noticed huge, discouraging signs. They came as shocking surprises. For example, I discovered a hidden option in an application I have used often before that had some of the functionality I was going to build. If that wasn’t enough, it was actually quite popular and many people probably knew that feature. As another example, I discovered another application that was more innovative in certain aspects than the application I planned to build. I got still more examples, but they aren’t worth repeating here.

As a habit, I reached for my next plan and the best tools I have available. I then realized that I would be throwing away about 8 months of work and the plans for this month, which worked out so well. Although I had no reason and no incentive at all to work on Fiddle Salad, I did so only because I enjoyed every moment of it. I believe that’s what we are all here for, the very drumbeat of the universe.

In the end, those serious signs got swallowed up by my project, as I managed to either include their ideas or integrate them right into it. Fiddle Salad is really the culmination and peak of all live web development environments, having the best features in all of them and in my imagination.

No responses yet

Firefox Could Not Get Domain for Worker

Mar 31 2012

While wrapping up the Fiddle Salad project and doing cross browser testing, I found that Firefox wouldn’t run my project, at least on a local host.

This was one of the reasons I couldn’t get a stable release all in one shot. I initially thought it was caused by the IP address or the port number, but others report it’s just a problem with not having a domain. So one way is to add a domain mydomain.com in the Windows host file in C:\Windows\System32\drivers\etc.

# Copyright (c) 1993-2009 Microsoft Corp.
#
# This is a sample HOSTS file used by Microsoft TCP/IP for Windows.
#
# This file contains the mappings of IP addresses to host names. Each
# entry should be kept on an individual line. The IP address should
# be placed in the first column followed by the corresponding host name.
# The IP address and the host name should be separated by at least one
# space.
#
# Additionally, comments (such as these) may be inserted on individual
# lines or following the machine name denoted by a '#' symbol.
#
# For example:
#
#      102.54.94.97     rhino.acme.com          # source server
#       38.25.63.10     x.acme.com              # x client host

# localhost name resolution is handled within DNS itself.
    127.0.0.1       localhost
    127.0.0.1       myapp.com

The changes are applied immediately after saving, and I’m able to run the site locally in Firefox.

Another way is to use one of the Worker polyfills that bypasses the Firefox security checks. I suggest fakeworker, but in my case I would need to rewrite some code to be compatible with the old API to use it. Of course, this was only one of the many problems I found on the uploaded version, so I had to prioritize which problems to fix first. It’s always better to go for the efficiency gains at the start and visible results at the end. So I made a Django media sync javascript debug processor for future debug purposes on the production site.

No responses yet

Installing Turnkey Rails on VMWare

Mar 04 2012

While working on the new Fiddle Salad, which required several languages with native Ruby compilers, I downloaded and installed Ruby on my Windows machine. Ruby by itself wasn’t a problem, but when I run gem install rails, some kinks come up, like missing a lib folder in a package. Then I moved on to installing therubyracer, which I guess is an optimizer, but it wouldn’t compile with mingw on Windows. After that, when initializing a rails app, it got stuck on bundler, which I used to install several packages. Like one user said on stackoverflow, Ruby isn’t meant to run on Windows.

So this morning I got ruby set up pretty quick. Just follow these steps:

  1. Download turnkey rails http://www.turnkeylinux.org/rails
  2. Install on VMWare Player
  3. Set Networking to Bridged
  4. Connect using SSH to the address shown after installation

That last part took a bit of searching on Google for me, because I expected a full Ubuntu installation, with GUI. But the download was only 253Mb. As I thought more about it, it made sense. I was glad how easy it was to set it up in a virtual machine. Once the VM was installed, I installed Rails with gem install rails and set up the build tools.

 Update: therubyracer is the Google V8 embedded within Ruby. There is a list of other engines here: https://github.com/sstephenson/execjs

No responses yet

Enumerables in JavaScript

Feb 18 2012

Creating clean, readable code is a primary imperative when working with large systems. Today, I have factored the knockout view model from a single object for both projects into an inheritance based hierarchy. Now in the middle of it, I’m replacing a programmingLanguage variable with a language class that also handles the language for the style sheet (LESS, SCSS) and document (Jade, ECO).

var Language = Class.$extend({
    __init__: function () {
        LANGUAGE = {
            PYTHON: 0,
            JAVASCRIPT: 1,
            LESS: 2,
            CSS: 3,
            HTML: 4
        };
        LANGUAGE_TYPE = {

            STYLE: 1,
        }
    }
})

Then I paused and thought it didn’t seem right. I remember Enums were more terse in C. So I decided to us an Enum class.

function Enum() {
    var obj = new Object();
    for (var i = 0; i < arguments.length; i++) {
        obj[arguments[i]] = i;
    }
    return obj;
}

var Language = Class.$extend({
    __init__: function () {
        LANGUAGE = Enum('PYTHON', 'JAVASCRIPT', 'LESS', 'CSS', 'HTML');
        LANGUAGE_TYPE = Enum('SCRIPT', 'STYLE', 'DOCUMENT');
    }
})

Now I like that much better. It’s like automatic numbering, except I don’t have to read it.

No responses yet

« Newer - Older »