Generating Possible Courses to Fulfill Prerequisites

Nov 06 2010

For this part of the project, the goal is to generate a list of courses that meets the requirements for a course from a string stored in the database in a specific format. The entire project was designed so that each component is modular. This way, each component could be written and tested independently, and the complexity of the task is greatly reduced. Likewise, the project consists of so many high level components it appears, at first glance, to be incapable of being one piece. To my surprise, I put together Scrapy + PLY + Django, and it all works beautifully.

Part of the reason these separate components can all come together is that the specifications were set in stone before design, and the design itself (choosing the web framework, scraper, compiler) is tailored with the whole in mind. So for this part, these are the requirements:

From a string of prerequisites separated by commas, with alternatives separated by slashes, generate a list of lists that represents all possible course combinations, without more than one course selected out of each group of alternatives.

"CS 241, STAT 206/STAT 230/STAT 240" -> [['CS 241'], [' STAT 206', 'STAT 230', 'STAT 240']]
[['CS 241'], [' STAT 206', 'STAT 230', 'STAT 240']] -> [('CS 241', 'STAT 206'), ('CS 241', 'STAT 230'), ('CS 241', 'STAT 240')]

Basically, it’s a combination of a list of lists of strings.
 

Once the goal was set, the coding phase was straightforward. Simply understand the meaning of the input, and use a python function to turn it into the specified output.

def calc_comb(t):
    l = t.split(', ')
    group = []
        group.append(s.split('/'))
    for s in l:
    return list(itertools.product(*group))

 

No responses yet

Coding Horror and the Illumination

Nov 03 2010

Can you imagine anything useful coming out of this line of code?

prereqPattern='''([A-Z]{2,5} \d{3}/[A-Z]{2,5} \d{3}\s*and\s*[A-Z]{2,5} \d{3}/[A-Z]{2,5} \d{3})|([A-Z]{2,5} \d{3}\s*and\s*\(*[A-Z]{2,5} \d{3}\)*)|(\)\s*and\s*\()|(One of [A-Z]{2,5} \d{3}\s*[, or\d{3}]*;|[A-Z]{2,5} \d{3}[, \d{3}]*;|.*,\s*[A-Z]{2,5} \d{3}.*;|[A-Z]{2,5} \d{3} or [A-Z]{2,5} \d{3}|[A-Z]{2,5} \d{3} or \d{3}|[A-Z]{2,5} \d{3})'''

I can, because I’ve seen it. Here’s a small slice:

MATH 137 or 147
 ACTSC 231, STAT 230 or 240
MATH 136 or 146
ACTSC 232
ACTSC 371
AFM 272;ACTSC 291
STAT 330, 333
STAT 330, 333
 ACTSC 331, STAT 330
AFM 372;ACTSC 391;ACTSC 231;ACTSC 231 and BUS 393;STAT 330;STAT 334
AFM 372;ACTSC 391;ACTSC 231;ACTSC 231 and BUS 393;STAT 333 or 334
ACTSC 331
AFM 101 or BUS 227
AFM 101

A master piece at its time, and quite decent to maintain with the documentation:

'CS 134 or 145 or a mark of 60% or higher in 136 or 138; Honours Mathematics or Software Engineering students only.'
'([A-Z]{2,5} \d{3}) or (\d{3})*'
('CS 134', '145')

'One of CS 134, 136, 138, 145; Computer Science students only.'
'([A-Z]{2,5} \d{3})([, \d{3}]*;)'
('CS 134', ', 136, 138, 145;')

'Computer Science students only.'
'(\D* students only)'
'Computer Science students only'

'CS 350 or ECE 354'
'([A-Z]{2,5} \d{3}) or ([A-Z]{2,5} \d{3})*'
('CS 350', 'ECE 354')

'CM 339/CS 341 or CS 350'
'([A-Z]{2,5} \d{3})/([A-Z]{2,5} \d{3})( or [A-Z]{2,5} \d{3})*'
('CM 339', 'CS 341', ' or CS 350')

'DAC 201/ENGL 203 and DAC 202/ENGL 204'
 '([A-Z]{2,5} \d{3})/([A-Z]{2,5} \d{3}).*([A-Z]{2,5} \d{3})/([A-Z]{2,5} \d{3})*'
('DAC 201', 'ENGL 203', 'AC 202', 'ENGL 204')

And a light comes through. There is a better way, better than I ever imagined. In fact, the ugly phase of evolution comes before the flowering. A metaphor for this would be the pollution during industrialization and a bloom of clean industries following it due to the emphasis on knowledge rather than machine labor.
Simply pass on the parenthesis in the lexer and rewrite the grammar so that it can see more of the input before having to make a decision for shift/reduce conflicts. This is similar to using a parser with more lookahead, an oracle that sees infinitely many steps ahead.

Yet, no matter how sweet the code,

it is time to part ways.

What comes next,

no one knows.

This age is past,

and the ground is laid for the next one.

No responses yet

Parsing with PLY: The Good, Bad, and Ugly

Oct 30 2010

PLY is a Python implementation of Lex and Yacc originally written for Unix. Previously, I used it for tokenizing prerequisite strings. The goal is to have a program that performs the following trick:

Input Interpretation

MATH 135 or 145, STAT 220 or 230 or 240 MATH 135/MATH 145, STAT 220/STAT 230

AMATH 250 and MATH 237 or 247; Level at least 3A;
Not open to General Mathematics students.
AMATH 250, MATH 237/MATH 247

BIOL 140, 140L, (BIOL 208 or 330) and (BIOL 308 or
330)
BIOL 140, BIOL 140L, BIOL 208/BIOL 330, BIOL
308/BIOL 330

(AMATH 341/CM 271/ CS 371 or CS 370) and (MATH 128
or 138 or 148)
AMATH 341/CM 271/CS 371/CS 370, MATH 128/MATH
138/MATH 148

MATH 135 or 145, STAT 220 or 230 or 240 MATH 135/MATH 145, STAT 220/STAT 230/STAT 240

AMATH 250 and MATH 237 AMATH 250, MATH 237

(BIOL 140,BIOL 208 or 330) and (BIOL 308 or
330)
BIOL 140/BIOL 208/BIOL 330, BIOL 308/BIOL 330

MATH 229 or 239 or 249 and (One of CO 227, 250/350,
355, CO 352/CM 340);
MATH 229/MATH 239/MATH 249, CO 227/CO 250/CO
350

(CS 240 or SE 240), CS 246, ECE 222 CS 240/SE 240, CS 246, ECE 222

CS 241 and (STAT 206 or 230 or 240) CS 241, STAT 206/STAT 230/STAT 240

At first glance, it looks like a simple task. Just replace “,” “and” with “,”, and replace “/” “or” with “/”. Regular expressions could do that. Then there comes the issue with the comma and its interpretation depending on the level in the hierarchy. So using PLY to do the same thing turned out to be much easier, since grammar rules are used instead of regular expressions. I put together a quick prototype in an hour.

import ply.yacc as yacc
import sys
sys.path.append("/Documents/Projects/ply/")
from prereqlex import tokens

def p_prereq_cp(p):
    'prereq : cp'
    p[0] = p [1]

def p_prereq_cp_and(p):
    'cp : course AND cp'
    p[0] = p[1] + ' ' + p[3]

def p_prereq_cp_or(p):
    'cp : course OR cp'
    p[0] = p[1] + ' ' + p[3]

def p_prereq_cp_comma(p):
    'cp : course COMMA cp'
    p[0] = p[1] + ' ' + p[3]

def p_prereq_cp_parens(p):
    'cp : LPARENS cp RPARENS'
    p[0] = p[2]

def p_prereq_cp_course(p):
    'cp : course'
    p[0] = p[1]

def p_prereq_course_extra(p):
    'course : course OR NUM'
    p[0] = p[1] + ' ' + p[3]

def p_prereq_course_default(p):
    'course : DEPT NUM'
    p[0] = p[1] + ' ' + p[2]

# Error rule for syntax errors
def p_error(p):
    print "Syntax error in input!"

# Build the parser
parser = yacc.yacc()

while True:
   try:
       s = raw_input('> ')
   except EOFError:
       break
   if not s: continue
   result = parser.parse(s)
   print result

> MATH 125 or 135
MATH 125 135
> MATH 125 or MATH 135
MATH 125 MATH 135

That’s pretty good for a start, but doesn’t get much better for interpreting the input correctly.

def p_prereq_cp_lp_ep(p):
    'prereq : cp SEMI lp SEMI ep'
    p[0] = p[1]

def p_prereq_cp_lp(p):
    'prereq : cp SEMI lp'
    p[0] = (p[1] , 'Level ' + p[3])

def p_prereq_cp_ep(p):
    'prereq : cp SEMI ep'
    p[0] = p[1] + ' Exclude ' + p[3]

def p_prereq_cp(p):
    'prereq : cp'
    p[0] = p[1]

def p_prereq_cp_and(p):
    'cp : cp AND cg'
    p[0] = ('AND', p[1], p[3])

def p_prereq_cp_comma(p):
    'cp : cp COMMA cg'
    p[0] = ('COMMA', p[1], p[3])

def p_prereq_cp_comma_and(p):
    'cp : cp COMMA AND cg'
    p[0] = ('COMMA', p[1], p[4])

def p_prereq_cp_semi(p):
    'cp : cp SEMI cg'
    p[0] = ('SEMI', p[1], p[3])

def p_prereq_cp_semi_and(p):
    'cp : cp SEMI AND cg'
    p[0] = ('SEMI', p[1], p[4])

def p_prereq_cp_defualt(p):
    'cp : cg'
    p[0] = p[1]

def p_prereq_cg_or(p):
    'cg : cg OR courses'
    p[0] = ('OR', p[1], p[3])

def p_prereq_cg_slash(p):
    'cg : cg SLASH courses'
    p[0] = ('SLASH', p[1], p[3])

def p_prereq_cg_comma(p):
    'cg : cg COMMA courses'
    p[0] = ('COMMA', p[1], p[3])

def p_prereq_cg_comma_or(p):
    'cg : cg COMMA OR courses'
    p[0] = ('COMMA', p[1], p[4])

def p_prereq_cg_parens(p):
    'cg : LPARENS cg RPARENS'
    p[0] = p[2]

def p_prereq_cg_one_of(p):
    'cg : ONE OF cg'
    p[0] = ('OR', p[3])

def p_prereq_cg_courses(p):
    'cg : courses'
    p[0] = p[1]

def p_prereq_courses_or(p):
    'courses : courses OR course'
    p[0] = ('OR', p[1], p[3])

def p_prereq_courses_slash(p):
    'courses : courses SLASH course'
    p[0] = ('SLASH', p[1], p[3])

def p_prereq_courses_comma(p):
    'courses : courses COMMA course'
    p[0] = ('COMMA', p[1], p[3])

def p_prereq_courses_default(p):
    'courses : course'
    p[0] = p[1]

def p_prereq_course_default(p):
    'course : DEPT NUM'
    p[0] = p[1] + ' ' + p[2]

def p_prereq_course_short(p):
    'course : NUM'
    p[0] = p[1]

def p_prereq_lp(p):
    'lp : LEVEL AT LEAST TERM'
    p[0] = p[4]

def p_prereq_ep(p):
    'ep : NOT OPEN TO'
    p[0] = ''

precedence = (
    ('left', 'AND', 'SEMI'),
    ('left', 'COMMA'),
    ('left', 'SLASH'),
    ('left', 'OR'),
    ('left', 'DEPT'),
    ('right', 'NUM'),
)

# Error rule for syntax errors
def p_error(p):
    print "Syntax error at '%s'" % p

import logging
logging.basicConfig(
    level = logging.INFO,
    filename = "parselog.txt",
    filemode = "w",
    format = "%(filename)10s:%(lineno)4d:%(message)s"
)
log = logging.getLogger()

parser = yacc.yacc(debug=True,debuglog=log)

prereqs = ('CHEM 120, 123; one of PHYS 111, 121; and one of PHYS 112, 122',
    'EARTH 121, 121L or 122, 122L or 153 or CIVE 153 or CIVE 253 or GEOE 153 or ENVE 153',
    'PHYS 112 or 122 or 125',
    'CHEM 254 or EARTH 421, CHEM 350',
    '(One of PHYS 112, 122, 125) and (one of MATH 118, 119, 128, 138, 148) and (one of AMATH 250, 350, CIVE 222, ENVE 223, MATH 218, 228, ME 203, SYDE 211)',
    'One of PSCI 250, 255, 260, 264',
    'One of ANTH 201/CLAS 221, ANTH 203, 233',
    'MATH 135 or 145, STAT 220 or 230 or 240',
    'AMATH 250 and MATH 237 or 247; Level at least 3A',
    'BIOL 140, 140L, (BIOL 208 or 330)',
    '(CS 240 or SE 240), CS 246, ECE 222',
    'CS 246, (SE 240 or CS 240)',
    '(AMATH 341/CM 271/ CS 371 or CS 370) and (MATH 128 or 138 or 148)',
    'MATH 135 or 145, STAT 220 or 230 or 240',
    'AMATH 250 and MATH 237',
    '(BIOL 140,BIOL 208 or 330) and (BIOL 308 or 330)',
    'MATH 229 or 239 or 249 and (One of CO 227, 250/350, 355, CO 352/CM 340)',
    '(CS 240 or SE 240), CS 246, ECE 222',
    'CS 241 and (STAT 206 or 230 or 240)')

limit = float("infinity");

i = 0;
for s in prereqs:
    if i < limit:
        result = parser.parse(s, debug=log)
        print result
        i += 1

Output:

('SEMI', ('SEMI', ('COMMA', 'CHEM 120', '123'), ('OR', ('COMMA', 'PHYS 111', '121'))), ('OR', ('COMMA', 'PHYS 112', '122')))
('OR', ('OR', ('OR', ('OR', ('OR', ('COMMA', ('OR', ('COMMA', 'EARTH 121', '121L'), '122'), '122L'), '153'), 'CIVE 153'), 'CIVE 253'), 'GEOE 153'), 'ENVE 153')
('OR', ('OR', 'PHYS 112', '122'), '125')
('COMMA', ('OR', 'CHEM 254', 'EARTH 421'), 'CHEM 350')
('AND', ('AND', ('OR', ('COMMA', ('COMMA', 'PHYS 112', '122'), '125')), ('OR', ('COMMA', ('COMMA', ('COMMA', ('COMMA', 'MATH 118', '119'), '128'), '138'), '148'))), ('OR', ('COMMA', ('COMMA', ('COMMA', ('COMMA', ('COMMA', ('COMMA', ('COMMA', 'AMATH 250', '350'), 'CIVE 222'), 'ENVE 223'), 'MATH 218'), '228'), 'ME 203'), 'SYDE 211')))
('OR', ('COMMA', ('COMMA', ('COMMA', 'PSCI 250', '255'), '260'), '264'))
('OR', ('COMMA', ('COMMA', ('SLASH', 'ANTH 201', 'CLAS 221'), 'ANTH 203'), '233'))
('OR', ('OR', ('COMMA', ('OR', 'MATH 135', '145'), 'STAT 220'), '230'), '240')
(('AND', 'AMATH 250', ('OR', 'MATH 237', '247')), 'Level 3A')
Syntax error at 'LexToken(LPARENS,'(',1,16)'
Syntax error at 'LexToken(RPARENS,')',1,32)'
None
('COMMA', ('OR', 'CS 240', 'SE 240'), ('COMMA', 'CS 246', 'ECE 222'))
Syntax error at 'LexToken(LPARENS,'(',1,8)'
Syntax error at 'LexToken(RPARENS,')',1,25)'
None
('AND', ('OR', ('SLASH', ('SLASH', 'AMATH 341', 'CM 271'), 'CS 371'), 'CS 370'), ('OR', ('OR', 'MATH 128', '138'), '148'))
('OR', ('OR', ('COMMA', ('OR', 'MATH 135', '145'), 'STAT 220'), '230'), '240')
('AND', 'AMATH 250', 'MATH 237')
('AND', ('OR', ('COMMA', 'BIOL 140', 'BIOL 208'), '330'), ('OR', 'BIOL 308', '330'))
('AND', ('OR', ('OR', 'MATH 229', '239'), '249'), ('OR', ('SLASH', ('COMMA', ('COMMA', ('SLASH', ('COMMA', 'CO 227', '250'), '350'), '355'), 'CO 352'), 'CM 340')))
('COMMA', ('OR', 'CS 240', 'SE 240'), ('COMMA', 'CS 246', 'ECE 222'))
('AND', 'CS 241', ('OR', ('OR', 'STAT 206', '230'), '240'))

This was the BNF grammar encoded:

prereq : cp
cp  : cp AND cgparens
	| cp COMMA cgparens
	| cp COMMA AND cgparens
	| cp SEMI cgparens
	| cp SEMI AND cgparens
	| cg
cg : cg OR courses
   | cg SLASH courses
   | cg COMMA courses
   | cg COMMA OR courses
   | LPARENS cg RPARENS
   | ONE OF cg
   | courses
courses : courses OR course
	   | courses SLASH course
	   | courses COMMA course
	   | course
course : DEPT NUM
	   | NUM

, which gives a surprising error whenever parenthesis appeared:

yacc.py: 389:Action : Reduce rule [course -> DEPT NUM] with ['BIOL','140'] and goto state 23
   yacc.py: 423:Result :  ('BIOL 140')
   yacc.py: 389:Action : Reduce rule [courses -> course] with ['BIOL 140'] and goto state 22
   yacc.py: 423:Result :  ('BIOL 140')
   yacc.py: 389:Action : Reduce rule [course -> NUM] with ['140L'] and goto state 24
   yacc.py: 423:Result :  ('140L')
   yacc.py: 389:Action : Reduce rule [courses -> courses COMMA course] with ['BIOL 140',',','140L'] and goto state 21
   yacc.py: 423:Result :  (('COMMA', 'BIOL 140', '140L'))
   yacc.py: 493:Error  : courses COMMA . LexToken(LPARENS,'(',1,16)
   yacc.py: 493:Error  : courses COMMA . error
   yacc.py: 493:Error  : courses . error
   yacc.py: 493:Error  : . error
   yacc.py: 389:Action : Reduce rule [course -> DEPT NUM] with ['BIOL','208'] and goto state 23
   yacc.py: 423:Result :  ('BIOL 208')
   yacc.py: 389:Action : Reduce rule [courses -> course] with ['BIOL 208'] and goto state 22
   yacc.py: 423:Result :  ('BIOL 208')
   yacc.py: 389:Action : Reduce rule [course -> NUM] with ['330'] and goto state 24
   yacc.py: 423:Result :  ('330')
   yacc.py: 389:Action : Reduce rule [courses -> courses OR course] with ['BIOL 208','or','330'] and goto state 19
   yacc.py: 423:Result :  (('OR', 'BIOL 208', '330'))
   yacc.py: 389:Action : Reduce rule [cg -> courses] with [] and goto state 18
   yacc.py: 423:Result :  (('OR', 'BIOL 208', '330'))
   yacc.py: 493:Error  : cg . LexToken(RPARENS,')',1,32)
   yacc.py: 493:Error  : cg . error
   yacc.py: 493:Error  : . error
   yacc.py: 493:Error  : . $end

It means the parser just refused to use the rule cg : courses and subsequently cg : LPARENS cg RPARENS. At that time, I thought it must be something wrong with the grammar. Forgetting the parenthesis would have been an option, only if the compiler gave the correct interpretation. With that in mind, I worked on getting the final result:

def p_prereq_cp_lp_ep(p):
    'prereq : cp SEMI lp SEMI ep'
    p[0] = p[1]

def p_prereq_cp_lp(p):
    'prereq : cp SEMI lp'
    p[0] = p[1] + 'Level ' + p[3]

def p_prereq_cp_ep(p):
    'prereq : cp SEMI ep'
    p[0] = p[1] + ' Exclude ' + p[3]

def p_prereq_cp(p):
    'prereq : cp'
    p[0] = p[1]

def p_cp(p):
    '''cp : cg
          | cp AND cg
          | cp COMMA cg
          | cp COMMA AND cg
          | cp SEMI cg
          | cp SEMI AND cg'''

    if (len(p) == 2):
        p[0] = p[1]
    elif (len(p) == 4):
        p[0] = p[1] + ', ' + p[3]
    else:
        p[0] = p[1] + ', ' + p[4]

def p_cg(p):
    '''cg : cl
          | LPARENS cg RPARENS
          | ONE OF cl'''

    if (len(p) == 2):
        p[0] = p[1]
    elif (p[1] == 'LPARENS'):
        p[0] = p[2]
    else:
        p[0] = p[3]

def p_cl(p):
    '''cl : courses
          | cl OR courses
          | cl SLASH courses
          | cl COMMA courses'''

    if (len(p) == 2):
        p[0] = p[1]
    else:
        p[0] = p[1] + '/' + p[3]

def p_courses(p):
    'courses : DEPT nums'
    p[0] = p[1] + ' ' + p[2]

def p_nums(p):
    '''nums : NUM
            | nums OR NUM
            | nums SLASH NUM
            | nums COMMA NUM'''

    if (len(p) == 2):
        p[0] = p[1]
    else:
        p[0] = p[1] + '/' + p[3]

def p_lp(p):
    'lp : LEVEL AT LEAST TERM'
    p[0] = p[4]

def p_ep(p):
    'ep : NOT OPEN TO'
    p[0] = ''

# Error rule for syntax errors
def p_error(p):
    print "Syntax error at '%s'" % p

I realize at this point that PLY couldn’t fly:

   yacc.py: 389:Action : Reduce rule [nums -> NUM] with ['121'] and goto state 19
   yacc.py: 423:Result :  ('121')
   yacc.py: 389:Action : Reduce rule [nums -> nums COMMA NUM] with ['121',',','121L'] and goto state 22
   yacc.py: 423:Result :  ('121/121L')
   yacc.py: 389:Action : Reduce rule [nums -> nums OR NUM] with ['121/121L','or','122'] and goto state 20
   yacc.py: 423:Result :  ('121/121L/122')
   yacc.py: 389:Action : Reduce rule [nums -> nums COMMA NUM] with ['121/121L/122',',','122L'] and goto state 22
   yacc.py: 423:Result :  ('121/121L/122/122L')
   yacc.py: 389:Action : Reduce rule [nums -> nums OR NUM] with [,'or','153'] and goto state 20
   yacc.py: 423:Result :  ('121/121L/122/122L/153')
   yacc.py: 493:Error  : DEPT nums OR . LexToken(DEPT,'CIVE',1,39)

It seems to be stuck on the rule nums : nums OR NUM. I did a test by deleting that rule. Then it parsed the rest of the string fine, just missing a few NUMs. For some reason, PLY just refused to use the rule cl : cl OR courses. I checked the PLY documentation. Yup, PLY turned out to be an iron balloon. “shift/reduce conflicts are resolved in favor of shifting (the default).” Of course, once the mystery is solved, the solution is not hard to find.

No responses yet

Finding a Feather Fallen from the Firebird (Part V)

Oct 24 2010

Is initiation a significant event? The only one I could find from this period was a legendary hack. It was done with Linux. Anyways, I downloaded a whole bunch of Linux distributions at school and tried them out. In the end, I decided on Sabayon because it was a Gentoo distribution with a lot to play around with.  When it came to formatting, I read a forum post and decided on ReiserFS. At first, I saw some files with size 0 and thought I could delete them. Later, I found out ReiserFS had tail-packing, which put small files at the end of a block, if they fit. This makes file access faster, but can increase fragmentation. The amazing performance of ReiserFS made me read some articles about Reiser4, which has not been accepted into the kernel to date.

The point of a Gentoo distribution is to build optimized binaries, resulting in speed increases. The Gentoo wiki had many articles on how to increase the performance from internet browsing to the x windowing system. I remember getting Firefox to display pages as soon as I click on the links and getting the boot down to a few seconds.

Another initiation came from reading The Fountainhead. It was a really inspirational book, built on Nietzche’s philosophy, along with Kant’s. I definitely enjoyed reading it. It helped me make sense of where technology and I fit into the picture.

All this comes down to finding out what I wanted to do for a while. I decided I might as well go with the accelerating trends if I’m going to study anything useful. Just reading a book like that put everything into perspective.

Later, Zen and the Art of Motorcycle Maintenance added a nice metaphysical touch when working with computers. It’s one of the books every programmer should read.

Boos Read:

  1. Beginner’s Guide to Reality
  2. The Catcher in the Rye
  3. The Fountainhead
  4. Zen and the Art of Motorcycle Maintenance
  5. Meditation Classics of a Roman Emperor

Next episode: blastoff!

No responses yet

Evolution Speeds Up (Part IV)

Oct 24 2010

From a physical point of view, everything that has ever happened, and will happen in the universe, is a part of evolution. It begins long before life appears, with the formation of stars and galaxies. To keep it short, Mr. Kurzweil introduced me to his logarithmic plots. I quickly absorbed the ideas of his book. Accelerating returns of evolution. Technology is a craft and a survival tool for humans. Luddites are groups of people who don’t adapt. The implications of the ideas were more profound on a philosophical point of view. It seems that each step of evolution only fixes the mistakes of the previous step. This is well supported by the book. Biological machinery can only be controlled by nanorobots. Nanorobots can turn the entire planet into a gray goo within hours unless there is artificial intelligence. So each step seems to be a step back, not only in the sense of fixing things, but also a reunion with the intelligence of the universe.

This is the point where I decide that a new philosophy was needed. Gnosticism was good for accepting the world we live in, but it’s not deep enough for a foundation. A much better one came from Nietzche, whose philosophy is particularly relevant to the 20th century. He proposed that we live a nihilistic society where values were non-existent and had a philosophy for it. Human values change with technology, and they do so at an increasing rate. Nietzche, in Thus Spoke Zarathustra, writes about enduring values for humanity, or rather for the superman. The superman is the ideal human, who is fully developed. He has complete free will, and the implications of that are outlined in the book. Of course, free will implies that there are no errors in perception. Thus, part of the subtitle of the book is “for none”. Delusions are the human way of seeing the world. Yet, if humans are to survive at all, or rather evolve, as that is an imperative, they must become supermen. The superman is at the end of the development stage of the mind. Although I’m not certain of the meaning of the eternal recurrence, I like to think of it as a reunion of ourselves with the universe. In this sense, none of us will be left. Thus, the book is “for all”.

On the other hand, even the author of GEB wrote a book on French poet’s poem. There is always the other side. It is perhaps best expressed in Ape and Essence, human race heading towards apes, or already apes?

The singularity book had practical implications, as all changes in perception do. I started using the fastest browser at the time. A funny incident happened many times when IE and Firefox were disabled by the teacher, and I was still able to use a browser. Global warming was a hot topic at the time due to the Gore film. I seemed to be the only one in the class to see the impossibility of such an event as the sea waters rising a few meters, due to the growth of technology. Yet another result was Gothic literature being perceived as backwards, Luddite phenomenon. The stories we read in class all seemed to involve someone going insane. The study of the literature seemed to be an obsession with the craziest illusions. It’s kind of hard to write a Gothic short story when I have trouble taking on the perspective. At some point along the road, I dropped the ape side of the story.

Books read:

  1. 20,000 Leagues Under the Sea
  2. The Singularity is Near
  3. Intuition Book
  4. Second Book on Habits
  5. The Bible Code II
  6. Thus Spoke Zarathustra
  7. A Purposeful Book
  8. Ape and Essence
  9. Quantum Programming the Universe
  10. Accelerando

Next episode: initiation

No responses yet

Three Produced the Ten Thousand Things (Part III)

Oct 24 2010

Let us three hold hands and dance.

The Tao Te Ching states, “one produced two, two produced three, three produced all numbers”. Ten thousand is 104.

Three came to my life with Canada. The expansiveness, symbolized by the three dimensions of space, was evident in Canada. A diary entry reflects the rediscovery of the world, a renaissance:

There were noodles, sushi, salad, turkey, juice, and more. There were spicy, sour, and sweet foods. The food was delicious, and I made some friends and learned about them.

Yet another entry detailing new discoveries in the world of science:

He showed me a shape memorizing metal, an electric fan, and a racecar. He twisted the wire spelling “ICE”, standing for the Institute of Chemical Engineering, into hot water, it instantly returned to its original shape. He showed me a solar panel linked to a triangular water transistor linked to a propeller. He was trying to fix the racecar for the kids.

The world was not flat any more, and it was obviously bigger than I had imagined. The world as I knew it changed. I was forced to throw away the old myths. No need to make everything perfect to obtain Total Transformation. After all, it is left behind, I leave it behind, when I leave.

No such thing, actually. There is no escaping the demon.

Not long after I arrive in Canada, an echo is heard from the other side of the Grand Canyon. Karma. Human err. It catches up, and I find myself reading the book without looking for it. It somehow stood out among the other books on the shelf. Was I trying to find the source of the bug that stopped the world? Without a doubt, I was still running from it, to free myself from the imperfection which always troubled me.

Within the constructs of the book, the world operated as a system. If certain techniques were followed and patterns were observed, then the performance variables could be adjusted.

At this point, I decided all of the answers to all of the questions I had could be found in books. The first problem I solved was the human imperfection. Next, I tackled the paranormal, unexplained phenomenon. That lead to another question, “Had everything in the universe already happended?”. Of course, the code on the sapphire stone was the ultimate answer to that last question anyone should ask, Z.

But if I just asked that question, was able to ask that question, then why am I still here? That was the first question, A, the question of a purpose behind the universe, myself, that starts the search for meaning.

Books Read:

  1. The Godfather
  2. Canadian Indian Golf Resort Story
  3. The Logic of Failure
  4. Yes/No Guide
  5. Elements of the Extraordinary Series
  6. The Bible Code
  7. Edgar Cayce on the Millennium
  8. The Mammoth Book of Prophecy
  9. First Book on Habits
  10. From Atlantis to the Sphinx
  11. The Dragon Book (Map of Bones)
  12. 101 Zen Koans
  13. The Golden Book (Godel, Escher, Bach)

Next episode: accelerating returns

No responses yet

An Attempt to Cut the Knot (Part II)

Oct 21 2010

At that time, I had a friend named Frank. I talked with him often on the phone. We had many of the same classes. I liked to discuss other stars looming in my mind. We shared many constellations this way.

He watched everyone loves Raymond often and told me about the Matrix trilogy, “It makes you think”.

In my waking life, I remembered a note about swallowing a demon.

I worked my way back into normal life. Occasionally, I still set special days on which I would perform the work of Transformation to free myself from the Primary Error that caused the Crash. But they never arrived. The sun shined through the window, announcing a new day. I rose, but I could not shine. No light could penetrate the depths of the underworld, where the cartoon caricatures in the world of the day was given animation. I gave up. A toony world was better than facing failure confronting the demon.

I started typing out vocabulary lists for my English class on my computer. My Indian friend, Navin, printed them out for me. It was a win-win situation for both of us. I managed to score 100% on the  vocabulary analogy section. My English teacher thought I was a genius.

Another interesting case from the period was a study guide I made from a list of questions titled, “A free copy with each purchase”. That’s a paradoxical statement. On the one hand, it could mean it comes free. On the other hand, it could mean 2 copies for a certain price, intended for distribution. I favor the second interpretation. The purpose was to get more copies out so more people would want to buy it.

One day, I slept in a little hut in the playground. Then I heard a dog barking, and the police shined a light through the window. I got up, and the police hand-cuffed me. It was a really tight lock with my arms bent back. They led me across the street, shined the light in my eyes a few times and asked some questions. Afterwards, they put me in a police car and drove me home.

In light of that event, I decided there were no laws for me. I told the story at school. The guy who sat next to me was shocked.  Thus, the knot was cut. I was no longer bound to society.

At this time, Yu-Gi-Oh was being played on TV with the Three Musketeers of Doom. The character I identified with the most out of the three was Raphael. He provided an understanding of the negative and positive aspects of the higher plane. Two is about duality, and it is fitting to discover an identification with this character during this period.

Books read:

  1. Gone With the Wind
  2. Artemis Fowl
  3. A virtual reality fantasy
  4. Fermat’s Last Theorem
  5. A Series of Unfortunate Events: the Vile Village

Next episode: a surprising turn

No responses yet

The Monkey's Paw – Keys from the Dungeon

Oct 11 2010

A pair of keys hung, dripping with dew. A shaft of blue light shined against the stone walls.

I re-read a short story from English class in grade 8. My English teacher did not explain the meaning of the story. She simply saw it flat. The plot was a linear sequence of events, and the moral of the story was simply not to interfere with fate. That was the moral of Macbeth, as taught in the class room. But if this was the moral, then why would the author write the story at all, let alone perform it?

Like any story with multiple layers of meaning, the key to decoding it is given at the beginning.

“I’m listening,” said the latter, grimly surveying the board as he stretched out his hand. “Check.”

The son, in fact, is one with the paw, life. There are only two characters in the story, the old couple, who are two sides of one psyche. The sergeant represents the force of change. The son plays the same role as the paw, that of equilibrium. There is no gain without loss. Listening to the winds of change, holding out his hand, symbolizing the paw in control of the game, and the checkmate cannot be mere coincidences. The entire story is about the endgame, the checkmate.

In another instance, the son predicts the sequence of events before he leaves.

“Well, don’t break into the money before I come back,” said Herbert, as he rose from the table.

In a metaphorical sense, when the son came back at the end of the story, the old couple still made no use of the money.

The father holds his right hand to the paw each time he makes a wish. It is unsurprising that each wish and its consequence can be found in advance.

“I should hardly think that he’d come tonight,” said his father, with his hand poised over the board.

The father was unable to make a move. It was checkmate. The son did return, unexpectedly.

“Hark at the wind,” said Mr. White, who, having seen a fatal mistake after it was too late, was amiably desirous of preventing his son from seeing it.

This time, the father realizes the mistake towards the end of the story and wants to prevent his wife from seeing it.

The second key to the story lies in the plot. A wish for gold (represented in this story as two hundred pounds, to be free from the burdens of life), results in the death of the dynamic part of life. This is further confirmed by the result of the two other wishes, which made no effect. As a last nail on this issue, the paw itself became ineffective at the end of the story. So the winds of change brought by the sergeant died with the sudden appearance of gold. From that moment on, things were incapable of changing for the old couple. Life would wear on, day after day.

The second wish, for the dynamic part of life to return, calls for the negative side, or dark side of life. This is the path taken by Darth Vader. Prospero, in the Merchant of Venice (aka Her Majesty’s Magician in real life), contacts angels to find something he lost in life. He gives up on the quest for gold, represented by alchemy.

The third wish, the whisper, is a secret. Star Wars, it does not happen until the last moments, when the father takes off his helmet. The old couple waited for it to happen until they forgot about the wish. That’s how long it takes in real life. Prospero returns to England after his adventure and gives up on his pursuit. There is something extraordinary about this wish compared to the other ones. For one, it does not obey the natural laws (of money or walking speed). This one alone did not have any side effects.

At the end of the story, the road is shone.

The streetlamp flickering opposite shone on a quiet and deserted road.

It is the same road that was there at the start of the story.

No responses yet

Tokenizing Prerequisites

Oct 06 2010

I’m taking a bite by bite approach to interpreting the course prerequisites from the scraper to make its storage in the database possible. Meanwhile, I already finished the part of it that retrieves the data recursively. The approach I decide to take is the same as building a compiler.

I previously favored a quick approach such as the interpreter pattern.

design

However, commas in sentences have different meanings depending on the context.

(BIOL 140, BIOL 208 or 330) and (BIOL 308 or 330)
(CS 240 or SE 240), CS 246, ECE 222

The interpreter pattern assumes a symbol only has one meaning. Thanks again to a course I took, I was already familiar with the problem and the solution. Recognizing and defining the problem was the hard part. A quick overview of the steps involved from start to finish:

First step in the compiling process is to tokenize input.  That takes care of blobs such as “Level at least 3A; Not open to General Mathematics students.”. Totally useless in terms of prerequisites. So it should not show up in the token string.

Building the lexer was as simple as specifying the token list, writing regular expressions for them, and setting ignored characters.

import ply.lex as lex
t_DEPT = r'[A-Z]{2,5}'
t_NUM = r'\d{3}'
t_OR = r'or'
t_COMMA = r','
t_SEMI = r';'
tokens = (
    'DEPT',
    'NUM',
    'OR',
    'COMMA',
    'SEMI',
    'AND',
    'LPARENS',
    'RPARENS',
)
t_AND = r'and'
t_LPARENS = r'\('
t_RPARENS = r'\)'
t_ignore  = ' \t'
def t_error(t):
        print "Illegal character '%s'" % t.value[0]
        t.lexer.skip(1)
lexer = lex.lex()
data = ' (CS 240 or SE 240), CS 246, ECE 222'
lexer.input(data)
while True:
    tok = lexer.token()
    if not tok: break      # No more input
    print tok

Running it gives the output:

LexToken(LPARENS,'(',1,1)
LexToken(DEPT,'CS',1,2)
LexToken(NUM,'240',1,5)
LexToken(OR,'or',1,9)
LexToken(DEPT,'SE',1,12)
LexToken(NUM,'240',1,15)
LexToken(RPARENS,')',1,18)
LexToken(COMMA,',',1,19)
LexToken(DEPT,'CS',1,21)
LexToken(NUM,'246',1,24)
LexToken(COMMA,',',1,27)
LexToken(DEPT,'ECE',1,29)
LexToken(NUM,'222',1,33)

No responses yet

I created the Universe on my birthday (Part 0)

Oct 02 2010

Friday the 9th. My birthday. A group of worshippers celebrated the holiday of a deity. It was the beginning and the end of a world. What had ended, and what had begun?

I frequently obtained perfect marks on those weekly math tests in grade 6. Except in grade 7, I got more attention for it. One guy rubbed my cheeks on Friday for good luck on the tests. On Monday, another guy always announced my grade. I kept answering all those questions and making comments in class that everyone thought I had the Divine Spark. “He knows everything.“ The essays I wrote at the time flowed like poetry. Gold was everywhere I touched, even in the higher realms.  I lived in the sunlit sphere among the leaves of the tree of life.

With gold, comes ritual. A methodical, disciplined approach to life flowed out of the gold and in turn revealed more gold hidden in rocks. The rituals are well preserved in my memory, however, there is not enough space here for it. Instead, a discussion of their contribution and effect on the later part of my life is well worth the effort. For the rituals, just pretend that they are a set of actions, if performed to perfection, brought the greatest of joy. Nevertheless, a less than perfect performance still brought much joy, since the fun was in everything I did. A set of mental repetitions accompanied some rituals to bring the mind to a highly focused state and to enhance performance.

There were some immediate results of a ritualized life: high scores on standardized tests, high score on standardized IQ test, an award for music performance. I outperformed my own expectations on nearly everything: the yearly TCAP, in the 85-95 percentile, IQ test for the gifted class,  99.5 percentile, test for knowledge master team. I was able to run faster than ever in the afternoons.

In light of the 7 years that followed, I must admit I was running. Fear behind me, joy before me. The man flying towards the sun cannot see his shadow. He is blinded by the light. He soars on his wings. What follows is that the wax on his wings melt, and he drops off into the sea.

Books read:

  • A biography of Einstein
  • Lord of the Rings
  • Til We Have Faces
  • Chinese Children’s Encyclopedia of Knowledge (A Hundred Thousand Why’s)
  • A Collection of Children’s Short Stories
  • Workbook of Mind Blowing Puzzles
  • Book of Fun, Games, and Puzzles
  • The Book of Three
  • Charlie and the Chocloate Factory

Next episode: gold turns into ashes, then gets blown away by the wind

No responses yet

« Newer - Older »