{"id":877,"date":"2011-06-24T21:34:48","date_gmt":"2011-06-25T01:34:48","guid":{"rendered":"http:\/\/yuguangzhang.com\/blog\/?p=877"},"modified":"2011-06-24T21:34:48","modified_gmt":"2011-06-25T01:34:48","slug":"coursetree-2-0-an-intelligent-backend-coming-soon","status":"publish","type":"post","link":"http:\/\/yuguangzhang.com\/blog\/coursetree-2-0-an-intelligent-backend-coming-soon\/","title":{"rendered":"Coursetree 2.0: An Intelligent Backend Coming Soon"},"content":{"rendered":"

The goal of coursetree 2.0 is to leverage the current cloud infrastructure to deliver semantic applications that help users find the information they are looking for.<\/p>\n

Features currently planned<\/strong>:<\/p>\n

    \n
  1. Course search that understands what the user wants<\/li>\n
  2. Filtering of irrelevant links<\/li>\n
  3. Pattern based degree data mining<\/li>\n<\/ol>\n

    Draft implementation strategy<\/strong>:<\/p>\n

      \n
    1. Let Google search index Wikipedia and video links<\/li>\n
    2. A\u00a0Bayesian classifier<\/a> will be used to categorize link content into subjects<\/li>\n
    3. Template induction<\/a> and template scraping<\/li>\n<\/ol>\n

      Features under consideration<\/strong>:<\/p>\n

        \n
      1. Adaptable prerequisite semantic analysis<\/li>\n
      2. Fully automated template learning and template extraction<\/li>\n
      3. Relevant course links\/suggested courses<\/li>\n<\/ol>\n

        Tenative ideas<\/strong>:<\/p>\n

          \n
        1. Genetic algorithm for grammar rule generation<\/a> with fitness score assigned according to the total number of parse errors<\/li>\n
        2. Use hashing algorithm<\/a>s to detect similarity in sections of a page, feed similar sections using wrapper induction<\/a> to generate template<\/li>\n
        3. Build map of courses using anti-requisites and display nearest neighbor<\/a>s<\/li>\n<\/ol>\n

          Unsuccessful incubation features<\/strong>:<\/p>\n

            \n
          1. Using genetic algorithms<\/a> to generate templates for wrapper induction<\/li>\n
          2. Switch to parse trees extract noun phrases for Wikipedia link candidates<\/li>\n
          3. YQL<\/a> for video link scraping<\/li>\n<\/ol>\n

            Lessons learned and salvaged<\/strong>:<\/p>\n

              \n
            1. Don’t use genetic programming methods where scores cannot be assigned to each individual “program”, as many of the templates were simply fails with zero scores<\/li>\n
            2. Although in some cases successful (with a comma separated list of noun phrases), in other cases single words were marked as noun phrases in the parse tree instead of a more desirable longer phrase<\/li>\n
            3. Due to frequent changes in video sites, nested JavaScript callbacks with closures to glue previews to links made the code a target to be recycled<\/li>\n<\/ol>\n","protected":false},"excerpt":{"rendered":"

              The goal of coursetree 2.0 is to leverage the current cloud infrastructure to deliver semantic applications that help users find the information they are looking for. Features currently planned: Course search that understands what the user wants Filtering of irrelevant links Pattern based degree data mining Draft implementation strategy: Let Google search index Wikipedia and […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_import_markdown_pro_load_document_selector":0,"_import_markdown_pro_submit_text_textarea":"","footnotes":""},"categories":[29,30],"tags":[134,133,136,53,22,135],"class_list":["post-877","post","type-post","status-publish","format-standard","hentry","category-coursetree","category-singularitarian","tag-genetic-programming","tag-javascript","tag-pattern-recognition","tag-ply","tag-python","tag-wrapper-induction"],"aioseo_notices":[],"_links":{"self":[{"href":"http:\/\/yuguangzhang.com\/blog\/wp-json\/wp\/v2\/posts\/877","targetHints":{"allow":["GET"]}}],"collection":[{"href":"http:\/\/yuguangzhang.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/yuguangzhang.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/yuguangzhang.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/yuguangzhang.com\/blog\/wp-json\/wp\/v2\/comments?post=877"}],"version-history":[{"count":0,"href":"http:\/\/yuguangzhang.com\/blog\/wp-json\/wp\/v2\/posts\/877\/revisions"}],"wp:attachment":[{"href":"http:\/\/yuguangzhang.com\/blog\/wp-json\/wp\/v2\/media?parent=877"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/yuguangzhang.com\/blog\/wp-json\/wp\/v2\/categories?post=877"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/yuguangzhang.com\/blog\/wp-json\/wp\/v2\/tags?post=877"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}