Today I cleaned up the database for Fiddle Salad and Python Fiddle. Both use the same Django back-end for code storage. While browsing tags, I noticed that often both CamelCase and lowercase spellings were used for tags. Since I was working on a tag suggest feature earlier this week, I decided to convert all tags to lowercase so that tag suggestions would not be redundant. An additional benefit is further normalization of the data. Fortunately, I found a fork of django-taggit, the Django app I used for tagging, that supported enforcing lowercase tags everywhere. Two management commands were already present for normalizing data, mergetags and lowercasetags. django-taggit had two fields for each tag, a name and slug. lowercasetags converted all tag names to their lowercase form. mergetags takes at least two tag slugs and merges all tags into a single destination tag. The result is that all associations are moved to a single tag. While mergetags is suitable for manually resolving redundant data, the number of tags on Fiddle Salad is too large. I wrote an command to automate this process:
from django.core.management.base import BaseCommand, CommandError
from taggit.models import Tag, TaggedItem
from django.core.exceptions import ObjectDoesNotExist
help = 'merges all tags automatically'
def merge(self, extra_slugs, dest_slug):
dest_tag = Tag.objects.get(slug=dest_slug)
raise CommandError('Destination Tag "%s" does not exist' % dest_slug)
for slug in extra_slugs:
tag = Tag.objects.get(slug=slug)
raise CommandError('Tag "%s" does not exist' % slug)
items = TaggedItem.objects.filter(tag=tag)
count = items.count()
for i, item in enumerate(items):
if i % 20 == 0:
self.stdout.write('Merging %s %d/%d\n' % (slug, i+1, count))
obj = item.content_object
if not obj:
self.stdout.write('Successfully merged tags into "%s"\n' % dest_slug)
def handle(self, *args, **options):
for tag in Tag.objects.all():
if Tag.objects.filter(name=tag.name).count() > 1:
tags = Tag.objects.filter(name=tag.name).order_by('id')
dest = tags.slug
extras = 
for tag in tags[1::]:
Because performance is not a concern for a single-time data processing script, I did not bother to optimize the queries nor run-time. This script would be useful for anyone who wants to normalize tags in the same manner, so it is in a git repository. Finally, I tested the new command on a clone of the production database.
bash-4.1$ python manage.py lowercasetags
bash-4.1$ python manage.py mergealltags
Merging jquery_1 1/46
Merging jquery_1 21/46
Merging jquery_1 41/46
Successfully merged tags into "jquery"
Successfully merged tags into "jquery"
Merging stylus_1 1/7
Successfully merged tags into "stylus"
Merging hello_1 1/10
Successfully merged tags into "hello"
Merging test_1 1/147
Merging test_1 21/147
Merging test_1 41/147
Merging test_1 61/147
Merging test_1 81/147
Merging test_1 101/147
Merging test_1 121/147
Merging test_1 141/147
Successfully merged tags into "test"
Merging me_1 1/2
Successfully merged tags into "me"
Merging no_1 1/5
Successfully merged tags into "no"
Merging one_1 1/16
Successfully merged tags into "one"
Merging things_1 1/3
Successfully merged tags into "things"
Merging learning_1 1/4
Successfully merged tags into "learning"
Successfully merged tags into "body"
Merging week-one_1 1/1
Successfully merged tags into "week-one"
Merging studio_1 1/33
Merging studio_1 21/33
Successfully merged tags into "studio"
Merging internet_1 1/36
Merging internet_1 21/36
Successfully merged tags into "internet"
Merging assignment_1 1/6
Successfully merged tags into "assignment"
Merging homework_1 1/6
Successfully merged tags into "homework"
Merging lessons_1 1/1
Successfully merged tags into "lessons"
Merging code_1 1/12
Merging tags_1 1/7
Successfully merged tags into "tags"
Merging two_1 1/6
Successfully merged tags into "two"
Merging salcedo_1 1/3
Successfully merged tags into "salcedo"
Merging page_1 1/15
Successfully merged tags into "page"
Merging music_1 1/4
Successfully merged tags into "music"
Merging table_1 1/7
Successfully merged tags into "table"
Merging band_1 1/9
Merging texas_1 1/1
Successfully merged tags into "texas"
Merging biography_1 1/2
Merging assignment-two_1 1/2
Successfully merged tags into "assignment-two"
Merging website_1 1/9
Merging a_1 1/2
Successfully merged tags into "a"
Merging words_1 1/1
Successfully merged tags into "words"
Merging section_1 1/2
Successfully merged tags into "section"
Merging header_1 1/1
Successfully merged tags into "header"
Merging ui_1 1/4
Successfully merged tags into "ui"
Merging first_1 1/8
Successfully merged tags into "first"
Merging random_1 1/1
Successfully merged tags into "random"
Merging internet-studio_1 1/5
Successfully merged tags into "internet-studio"
Merging angularjs_1 1/11
Successfully merged tags into "angularjs"
Merging i_1 1/2
Successfully merged tags into "i"
Merging lines_1 1/1
Successfully merged tags into "lines"
Merging row_1 1/1
Successfully merged tags into "row"
Merging alex-alpha_1 1/1
Successfully merged tags into "alex-alpha"
Merging assignment-one_1 1/2
Successfully merged tags into "assignment-one"
Merging google_1 1/1
Successfully merged tags into "google"
Merging man_1 1/4
Successfully merged tags into "man"
Merging nick_1 1/1
Successfully merged tags into "nick"
Merging cartoon_1 1/1
Successfully merged tags into "cartoon"
Merging batman_1 1/2
Successfully merged tags into "batman"
Merging code_1 1/7
Merging the_1 1/1
Successfully merged tags into "the"
Merging animation_1 1/2
Successfully merged tags into "animation"
Merging band_1 1/4
Merging assignment-one-of-three_1 1/2
Successfully merged tags into "assignment-one-of-three"
Merging status_1 1/1
Successfully merged tags into "status"
Merging python_1 1/2
Successfully merged tags into "python"
Merging cat_1 1/1
Successfully merged tags into "cat"
Merging none_1 1/7
Successfully merged tags into "none"
Merging adam_1 1/2
Successfully merged tags into "adam"
Merging school_1 1/3
Successfully merged tags into "school"
Merging website_1 1/9
Merging biography_1 1/2
Merging bootstrap_1 1/8
Successfully merged tags into "bootstrap"
Merging datamill_1 1/5
Successfully merged tags into "datamill"
Merging gentoo_1 1/2
Successfully merged tags into "gentoo"
Merging dobschal_1 1/1
Successfully merged tags into "dobschal"
Merging weimar_1 1/1
Successfully merged tags into "weimar"
When all went fine, I ran lowercasetags and mergealltags on both Fiddle Salad and Python Fiddle. Now I was really impressed with the results as I clicked through the tags on both sites. The tags on Fiddle Salad were much better organized as they were ordered by popularity. While looking through the tags, I noticed that “test” was among the top. I decided to add ‘test’ to the list of stopwords for django-taggit. These stopwords are removed during save so that they are not associated with new snippets.
Now that the tags are normalized, I am ready to move on and deploy tag suggestions.
After launching the a new tips panel on Fiddle Salad that shows on top of other dialog windows whenever a fiddle is opened, I found a decline in traffic. Specifically, page views decreased. Does that mean the tips were poorly done? To investigate, I compared the views in the past 30 days to the previous period and looked at the traffic for different pages. The top pages ranked by page view saw an increase in ratio compared to the rest. My hypothesis is that when opening a saved fiddle, users were deterred by the tip panel that opened. I found the data to back up the hypothesis.
The analysis is that there were 0 visitors to the pages in the past 30 days because they just didn’t bother to check off “Show tips on startup”. This made sense, as the tips panel often blocked pieces of code. My plan is to just show the tips panel once a day so that opening saved fiddles wouldn’t show it.
I found out that the new tab page in Chrome stable was changed, removing the apps. This would explain the decline in traffic on the CoffeeScript IDE page. This change has been there for 4 months on the Chrome beta channel that I use on Windows.
Angular JS is a popular MVC framework by Google, and I received a bug report that Fiddle Salad wouldn’t reload the preview as it does in all other cases. This problem was noticed before, but it wasn’t a priority since pressing the refresh button as required in all other web development methods solves the problem. However, this was because I was attempting to solve it at the wrong level by modifying a method. As I’ve learned, there are dependencies between the execute, reset, and initiation of the class involved.
Fiddle Salad’s CodeRunner class has just 5 public methods. They are
as in the class diagram on the wiki at https://github.com/yuguang/fiddlesalad/wiki/2.-Classes. Therefore, any implementation of a CodeRunner just needs to support these 5 methods. The debug method could be refactored into a super class because it would generate the template for the new execute() method.
Debug opens a new window with the code loaded in the page. External CSS and JS files are loaded through head tags.
It assumes all external resources are stored in the view model.
css: _.template '<link rel="stylesheet" type="text/css" href="<%= source %>" />'
html: _.template """
<title>Fiddle Salad Debug View</title>
<%= css %>
<%= body %>
<%= headtags %>
# initialize array of head tags
headTags = new Array
# for each external resource in view model
_.each viewModel.resources(), (resource) =>
# get the file type of the resource, call mapped template with resource, and append generated HTML to head tags
headTags.push template[@filetype resource.source()](source: resource.source())
headtags = headTags.join('')
body = engine.get_code LANGUAGE_TYPE.COMPILED_DOCUMENT
css = engine.get_code LANGUAGE_TYPE.COMPILED_STYLE
# call the template for the window with the head tags and code
# open window with generated HTML
window.open = 'data:text/html;charset=utf-8,' + encodeURIComponent(html)
If I just change the last line, the debug method turns into a reset method.
@window.location = 'data:text/html;charset=utf-8,' + encodeURIComponent(html)
The rest of the class still needs to be completed, but they are as obvious as appending to an array when adding JS/CSS files.
TipsPanel = ->
@startup = ko.observable(not store.get('hideTipsOnStartup'))
store.set('hideTipsOnStartup', not checked)
Next, I add a manual subscription to the observable so that its value is stored each time the checkbox is checked or unchecked.
Finally, the checkbox is bound to tips.startup where tips is a TipsPanel instance.
<label><input data-bind="checked: $root.tips.startup" type="checkbox"/>
Show tips on startup</label>
I think the demise of the Python-only fiddle came as a combination of low traffic and no saves. If nobody ever sees it or uses it, then it is better gone. Fortunately, I had the chance to design and develop the next generation of Fiddle Salad to support other languages.
The real Fiddle Salad was launched in April of 2012. I remembered calling the previous version Python Fiddle. Last January, I decided to do an experiment with the CoffeeScript page by creating a separate app in the Chrome App Store. It turned out to double my number of daily visitors. Last May, it was featured on Lifehacker as one of the best apps for development. So why not do it again?
I waited until the right time. Last month and the previous one, the project received major updates which I will be discussing on the Fiddle Salad blog. Meanwhile, I published more apps in the Chrome App Store.
Fiddle Salad is an evolution of both the original idea and code base that belonged to Python Fiddle. Now it is really Fiddle Salad that’s driving the development of Python Fiddle, because they share much of the code base.
So this is the third major milestone, which I almost gave up on before I embarked on it. Before I started work on this milestone, actually a day or two before I planned, I suddenly noticed huge, discouraging signs. They came as shocking surprises. For example, I discovered a hidden option in an application I have used often before that had some of the functionality I was going to build. If that wasn’t enough, it was actually quite popular and many people probably knew that feature. As another example, I discovered another application that was more innovative in certain aspects than the application I planned to build. I got still more examples, but they aren’t worth repeating here.
As a habit, I reached for my next plan and the best tools I have available. I then realized that I would be throwing away about 8 months of work and the plans for this month, which worked out so well. Although I had no reason and no incentive at all to work on Fiddle Salad, I did so only because I enjoyed every moment of it. I believe that’s what we are all here for, the very drumbeat of the universe.
In the end, those serious signs got swallowed up by my project, as I managed to either include their ideas or integrate them right into it. Fiddle Salad is really the culmination and peak of all live web development environments, having the best features in all of them and in my imagination.
I just had an aha! insight 5 minutes ago when working on the design to be implemented next month. The previous storage format used string splitting and joining. Everyone who’s worked with it recognizes the mistake immediately, though the solution didn’t come until I spent a bit of time designing the system.
Previously, get_code and set_code belonged to the abstract factory. Looking at the way things are used, it may better facilitate the implementation of the local history feature if a code storage class was used. I took a snapshot when I got the idea for the JSON format, so things are still a bit disorganized:
The realization is that this design doesn’t require the back-end to store all the different languages used in the fiddle. It allows both PythonFiddle, which stores only Python, and FiddleSalad, which mixes languages, to use the same storage backend. They already do, but I’m getting ready to go to the next stage. The /python/ and /coffeescript/ URL structure is still good for SEO and linking purposes.