Geeks By Nature

Some hours ago, a reddit user suggested an extension for Firefox that I thought would be a good example of why Jetpack is going to change the way users think about modifying their browser experience.

It took me literally minutes to complete this extension, and all in 7 lines of code. I can’t seem to make wordpress play along, so go to this page and click “install” to try it out.

This is how easy modifying your browser should be. This is how easy it is going to be for everyone in the not too distant future. Jetpack is here to stay!

Note: other people have written similar extensions the old fashioned way (Mobile Barcoder, and probably other), however this is an illustration on how easy it can be to do it yourself.

Sparked from a conversation I had with a friend and fellow programmer some odd hours ago, I decided it would be fun to try to make a program that would run in more than one language. Initially, I thought of Brainf*ck and C(++), but my friend suggested I add Whitespace to the mix too. As it turned out, I managed to throw in Python as well.

After some kind people in #esoteric@irc.freenode.net (I joined originally because it was my first foray into the madness that is Whitespace) told me that this is what is called a polyglot, they helped me shave the last fat off and this is the result. A 384 byte “Hello World!\n” that works in Brainf*ck, Whitespace, Python and C++. Not the worst thing I’ve spent a sleepless night on.

As for acknowledgements, I admit that the bf version was one I had lying around, and I don’t know if it was my own or someone else’s. I did the ws version myself because I couldn’t find a working version anywhere. I do owe thanks to coppro and immibis from #esoteric, they were most helpful in the final stages of the process.

UPDATE: reddit users mallardtheduck, isionous and youreameme have suggested further enhancements, which have resulted in shaving off a further 28  bytes, so here is an updated version weighing in at 356 bytes. Many thanks, reddit!

I just stumbled upon this article (since taken down, see google cache) by Catch My Fame, and figured I would take him up on the challenge to break it. It was a good opportunity to play around with canvas and bookmarklets, neither of which I had tried before.

I won’t bore you with too many details, but the gist of it all is to transfer the image into canvas, then do some edge matching and figure out the best solution.

Without further ado, here are the solutions as a bookmarklet (follow the link to see it, WordPress wouldn’t let me link directly) and as a Ubiquity command

It does have it’s limits, but I’ve only seen it fail once, so it has a success rate of about 98% (which is better than my success rate when using Facebook’s CAPTCHA.) Also, I have only tested in Firefox, so no promises if you’re using something else.

If you want to understand what’s going on, I’d advise you to look at the Ubiquity command first; very little is related to Ubiquity.

Does this mean I caught his fame?

UPDATE: It appears Mr. Catch My Fame didn’t want to display his post touting a broken CAPTCHA system, so he took the post down – the demo is still up, so if you hurry you can still see it in action: http://www.catchmyfame.com/jquery/slider_captcha/

UPDATE 2: I have made a small change in the fitness function, making it much more reliable — you can track changes by looking at the Ubiquity command at github

UPDATE 3: The demo has since also been taken down, but as they say, once burned twice shy, and I made a copy well in advance: sliderCAPTCHA. Also, the original blog post appears in google cache (thanks semanticist/Reddit)

Lately I’ve been spending some time working on Bespin integration in Jetpack and Ubiquity. “But Jetpack already had Bespin support!” some of you might think, an yes it did – but only for people running OSX. The rest of us was stuck with a textarea, styled to look slightly like Bespin (and indeed, most people didn’t even notice it wasn’t the real one). Ubiquity had CodeMirror for certain version of Firefox, but for most users it would be disabled due to an incompatibility between later versions of Firefox (3.1b1+ I think) and the CodeMirror project.

The problem — and the reason Ubiquity didn’t change over sooner — was that the official version of Bespin embedded was running a quite old Bespin (0.2.2), while the current version is 0.4.3, and the former didn’t support clipboard copy/paste in anything but OSX.

Long story short: a few bugs later and a lot of help from some very friendly Bespin devels (thanks Alex and Kevin!), inquisitive readers can now try out Bespin 0.4.3 in Jetpack and Ubiquity. It will require you to apply patches by hand against the latest development versions of the relevant project, or wait until the changes land.

EDIT: here is an updated version of the Ubiquity Bespin patch, that includes autoindent, closepairs, and other useful settings.

Feedback (and bug reports) would be greatly appreciated.

Now that Ubiquity 0.5 is on the verge of release, we have scrambled to demonstrate the localization-possibilities by including a couple of alternative languages for the standard feeds (and for the new parser.)

By the looks of it, the languages Catalan, Danish and Japanese are going to make it in, but we hope more will join them before the 0.6 release.

Being a localizer myself (I did the Danish translation) I know the pain of keeping up with ever-changing-, new- and deleted keys. Mostly for my own benefit, I have put together a script that helps you track changes between the templates and the translations. If you place this script in your ubiquity/localization/ folder and run it, it should list relevant information for each language.

Examples:

If you want all the information, simply run the script:

python localization.py

In the current development version, this produces quite a lot of output, so I’ll paste a representative section only:

$ python localization.py
! Missing translation: da/developer.po
! 2 keys are missing in pageedit(da):
----------------------------------------------------
["If you used the 'edit page' command to put the page into editable mode, use this command to end that mode and go back to normal page viewing. If you want the changes to persist on page reload, issue the 'save' command first."]
----------------------------------------------------
["Saves edits you've made to this page in an annotation. They will persist on page reload. You can remove them with the 'undo page edits' command."]
----------------------------------------------------
! 2 keys are superfluous in pageedit(da):
----------------------------------------------------
["If you used the 'edit page' command to put the page into editable mode, use this command to end that mode and go back to normal page viewing."]
----------------------------------------------------
["Saves edits you've made to this page in an annotation. Undo with the 'undo page edits' command."]
----------------------------------------------------

If you just want an overview and not all the details (I imagine this will be useful for project leaders) you can run the command:

python localization.py | grep -E ^\!

which again produces a lot of information but looks something like:

$ python localization.py | grep -E ^\!
! Missing translation: da/developer.po
! 2 keys are missing in pageedit(da):
! 2 keys are superfluous in pageedit(da):
! 1 keys are missing in email(da):
! 1 keys are superfluous in email(da):

The future

The script is not yet added to the ubiquity source distribution, as there are still a lot of changes that I feel need to go in before it is production-ready. I plan to integrate more options into the script, making it easier to get the overview, getting info about only one language, or about only one template. It will also become smarter, and realize when a key has changed, and what it has changed to. Currently as the above example show, it lists each as a missing and superfluous key. As the deadline for 0.5 is looming, I don’t expect it to be ready by then.

For the adventurous localizer, I do think it will already in its present state be a big help, so feel free to try it out and report back with any suggestions, improvements or bugs you may encounter.

Happy hacking and translating,

– cers / Christian Sonne

Another couple of days have passed, so another round of updates is in order. Some of the changes I’ll mention here affect the previous articles: Making a search command – the easy way! and Making a search command – part 2, so if you’ve read them: read this one too!

This is the part that broke your commands

And in more than one way. If you’ve tried some of the commands before the recent updates, you might have noticed that for some reason, certain queries wouldn’t return any previews, while other would work just fine. This is because some search engines return “strange” results in the middle of the normal results – and only do it sometimes – but the only way the parser knew which titles, previews and thumbnails fit together, was by their index in each of their own lists. This was a very fragile system, and while it it still available as fallback, using it isn’t recommended.

The main difference is that the parameter options.parser.container should no longer select the element that contains all the results, but rather it should select a list containing each result. Each results title, preview and thumbnail will then be found (if available) inside the result, and grouping is no longer dependant on indexes. This means that any given result can have a missing preview or thumbnail, and it won’t affect any of the others.

Obviously, changing the meaning of the parameter means you will have to rewrite any search command you have that uses it. I hope the benefit of stability will outweigh this initial annoyance.

JSON as promised

In my last post, I said I was going to be working on JSON support, and as of about 4 hours ago, initial support has landed in hg. I have successfully rewritten our default google and yahoo commands to use this new system, and still use the JSON API provided by the services, but as I don’t have much experience with this sort of interface, it is entirely possible that I have in some way limited the functionality and thus the compatibility with some services. If this is the case, the JSON support will of course be expanded.

A slight note on the usage is perhaps needed here. Let us take google as an example:

CmdUtils.makeSearchCommand({
  name: "Google",
  url: "http://www.google.com/search?q={QUERY}",
  parser: {type: "JSON",
           url: "http://ajax.googleapis.com/ajax/services/search/web?v=1.0&q={QUERY}",
           container: "responseData.results",
           title: "titleNoFormatting",
           preview: "content",
           href: "url"}
});

First you might notice that there are two fields called url, one outside the parser and one inside. In that order, the first one does exactly what it usually did, except it’s only used if the user presses return to go directly to the search page. The second is the one used for the actual searching if you pass type: "JSON" to the parser. If the API requires you to use POST, this works the same way as you’re used to.

Secondly, if you happen to be uncannily familiar with the structure of the JSON google returns, you might notice that the container parameter I talked about earlier in this very post doesn’t seem to be pointing to each result. Instead, it’s pointing to the container of all the results. If you think about it however, in the HTML-case, jQuery makes sure we are returned a list of results, but in JSON, the parent element holding the results is already a list of results – so it sort of makes sense.

The only other new thing is the addition of the options.parser.href parameter, which was automatically calculated in the HTML version. In JSON, the url is usually held in a separate field, so this is needed to make sure we actually link to the results.

What you can do to speed things up

These changes that break backward compatibility will happen less often – or hopefully not at all – if I get fast feedback and suggestions. The more you complain, the more likely I am to do something about it fast

To try these changes out, run Ubiquity from source, or wait for 0.1.7 or 0.2pre15 where they will most likely be included.

A few days have passed since my first post Making a search command – the easy way!, and a few new features have been implemented, most of which were suggested by readers – so keep up filling the suggestion box!

Picture this

Most notably, it is now possible to include a thumbnail in the preview, which will be automatically formatted and inserted to the left of the preview text. This is especially useful for search-sites which display images to help identify the result – such as books, cd, movies, etc…

As an example, here is the Wikipedia command redone with this enhancement:

CmdUtils.makeSearchCommand({
  name: "Amazon",
  url: "http://www.amazon.com/s/ref=nb_ss_gw?url=search-alias%3Dstripbooks&field-keywords={QUERY}",
  parser: {container: "div.listView",
           title: "div.productTitle a",
           preview: "div.productTitle span.ptBrand",
           thumbnail: "div.productImage"}
});

You will also note the lack of an icon and description, but if you try it out (in latest source) you’ll see that it makes no difference. If left out, they are automatically generated from url and name respectively.

Search preview with thumbnails

Correcting errors

Some sites (like Amazon) have a tendency to return weird results in the middle of the set, without any real indication that it isn’t an actual result, and this can cause the parser to become confused (say, if the number of titles and the number of preview do not match up, which belongs to which?), and before, it would give you a warning in console and then most likely die before returning any output to the user. Now, it will fall back to just supplying a list of titles, and explain to the user why the previews are missing.

Search with broken preview

What’s next?

The next thing I’ll be looking into, is allowing makeSearchCommand to use services that return JSON encoded data. This will enable us to use APIs made public by many search engines instead of scraping the HTML, and is a feature that has not only been requested by early adopters, but will also make it possible (and/or easier) to switch over to this new parser system for all of our default search commands.

As usual, I encourage you all to try out the new features, and see if anything breaks or behaves unexpectedly. And of course, if they do: let me know!

Ubiquity prides itself on not only making it easy for the average user to understand how ubiquity works, but also for making it easy for command authors to enhance it – and for good reason. Anyone should be able to write a simple command, to make their day just a little less tedious, just a little more efficient.

How it used to be

Now, making a search command has been very easy for a while, thanks to the excellent work of a handful of the core devels. To show how simple it has been, I’ll develop a command to search our own trac, and use this process to illustrate the recently added features.

First thing we do, is head over to Trac and do a search to see where it takes us. This url is quite human readable:
https://ubiquity.mozilla.com/trac/search?q=example
We can then create the search command with this simple code:

CmdUtils.makeSearchCommand({
  name: "Trac",
  url: "https://ubiquity.mozilla.com/trac/search?q={QUERY}",
  icon: "https://ubiquity.mozilla.com/trac/chrome/common/trac.ico",
  description: "Searches Trac for your words."
});

and that would work just fine. Go ahead, try it – just paste it into your command editor (note: wordpress is refusing to let me link this, so here you go instead: chrome://ubiquity/content/editor.html .) Now just bring up ubiquity and type “Trac example” and hit enter – you should end up at the same place as the link above.

Here is a screenshot of how it looks:

Screenshot of simple Trac search

“But what about a decent preview of the results?”, I hear you cry, “like the one the google search has, or wikipedia – you can even navigate it by using your keyboard!”. Well here’s the thing – those aren’t that simple to do. The more technically inclined can take a look at how it’s done here (note: you have to view source.)

As if this hurdle wasn’t enough – we should also be thinking about standardizing the way search commands look and feel. This is key in user experience: consistency.

Easy does it

Instead of having the user write a lot of semi-redundant javascript each time he or she wants to make a search command, or indeed at all, it is now possible to pass a simple parser argument to makeSearchCommand which makes Ubiquity do all the hard work for you – now doesn’t that sound nice?

The argument is called parser, and is an array of options: (note: this feature has only recently landed in hg, so you’ll have to run either ubiquity from source, or wait till version 0.1.7 or 0.2pre14, which will almost certainly include this feature)

container

a jQuery selector that can help you narrowing in your search for the actual results on the page

title

a jQuery selector that will match each of the titles in the results

preview

same as above, except it will match the preview data returned by the search engine (if left out, only titles will be previewed)

baseurl

a string prepended to the link in the title (should only be passed if the search engine returns relative paths)

Finding the right jQuery selectors to pass is still a bit more technical than what might be expected of the average user, and we hope to alleviate that problem sometime in the future, perhaps with a neat UI that helps you pick out the titles and previews with a simple mouseclick. For now though, let’s settle for this and move on with the demonstration.

For Trac, where the HTML is fairly clean, finding the right values was a breeze, and for the avid reader wanting to try this out for themselves, I recommend using Firebug along with its “inspect” feature.

As soon as we pass the parser option to makeSearchCommand like so:

CmdUtils.makeSearchCommand({
  name: "Trac",
  url: "https://ubiquity.mozilla.com/trac/search?q={QUERY}",
  icon: "https://ubiquity.mozilla.com/trac/chrome/common/trac.ico",
  parser: {container: "dl#results",
           title: "dt",
           preview: "dd.searchable",
           baseurl: "https://ubiquity.mozilla.com"},
  description: "Searches Trac for your words."
});

Will do a pretty good job of emulating all that hard work that would otherwise have gone into creating the preview. Again just paste the above into your command editor, or simply insert the parser argument by hand.
This is a screenshot of the new behaviour:

Trac search with preview

Of course in the future, this might very well be improved upon, better looking template, better error-handling, there are lots of potential improvements out there. And that is where I leave you – with the freedom to play around with this cool new feature, and the opportunity to help evolve this beyond its current state, and hopefully towards something better for everyone. The easiest way to do this is to try it out on your favourite sites, and report back here any bugs or shortcomings you may find.