I love my job

If you are using Firefox or Chrome, and your video drivers are very up-to-date, you should see something more than a couple of non-functional buttons below. Putting this together took me a little more than half an hour (I had to dig up an undocumented feature of X3DOM) and it totally counts as work. Awesome.

Don't dis *******

The other day I had somewhat of a ranting post about a certain version control system which will remain nameless. It's very rare that I discuss programming issues at all on this blog, and that was the first time I had mentioned something that was even remotely connected with my job. Although I use my real name, I have meticulously avoided any mention of my employer whatsoever. It goes without saying that the commentary on this blog only represents my own personal opinion, not those of my employer, my wife, my dogs, my state representative, or my guitar.

If you Google around for this certain version control system -- which you can't because I'm not going to say the name of it -- you will find that their senior engineer rapidly identifies any blog or other web presence that is dissing their product, and tries to address the person's concerns. In some ways, this could be a good thing, and I can respect that. If someone had a problem with a product I worked on and was passionate about, I would both want to help them, as well as try to ensure that any public criticisms were at least honest. And in fairness, the senior engineer in question does seem to honestly want to assist people, and if you read his replies to other people who have complained, he is at times willing to acknowledge shortcomings in the product.

This is all fine and good, but I got a phone call yesterday that -- in light of some new information this morning -- has retroactively scared the shit out of me.

One of the guys from our local ******* support team called me up, and the following conversation ensued:

OTHER GUY: So uh, I heard you've been having some problems with *******...

ME: Oh yeah, [co-worker who was helping me try to track down a solution the other day] must have left a message for you.

OTHER GUY: Um, uh, no, it wasn't him.

ME: Huh. Well, okay, anyway, my problem was that...

A discussion of the problem ensued, which was reasonably productive (though my frustrations with ******* remain), but I was a little disturbed even at the time, because there was a subtle undercurrent of tension in the whole thing, and the poor guy seemed really eager to make sure any and all of my concerns with ******* were fully addressed.

Found out why this morning... the senior engineer in question replied to my post -- and actually I think there was a potential for a useful dialog there, despite the fact that I think he misunderstood the specific issue I was having the other day -- and alluded to "your company's excellent ******* support team."

So yeah. Apparently the senior engineer saw my blog post and wanted to address it -- which is fine -- but first he figured out where I worked, which I had thought was no mean feat (try Googling me; you won't get far as my name is rather common) and talked to the local support team. As in, he talked to my employer. Because of something I wrote on my blog. Oh fuck.

I am quite certain he was just trying to help, but this is total Panic Button time for me. There have just been too many horror stories about people getting fired because they expressed their personal opinions on their blog. I knew from the start I was taking a risk by using my real name, but since as a rule I don't discuss issues relating to my employer, and since it's non-trivial to tie my name back to the name of my employer, I felt reasonably safe being open about my identity. I guess not.

If you Google around about this product and the senior engineer's omnipresent defending of it -- which again, you can't, because I'm keeping my goddamn mouth shut from now on -- you will see that I am not the first to be skeeved out by his approach. As I say, I know he is trying to be helpful and is being passionate about his product, but if you were able to Google, you will see multiple people complaining that they feel they are being "put on trial" or whatever for complaining about the product. Nuh uh, I'm not going there. I have a 1-year-old son, I don't need to put my family's financial security in jeopardy just because I happen to dislike some of the design choices made by a certain tool that my employer uses.

If the senior engineer had replied to my blog post first, and without attempting to uncover personal details I had not explicitly revealed, we could potentially have had a dialog. But instead, he contacted my employer without giving me a heads-up first. It is very very very very important to me that nothing about this blog be connected in any way to my employer, since they don't have fuck-all to do with each other, and I would like to be free to express my personal opinions without fear of it impacting my family's future. I strongly regret having used my real name now, but it is probably too late (even if I changed my name in my profile, all of the old comments apparently retain my original name).

So I have deleted the apparently-controversial post, and I will not being saying anything publicly about ******* any longer. I have disabled comments on this post, because I don't wish to continue any conversation which connects my blog and my employer. If the senior engineer in question reads this, I would ask him to please not follow up on this. I'm just too freaked out by this whole thing, and would like to just put it behind me.

Robots evolve to deceive -- how could they evolve to cooperate?

Note: I am pretty sure I use the term "Bayesian network" completely incorrectly throughout this entire post. Just pretend I used the more general term "neural network". Or, if that's wrong too, just pretend I said whatever thing that wouldn't make me sound stupid and pretentious.

I have this vision in my head from school days of a bunch of inputs and outputs connected by some number of intermediate nodes that do a weighted sum of the inputs -- with the weights being the coefficiencts that change over time, whether by "training" the network or via genetic algorithm. Whatever that thing is called, that's what I meant. (That'll teach me to try and talk authoritatively about stuff I learned eight years ago and never applied...)

The scienceblog-o-sphere is mildly abuzz with a fun new study about Bayesian-network controlled robots evolving the ability to deceive each other. Click the link for details, but to very briefly summarize: The robots can only detect a "food" source at very close range, but they also have the ability to flash a light which the other robots can detect at a distance. They start off flashing randomly and searching randomly. Within a few generations, some robots evolved an attraction to the flashing lights, since it might indicate the discovery of a food source. Within a few more generations, though, many of the robots started to control when they flashed the light so as to "remain silent" when in proximity to a food source, thereby preventing others from discovering it and crowding them out for resources.

Neat stuff. I can't help but think the same thing probably could have been done with a computer simulation, but doing it with robots is just so cool. (It probably also has the advantage of removing a number of possible experimenter biases in the computational model of how to represent movement, sight, overcrowding, etc., since you can just let the laws of the physical world handle it.)

This got me to thinking, how would the conditions have to be different in order for the robots to evolve cooperative behavior? Could the conditions of the experiment be modified so that, after enough generations, some robots intentionally signaled the presence of a food source to their peers?

It is very difficult to see this happening in the present setup, where all the robots can see the "signal" emitted by all the other robots. A "gene"¹ would have no selfish reason to alert its peers to the presence of food, because there is no mechanism within that experimental framework to signal robots who shared the gene at a higher frequency than those who did not share the gene.²

But what if we change the experiment to involve several different "species" of robots. Some of them emit blue light, some green light, some red light, etc. Furthermore, a robot can only see the color of light it itself emits, i.e. it can only "see" the signals of members of the same species. The species all compete for food in the same common area, and the selection criteria applies cross-species, i.e. survival is awarded to the 200 fittest robots regardless of species. "Mating", on the other hand, is constrained to within species.

Now let's assume that "somehow or other" -- and I'll come back to this, because it's quite a large bit of hand-waving -- one of the species develops a significant sub-population with the "cooperative" gene. Call it the Green species, for sake of convenience. Robots of the Green species would, on average, be alerted to food sources much quicker than robots of other species, and so ought to tend to outperform them in each round. While the within-species selection would be against the "cooperative" gene, it might in the short term tend to out-reproduce the various genes of other species simply by virtue of random mating with more selfish Greens. In other words, a sub-population of selfish Blues might be more individually fit than cooperative Greens, but the selfish Greens are crowding out the selfish Blues and forcing them to disappear. Meanwhile, the few cooperative Greens that do manage to survive could continue to sprinkle the "cooperative" gene amongst the burgeoning Green population.

I think this might be a bit fanciful in that form. It doesn't sound particular sustainable to me, and more damning is that I can think of no evolutionary process by which the Greens would develop a significant sub-population of cooperatives to begin with. (This is the frantic hand-waving I referred to earlier) If a Green mutated to show cooperative tendencies, the selective pressures would be strongly against that mutant. The species-wide benefit would have to be immense in order to facilitate the by-chance propagation of a gene that was so clearly bad for the individual, but a large species-wide benefit would not be able to develop until the "cooperative" gene constituted a sizeable sub-population. It seems to me this is an unsurmountable can't-get-there-from-here problem.

Unless the inheritance were somehow tweaked to behave in a Mendelian fashion, with the "cooperative" gene being a recessive trait. Now I think maybe we are getting somewhere. A "cooperative carrier" can still behave selfishly, but it also benefits from the fact that 1/4 of its brethren are pure cooperatives. There is still a bit of a jump start here, because cooperative carrier Greens are, within the species itself, no better off than pure selfish Greens. But if one species were to develop even a relatively small population of cooperatives and cooperative carriers, that species could suddenly start to significantly outperform other species. The selective benefit to the cooperative carrier is clear: it surrounds itself with cooperatives that it can exploit in order to out-compete other species.

Hmmm, this makes me want to try to duplicate the experimenters' results in software and then start playing with the model... Of course, the fact that I've never directly worked with Bayesian networks or genetic algorithms could put a damper on that...

¹Depending on the experimenters' exact implementation of the neural network and the inheritance scheme, it is probably not quite accurate to use the word "gene" here. At this time, by "gene" I am simply referring to some group of coefficients in the Bayesian network which produce a particular behavior. I am also going to pretend that this "gene" is atomically heritable, for purposes of this thought experiment, even though that may not reflect the experiment at hand.

²We can maybe envision a group of robots which devised a special "code", i.e. a series of pulses of light, that signified the presence of food only to others who were genetically "in the know", but this is fraught with all sorts of difficulties. For one, even a brief code would likely attract robots who did not comprehend the code. For another, robots who did not have the "cooperative" gene could still develop the capability to understand the code and exploit the cooperative robots. Perhaps most damningly, the development of this code is too complex to evolve all at once, and looking only within the context of food-signalling, there is no evolutionary advantage to a partial or weakly-comprehended code. In order to surmount this, the robots would have had to evolve a coded communication ability for some other purpose, which was then co-opted into food signalling -- and that is going way beyond the scope of even this quite fanciful thought experiment.

Internet Explorer Epic Fail

Oh shit, I just discovered that all of my custom Javascript has been failing in IE, which caused the "Continue reading..." link at the bottom of longer posts to get omitted entirely. It may also be breaking comments in IE as well.

Fucking Internet Explorer... WTF... Anyway, the blog may be unstable for the next little while as I try to fix this. Goddamn it..

Update: It works in IE8, but not in any version before that. It also seems to fail in versions of Opera prior to 9.8. So it looks like some of the Javascript I used is fairly new. (Although, it works fine in both Firefox 0.9.3 and Chrome 0.4.154.33 -- even though Firefox 2 came out around the same time as IE7. Just sayin'.)

I'm thinking it should be possible to either a) detect whether it's going to work or b) failing that, detect the most common too-old browsers¹, and insert a fallback. The fallback in this case would likely be that every single post will have a "Continue reading..." link, regardless of whether there is anything below the fold or not. Boo, hiss!

¹I adamantly refuse to detect browsers I know are compatible, and then use the fallback if the browser is unrecognized. I have an ideological problem with this. If the website doesn't recognize the browser, it should (maybe) warn the user, and then chuck the HTML at 'em anyway. If it works, great. If it doesn't, use a different browser.

Blogging via text message

The problem with blogging via text message is that Verizon always puts "This message sent using yada yada" at the end. To see how I resolved this problem, check out the HTML source of this post...

This message was sent using the Picture and Video Messaging service from Verizon Wireless!

To learn how you can snap pictures and capture videos with your wireless phone visit www.verizonwireless.com/picture.

Note: To play video messages sent to email, QuickTime� 6.5 or higher is required.

Reinventing the Wheel

There were a couple of web-related things I recently puzzled through before finding out somebody had already solved the problem, but I'm kind of proud of them anyway.

The first one was a Perl script to convert from LiveJournal's export format to the Atom format for import into Blogger. I wrote it so my wife could transfer her old LiveJournal blog. After some finagling, I even got it to import the comments correctly and get the timestamps right.

I later found out some other guy had actually written a much more full-featured application for managing and converting LiveJournal posts. The punchline, though, is that he also wrote it for his wife. Ba-dump!

The second cool thing I figured out you can see by clicking on the link below.

Woo hoo! Now, I know everybody's blog already has this, but I ended up doing my own homebrew solution without peeking. It was doubly satisfying because I didn't know any Javascript prior to this, and I ended up with a JS-heavy solution.

Blogger's help tells you how to do a rudimentary fold by defining a style called "fullpost" that doesn't do anything if you are on a post page, but causes the text not to be displayed if you are anywhere else (e.g. like on the main page). So then you just wrap everything below the fold in a <span class="fullpost"> tag and that makes it appear in posts but disappear on the main page.

The problem is, they then have you put a fixed "Continue reading..." link in the HTML that appears at the bottom of every single post on the main page. So either you have a fold in every post (blech!) or else you deal with having a "Continue reading..." link that doesn't actually give you any more content (double-blech!). They say that fixing this is left "as an exercise for the reader".

My first solution did not involve any Javascript, but instead used a conditional based on Blogger's custom tags. I had it scan the labels for each post, and if it found the label "more", then it inserted the "Continue reading..." link. Note that it used no standard scripting language, only the custom tags provided by Blogger, meaning that all of this was executed on the server-side. The two big disadvantages of this approach is that a) I have to remember to add the label to every post where I use the "fullpost" span, and b) the label "more" was showing up all over the place. I partially solved the second problem by modifying the code for listing the labels so as to forcibly exclude "more", but this occasionally messed up the formatting of the labels, and also caused some weird bug in the tag cloud code that I ganked.

So on to Javascript! First, I changed the HTML for each post entry on the main page to wrap the entire post in a span tag with an id that matched the post ID. Then, I added a script that executes at the bottom of each page. The script grabs the HTML DOM element with the id matching the post ID, then searches that element for span tags, and compares the class of each to "fullpost". When it finds a match, it inserts the "Continue reading..." link. Voila! Now, any time I use the "fullpost" span, your browser figures out I did it and adds the link. Nice.

The finishing touch I added today. In many blogs, the "Continue reading..." link will not only take you to the post page, but will jump to an anchor marked right where you left off. I wanted it to do this, but not have to actually add the anchor in myself, i.e. I wanted it to somehow put the anchor in for me wherever I put the "fullpost" tag in.

It would be nice if CSS allowed you to put macros in for certain classes, i.e. if I could just tell it the text I wanted to appear whenever I had something of class "fullpost". But no dice.

So instead, since I had code to find the DOM element corresponding to "fullpost" span tag anyway, I decided to see if I could just modify the DOM tree to insert the anchor. I thought I might have to do something fancy, like instantiate an anchor element and insert it into the linked list. But I forgot that Javascript is, well, a scripting language and not a programming language, so they try to make it intuitive. And sure enough:

spanElem.innerHTML = "<a name='more'/>" + spanElem.innerHTML;

Yep, that was it. Just access the damn HTML and prepend a string to it. Too simple!

After I wrote that, it occurred to me that I wasn't sure if the Javascript would execute before or after the browser searched for the anchor tag, but it seems to work in Chrome. Ping me if it fails in any other browsers.

The only thing that worries me is that it's doing a lot of important formatting on the client side... it seems like putting anchors and links in place ought to be done server side, or at least that's what it feels like to a non-web developer like me. The pages still seem to load pretty fast, though, and it's not like you can view Blogger without Javascript support anyway, so I think it's okay. Hmm, maybe when I said "ping me if it fails in any other browsers," I should have mandated that they be, you know, modern browsers. I just know somebody is going to be like, "It doesn't work in Links!"

Displaying my web development idiocy

I was messing around with creating a Widget here, and I wanted to take a remote web page, extract some info from it, and display it. I am just familiar enough with the concept of DOM, and recently educated myself enough on Javascript, that I am confident I could do it in about five minutes if I could just get the DOM of the remote web page loaded into a Document object.

But damned if I can't figure out how to do that...

First I tried putting it in an invisible iframe, but the problem is that the iframe might not be (actually, won't be) loaded when you go to start traversing its DOM tree. Then I thought maybe I could have an onLoad event for the iframe document, and go back and fix the HTML then... but it turns out that doesn't work, and a lot of people make that mistake. You have to put the onLoad handler in the frame itself, and... I can't do that, because I don't have control over the remote web page.

So then I started looking into Ajax, which I am pretty sure is the right thing, but I couldn't for the life of me get it to work. I don't really understand how to debug web stuff, so I am stuck. Gah!

On the plus side, I think I learned enough about DOM that I should be able to create a spiffy Javascript thingy to replace the horrible workaround I did to make those "Continue reading..." links appear at the bottom of the appropriate posts. Right now, I put a special label called "more" on each post where I want it (blech!) and then I hacked all the label-referencing code to omit that label. This has numerous disadvantages though.

However, with JS I should be able to get the div that includes the post, then search to see whether it contains the special div tag to hide the text below the fold. Shouldn't be too hard...

Update: Woo hoo, after much fumbling it worked! It probably would have taken me no time at all if I hadn't got confused and started searching for a div tag when I was actually looking for a span tag. D'oh!

Getting rid of that stupid "more" hack also fixed a problem I was having with my tag cloud. Ah, much better... now only if I could figure out AJAX...