mason

why you must use pinned transfers to compare CUDA device bandwidth

2010-01-19T04:46:28Z

correct me if i'm wrong, but NVIDIA's bandwidthTest program included in the CUDA SDK takes single timing measurements and reports them. if there's any noise in the measurements, these single reports may be misleading. i wrote my own bandwidth test program that takes 100 measurements and spews out the resulting data for analysis. with my program at least, there is plenty of noise if you measure unpinned rather than pinned transfers.
below are some plots to show it. each plot shows the percentage of transfers that completed in <= the amount of time on the x axis. so the closer a line is to being purely vertical, the less variation there is in the transfer times for that device. the farther left a line is, the faster the transfer times for that device. the plots compare transfer times on my two systems--maul and vader, to transfer times on a friend's system. note that i've restricted the plot so the y-axis only goes to 0.99 (99%) so that outliers don't squish the x-axis.

in this plot, the green and orange lines are all over each other, but in the next one...

...they separate neatly (up till 90% or so).

if you want my measurement code or the R code to analyze its results, shoot me an email or comment.

]]>

notes on The Mythical Man Month

2009-06-09T10:56:16Z

For tips on designing a software system, I read The Mythical Man-Month. I recommend it, but here are my notes in case you just want the juicy bits:]]> ch1 the tar pit

a bunch of software engineering projects run over budget and schedule

ch2 the mythical man month

he estimates

1/3 time for planning

1/6 for coding

1/4 for component test and preliminary system test and debugging

1/4 for final system test and debugging

ch3 the surgical team

even a small team of really good programmers won't be able to churn out a large system as fast as a hulking team of crappy programmers

there are combinatorial costs as you increase team size, due to all the communication that has to be done between subsets of members. you can solve this by making specialized teams supporting one star--to minimize communication--rather than making teams of equals who all have to talk and argue

ch4 aristocracy, democracy, and system design

have a few architects design--the builders will get to be creative in satisfying that design, even if they miss out on the fun of initially architecting it, and the product will be better as a result of having that cohesive design by a small group than it would've been if done democratically

ch5 the second-system effect

the first system you design is necessarily simple and clean because you need to get it out the door. so you make notes of all the neato features you'd want to have if you had more time. and those make it into your second system. but you already know they were unnecessary because you built the first one without them, and even worse, they may have been made obsolete by a basic difference between this second system and the first, which generated these feature-ideas. so watch out buddy

ch6 passing the word

decide what parts of your spec are just undefined

making two implementations is a good way to ensure the spec is followed, rather than changed to match the implementation

when an implementer has a question, he must ask the architect, and the architect should record these questions and answers and disseminate them to all implementers

ch7 why did the Tower of Babel fail?

the OS/360 project kept a daily-updated workbook that contained all the specifications. but the workbooks couldn't just be updated silently, there needed to be a changelog describing the updates, and the updates needed to be marked (this sounds very familiar...)

ch8 calling the shot

in estimates, you must budget time for stuff that's totally unrelated to coding and debugging: interruptions, meetings, ...

there are some conclusions drawn here from thin data... i don't trust 'em

ch9 ten pounds in a five-pound sack

when your code can trade off space and time, make two versions: one that's light on space and one that's light on time, then use the appropriate ones

ch10 the documentary hypothesis

writing out your ideas and estimates and budget, ... forces you to make a ton of small decisions that are essential, so do it.

ch11 plan to throw one away

think of the first system you build as an instructive exercise that teaches you what you actually need to build, then build that new thing.

ch12 sharp tools

they used to automatically schedule (scarce) machine time in 4-hour chunks--so you'd submit a job and it would get scheduled and run, then you'd get the results and could try again if it failed. but they found it was way more effective to just give a whole team unlimited access to the machines for 6-hours in which they could run as many jobs as they needed. that way if your first job failed immediately you could just fix it and start a new one.

let developers build their components freely, but when ready to drop into the system, they give that version to the integration manager who is the only one with the power to allow changes to it. after testing the integrated system, that version of the codebase can be released

build tools to test the performance of your system automatically

use high-level languages (he likes PL/I for system programming, with APL to prototype algorithms) and interactive programming environments

ch13 the whole and the parts

designing top-down will help you to dodge bugs that result from poor specification

test your components well and individually--even though it's tempting to use one component to test that another works

don't throw stuff together and start system testing until you've done component (unit) testing

ch14 hatching a catastrophe

prefer specific, well-defined milestones to ones that are more vague but easier for outsiders to understand

when a component gets behind schedule, its manager doesn't want to tell his boss, because the boss might step in. but the boss needs to know this stuff, so if you're the boss, don't step in when the manager can solve the problem--you want to know this stuff so don't give the manager a reason to hide it from you.

lay out your milestones in a chart showing their dependencies (like a PERT chart), and regularly review your progress through that chart

ch15 the other face

document your code.

describe the program at a high level, in prose--you should probably do this before you even code it

you probably don't need to make flow charts, and definitely don't make a flow chart that's bigger than 1 page

put yor documentation in the same file as your code

make more paragraph-length comments and fewer single-line comments

document as you go

ch16 no silver bullet-essence and accident in software engineering

it's hard to visualize a piece of software--there are too many dimensions of stuff going on

big software systems are too complicated to fully specify up-front--instead, rapidly prototype the system, then figure out the details

grow your software system bit-by-bit so you have working versions all along the way--don't set out to build it through incremental progress toward a grand goal

ch17 no silver bullet refired

not much content here

ch18 propositions of the mythical man-month true or false?

this chapter is a big outline of the whole book with references to current data. i skipped this because i was more interested in his personal experience than in research on software engineering, which i tend to find questionable. how generalizable are the results of analyzing one software engineering project going to be when the studies are purely historical and don't include experimental manipulation? how comparable are lines of code statistics between projects? how interesting is lines of code as a summary statistic anyway?

migrate from Google Docs to Evernote with gdocs2evernote

2009-01-08T18:54:29Z

Google Docs and Evernote will both store your text data on their servers. I've used both for keeping my diary, but I'm moving over to Evernote completely now. Here's a python program I wrote that downloads all of your Google Docs from a particular folder and puts them into an Evernote notebook: gdocs2evernote.zip (Windows only)

do arithmetic on time with Timelab

2009-01-04T15:55:23Z

A calculator that could do arithmetic on time, like 5:13*2 + 10:50 = 21:16, would be handy. So I built it. Download Timelab to try it! You'll need python to run Timelab (macs already have this installed).

Eigenfaces for art

2009-01-02T17:27:07Z

If you discard color, a 100x100 pixel image has 10,000 pieces of information--one for each pixel. Consider each image a data point. To plot them, you'd need graph paper with 10,000 axes, because each pixel's value tells you the position of the image (data point) along the corresponding dimension. And you thought images were 2-D.]]>

The reason for putting images in this 10,000 dimensional space is to find how far one point is from another; in terms of images: to find how similar one image is to another. This can be useful for face recognition. But 10,000 dimensions are too much (google "the curse of dimensionality"). Intuitively, we don't need that many, because to describe how one person's face differs from another, we say things like "his nose is bigger than hers", "her mouth is wider than his", ... and these high-level differences aren't captured by any single pixel.

Now forget about faces for a moment, and think about colors. All the colors your computer screen can display are formed by mixing red, green, and blue. So plot each color you're interested in using 3 axes--one for red, green, and blue--and look at how close they are to each other in 3-D.

The critical step with colors was to recognize that each is a mixture of 3 components. Faces (and other images) can be analyzed this way too. Now the components aren't obvious like they were for colors--you have to choose them yourself, and you'll probably want more than 3. Principal Components Analysis (PCA) is the best way to choose them because it picks the components that express the most difference between faces--so just knowing that face A is 40% composed of component C1 (the best) while face B is 70% composed of C1 should tell you that A and B are very different. In the same way that red, green, and blue are themselves colors, each component is itself an image, like this one:

These "eigenfaces" are just an intermediate step in the technique (I've partially described) that Turk and Pentland developed to do face recognition. But I found some of them stunning, like the ones above and below.

David Billington taught me that good engineering often produces good aesthetics, and that happened in this case. My next idea for eigenfaces as art is to incorporate color, which will take them out of the engineering realm (because color varies too much due to lighting to be useful for recognition, I've heard), but I'm curious to see what colorful eigenfaces look like.

real-time seam carving with openFrameworks

2008-12-13T22:01:28Z

As a final project for my computer graphics class last semester, I implemented a fast version of a technique called "seam carving". Seam carving resizes an image by removing the least noticeable bits. I designed my seam carver to do this resizing on video streaming in from a webcam in such a way that you could remove an object from the video. Read on for my writeup, or just download the code and try it for yourself.]]>

How to solve "unknown error" when installing a device driver in Windows Vista

2008-10-28T18:27:17Z

Windows Update kept trying to upgrade my graphics card (Intel 945GM in Lenovo X60 tablet) and failing. I tried manually upgrading the driver with ones from both Intel and Lenovo, and neither worked. Here are the steps that solved my problem: 2) Right click "INFCACHE.1" and select "Properties"

3) Go to the "Security" tab and click "Edit"

4) Select "Users" and then check "Full control" under the "Allow" column below, then hit "OK"

5) "OK" again

6) Now right click "INFCACHE.1" and "Rename" it to "INFCACHE.1.old"

I also did this for "setupapi.ev1", "setupapi.ev2", "setupapi.ev3", and "drvindex.dat", but I'm not sure they're necessary.

7) Try installing your driver again (you might need to restart before doing this, I'm not sure)

If that works it's probably safe to delete those files you renamed to ".old" (Windows should have made fresh ones, minus the ".old" extension). Apparently that INFCACHE file can get junked up, but Windows doesn't know to recreate it unless you manually remove (or rename) it.

These links helped me out:

http://support.microsoft.com/kb/940199

http://forums.logitech.com/logitech/board/message?board.id=bluetooth&message.id=3755

http://www.broadbandreports.com/forum/r18594702-Vista-Corrupt-INFCACHE1

]]>

Random grammatical sentence generator using CFG

2008-10-18T06:47:09Z

Context-free grammars (CFGs) can be used as a model for natural language. That means they capture the structure of language in a relatively simple form that can be reasoned about and used to generate new instances of natural language. My natural language processing book says CFGs were proven insufficient for modeling German spoken in Zurich (Speech and Language Processing (2nd Edition), p.538), but they're good enough to bring you the following silly, random, grammatical sentences:]]> function randInt(low_bound_inclusive, high_bound_inclusive) { var range = high_bound_inclusive - low_bound_inclusive; var r = Math.random(); r = r * range; r = r + low_bound_inclusive; r = Math.round(r); return r; } proper_nouns = [ 'John', 'Mary' ]; nominals = [ 'fish', 'cat', 'dog', 'zebra', 'horse', 'donkey', 'giraffe', 'walrus', ]; determiners = [ 'the', 'a', ]; past_tense_verbs = [ 'ate', 'dragged in', 'befriended', 'saluted', ]; // cfg maps between a non-terminal symbol and the list of expansions it can have cfg = { 'S': ['NP VP'], 'NP': ['DN', 'PN'], 'DN': ['DET N', 'DET N that NP V'], 'PN': proper_nouns, 'VP': ['V NP'], 'V': past_tense_verbs, 'DET': determiners, 'N': nominals, } var nonterminals = [ 'S', 'PN', 'VP', 'V', 'NP', 'DET', 'N', 'DN', ]; function generate(g) { var tokens = g.split(' '); for (var i = 0; i < tokens.length; i++) { var token = tokens[i]; for (var j = 0; j < nonterminals.length; j++) { var nonterminal = nonterminals[j]; if (token === nonterminal) { // select one of the possible expansions of token to use var expansions = cfg[nonterminal]; var expansion = expansions[randInt(0, expansions.length-1)]; // now replace this instance of token with a recursive expansion of its expansion g = g.replace(token, generate(expansion)); break; } } // if no nonterminal matched, then token was a terminal symbol } // all nonterminal symbols have been replaced by terminal ones return g; } document.writeln('

' + sentence + '

'); They may not make sense. They may be hard to understand. But they should be grammatical.

Refresh to see more. View source to see how it's done.

Is the Problem of Other Minds a problem?

2008-09-27T17:37:51Z

The Book of Tea contains an excerpt I love:

One day Soshi was walking on the bank of a river with a friend. "How delightfully the fishes are enjoying themselves in the water!" exclaimed Soshi. His friend spake to him thus: "You are not a fish; how do you know that the fishes are enjoying themselves?" "You are not myself," returned Soshi; "how do you know that I do not know that the fishes are enjoying themselves?"

Find yourself in photos of the SF Marathon (more coming soon)

2008-09-11T16:10:57Z

I ran the half-marathon in San Francisco this past summer. It was amazing. Afterwards I wanted pictures to remember it by--specifically, pictures that had me in them to remember my experience. Before the race, MarathonFoto informed me that "you and your family will not have to worry about trying to capture your special race moments" because they would take pictures. Great! They neglected to mention those pictures would cost $40 each to download.

I liked the pictures, but that heads-up they gave before the race was slimy. I wondered, "how can I make a better, cheaper service for finding photos of myself (and put these guys out of business :P)?" The result is bibfind.com: a crowd-sourced photo tagging system that uses the bib numbers worn by runners to identify them in photos.

]]> I like to say it's powered by our collective narcissism. When you go to bibfind.com you get to select the race you ran and enter the bib number you'd like to see photos of. But before you get to see those photos, you've got to look at some random photos from the race, and tag them with the bib numbers you see. Then you are shown the photos you searched for (assuming such photos exist and have been tagged).

At this point the only photos in the system are ones posted on flickr of the SF Marathon, but I'll expand to other races and photo services when the site is ripe enough. If you ran the SF Marathon or know anyone who did, try to find yourself at bibfind.com. If you just want to check the site out, try searching for bib # 8685; you'll see a hilarious photo.

If you've got any questions or suggestions, leave a comment here or use the link at the bottom of the site to put in feedback.

Automatic style-checking with Strunk and White's rules

2008-09-05T18:15:49Z

I read The Elements of Style by Strunk and White to get better at writing. The biggest section is on common misuses of words and expressions in English. I thought the section was valuable, but more than I'd be able to keep in my head while writing. To solve that problem, I made a script that analyzes some given text and then annotates each potential misuse with a link to Strunk's guidance.

]]> // this is a list of commonly misused expressions' names and definitions (in regex form), each with a link giving detail on usage var cm = [ ['All right', /\bal(l )?right/gi, 'http://crockford.com/wrrrld/style5.html#1'], ['As good or better than', /\bas good or better than/gi, 'http://crockford.com/wrrrld/style5.html#2'], ['As to whether', /\bas to whether/gi, 'http://crockford.com/wrrrld/style5.html#3'], ['Bid', /\b(?:bid|bade)\b/gi, 'http://crockford.com/wrrrld/style5.html#4'], ['Case', /\bcases?\b/gi, 'http://crockford.com/wrrrld/style5.html#5'], ['Certainly', /\bcertainly\b/gi, 'http://crockford.com/wrrrld/style5.html#6'], ['Character', /\bcharacter\b/gi, 'http://crockford.com/wrrrld/style5.html#7'], ['Claim, vb.', /\bclaim(?:ed)?\b/gi, 'http://crockford.com/wrrrld/style5.html#8'], ['Compare', /\bcompare\b/gi, 'http://crockford.com/wrrrld/style5.html#9'], ['Clever', /\bclever\b/gi, 'http://crockford.com/wrrrld/style5.html#10'], ['Consider', /\bconsider(?:ed)?\b/gi, 'http://crockford.com/wrrrld/style5.html#11'], ['Dependable', /\bdependable\b/gi, 'http://crockford.com/wrrrld/style5.html#12'], ['Due to', /\bdue to\b/gi, 'http://crockford.com/wrrrld/style5.html#13'], ['Effect', /\b(?:effect|affect)\b/gi, 'http://crockford.com/wrrrld/style5.html#14'], ['Etc', /\betc\.*\b/gi, 'http://crockford.com/wrrrld/style5.html#15'], ['Fact', /\bfact\b/gi, 'http://crockford.com/wrrrld/style5.html#16'], ['Factor', /\bfactor\b/gi, 'http://crockford.com/wrrrld/style5.html#17'], ['Feature', /\bfeature\b/gi, 'http://crockford.com/wrrrld/style5.html#18'], ['Fix', /\bfix\b/gi, 'http://crockford.com/wrrrld/style5.html#19'], ['He is a man who', /\bis a \w+ (?:who|which)\b/gi, 'http://crockford.com/wrrrld/style5.html#20'], ['However', /\bhowever\b/gi, 'http://crockford.com/wrrrld/style5.html#21'], ['Kind of', /\b(?:kind of|sort of)\b/gi, 'http://crockford.com/wrrrld/style5.html#22'], ['Less', /\b(?:less|fewer)\b/gi, 'http://crockford.com/wrrrld/style5.html#23'], ['Line, along these lines', /\balong (?:these|those|the)(?: same)? lines?\b/gi, 'http://crockford.com/wrrrld/style5.html#24'], ['Literal, literally', /\bliteral(?:ly)?\b/gi, 'http://crockford.com/wrrrld/style5.html#25'], ['Lose out', /\blose out\b/gi, 'http://crockford.com/wrrrld/style5.html#26'], ['Most', /\bmost\b/gi, 'http://crockford.com/wrrrld/style5.html#27'], ['Nature', /\bnature\b/gi, 'http://crockford.com/wrrrld/style5.html#28'], ['Near by', /\bnear ?by\b/gi, 'http://crockford.com/wrrrld/style5.html#29'], ['Oftentimes, ofttimes', /\b(?:oftentimes|ofttimes)\b/gi, 'http://crockford.com/wrrrld/style5.html#30'], ['One hundred and one', /\bone[ -]hundred(?:[ -]and)?[ -]one\b/gi, 'http://crockford.com/wrrrld/style5.html#31'], ['One of the most', /\bone of the most\b/gi, 'http://crockford.com/wrrrld/style5.html#32'], ['People', /\bthe (?:people|public)\b/gi, 'http://crockford.com/wrrrld/style5.html#33'], ['Phase', /\bphase\b/gi, 'http://crockford.com/wrrrld/style5.html#34'], ['Possess', /\bposses(?:ed|or)?\b/gi, 'http://crockford.com/wrrrld/style5.html#35'], ['Respective, respectively', /\brespective(?:ly)?\b/gi, 'http://crockford.com/wrrrld/style5.html#36'], ['So (for emphasis)', /\bso\b/gi, 'http://crockford.com/wrrrld/style5.html#37'], ['State', /\bstate\b/gi, 'http://crockford.com/wrrrld/style5.html#39'], ['Student body', /\bstudent body\b/gi, 'http://crockford.com/wrrrld/style5.html#40'], ['System', /\bsystem\b/gi, 'http://crockford.com/wrrrld/style5.html#41'], ['Thanking you in advance', /\b(?:thanking you in advance|thanks in advance|TIA)\b/gi, 'http://crockford.com/wrrrld/style5.html#42'], ['They', /\b(?:they|he or she|him or her|his or hers?)\b/gi, 'http://crockford.com/wrrrld/style5.html#43'], ['Very', /\bvery\b/gi, 'http://crockford.com/wrrrld/style5.html#44'], ['Viewpoint', /\bviewpoint\b/gi, 'http://crockford.com/wrrrld/style5.html#45'], ['While', /\bwhile\b/gi, 'http://crockford.com/wrrrld/style5.html#46'], ['Whom', /\bwhom\b/gi, 'http://crockford.com/wrrrld/style5.html#47'], ['Worth while', /\bworth ?while\b/gi, 'http://crockford.com/wrrrld/style5.html#48'], ['Would', /\b(?:would|should)\b/gi, 'http://crockford.com/wrrrld/style5.html#49'], ]; var N = cm.length; N = 48; // for some reason Windows Live Writer was counting wrong so I had to manually set this function analyze_and_annotate_text(text) { text = text.replace(/\n/g, '
'); var out = document.getElementById('output'); for (var i = 0; i < N-1; i+=1) { var name = cm[i][0], regex = cm[i][1], link = cm[i][2]; text = text.replace(regex, '$& ['+name+']'); } out.innerHTML = text; }

In the box below, type or paste the text to style-check.

The text you entered above is reproduced below. Inserted after each potential misuse is an annotation pointing you to the relevant comment by Strunk.

Making whitespace meaningful in order of operations

2008-08-30T04:23:06Z

Of all the mundane details that programming bothers me with, operator precedence/arithmetical order of operations pisses me off the most. Pardon the vulgarity, but if you've ever had to wonder whether 5 + 6 * 2 was equal to 22 or 17, or even worse--had a bug caused by guessing wrong in that sort of situation, you know how I feel.

]]> I like using whitespace to disambiguate that sort of expression for myself, as in 5+6 * 2 (note the lack of space between "5", "+", and "2"). This works great; the "5+6" groups together to our eyes because of the lack of space, and is clearly separate from the "*" and "2" because of the intervening whitespace. (If you're interested: this sort of thing was studied by the German Gestalt psychologists.) The problem with using a scheme like this isn't that it doesn't work for us--it works great; the problem is that all programming languages I know of disregard the information contained in the whitespace. So while I'd say 5+6 * 2 equals 22, the computer would say otherwise.

Steve McConnell recommends you "use more parentheses than you think you need" (pg738 in Code Complete, 2nd edition). That solution is very practical, but it amounts to giving in to the language. I'm an idealistic young kid, and I think we should make the languages suit us instead of changing ourselves to suit the languages. I've also taken enough philosophy courses that just thinking bores me, so here's some action:

Type an arithmetic expression in the box below and then click "evaluate".

Below, you see the results of evaluating the above expression under two different schemes: JavaScript's standard order of operations and my "thoughtful" order of operations which groups expressions based on whitespace.

standard
thoughtful

The code that accomplishes this feat is simple; view source to see exactly how it's done. The heavy lifting is accomplished in under 10 lines of code, and those aren't very dense lines either.

Now the real question is, will I find some open source language and implement this thoughtful grouping? We'll see...

Spatial Frequency in Vision and (a bit about) Art

2008-08-15T22:19:44Z

I just gave a presentation in my Cognitive Neuroscience course at Stanford about Oliva and Schyns' paper "Coarse Blobs or Fine Edges? Evidence That Information Diagnosticity Changes the Perception of Complex Visual Stimuli", doi:10.1006/cogp.1997.0667. I was happy with the presentation, so I post my slides here for anyone who's interested.

]]>

iphone call data visualization

2008-07-31T05:54:40Z

I read Edward Tufte's book The Visual Display of Quantitative Information, and afterwards I was burning to visualize some data. I settled on charting the time of day that I make phone calls at.

]]> the graphic

Thinking about this data, it occurred to me that a simple bar chart or histogram would be inappropriate, because I'd have to choose some arbitrary point at which to start drawing the bars, say midnight, with the result that 11pm would end up on the other side of the chart. That would be wrong, because in reality one comes right after the other. To solve that problem, I decided to plot my calls in a circle. The result is that the angular distance between two data measures on the chart corresponds directly with their temporal distance (fancy term isn't it?).

Similarly, the "small multiple" (Tufte's term) plots for each weekday are arranged in a flattened clockwise loop to keep temporally adjacent days near to each other.

Initially, I envisioned this plot as a circular histogram, and that's how I coded it. Then I had the idea of making those small multiple plots for the individual days, and I realized that I'd need to use a consistent scale for comparisons between those plots to be meaningful. So it turned into a circular bar chart.

I liked the way that looked, but one of Tufte's points kept nagging me: he says that the amount of visual real-estate you give to the representations of your data points should be directly proportional to the magnitude of those data points (I'm paraphrasing, accurately I hope). Looking at my circular bar chart, I realized that I was violating that principle, and that he was right about it. I varied the radius of each circular "bar" in proportion to the amount of calls during the corresponding hour, but the area of a pie slice (segment of a circle) is proportional to the square of the radius of the pie. That means that the area of the circular bar for an hour with 5 calls would be less than half the area for the circular bar representing 10 calls. So I scrapped the bars in favor of lines; they're one-dimensional, and so are the data they represent.

The lines for daytime hours are colored in shades of yellow, and nighttime ones in shades of blue. Tufte says that most people can distinguish blue from other colors, even if they're colorblind, and as a bonus, there's some semantic relationship between those colors and the hours of the day they correspond with (skycolor). Unfortunately the yellow is hard to see against a white background. Expletive deleted.

the making of

To get at my phone's call data, I mangled mrflip's perl code for extracting the SQLite database from the iphone's backup files (see this thread). Then I put together some python scripts to digest the call data into a form that's conducive to charting and send it down the pipeline to a buggy little piece of Processing code that draws the actual graphics. The bugs have to do with the calculation of the low, median, and high values that are labeled on the charts, but I touched up the graphic in Photoshop to mask the issues.

That cover-up was a profound philosophical moment for me. I had the option of tightening my code to handle all cases, pass all unit tests (hah! as if I wrote those), and be prepared for use by total strangers. But as the gears in my mind spun up to design this bulletproof algorithm, the monkeywrench of cool rationality fell in. Taking that code from functional with bugs to bugless would have taken a disproportionate amount of time. I'm ok with mostly bugless, because this code is throwaway; it was just a tool I made as one step in the process of generating the chart.

This is what I like most about being able to code: if I don't have the tools I need to make something, I can make the tools.

WANTED: Engineer to make wildest fantasies reality

2008-07-07T18:33:35Z

As a software engineer with experience doing web development, I hear a lot of people with ideas for hot new websites. In most of these situations where there's an idea looking for someone to implement it, I get the sense that the software engineer's point of view isn't understood. Here's my edited response to a request for someone with spare time to implement an idea that was described at the level of detail of requiring community features and that users be able to upload content:

]]> --------------------------

Let me start by saying that I think it's great that neat opportunities for web developers are posted on this list. Keep 'em coming! But as a software engineer I tend to have reservations about "I have an idea; now I need an
engineer" type requests, so I thought I'd lay out my thought process here for the benefit of everyone. Before I start, a caveat: I may not be representative of all web developers; I believe there are websites where you can go to post a job description and have someone remotely code your site for you, but I'm not the sort of engineer that looks for jobs there, so please take what I have to say with this in mind.

First question I usually have is "what's the idea?" I have a mountain of ideas to implement whenever I find the time and inclination, so an overly vague description is unlikely to get me interested. Of course I understand the desire to protect your idea, but for me at least, the description has to be more of a sales pitch than a "need handyman to build 4'x6' doghouse"-style ad in the paper.

The secrecy also raises a big red flag for me. I think: he's protecting the idea, so he's in love with it, and without any detail on his background, I have to assume that the idea person ("Idea" from now on) is non-technical.
From that combination I conclude that I'll have to do more work than Idea realizes and thus suggests, Idea will be frustrated by the slow rate of progess relative to his expectations, AND Idea will believe that the idea is worth at least as much as my implementation (which is taking so damn long!), so he'll be unwilling to give up an appropriate amount of equity/cash. All of these conclusions might be wrong. Idea might be a PhD in CS who got an MBA, now works at a nice consulting firm, and just wants to do this idea on the side but doesn't have time to code it himself. But I need to hear this up front, otherwise I assume the worst (described above) and nothing is going to get me to play a role in that sad story.

After that initial stuff is out of the way, I'm curious what I can expect from this project. Is it going to be fun from a technical standpoint? Web development involves a lot of mindless drudgery, particularly when it comes to tweaking the visual aspects of a website. So either the idea should be so exciting that I won't mind that stuff, there should be a graphic designer taking care of front-end details, or the financial prospects have to be good enough that I can slog through it (but this option is probably not the most preferable). Which brings me to...

What's it going to pay? I'm not necessarily interested in exact dollars, but more, what sort of cash-equity combination is it. If it's all cash, this might be the sort of job for one of those rent-a-coder websites; if it's all equity, then how will my equity compare to Idea's? If it's not all one or the other, what sort of mix is it (and I'm still interested in how my proportion compares to Idea's)? If the equity is significant (and it probably should be to get our interests aligned) then sell me on the business merits. I've already got a fancy piece of paper entitling me to thousands of shares in a tech-startup, so if you want to sell me another, it had better be even fancier (I'm joking :P) or I also need to be convinced that yours will be worth the ink it's printed with.

--------------------------

Hopefully that gave you a peek into the software engineer's perspective on ideas needing an engineer, because propositions that take this perspective into account are way more likely to get my attention (and other engineers too I'd guess). It probably sounded one-sided, because I'm sure I'm biased here. I understand that we engineer's need business people too; there are plenty of business administration tasks that I'm sure I'd find even less interesting than making the corners on some box render correctly across all the browsers out there. So good luck with whatever idea you're pursuing!