Friday, December 31, 2010

Co-lo

We've been going for just over ten years, and in all this time we've housed our own servers. Most of the work we do is either on-site at our clients, or back at base but using client hardware. So our own kit has been primarily for training purposes and running things like a Twiki.

However, primarily as a result of the pretty underwhelming experience we've had with so-called "business" grade internet from our local provider, we've decided to move our stuff to a local data center. In fact we took the opportunity to do a much needed upgrade of some core machines, and bought ourselves some Dell rack-mounted servers. We chose CoreNap in Austin. For $400/month we get 1/3 of a 72" cabinet, housed in a secure data center with 24x7 on-site support, heavy-duty a/c, multiply-redundant, clean power supply, and a chunk of bandwidth. So we get to ditch our cable service, UPS, spike protection, and an old rack we'd been using to house our PC towers.

So far so good. Should have done it years ago.

Wednesday, December 29, 2010

Page not found

One of my team pointed me at an old Joel Spolsky article on measuring software team goodness. I thought it very apt on finding what was on the end of the SEMA link that Spolsky mentions in the first line (but advises the reader against following).

Friday, December 17, 2010

Sunday, November 21, 2010

How to sell high rate professional services and still save your clients money

How much does effectiveness cost? For example, how much per hour should one pay for a good accountant, or lawyer. My field is programming and engineering, so I'm going to talk in those terms. But it applies to amost all billable hour professionals.

So, we know that at very low costs, you may get someone who is just utterly useless for your purposes. They may not be forever broken -- maybe they're just a smart but inexperienced graduate. We all were once. But while they're at that stage of learning, they are barely worth the paper you wipe their noses with.

But the good thing is, they improve quickly. Start spending more and you get a little bit of improvement. Spend more and the increase gets even bigger. Something like this:
 1

Gnus - multiple imap accounts on the same server

Quick config snippet for Gnus. I have two primary email accounts -- one on Google Apps for work, and the other just plain gmail for personal. In moving to Gnus, initially I had a problem getting them both to work. The lisp for .gnus is simple. I'm using gnus-secondary-select-methods to configure my mail servers, like this:

(setq gnus-secondary-select-methods
   '(
       (nnimap "work"
           (nnimap-stream ssl)
           (nnimap-address "imap.gmail.com")
           (nnimap-server-port 993)
       )
       (nnimap "personal"
           (nnimap-stream ssl)
           (nnimap-address "imap.gmail.com")
           (nnimap-server-port 993)
       )
    )
 )

That works fine but it prompts for usernames and passwords, so I need some entries in my .authinfo file to automate that. Unfortunately, a simple view of that would have me writing a .authinfo like this:


machine imap.gmail.com login     work-email password     work-password port 993
machine imap.gmail.com login personal-email password personal-password port 993

But of course that won't work because both lines get triggered when Gnus tries to access either account, since both are on "imap.gmail.com". In practice I think it means it would just access my work account twice. But, according to Simon Josefsson:
nnimap has solved this problem by letting you use, instead of the server address, the virtual Gnus server name when specifying username/passwords in the authinfo file.
So that means I should be able to change the above to:

machine     work login     work-email password     work-password port 993
machine personal login personal-email password personal-password port 993

However, that didn't work. Gnus acted just as if it didn't recognize either of those and insisted on me authenticating interactively. But, helpful people at gmane.emacs.gnus.general (thanks Tassilo) provided the simple solution. All I had to do was append the "force yes" command to the end of each of those lines, like this:

machine     work login     work-email password     work-password port 993 force yes
machine personal login personal-email password personal-password port 993 force yes

Now it's working fine.

Thursday, October 28, 2010

Excel 2011 for Mac

I've toggled back and forth between Windows and Linux over the years. I much (*much*) prefer Linux, but a few applications forced me always to keep at least one foot in Windows. Those apps were: QuickBooks, Projector (a Professional Services Automation tool), and Microsoft Office. In fact only the first two really matter in and of themselves. For most of MS Office, you can shut it in a box with that bloody paperclip. I need Word and Powerpoint only to the extent that other people insist on using those proprietary file formats. But Excel is different. Excel 2007 kicks OpenOffice calc's a*se (I've used them both a lot). I do use Excel for stuff.

But I really don't like living in Windows. Apart from its exposure to Malware, the instability that's probably inevitable given its wide use just drives me nuts. Add to that collateral issues like the sheer amount of utter garbage trial/demo/ad/nag-ware that is inflicted on you when buying a new non-business machine, and also its constant attempts to "help" by guessing what I really want to do (which are never ever useful) and ... well, it sucks.

But about a year ago I decided that a reasonable compromise could be switching to Mac. A unix base, but able to run MS Office natively. Sounds cool. I'd still have to run Windows under Parallels or similar, but I'd get to live for most of my work in a much nicer OS.

But then I tried it and found that Office 2008 on the Mac is to Office 2007 (*The* Office) what the US economy is to a free market. Not Remotely The Same.

Well today I bought and installed the new Office 2011 for Mac. And after a *very* brief test I can say that I think that problem may have gone away. A key part of the test was to load up some Excel 2007 files, including some pivot tables from Projector, and some pretty intense INDIRECT()-based gaskets around those pivots. It all worked perfectly without even a warning.

Now although there were some macros in there (probably by virtue of the pivots) there was nothing in the way of VB. (VB? Messing up my lovely functional-esque pure Excel? Are you trying to be funny? If you need VB you need to go and learn more Excel. And if you *really* need VB, then you probably should be in Python or Ruby or something and dropping Excel completely.) So treat this, as I say, as a very brief test.

But so far, so good.

Saturday, October 23, 2010

Complexity management? You think it's gone? It's never gone!

On seeing my Sheldon impression, one of my guys helped me over the last hurdle with advice on getting that PATH setting for when I'm not launching emacs directly from a terminal (where the PATH is already set). Gordon said:
If you want to set PATH variables for GUI apps in OS X, there are two ways to go

Set up an ~/.MacOS/environment.plist
http://developer.apple.com/library/mac/#documentation/MacOSX/Conceptual/BPRuntimeConfig/Articles/EnvironmentVars.html
You should have a Property List Editor in /Developer/Applications/Utilities/Property List Editor.App  (or quicksilver search for plist)

Or drop an entry into /etc/paths.d/
http://littlesquare.com/2008/01/upgraded-to-leopard-making-use-of-etcpathsd-and-path_helper/

First option will require restarting your user session, second option requires restarting the whole enchilada
Cool, thanks dude. I knew I hired you for a reason :-)

But the question is -- is that really the last hurdle? See, these kind of sysadmin-ish issues are to the geek what Bill Murray's Bob was to Richard Dreyfuss's Dr Leo Marvin in What About Bob?



As soon as I read through Gordon's suggestions the next thought was, but what happens when I move to a new Mac one day; how will I remember that I made these mods? I could always write it down somewhere I suppose, or I could back it up along with everything else. But I don't want to back up *everything*. How do I capture this important piece of information?

But why is this still a question, in 2010? It is now many years since another pal of mine first developed Arusha. We're now well into the age of such machinery as Puppet. Why am I even asking about managing my one Mac? Maybe there's some comfort to be had from things like the forthcoming Mac App Store. But it stills feels awfully GUI-ish and clicky, and I can't help the hairs on the back of my neck raising a little in concern -- probably unjustified -- that an optional Mac App Store eventually becomes an obligatory-without-a-possibly-DMCA-violating-jailbreak Mac App Store.

Of course this goes beyond just sysadmin. One of the approaches my team takes in our professional lives as experts in chip verification, in many ways an abstruse form of software test, is that everything should be automatable in the sense that it is written as a program. On the Big Day (called "tape out"), when person-years of design efforts are to be released to over $1m worth of manufacturing, we'd like to do the equivalent of typing two simple words -- "make tapeout" -- and have, via however many hours of building and processing and checking, the single digit "0" telling us we're good to go. But in over ten years of working with over 100 clients on over 200 projects across the planet, we have yet to see that ideal setup.

Why? Why is "configuration management" -- not just in version control, but in the other aspects of build management, release control, issue tracking, and workspace management -- still the poor cousin to fancy GUIs and IDEs and other such eye candy?

As Fred Brookes said, "Complexity is our business". And complexity, like Bob, is never gone. Why won't we accept that?

Friday, October 22, 2010

I'll hack if I want to

Being a wichtig CEO, I'm not supposed to hack anymore. Excel's OK, but even there I probably make far too much use of INDIRECT() than is respectable. I mean, I'm a captain of industry an' stuff. Officer class. Hacking's for the enlisted men. But wait. It's 5:00pm. Friday evening. The house is quiet. Wife out with friends. No one will know. Just a wee bit -- where's the harm.

Ha. Wee bit? A wee bit? It's never a wee bit!

And what makes it worse is that the more time you leave between exposures to code and all the trimmings, the more you feel sick and decrepit on the odd occasion you try to dive back in. And I'm actually not even talking about real coding here -- just messing with my new Mac's setup in a way most non-geeks wouldn't.

Here's a quick stream of (the lack of) consciousness I'm following at the moment.

Thursday, October 21, 2010

Eh, Yammer; gonnae no.

We use Yammer, the company-internal-use-only version of Twitter. We've been Yammering for a while now. It's cool. We have teams in Scotland, England, Germany, and Texas and Yammer helps provide a virtual version of popping your head over the cube wall and yelling, "ANYONE KNOW HOW TO DO X!?" But today, I got a status email update that made me sigh. And not in a good way. Snippets follow:
Hi Tommy, We noticed you are Following Everyone in your Yammer network.
Ehm, OK. A bit creepy, but fair enough. In these days of Eric Schmidt, we've come to expect such.
Your interest in viewing all the messages in your company is fantastic.
Well thank you. Vote of confidence from my SaaS people. And lest someone accuse me of being Schmidt-like, everyone in Verilab is on the main feed (and there's at least one feed I am not on -- mainly because I don't need to know if the Munich guys are planning another dinner of pancreas fajitas and beer). But anyway, nice as Yammer is being, doesn't it sound like they're setting me up for a "But ..."
Here's the thing:
Ah, see! As Rachel told Josha about "However", that's just just a fancy "But...".
as Yammer networks grow in companies, we've learned a few things. We've learned that as your company network gets bigger, Following Everyone can clutter up your MyFeed and impair your ability to find what's most relevant to you.
Yeah I suppose that could happen in a big company with hundreds of users. But we don't have hundreds. And you know that, don't you, 'cos you already know I'm following them all. No?
On Friday, October 29th, you will stop following everyone in your network. ...
Excuse me?
This way your MyFeed will be focused on the messages that matter to you most.
Excuse me?? You have a "matters most to Tommy" sniffer?
...We understand this is a slight change in how you use Yammer, but we know this will greatly improve your ability to find important information from your colleagues -- and help you get the most from your network.
ExCUSE me?? Would you like to add some bad-word catchers too while you're at it? Maybe add some filters to my email?

Sigh. Yes, yes, I've snipped some bits about how I can get back to my "follow everyone" situation, but why should I have to bother? And exactly what are they going to do anyway, on October 29th, the day of The Great Unfollowing? Are they just going to forcibly unfollow me from everyone in Verilab? If so, is that only because I currently follow everyone -- all 18 of 'em? And could I avoid the problem by temporarily unfollowing one person (and then re-following them on the 30th)? Or are they going to have everyone unfollow everyone no matter who, when or where? Surely not? I'm sure that will piss off a lot of users within large companies who have carefully selected their to-be-followed subset.

Sigh, again. Google. Yammer. All of you. Just gonnae no.
as Yammer networks grow in companies, we've learned a few things. We've learned that as your company network gets bigger, Following Everyone can clutter up your MyFeed and impair your ability to find what's most relevant to you.
Yeah I suppose that could happen in a big company with hundreds of users. But we don't have hundreds. And you know that, don't you, 'cos you already know I'm following them all. No?
On Friday, October 29th, you will stop following everyone in your network. ...
Excuse me? 
This way your MyFeed will be focused on the messages that matter to you most.
Excuse me?? You have a "matters most to Tommy" sniffer? 
...We understand this is a slight change in how you use Yammer, but we know this will greatly improve your ability to find important information from your colleagues -- and help you get the most from your network.
ExCUSE me?? Would you like to add some bad-word catchers too while you're at it? Maybe add some filters to my email?
Sigh.
Yes, yes, I've snipped some bits about how I can get back to my "follow everyone" situation, but why should I have to bother? And exactly what are they going to do anyway, on October 29th, the day of The Great Unfollowing? Are they just going to forcibly unfollow me from everyone in Verilab? If so, is that only because I currently follow everyone -- all 18 of 'em? And could I avoid the problem by temporarily unfollowing one person (and then re-following them on the 30th)? Or are they going to have everyone unfollow everyone no matter who, when or where? Surely not? I'm sure that will piss off a lot of users within large companies who have carefully selected their to-be-followed subset.
Sigh, again. Google. Yammer. All of you. Just gonnae no.

Wednesday, October 20, 2010

org-mode. toodledo. aw naw.

The Healy interview I mentioned led me to Mike Hall's org-mode in your pocket is a GNU-shaped Devil. In it, Hall shares his suffering at the hands of org-mode, an emacs-based plain text approach to, well, everything. Been there Mike. I feel your pain. And yes, just because it's plain text doesn't mean you won't curse the day you began the learning curve of *'s in headers, and drawers, and properties and ... Excuse me. <sniff> No, I'm OK. I'll get through this. I'm OK.

Mike thought he was free when he met his iPhone. Hostile to emacsen, the iPhone ecosystem nevertheless offered plenty of shiny "personal information management" alternatives to org-mode. So he gradually let his emacs use atrophy, and allowed himself to be seduced by iPhone-ness. And Mac-ness. And mobility in the form of MobileMe. And iDisk. And he didn't see his peril until it was too late. org-mode found mobility in MobileOrg. And that would have been OK had Mike not realized that the only thing stopping his old ways reappearing was that MobileOrg needed a WebDAV server. But oh, horrors, what does iDisk provide? WebDav :-( As Mike says, it was as if emacs and org-mode were taunting him with:
“Sucker … walked away from Emacs and even took the extra step of slurping the MobileMe kool-aid thinking it’d harden your resolve against ever returning. Well … fine … keep your precious iDisk … it will become the tool of your re-liberation.”
Well, in my own case, my escape from org-mode came from toodledo. It too has a cute and shiny iPhone app, guarding me from the occasional madness of the form "but we like plaintext precious, yes...".

Yeah, well look what that wee emacs fiend Sacha's been up to. 


Sunday, October 17, 2010

Productivity treadmills

Nice The Setup interview with sociologist, Kieran Healy. I particularly like that he shares my frustration over the fact that modern computing hardware and software can still leave us less productive than people in the past who had little more than two sticks to rub together. Healy says:
"On the hardware side, there's that absurd productivity counterpart to the hedonic treadmill, where for some reason it's hard to get through the to-do list even though the café you're at contains more computing power than was available to the Pentagon in 1965. ... One of the smartest people I've ever known spent half of his career writing on a typewriter and the other half on an IBM Displaywriter. His backup solution for having hopelessly outdated hardware was to keep a spare Displaywriter in a nearby closet, in case the first one broke. It never did."

Tuesday, October 12, 2010

Email things

Couple of email thoughts. This stems from my never-ending attempts to, well, stem the flood.

First, I'm on the verge of deciding that any bulk emails that use mail-merge style methods to make them look like they were addressed personally to me are a form of lying. If I'm validly on a marketing list, and you're sending out mailshots to your audience, no problem. But then be up front about that. Say "Dear <servive-or-product> users", or "Hey people who might be interested...". Just don't pretend I'm special by calling me by by first name. If you do, and I rumble you (and the context usually makes that easy), I'll be annoyed.

Second, here's an approach I'm trying out with gmail. The aim is, as I say, to stem the flood. Well, not really. I get around 30 to 70 emails a day (excluding spam caught by Google) but that's not a flood. Most are quickly dealt with. No, what I'm really trying to do is tame by ADD-esque brain. I don't want to have temptation present itself throughout my day in the form of new unread emails looking seductively at me, eyelashes fluttering from my Inbox. I know, it's a sickness. I'm ill. Sue me.

Of course, one obvious solution is to shut down my mail client. The problem with that is, a lot of my work involves email conversation. So while I want to be able to shut off the receiving of new email, I still have to be able to send stuff.

Now in the olden days, with POP email, that was easily achieved. Just tell your client not to check the server until you were ready. But with IMAP and also with whatever it is that Gmail is doing, it's pretty much always on from the receiving point of view. I imagine there are ways to control that too, but I haven't found them.

So, here's what I'm trying. There are three components: a label (could be a folder if you're not in Gmail), a filter, and a working practice.

The label/folder. It's called "Pending" and, most important, it's hidden. In Gmail that means it resides under the "<n> more v" link at the bottom of Gmail's label list. Since I have only one hidden label, my link says "1 more v". Only if I click on it, do I see "Pending".

The filter. Well actually there are a few I use to catch well-behaved mailing lists and stuff -- things I want to read at the weekend and so on. And I have another set aimed solely at trapping spam that Google can't see. But other than that, I have one all-important filter and it goes like this:

IF the incoming message has an attachment OR the incoming message does not have an attachment THEN leave it unread, skip the inbox, and apply the label "Pending"

In other words, pretty much ALL MY MAIL GOES INTO PENDING.

The working practice. Every morning -- ONCE A DAY (that's my preference, but you could do more or less frequently) -- I do the following:
  1. Go to the "Pending" label
  2. Select every email in one go (i.e. hit "Select: All")
  3. Move them all to Inbox
  4. Click the 'Remove label "Pending"' button
  5. Go to my Inbox and run it through a GTD-style "get to empty" flow.
DONE. No more email handling for today. And no peeking at "Pending" just in case there's something juicy in there. (Do I always obey that rule? No. But it's a lot less distracting than if they arrive in the Inbox)

Now obviously this has implications for people who use email and want a fast response. But my theory is that this system will improve my overall average response, because it's lowering my distraction level. I suppose I could tune the filtering to allow urgent replies to get through. For example, I could provide a code word to important people who won't abuse it, and have my filter allow those emails through immediately. But for now -- well they can always pick up the phone, or IM/Skype me. There's more to communication than email!

Sunday, April 4, 2010

Coincidence? I don't *THINK* so (Part 2 of 2)

So, how come y'all are being overrun by foreign engineers? Well, let's look at some options.

The Indians are smarter than y'all? Well, there are certainly some bloody smart Indians, but my experience is that they're much the same as the rest of us. Sure they'll throw up the odd Ramanujan, but they don't have a monopoly on brains. Ditto the Chinese. Smart, but no smarter.

OK, so maybe they're more experienced? Absolutely not. Here, the Americans do win. Experience in semiconductor design means "tape outs" -- or, pretty much, numbers of designs. In my experience, your average Indian (based in India anyway) hasn't had enough time to get as many tape outs under his belt. It doesn't mean they're bad, just young.

So something else that makes them "better" in some way? Better at learning quickly? No. Better at explaining their ideas? No.

No, I think we'd have to conclude that from the point of view of actual ability to get the job done, your average Indian or Chinese engineer is no better than your average American .

So is it cost then? Yes, I think maybe that could be it. It makes sense. If the foreign engineer is pretty much the same as the local engineer in capability, but is lower in cost, then that could account for at least some of the difference, no? But the trouble is, immigration rules are supposed to deal with that. I've hired "foreign" engineers, and the way I hire, the way I advertise, and the way I pay is scrutinized. You can't just pay foreigners $4.00 and a chocolate cookie an hour.

But there's more than one way to skin an immigrant. Let me tell you of my own experience at the hands of US immigration.

Coincidence? I don't *THINK* so (Part 1 of 2)

Let's do an experiment. You can just do it in your head if you like, or go out and do it for real if you're feeling scientisty.
  1. On a clear, starlit night, go stand in the middle of a field, with your arms hanging loosely by your sides
  2. Looking up at the stars, start to spin around (Try not to fall!)
  3. Notice your arms going up. Notice the starscape spinning above you.
  4. Think, "Coincidence? I don't THINK so"
That's Mach's Principle. The arms raising while the starscape spins is not a coincidence. Isn't this fun! Here's another one. You probably won't want to do this for real:
  1. Go find a copy of each of the Royal Society's Philosophical Transactions published from 1665 to 1850
  2. Arrange them into piles, against a wall. One pile per decade, with the 1660s on the left and the 1850s on the right
  3. Take a marker, and put a dot on the wall at the top of each pile
  4. Return the journals to from wherever you stole them
  5. Join the dots with your marker
  6. Stand back, and notice the way the piles grow in size (Hint: it looks vaguely exponential)
  7. Think, "Coincidence? I don't THINK so"
That's roughly what triggered Derek de Solla Price to develop a theory of the exponential growth of science. So, the moral of the stories so far is, be careful when coincidences appear, because they're not always so.

So, now consider: