Sunday, May 20, 2012

On video blogging - not

Because Verilab is spread over a chunk of the western hemisphere, getting regular news and status updates out to the teams can be a challenge. I've tried various methods, one in particular being short videos. I create them using Camtasia, which lets me capture not only my video and audio, but also screen activity like a PowerPoint or Keynote presentation, or maybe just the movement of the mouse across a spreadsheet as I describe the numbers. After minimal editing (I usually do the whole 10 minutes or so in one "take"), I export it to an mp4 file and then  upload it to our business Google Video area within our Google Apps account. Usually the videos are made visible for everyone in the company, and there's an easy default setting to allow that. But on the occasions where I'm addressing only one or two folk, that's easily done too. So far, so good.

But more recently I thought it would be useful to have those videos displayed in the context of a blog. The two big advantages of that are:
  1. It would let me include text and other media (like photos)
  2. It would allow the team to comment on what I was saying and ask questions
Another little advantage is that a blog provides an easy feed mechanism (e.g. RSS) so that the team can know when there's something to see. I know they hang on my every word and can't wait to see the next excitement installment of "Tommy Does Excel".

Of course in theory all of the above can already be done, given that the hardest port -- hosting the video -- is covered by Google Video. I could simply email the team when a new video was ready, and use that email for any text message or photos. And then they could reply to that email as a form of comment stream. But anyone who has frequented the "blogosphere", as either a blogger or a commenter, will know that an important component in determining whether or not you will participate is "friction". That is, all the little, even tiny obstructions that get between you and your thoughts being immortalized in text. There's too much to discus on that topic to fit here so all I'll say is that patched-together combinations of emails, hosted videos (that are linked from but not embedded in the emails), and so on all constitute too much friction.

So I've decided what I want is a decent company-internal-use-only video blogging setup that meets the following criteria:
  1. As easy to use, as an author or commenter or reader, as Blogger, or Wordpress, or Typepad, etc.
  2. Able to host videos of between 5 and 20 minutes duration, and to have those embedded within blog posts
  3. Uses EITHER one of our existing company authentication mechanisms OR a "stand-alone" authentication mechanism wherein the user is not required to create an account on some web service somewhere
  4. Allows for user-by-user authentication
Well, to cut a long story short, I failed. I simply cannot find, certainly among the widely available platforms, any combination of services to do the above. Here, for anyone wanting to avoid blind alleys, were the prime contenders and why they failed.

Wordpress hosted on

Wordpress is blogging software. But to use it, you need a place to run it and on which to keep the blogs themselves. One option for that is On its own, this fails 2 -- won't host videos. But that could have been solved using the not-too-expensive plugin. But the real problem was 3, authentication. Wordpress blogs do allow you to restrict access to certain users but the authentication credentials -- the information you provide, usually a username or email and a password, to tell who you are -- are the username and password of a account. In other words, to restrict my blog to be readable only by my team I'd have had to lock the whole thing down, and then open it up to my team based on each of them having their own account. That's a fail.

Wordpress hosted on our own machines

This is similar to the above in that it would use Wordpress software, but we'd install it on our own machines. Point 1 passed. Crucially, we all already have authentication credentials for that (because we use our company machines for a lot of other stuff), so this passes points 3 and 4. But the problem is hosting the videos, Point 2. We realized that in fact we didn't actually need because we already have a hosting solution -- Google Video for Business (i.e. within our overall Google Apps setup). All we have to do then is figure out how to embed those Google Video videos in wordpress posts. And can we figure that out? Nope. The Google Videos provide the necessary HTML snippet to share the video. And we even tested that that HTML worked by embedding it within a Blogger blog and a Typepad blog; both worked fine. But after several days of hacking, we can't get Wordpress to embed the video.  Of course this leaves as a loose end the option of self-hosted Wordpress combined with the plugin.  I may revisit that, but I don't hold out much hope for getting that to work easily when we've not managed to get Google Video embedding to work. I've no doubt we'd get there eventually, but getting a video blog is not the only thing we have to do with our time, and there comes a point when it makes sense to take the loss and move on. So, another fail.

MovableType hosted on

Analogous to Wordpress, MovableType is blogging software; and analogous to is, a hosting service. And in some ways this is the most promising of all. MT on handles embedding of the Google Videos fine. Authentication to *watch* those videos is achieved via our already-in-use Google Apps authentication. So all we need is something to restrict access to the blog itself (otherwise while unwelcome visitors may not be able to see the videos, they would be able to see everything else). And in fact, has precisely that -- you can simply password a blog. Yay! So it's passing criterion 3. The problem is criterion 4. Because the protection mechanism is a simple password *for the blog* (i.e. it is not authenticating on a per user basis), if someone leaves the company, I'd have to change the password for everyone. And in fact it would probably be wise to be changing the password every few months anyway, even if no one left, just to cover password "leakage" into the wider world. Fail! :-(

OK, but then I had a bit of a Homer Simpson "Doh!" moment. I've noted that we already have the video hosting part of the solution -- Google Video, part of our overall Google Apps setup. But in fact, we now also have that for the blogging engine too; Blogger, now owned by Google, is now (and has been for some time) available within Google Apps too. So:

Blogger under Google Apps

Well at first glance this looks perfect. Authentication is going to be the same as for the rest of Google Apps. Hosting of the video is also Google Apps. And embedding works (I tried it). I think we have a winner. Yeah, if only. To understand the problem it's necessary to understand that while Blogger is owned by Google and the service is now included within Google Apps, it is not -- unlike Gmail, Calendar, Video, and so on -- a core Google Apps service. In theory, all that means is you don't get support for it from Google. But in practice the fact that Blogger is somehow less part of the Google Apps family has made itself known via a nasty little hole in Blogger authentication. Basically, what happened was this. After checking that the core functionality -- blogging, video embedding -- worked, and that the security appeared to do what I expected, I then enabled all my team to read the blog. That's done within the Blogger dashboard and the result is that each user receives an email containing a link that they need to click to accept the invitation. Once that's done, they can then read my blog, and watch the embedded videos, providing their browser is authenticated with their Google Apps credentials.

The problem appeared when the Blogger invites were accepted by a few of my users who also have personal gmail accounts. It turns out that if someone is reading the invitation email and accepts the invitation while in a browser that is also authenticated for (i.e. logged in to) their personal google account, then although the invitation may have been sent to <persons-name-at-verilab>, it gets accepted by <persons-name-at-gmail> Worse, Blogger takes that acceptance as valid! In fact, if I, the inviter, look back at the list of people to whom I've given blog-reading permissions, the entry corresponding to that person has changed -- from <persons-name-at-verilab> to <persons-name-at-gmail> Failalamadingdong!

The only point in Blogger's favour here is that at least I -- the Blog owner -- get an email with subject "Your invitation was accepted using a different email address" and highlighting who the user in question is. But at that point, all I can do is remove their personal gmail from the list, re-insert them under their company address, and try again. Unfortunately if they again accept under a multiply-authenticated browser, the same thing will happen again.

Now, one of my users did argue that maybe Google were doing The Right Thing here. I invited Fred, and Google is permitting Fred access. OK I invited and they're permitting, but they know (not sure how, but they do) that those are one and the same Fred, so they're not being unreasonable. I have two problems with that.

First, from my point of view, and should be regarded as two different people from a security point of view. The reason is that it's conceivable that a person would apply different levels of "security in use" to their personal email versus their work email. For example, someone may share their personal email credentials with a family member, but not do the same with their work email. 

Second, even if I did allow for Blogger seeing two Freds as being the same, Google Video does not see that. So while either or would be able to read the blog, only would be able to see the embedded videos.


Well, the conclusion is, it can't be done. Of course, clearly it can -- all of the bits and pieces are out there waiting to be bolted together. But the same could be said about plain old text-only blogging in the days before blogging machinery. But blogging didn't take off until someone bolted together those blogging machines. So today, for your average small company, even your very technically able small company, getting a company-internal-use-only video blogging setup working may be more bother than it's worth.

But make no mistake, it's worth something. If I'm the only person who'd use this kind of thing, then I fully expect to be ignored and to see nothing change. But if there are enough of me out there (and YouTube, Google Video itself, and even wee curiosities like ConnectNote, suggest that there's all kind of demand for video comms bubbling under the surface), then it looks like an opportunity for some college-dormed entrepreneur or other. When you've got it up and running, I'll be your first user.

Thursday, March 29, 2012

On mats, and cats, and programming

You are a teacher of C++ or any of a number of similar languages. You are marking an exam. The first question is, "Write code to add 1 to variable x."

Candidate A's answer consisted primarily of the following:
x = x + 1;
Candidate B had, by contrast:
I'm guessing, but I reckon the majority of examiners would find no fault with Candidate A's answer. They'd do that because by most standards, there is nothing wrong with Candidate A's answer. It means exactly what it was supposed to mean. It says what it was supposed to say.

Now; you are a teacher of English As A Foreign Language[1]. You are marking an exam. The first question showed a picture and the following jumble of words: "mat", "the", "sat", "cat", "on", "the".

The candidate had to "Rearrange the word jumble to match the picture."

Candidate A wrote the following:
On the mat the cat sat
Candidate B had:
The cat sat on the mat
I'm guessing, but I reckon the majority of examiners would find fault with Candidate A's answer. They'd do that because by most standards, there is clearly something wrong with Candidate A's answer. Although it means exactly what it was supposed to mean, and says what it was supposed to say, it's wrong. It's not how English is spoken by English speakers.

The difference is simple. In natural languages, idiomatic correctness is seen as being part and parcel of overall correctness and we don't stand for it when it is missing. By contrast, we seem to tolerate its absence in programming languages?


[1] I chose EFL instead of just English for my example, because lets face it, the more enlightened examiner may give more marks to "On the mat the cat sat" to reward its more poetic quality, a quality lacking in the idiomatically correct but more mundane, "The cat sat on the mat". But EFL is, sadly perhaps, more about simply getting on in English speaking environments than about writing poetry.

Tuesday, March 13, 2012

(Trying to do) Business in Canada

Over six months since my last post and what is it that gives me the kick in the pants needed to blog again? Not some advance in the field of chip verification; not an epiphany on how to spot future world class programmers from the way they play Plants versus Zombies. No, nothing so valuable. What's got me hitting the keys again is what always gets me hitting keys hard: moronic government and how it gets in the way of Just Doing Business. I annoyed some people a few years back when I lobbed a few barbs at Germany for its anti-business funkiness. Well have a break Germany, cuz it's Canada's turn now.

Enter Regulation 105. It goes something like this.

If a non-Canadian firm goes into Canada to deliver a service to a Canadian client, then 15% of the fee may have to be paid to the Canada Revenue Agency to cover possible tax. Now it's important to note that the 15% is merely provision for *possible* tax. It is entirely possible -- likely even -- that the US firm will be paying the relevant taxes back in the US, and if it's affairs are in order (specifically, if it is not deemed to have a "Permanent Establishment" in Canada) it simply will not owe tax to Canada. In that case, at the end of the year, the US firm will account for itself and will receive a refund of all the withheld money.

But it's a pain in the arse. Here is why.

First, it hurts cash flow. You have to pay money up front where normally you'd pay it later. And you have to pay it from revenue, not from profit (that can be ameliorated, but not simply).

Second, it's just another taxation straw threatening to break the back of the entrepreneurial camel. On its own it's annoying enough, but it's yet another one of a myriad of pieces of crap that distract companies from doing what they do -- serving their customers (and, as a result, making the money in the first place that can then *be* taxed). And Canada is far from alone on this front. The new 1099 provisions, anyone?

Third, Canadian Reg 105 is poorly understood, even by so called experts. So even the honest and diligent firm has to work hard to obey it. In investigating how to handle this, I have had three opinions, two contradicting each other, and the third not agreeing entirely with the others. And the two contradictory views came from the same office of the same Big Four accounting firm. So even having found The Truth, a firm always has the nagging doubt that it may, despite its best efforts, have made a mistake and as a result still be carrying a liability.

Fourth, it's not just inconvenient and time consuming, it's expensive. Big Four accountants aren't cheap, and they charge you even when they're learning stuff.

But finally, the best bit. Consider the following scenario (very simplified to keep us all awake):
  1. A particular firm, for various reason,  is not liable for tax, but is liable to pay the withholding (and then reclaim it)
  2. They don't realize this and fail to pay the withholding (of course neither do they reclaim anything)
  3. Five years later they are audited, and the CRA spots the error, demanding the withhold. Notice: no tax was ever due, but the CRA's rules are rules. Even though there's going to be a reclaim, the payment must first be made
  4. So the firm pays the withhold
  5. The firm duly applies for the refund. Remember, they were not liable for tax
  6. The CRA refuses the refund. The reason is that there is a statutory limit on the length of time in which certain reclaims may be applied for. And in this case, the limit is *three* years.
The end result is that the CRA gets to keep a portion of tax which they were not due, and that's because they allow themselves more than three years to get it but allow the victim only three years in which to reclaim it.

Now in fact the above scenario may not be correct. Lost in the simplification is the fact that the firm liable for the withhold in the first place may not be liable at the time of audit. But to figure out whether that "may not" is a definite "is not", my Big Four accounting firm needs to charge me around $5K so they can go figure it out. 


Wednesday, June 8, 2011

Clouds of EDA

Todays DAC panel session on EDA and Cloud Computing was interesting and the ensuing discussion was fairly lively (I'm speaking relatively of course. it's a conference of chip design geeks; how lively do you think it could ever get?) But while cries of "The Cloud, The Cloud, My Kingdom for The Cloud" are all very well, I'm only going to get excited when I see some important enabling technology show its face first. Here are four:
  1. Mobile apps. This is really what Cloud computing is all about. Sure you get to stuff your stuff up on Amazon, somewhere in, well, in the Cloud. But it's how you interact with it when it's up there that's important. You have a LinkedIn app on your iPhone? Or a DropBox one on your iPad? Well, you'll want a Regression Dashboard one too. 
  2. Data Visualization Technology. It's already bad enough trying to wade through a jillion lines of plaintext logfile coming off one or two or five simulations. Imagine what it's going to look like when you have five thousand simulations running in parallel, the log outputs combining into a flood. I'm not just looking for a glorified grep I can run during a boring DAC talk (the Cloud one wasn't boring). We need new ways to look at and in other ways analyze simulation data. We don't, for the most part, know how to do that yet. People like Edward Tufte might. We should ask him. (Here at Verilab we already sent one of the team to do exactly that.)
  3. Virtualization. Partly because the chip design environment is so heterogeneous (lots of different things as opposed to lots of things-the-same), partly because the EDA tools are not always as well-behaved citizens of the GNU/Linux environments in which the live as we'd like, and partly because it's not easy making a large number of GNU/Linux boxes behave in a sane and stable way anyway, the last thing we want to do is in anything like a manual fashion try to configure five thousand in-the-cloud servers so they are ready to run big chip sims. One way to solve this is to rely on virtual machines which are then cloned and farmed out to the farm.
  4. Much *much* more robust GNU/Linux configuration. Even with the help of virtualization, that merely helps the scaling, You need a stable system in the first place. Unfortunately GNU/Linux sysadmin is often treated as little more than a glorified (if that) backup-tape-changing, email-fixing job. Its not. If we compare looking after GNU/Linux systems in a chip engineering environment to looking after financial systems, then too often GNU/Linux admin is barely at the level of book-keeping. What it should be compared to is the role of the sophisticated M&A specialist, or investment quant. The GNU/Linux infrastructure needed to design and verify a modern SoC is a significant piece of engineering in its own right, and it needs a significant piece of engineering brain power to run it. Every chip team worth its salt needs such a large-brained sysadmin. Verilab has one. If you don't, go get one. If you don't know what one looks like, ask me and I'll tell you. But a clue: if your guys aren't running Puppet, or something similar, ask why.

Of course there's more to it than even that. We'll want FaceBook "Like" buttons next to simulation output displays, and the ability to retweet a successful regression will be cool too. All of this is why, unlike what one questioner suggested at this evening's panel, "The Cloud" is much more than the same simple build-versus-buy decision we've contemplated for years. No; done right, Cloud-based EDA -- Cloud-based anything -- will be a game changer. Why else would Big Brother Steve be so excited about it, and Brother Richard so not?

Saturday, June 4, 2011

Programming as Soulcraft

I'm always looking for "proxies for greatness" in potential new members of the Verilab team. It can take some time to find out if someone is good, because in the end "good" in this context means something like "consistently delivering desired results", and you can only see that over time. But there are clues early on that a person may be, or may become, good. Those are the P'sforG.

One is mentioned by philosopher-turned-mechanic, Matthew Crawford in his "Shop Class as Soulcraft". In chapter eight, "The Further Education of a Gearhead: From Amateur to Professional" he tells the story "Of Madness, a Magna, and Metaphysics" in which he takes on the task of bringing back to life an old and neglected-by-underuse 1983 Honda Magna V45. A key part of the repair was fixing the clutch hydraulics.

As he burrows in through the layers of grease and grime, he encounters a suspicious oil seal that he suspects is the culprit. But he can't be sure without a lot of extra work. This triggers a debate within himself as to the sense of digging into that oil seal when he knows he can perform a reasonable if temporary fix by simply focusing on the slave cylinder:
"It occurred to me that the best decision would be to forget I'd ever seen the ambiguously buggered oil seal. With a freshly rebuilt slave cylinder, the clutch worked fine. Even if my idle speculation about the weeping oil seal causing the failure of the slave cylinder was right, so what? It would take quite a while for the problem to reappear, and who knows if this guy would still own the bike by then. If it is not likely to be his problem, I shouldn't make it my problem."
Note that he's not idly trying to avoid work. His concern is for the client (who'll have to pay for the extra work), and for sheer economic sense in to the bargain. But there's more at work here than simple economics:
"But as I walked back into the fluorescent brightness of the shop, I wasn't thinking about the owner, only about the bike. I just couldn't let that oil seal go. The compulsion was setting in, and I did little to resist it."
It's that word "compulsion" that intrigues me. Only sentences later he mentions it again:
"There is something perverse at work here, and I would like to understand it. The oil seal was the opening to Pandora's box: I felt compelled to get to the bottom of things, to gape them open and clean them out. "
 Not money, not because the boss or client wants it. A compulsion, and thinking only about the bike. I agree with him; it's perverse, and I too would like to understand it. And his inner conflict, and outright guilt at the perversity of it is clear:
"But this lust for thoroughness is at odds with the world of human concerns in which the bike is situated, where all that matters is that the bike works... [The] more ... pragmatic view of the motorcycle ... grounds the fiduciary responsibility of the mechanic to the owner. In digging at that oil seal needlessly, I was acting out of some need of my own."
But if he didn't intend to be ironic there, he should have. And that's my point. It is the very compulsion he demonstrates that I believe is a Proxy For Greatness. I see it (and its sad opposite, a robotic lack of compulsion) all around in software.

The Proxy -- the sign you may be standing in the presence of programming greatness, or the potential for it -- is the tendency in some individuals to write clean, well-nigh poetic code not because it's useful (although it usually is) or reusable (ditto) but because they cannot *not* write such code. They are compelled to do so. They avoid, where possible, writing the same lines of code in more than one place not because a coding standard tells them not to, but because they are compelled to be succinct. Saying the same thing twice is icky -- it *feels* bad. Or they look on existing code, and cannot help but notice that those four "different" functions are really the same function. The itch to refactor develops, and may well become irresistible.

The correlation is not 100%. There are potentially fine programmers out there who don't have this instinct but can learn it. And I've met a few who go too far to the extreme, sacrificing all real concerns for an elusive platonic form of every program. But I reckon it's a very strong predictor. And I'd recommend erring on the side of too much of it than too little.

Verilab is hiring that kind of people in the field of chip verification. If you're looking for a place where Programming as Soulcraft is appreciated, where the beauty of a solution is part of its merit, and where you would be part of an elite team who understand the compulsion to Just Do It Right, give us a call.

Friday, May 27, 2011

Egnyte - update

So, a local person (well, US) called me. Turns out the latest local cloud client is busting something on Mac SnowLeopard. They gave me an older version and I'm now in action. Now I can start comparing.