Mark Carroll's Journal

I realized a little of why I do not make much progress on the projects I want to get on with on computers at home. I am comparing with my productivity at work but there I get to focus for a few hours at a time on a task. The kinds of projects I want to pursue at home take enough thought, and use rusty enough skills, that it takes me an hour's work just to properly get back into them and gain some momentum. By that time it is pretty much time for me to wrap up and come do something with my family, with whom I do want to get to spend time.

This is why I still manage to do jobs like posting entries to this journal: it is the kind of thing I can switch right into with little preparation and handle in a shorter span of time. Yesterday mst3kmoxie kindly handled the children's afternoon visit to Dundee which gave me time at home that mostly went into upgrading our OpenBSD machines to v6.1, a slightly fiddly process. I manage to keep on top of these maintenance tasks, I just don't get to make the additional time around job and family and sleep to really push forward on anything. I do need to do something because my present job is not secure and by this point is not really advancing my own career, though I am likely to soon be learning about Vert.x and at least my paid work helps scientists.

On reflection, there are a couple of solutions. I do not make the progress even with smaller-scale easier-to-start tasks that I should, even if I enjoy them, so there is something to address there; maybe I could better notice what it is that I am doing instead though that may often simply be feeling tired. I also waste time in being available to my family at times when none of them feel interactive: perhaps I need to actually plan family time a bit better and remind of my interruptibility at other times. Further, I might need to more firmly set aside a long solid chunk of some weekend day toward useful ends so that the getting-started time does not dominate.

When I was starting out in computing ( hardware and software was often a joy to behold. )

Coincident with the rise of Microsoft Windows I have seen software quality go down. Now it is entirely normal to run into bugs. It is also normal for software to frustrate me: it insists on doing something I don't want or I can't get it to do something it clearly ought to be able to. ( Mostly things went downhill … )

It isn't all bad. ( … there are some pleasing exceptions. )

It feels like most software developers now care about little more than getting things working well enough so that their employers can sell them comfortably. The developers wanting to use their mathematical gifts to create systems that are both solid and flexible are largely dwarfed or otherwise out-competed by the many who just want to deliver something that appears to work sufficiently well that everybody gets paid and management either do not appreciate or do not care about the difference.

Maybe the change is partly in the user community: ( they increasingly find poor quality acceptable. )

I don't have any answers, I think it's just how the world works. I would look at retreating to high-assurance systems but look at the Department of Defense's move away from Ada or Ericsson's from Erlang: I think the only refuges are in the past. I could be frustrated to know that modern computer systems are typically no longer built anywhere near as well as they could be but I instead find myself grateful to be able to recall a time when it was normal for them to be both useful and reliable.

I have been quieter lately through feeling tired and irritable. I am usually at my desk by 8h or so and, in fasting, I don't take time for lunch, but I don't want to leave too weirdly early so I usually stay until 16h30 or so. Sometimes I stay later, for example on Tuesday I stayed beyond 17h because, as I was wrapping up, a series of three people came to ask me things. This is okay, I like to be helpful, but it did mean a longer workday.

Unfortunately, I awoke rather prematurely on Wednesday. I do not make much use at home of unexpected morning time because I am distracted by knowing that I have to leave the house so I just head to work unusually early. Over the afternoon I had a migraine, though not quite bad enough to prevent me from working, and on the way home I had a detour via the doctor's office which added maybe a half-hour to my trip, so that became a long day indeed. Yesterday I again awoke well before the alarm and started my workday correspondingly early.

( An early workday makes sense for me. )

I have wondered if, though I do not eat at lunchtime, I could instead use the time well for some other personal task. I am not sure what that might be, though. Especially, I like to keep my own work separate from my paid work, on different computers, but I do not want to risk routinely bringing my own laptop into work. I do at least sometimes take a brief walk, as I did today to the mailbox.

( Recently my work has been difficult. )

I have also been unusually busy at home. For example, last Saturday one of my children needed to be in Dundee all day and tomorrow they both need to be. So, that is basically a day of the weekend rather perturbed. I thus wonder if everything has added up to push me a little over a tipping point lately.

Feeling myself to be much inclined to utter screw this and let chips fall, I skipped exercising for a couple of days and have instead treated myself a little: for instance, after exercising yesterday I took a bath instead of a quick shower. Today I felt somewhat better. Not only did I sleep for longer but I also felt a little more enthusiastic. I did well in my exercising. Perhaps I am returning to a more tolerable state of mind but I will try not to push it too far.

An online conversation earlier today reminded me of some work I had done ten to twenty years ago back when I was mostly working on government-funded research and development projects. It is now long enough ago that sometimes I forget how very much I enjoyed the challenge of solving various problems in science and engineering, inference and decision-making by devising software-based approaches. I turned out to be very good at it but haven't had to much do it for what must now be a decade or so. It is hard to get good applied research work without a doctorate and while needing enough income to support a family especially as my better work was also defense work that I cannot talk about especially outside the US. So, over the past decade I have done more regular software development: I am good at it but not as great as I am when faced with more need for creative solutions to technical problems.

Having been decidedly obese when I moved to Scotland I find it strange now to be in a position when on many nights I am positively trying to eat a bit more. I suppose that my diet may have had lasting effects on my appetite. For the past couple of days I could have easily eaten comfortably less than what I allowed myself when trying to lose weight. Last night after a perfectly satisfactory dinner I later decided to also have a heavily buttered piece of toast with a generous amount of Gouda, then a few mini creme eggs left over from Easter, but it is not like I was ravenous at the time, the challenge was more to think of something that seemed appealing. On the one hand, I do want to listen to my body about when it feels hungry or not, but I have still been losing a little weight on my maintenance plan so I do not want to be drifting far below even that.

I got to thinking about why, when I have been overweight for so very long, it is only in the past couple of years that I seriously addressed the issue. What switch flipped in my head? In watching a documentary about schizophrenia I came to wonder if it was about control: my best talents are not greatly exercised in my current work, I barely earn enough to get by, I live in a small, cluttered house, maybe my effort toward improving my weight and fitness stems from wanting to successfully grasp at least some facet of my existence and addressing my health was the most plausible remaining option? I do not know. Now that I have generated the hypothesis it doesn't feel as if it obviously rings true but it is also not assaulted by competition with rival explanations. (One of my military projects involved creating software for generating, evaluating, assembling hypotheses.) I thus do now wonder if my diet and exercise might indeed partly be about gaining some control and feeling good about something.

The mention of schizophrenia reminds me of a lady who used to roam the streets around our apartment in Providence, RI. She was often off her medication and would indignantly berate pedestrians and drivers for all manner of creatively imagined slights, sins like trying to sell her into prostitution to workmen down the road. While she was not causing anybody any real harm then it was her legal right to be in such a state. When I would first spy her ranting from the window I initially thought that she was speaking on her cellphone via a Bluetooth headset but her interactions subsequently made the unfortunate situation quite clear.

I was chatting recently with somebody who shares my observations on the apparent decline of the quality of software and figured that I may as well capture the broad strokes of my thoughts here.

Certainly back in the 1980s, maybe also the 1990s, software often impressed me. I could easily figure out how to use it, how to get it to do what I wanted, and it was genuinely a surprise to discover a bug. These days I often find software to be difficult to get on with, trying to force me into specific workflows of what somebody thinks users ought to want to be able to do, and positively bug-ridden.

One thing I am curious about is to what extent this trend is quantitatively corroborated. Organizations like NASA put research money into assessing software quality. I wonder what they have found.

Another is, what's the cause? Is it that there are so many people now entering the software industry that their typical ability level is much lower? Is it the false economy of building one's software atop a whole stack of complex third-party libraries that are all too often both buggy and barely maintained? I suspect that those two are connected: that it may take a certain skill level to recognize a steaming pile of festering liability just as it takes a decent programmer to be able to recognize one in recruitment interviews. Or maybe they have to use the third-party cruft because they don't see how they can easily just build the part they need themselves and wouldn't recognize clean elegance even if it lived next door to them.

Or maybe it's a social change in what's acceptable: professional pride giving way to commercial expediency, a market failure of rewarding feature lists over reliability, etc. Perhaps I am a poor fit for the modern world in caring more about software being clear and correct rather than featureful.

I see so much ( poor quality software )

I am not against learning new things, I just find few of them worthwhile. ( some of them are, though )

For around a decade I worked for Aetion. ( I helped to keep the business alive via military research and development projects )

And, eventually, the music stopped. Fortunately, I had seen that coming and arranged to live near Boston before I was made redundant; Aetion won no further work.

There were plenty of computing jobs around the Boston area, especially in Cambridge which for me was a short bicycle ride away. However, in my mid-thirties I found that my age was already counting significantly against me: in describing me employers put scare quotes around experienced and feared that I would not easily take instruction from those who, in experience, were considerably my junior. I also found military research projects closed to me through my not having, nor wanting, higher security clearance. I did manage to find work but more suiting a new graduate than a creative well-rounded professional with a good track record.

Now I find myself in Scotland. ( in a nice job )

However, from a personal point of view, the situation is uncomfortably precarious. My team is funded by a series of fixed-term grant proposals. ( I want to keep my children in their local school but there are few local jobs )

My contract is not permanent. I pray that our group leader continues to be so good at the hard work of winning funding but I wish it were more secure: I dare not consider buying a house here in case the music stops again. If I do have to look for work then I may well be in my mid-forties and living near far fewer employers with a recent work history rather less impressive than the previous.

These issues were weighing upon me last winter too; gerald_duck pointed out that telecommuting might be viable and indeed I am more productive in working from home than I am from an open-plan office; also, that currency of skills may matter more than age. The thoughts about the above are back in my head though so maybe writing them out in this new entry will palliate them while I focus on other issues.

Benjamin likes the collectible card game Yu-Gi-Oh! and indeed there are now many more such cards than people in the world. I am okay with games like Magic: The Gathering and I usually do well with complex systems of rules but Yu-Gi-Oh! overwhelms me with tedious detail: there are so many cards each with their own special rules and interactions with other cards that the optimization problem of deck-building seems thorny indeed.

I have wondered if this suggests any future career for Benjamin. The larger challenge is probably that of finding something that interests him sufficiently but I do think that it may be important to note if it is the kind of thing where Benjamin's facility with sizable rule systems affords him unusual advantage.

Apart from my GCSE English oral in which I spoke for five minutes on UFOs, without slides, I didn't have to give a presentation until it was actually warranted in my full-time work in Ohio after graduation. Indeed, it was less than a year ago that I finally had to use PowerPoint: previously I had used LaTeX's Beamer class. (Currently for things like workshops for users I use reveal.js.) In applying for jobs in Boston I didn't have to give any presentations: I largely took written and oral tests and people bought me lunches. Benjamin made it partway through middle school there without having had to give a presentation.

However, presentations seem ubiquitous in Britain. I had to give one as part of the interview process for most jobs I applied for; for my present job, they forgot to tell me to, and I hadn't even brought a laptop computer; they mentioned the night before that they expected a presentation in the morning, but that seemed to go okay: they got an extemporaneous talk on the selection of spare parts to include when the US Air Force deploys a set of aircraft to a different base. Another British employer got one on multi-sensor fusion for reconstructing situations in 3D: fortunately I had written code to output the actual data structures as PostScript so the sensor views and projections of the target situation were easily translated into PDF diagrams for the slides. I think it's mostly just about that one can speak intelligently and coherently.

It's not just British employers: recently Miranda has been working on yet another set of slides, even in primary school, this time on her achievements outside school. Her previous one was on salmon's contribution to Scotland's economy. Since moving to Scotland both our children have had to prepare many PowerPoint presentations. I suppose it goes hand-in-hand with employers apparently now expecting such. Why and when this love of PowerPoint occurred here I've little idea but it has been an unexpected facet of our return to this country.

For some years I led technical recruitment at a small company, basically hiring sysadmins and programmers. While we were quite small and I am not in favor of quota systems or suchlike I hoped that we might nonetheless end up with a diverse group of employees. I took some action toward this: for instance, I would mail posters advertising vacancies out to historically black colleges and universities. To my disappointment there was a significant shortage of credible minority applicants getting even as far as initial interview. It's not a large sample size but I have to guess that whatever principally causes inequality in hiring might thus start at a rather earlier stage.

In the end every one of my hires was a young white male. The most-nearly-exceptions I can think of include,

• There was one middle-aged guy who unusually was largely hired on a probationary basis outside my process; he lasted a few weeks before we all agreed that he just didn't have what we needed.


• There was an older man, actually now deceased, to whom I took a liking, so I held on to his resumé, but at the time there was a clearly better candidate for the vacancy.


• There was a young Chinese chap where I may be guilty of some cultural insensitivity here: he got as far as in-person interview but I just couldn't get him to do anything other than agree with us, and I was looking for people who think for themselves.


• There was one young white guy who has attention-deficit disorder whom I did hire. It seemed to work well enough to be sure to summarize any non-trivial instructions in followup e-mail.

So, I suppose, a couple of older white guys got close, and one young white guy with a kind of disability did get in but, for instance, no black female applicants were at all qualified, and goodness knows Columbus, OH, is a fairly diverse city, indeed I was friends with a black female student in the computer science program at the university.

I don't know what to make of that. I wonder if I was just unlucky in what fell across my desk. (Not that unlucky, a couple of our white guys were great hires.) Maybe there are more good minority candidates out there but (admittedly without any official mandate) I somehow failed to reach them.