Breakfast, Dinner, Tea – Page 2 – Where Simon Scarfe writes stuff

Exporting my Outlook daily agenda to Roam using Hammerspoon

Post author By Simon Scarfe
Post date December 22, 2020

Recently, I posted on here about using Hammerspoon to scratch a personal itch. It was a nice exercise in documenting stuff for myself while hopefully putting something out into the ether that might be of use to others. So I thought I’d try it again.

The problem

During the summer, I started using Roam Research for all of my note-taking and personal knowledge management. You don’t need me to tell you why you should or shouldn’t be using it, there are enough of us out there already proselytising about this thing. But the long and the short of it is that absolutely everything I come across and think about is going in there – personal stuff, work stuff, little snippets of data that might one day be useful to me again. It has become a vital tool in my day-to-day thinking and planning.

One of the templates I’ve been using to take personal notes for meetings looks like this:

- [[Meeting/MEETING TITLE]]
  - Category:: #meeting #OTHER #KEYWORDS
  - Attendees
    - [[Simon Scarfe]]
    - [[Unfortunate saps talking to me...]]
  - Agenda
  - Notes

I find this fantastically useful, the “Attendees” section is particularly good for some of the natural cross-referencing it provides. But when you have a day of six or more meetings, filling in those can be tiresome, to the point where I actively avoid doing it (even with TextExpander shortcuts). When this happens, I both lose the value it provides, and I feel like a bit of a failure – which is never nice.

The solution

I decided to see if I could script exporting those meetings to markdown – trying to turn what is a bit of a laboured, several-times-a-day process into a simpler, single keyboard shortcut. And without wanting to spoil too much, it turns out I could. Not only that, but it was pretty straightforward.

I started out by using MacOS’s Script Editor to hack around the Outlook API so I could see what I might query on. When researching this, I found several pages about getting information using the frankly-quite-weird AppleScript language, but very few using Apple’s JavaScript for Automation extensions (JXA). After about 20 minutes of screwing around, most of what I required could be retrieved using Application("Microsoft Outlook").selectedObjects() (seriously). The trick to enable this is to activate Outlook Calendar’s “list view” (CMD+CTRL+0), so you can select several meetings without triggering extra UI elements.

Once I’d accessed those events, I turned them into plain ol’ JSON with the intention of letting Hammerspoon deal with them – before realising that Lua is not a strength of mine – so I continued using JavaScript to transform those objects into Roam-compatible markdown. You might wonder how bad Lua not being “a strength of mine” could possibly be by looking at the digital “chicken scrawl” that is the chained JavaScript I assembled to do that here.

Finally, it was a matter of hooking it all into Hammerspoon. Hammerspoon provides hs.osascript.javascript to execute JavaScript, hs.pasteboard.setContents to copy the results of that JS to the clipboard, and hs.hotkey.bind to hook it all up to a keyboard shortcut. I’ve probably overused the sentence a bit in this post, but it was all rather straightforward. Honestly, look.

Limitations

This isn’t without some slight flaws, the biggest one is Roam’s (or maybe it’s MacOS’s?) treatments of soft line breaks. I couldn’t find a way in Roam to paste in soft breaks, which leads to agendas pasting in as multiple blocks. That may be less problematic for more traditional outliners, but a lot of Roam’s power comes from blocks being first-class citizens that stand on their own. To get around this, in the short term I replaced them with `------` text, which I can quickly edit around if/when I find it appropriate. Fortunately, most agendas are a bit rubbish, and I only need a couple of lines of a LOT of (usually Zoom-related) bumph.

Also, none of this has been tested on “New” Outlook 365. I am a bit worried that if I upgrade (forced or otherwise), this functionality will quickly disappear. Maybe then I’ll finally start using a more portable/flexible email/calendar client?

And on a similar theme, as I have been reading around Applescript / JXA, those technologies might not be in Apple’s long term plans. That the information on how to do all of this was hard to find, could probably be taken as a symptom of this. However, if you are interested in digging more into JXA, the JXA Cookbook is a decent starting point

But all of that aside, once I’d got my head around the JS/Applescript nastiness and weird Osascript APIs, it was a pretty simple experience. Even if it all went away tomorrow, I’ve likely already saved some serious time manually adding people’s names to [[Meeting/]] posts.

Tags dotfiles, hammerspoon, lua, outlook, roam, workflow

Links Technical

The Lean Web

Post author By Simon Scarfe
Post date November 27, 2020

Chris Ferdinandi has thrown in some absolute gems in this talk he put together for wordpress.tv – The Lean Web, well worth 48 minutes (or less at > 1x) of your time.

Link – The Lean Web

Tags javascript, react, web performance, webdev

Links

Post author By Simon Scarfe
Post date November 26, 2020

Lovely piece from Terence Eden on squeezing more out of tech: What would happen if computers never got any faster?

Links Technical

Post author By Simon Scarfe
Post date November 24, 2020

TIL: Use a Submit Button Outside of a Form!

Tags TIL, webdev

Personal Social Stream Technical

Review/Recap – “Accelerate: The Science of Lean Software and DevOps: Building and Scaling High Performing Technology Organizations”

Post author By Simon Scarfe
Post date September 6, 2020

I originally gave this book 3/5 on Goodreads, however, now I’ve revisited the book and made some personal summarisations from my highlights – I feel I need to revisit that.

My original just-finished-it instinct was that the research part of the book was too deep and took me into methodologies that I ultimately don’t care about. In fact, I nearly dropped out during that section. Having battled through, I realise that would have been a mistake – after that, there is a great leadership case study that really hits home some of the messages around the culture the book promotes needing to be developed and learned, rather than being the result of blindly applying archetypes.

Still, the research methodology part is a third of the book – at least structurally. I would advise any potential readers that that is entirely skippable if you’re prepared to trust the source and take the advice it’s pushing at face value. After all, the advice is backed with logic and reasoning – which for me, is more persuasive than “we spotted correlation in these heuristics for high performing organisations”, which isn’t as compelling for me as it may be for others.

The first part of the book is full of absolute gold, end to end. Mostly backing up a lot of the literature around the benefits of things like Continuous Delivery, Lean Product Management and working in agile teams. But it presents it in a nicely joined-up way, built around the hypothesis that high performing teams deliver quickly and build stable systems – offering four simple heuristics that theoretically cement those two features.

It digs a lot into building organisational cultures – citing some of the good research into what makes for a performing team at Google (Westrum generative cultures, learning organisations, etc).

It examines technical practices that contribute to the above – why and how continuous delivery works, the benefits of automation, versioning everything, test strategies, embedded disciplines (devops, devsecops, the job of testers in a highly automated world etc).

There’s large importance put on security, reserving a full section to it – where the conclusion (naturally, given some of the authors’ previous DevOps literature) is that you should build it into your process as early as possible.

I found the culture and leadership sections to be particularly good – backing up a lot of my own personal thoughts and biases. Lots about organisations forming their own paths and not just mimicking their way to culture change – which feels obvious, but probably isn’t given how much Cargo Cultism there is in tech.

I think that all in, there aren’t any new conclusions drawn in this book, but it is great to have them in one place. The book sees its own USP as the exhaustive research, but as I say, I found it exhausting, and not to be its real strength. The density and jumping off points & onward knowledge journeys are what really made it a great read for me.

Tags book, continuous delivery, devops, management, Read, review

Technical

HOW many tabs?

Post author By Simon Scarfe
Post date August 28, 2020

Update – 12/11/2021: A while back I switched to a Chromium-based browser, I’ve documented the new approach here.

My Problem

Everyone I work with at some point mentions my browser tab mismanagement. As of right now, I have 135 tabs open, but that’s definitely at the lower bounds of where I am typically at.

For years I suffered under of Chrome’s tab Toblerone of death (“Tablerone”?), which constantly reminded me of my dysfunction with only an icon to identify which tab was which.

Recently, I started using Firefox again and it enabled my problem by presenting my open tabs in a carousel UI. So it wasn’t until my Mac told me I had no memory left that I realised how bad it had got.

The carousel obfuscates the exact number of tabs I have open – leaving the quickest (and only obvious) way to find out in Firefox through hitting CMD+Q – which triggers the above dialogue.

It may sound like this is kind of a weird brag, but it’s not. This problem isn’t one I’m delighted with having. My open tabs present a combination of lackadaisicalness (in that sometimes I just plain forget to quit out of them) and flawed aspiration (“I’ll get to reading that imminently”). By the time I get to declaring tab-bankruptcy and bookmarking them (using a combination of the OneTab extension and Pinboard ), they’ve amassed to a point where I sometimes no longer know why they’re there.

The Tool

Yes, I obviously have a problem. And once I’d admitted it, I started looking into ways to manage it. My chosen approach would be to better surface the above number, so at least it would become more quickly apparent that I’d gone too far.

I don’t want to move away from Firefox, and I don’t want to have to threaten to quit anytime that I want an update on my tab count. Nothing obvious was coming up in my searches until a little later I was pursuing another itch (hyper customisable window management, if that matters) where I stumbled upon some articles about a tool called Mjolnir, which had been forked to a more batteries-included tool called Hammerspoon.

The Hammerspoon splash page does a better job of describing it, but the tl;dr is it adds a scripting layer to MacOS using Lua. This seemed powerful enough for me to craft something that might suit my needs here.

Crafting A Solution

So with my hammer in hand, I needed to craft a way to address my “issue”. I’m a big fan of “Quantified Self“, and Continuous Improvement – both philosophies that involve surfacing and using data about yourself to improve a situation. So I figured – if that number is always in my face, shouldn’t I naturally try to optimise it now and again?

With that question in mind, I started looking at the Hammerspoon API, and initially, I figured this would be easy. Hammerspoon surfaces a method on a Window called tabCount, so this should be incredibly easy to integrate. Unfortunately not – the data it returns in Firefox isn’t representative of how many tabs you have open. At a guess, I expect Firefox isn’t using a native macOS tab implementation, and so this was an immediate dead end.

Then I started looking at methods in Firefox for surfacing this number. I’m sure I could automate a brute force counter somehow, but I don’t know how I might implement that in the background without interrupting the user. Firefox tracks “telemetry” for reporting browser usage. I found that stuff dead interesting, I’d encourage you to give it a read (all of the data is viewable in your instance at about:telemetry).

However, the only relevant number I’ve found is `browser.engagement.max_concurrent_tab_count`, and that only reflects the maximum number of tabs you’ve opened in that session. Which isn’t great when you’re trying to get that number down. Also, it doesn’t seem simple to query that data – you can get all sorts from FF’s SQLite databases, but I never landed on this value.

After lots of faffing, I finally discovered the session restore file found in your Firefox Profile directory at `sessionstore-backups/recovery.jsonlz4`. It’s the file that Firefox uses to restore your session when it runs out of memory because you’re an idiot who had too many tabs open. The file is massive – particularly if you have large amounts of tabs open – and in a slightly arcane format – lz4 – which on reading, you’ll see is a suitable compression format for the problem at hand. I expected that if we can parse that file quickly, it should be suitable for the job we have here – reporting regularly on the current number of tabs open.

Implementing a Firefox Tab Counter in Hammerspoon

First of all, I needed a fast way of reading that file. After some digging and pissing on with weird different libraries, I managed to get a fork of lz4json compiling locally – not only did it work, but it was fast:Then I needed to parse it. Instinct got me going down the “I’ll write some javascript for this” path, but I quickly realised that for fast feedback, I needed something close to the metal, simple, and again fast. So I dived into what describes itself as “sed for JSON data”, jq, and realised that this type of problem was its bread and butter.

Piping the above catted file into this strange incantation:

| jq 'reduce (.windows[] .tabs | length) as $length (0; . + $length)'

gives you a value that matches up exactly with the dialog box we get when hitting CMD+Q. Woop!

To try and explain the above, `.windows[] .tabs` gives you an array of tabs in each window. `| length` gives you the result of the length method applied to that array. And reduce applies a reduce operation to each of those windows, in our case adding them to get a total amount of tabs open.

Combine all of that into a little shell script, and by applying our ultra scientific `time` method of measuring the speed of it all, we still get sub-tenth of a second times:

That strikes me as a fast and feasible way of retrieving a tab count. So next step – surfacing it somewhere obvious. Working through the Hammerspoon getting started guide, you quickly come to a tutorial around simulating Caffeine in the menubar. This was incredibly straightforward and seemed like a nice obvious first step for getting stuff somewhere. So that’s what I did – we run the get open tabs script, and put the result of that in the menubar, using Hammerspoon’s PathWatcher utility to update that number whenever our session restore file changes.

I added a couple of enhancements, but nothing hugely complex – a limit constant that turns our number red when our open tabs are greater than it (at time of writing, I’ve exceeded it by 70 😬). And I had a play with creating an icon, which was incredibly straight forward. It’s based on this “format” (if you can call it that), which is self-documenting enough that I figured it out from looking at a couple of other instances of icons.

The script itself currently lives in my dotfiles, and I’d be happy to help anyone else trying to do something similar if they get stuck trying to replicate any of this. When it all comes together, my menubar looks a little bit like this:

Which is everything I had hoped for. Has it fixed my problem? Goodness me, no. But it has been a fun learning exercise, and it makes me think twice before randomly middle-clicking links.

Tags dotfiles, firefox, hammerspoon, lua, Quantified Self, tabs, workflow