usher

Email is annoying. Doubly so when you're a contractor and have a separate inbox for every client. Having to keep 4 Gmail tabs open to keep track of incoming requests feels like a waste of brain power and space. I've spent a long time looking for a solution that solves all my needs: - Gives me a unified inbox across all my email accounts on my Linux desktop - Lets me receive notifications for important emails anywhere (desktop, laptop or phone) - Allows me to quickly scan and archive emails that I need to see, but don't need to action (notifications, alerts, etc.) - Supports authentication with Gmail accounts via OAuth The first and most obvious solution is a GUI mail client. Mailspring, Thunderbird, Evolution and many of others have crossed my desktop. These are all great tools, but ultimately I find that: - Configuring them with Gmail OAuth is awkward or not possible - They are slow, especially with a lot of emails cached locally - Search performance isn't great - I end up having to login to Gmail whenever I need to search I'm not the sort of person who lives in the terminal, I like a good GUI when it serves a purpose and having previously ventured in to to trying mutt and not wanting to invest the time to learn the keyboard shortcuts, I wasn't particularly inclined to go down that route again. How naive I was. Some time ago I had configured isync to keep local backups of my mail. If all my mail was being regularly synced for backups in maildir format, couldn't I read it directly from there? - Thunderbird has Maildir support but has some scary warnings about it being an experimental feature. - mutt/neomutt are nice, but the aforementioned fear of learning a bunch of new keybindings kept me away - it's also a bit of a pain to configure with multiple inboxes and search is still slow. Eventually, I rediscovered notmuch and a well-supported client, alot - which I've been using for a few months very happily. Let's tackle the setup: --- ## Step 1: Grabbing email The first thing I need is to grab all my email from my various mail providers and copy those emails locally. This gives me local backups, the ability to search (I'll come back to that in a bit), and a way to quickly access, read and manipulate email locally. To do this, I use isync (the tool is called isync, the binary is called mbsync) to synchronise a local copy with each IMAP server. My config looks a bit like this: #### ~/.config/mbsync/mbsyncrc Expunge None Create Near SyncState * IMAPAccount my-email@gmail.com Host imap.gmail.com User my-email@gmail.com AuthMechs XOAUTH2 PassCmd "oauth2get google my-email@gmail.com" SSLType IMAPS SSLVersions TLSv1.1 TLSv1.2 IMAPStore my-email@gmail.com-remote Account my-email@gmail.com MaildirStore my-email@gmail.com-local Path ~/mail/my-email@gmail.com/ Inbox ~/mail/my-email@gmail.com/Inbox SubFolders Verbatim MaildirStore my-email@gmail.com-archive Path ~/mail/archive-my-email@gmail.com/ Channel my-email@gmail.com-archive Far ":my-email@gmail.com-remote:[Google Mail]/All Mail" Near ":my-email@gmail.com-archive:Archive" Sync All Channel my-email@gmail.com-trash Far ":my-email@gmail.com-remote:[Google Mail]/Bin" Near ":my-email@gmail.com-archive:Trash" Sync Pull Channel my-email@gmail.com-drafts Far ":my-email@gmail.com-remote:[Google Mail]/Drafts" Near ":my-email@gmail.com-local:Drafts" Sync Pull Expunge Both Channel my-email@gmail.com-sent Far ":my-email@gmail.com-remote:[Google Mail]/Sent Mail" Near ":my-email@gmail.com-local:Sent" Sync Pull Expunge Both Channel my-email@gmail.com-inbox Far ":my-email@gmail.com-remote:INBOX" Near ":my-email@gmail.com-local:Inbox" Sync All Expunge Both Group my-email@gmail.com Channel my-email@gmail.com-trash Channel my-email@gmail.com-inbox Channel my-email@gmail.com-drafts Channel my-email@gmail.com-sent Channel my-email@gmail.com-archive # ... repeat for every mail account Group all Channel my-email@gmail.com-trash Channel my-email@gmail.com-inbox Channel my-email@gmail.com-drafts Channel my-email@gmail.com-sent Channel my-email@gmail.com-archive # ... repeat for every mail account For each of my email accounts, this defines: 1. An IMAPAccount containing the server details and credentials for the account. 2. An IMAPStore for that account. This tool syncs between two stores - so we define this to give it an entity from which to sync. 3. Two MaildirStore entries - one for 'active' email, and one for the archived email - keeping them separate makes it easier to backup only the archives and gives me the option to access only the 'active' emails if I want to use clients like mutt. 4. A set of Channels - the config entry used for defining how two stores are synchronised. I create a seperate channel for each key folder within my remote IMAP store so that I can keep them consistently named locally across different mail providers. 5. A Group listing all my defined channels, so that running mbsync all syncs everything. I have mbsync all configured to run on a systemd timer every 2 minutes: #### ~/.config/systemd/user/mbsync-run.service [Unit] Description=Sync mail with mbsync OnFailure=status-email-user@%n.service [Service] Type=oneshot Nice=10 ExecStart=/usr/bin/mbsync --config /home/tom/.config/mbsync/mbsyncrc all Environment="HOME=/home/tom" #### ~/.config/systemd/user/mbsync-run.timer [Unit] Description=Sync emails with mbsync on schedule [Timer] OnBootSec=2m OnUnitActiveSec=2m Persistent=true [Install] WantedBy=timers.target --- ## Step 2: Indexing Now all my emails are being synced locally on a schedule, I need some way to make sense of them. This is where notmuch comes in. notmuch scans your maildir formatted directory and adds everything in to a Xapian index giving you a nice CLI tool to search through all your emails really quickly. Running notmuch new will walk you through getting it all set up - you can then edit your config file as you need. notmuch doesn't read its config from XDG_CONFIG_HOME by default, but it will read a config location from the NOTMUCH_CONFIG environment variable. If you like to keep your home directory clean, add: export NOTMUCH_CONFIG=$XDG_CONFIG_HOME/notmuch/config to your .zshenv, .bash_profile or equivalent. #### ~/.config/notmuch/config [database] path=/home/tom/mail [user] name=Tom Usher primary_email=tom@tomusher.com other_email=my-email@gmail.com [new] tags=new; ignore= [search] exclude_tags=deleted;spam; [maildir] synchronize_flags=true The most important thing here for me is the tags definition in the [new] section - this tells Notmuch that when it finds new mail, it should mark it in its internal tagging system as new. The default option here is to set both the new and unread tags, but as I may have read this email using a different client, I prefer the unread flag to by synchronised from the maildir using the synchronize_flags setting. To tell notmuch to index new mail when it comes in, I add an ExecStartPost command to my systemd unit: #### ~/.config/systemd/user/mbsync-run.service [Unit] Description=Sync mail with mbsync OnFailure=status-email-user@%n.service [Service] Type=oneshot Nice=10 ExecStart=/usr/bin/mbsync --config /home/tom/.config/mbsync/mbsyncrc all ExecStartPost=/usr/bin/notmuch new Environment="HOME=/home/tom" Now I can do all sorts of handy things with the notmuch CLI: - notmuch search tag:unread - list all the emails I haven't read - notmuch tag -unread -- tag:unread - remove the unread tag from all the mail tagged unread. - notmuch tag important -- tag:unread and critical - tag anything unread containing the term 'critical' as important. - notmuch search date:2020-12-01..2020-12-31 and from:auto-confirm@amazon.co.uk - find all my order receipts from Amazon in December 2020. This is useful, but it's not the nicest way to regularly read email. That's where the number of available notmuch frontends comes in. --- ## 3. Reading Email Of the frontend applications supporting notmuch that I've tried, I ended up sticking with alot which I find to be a great balance between simplicity and customisability. alot has a bunch of neat features, but it does what I want particularly well: - A simple overview of mail across all my inboxes in one panel - Very quick to start and flick through emails - Lets me archive emails quickly with the default a keybind - which toggles the inbox flag I don't customise it much, other than defining an 'account' for sending email, and adding a useful open in browser custom hook - which lets me press a key to view an email in the browser - useful for those complicated HTML emails. [accounts] [[personal]] realname = Tom Usher address = tom@tomusher.com sendmail_command = msmtp --account=personal -t sent_box = maildir:///home/tom/mail/my-email@gmail.com/Sent draft_box = maildir:///home/tom/mail/my-email@gmail.com/Drafts [bindings] [[thread]] b = call hooks.open_in_browser(ui) --- ## 4. Sending email I don't actually do much email sending - for better or for worse most of my communication has moved to messaging platforms. It is handy to have something to easily send emails without launching Gmail, and for that I use msmtp. This can be configured using a relatively straightforward config file: #### ~/.config/msmtp/config defaults auth on tls on tls_trust_file /etc/ssl/certs/ca-certificates.crt host smtp.gmail.com port 587 auth oauthbearer # Personal account personal from tom@tomusher.com user my-email@gmail.com passwordeval oauth2get google my-email@gmail.com # ... repeat for each account account default : personal I can then use the 'account' I previously configured in alot to send and reply to emails. --- ## 5. Making it work with Gmail Most clients I work with use GSuite for email - it's quick to set up, works well and provides an excellent web-based client. Because I've gone down this whole weird route, I do need to make some adjustments to cope with Gmail's oddities. First up; 'archiving' an email via IMAP in Gmail might not be what you expect. Incoming emails end up in both your All Mail folder as well as your Inbox. To archive something, you need to delete it from your Inbox, leaving the copy in All Mail intact. I have a script which does that on the local copy of my inboxes by searching for files in any Inbox folder that no longer have the tag inbox (i.e. I've archived them in alot), use grep to filter the results to only show mail from my Inbox (as notmuch search will return every copy of the email it found, including the one in my Archive folder), then delete the remaining items. Once they've been removed, mbsync will synchronise those changes back to Gmail, removing the copy from my Gmail Inbox, effectively archiving the email. #### ~/.mail/scripts/notmuch-move.sh #!/bin/bash echo "Removing emails deleted from inbox" notmuch new --no-hooks notmuch search --output=files --format=text folder:/Inbox/ -tag:inbox | grep Inbox | xargs --no-run-if-empty rm notmuch new --no-hooks The next problem is that if I have the Gmail web client open, or am checking emails on my phone, I'm likely to have read and archived an email before it hits my computer. All new incoming email will be marked as new by notmuch, even if it has already been read or archived. To resolve that, I treat the new tag as 'to be processed', and do all my post-processing on emails with that tag. For example, this script tags everything new and in an Inbox folder with the inbox tag, and then removes the new tag from everything: #### ~/.mail/scripts/notmuch-tag.sh #!/bin/bash echo "Tagging new inboxed mail" notmuch tag +inbox -- folder:/Inbox/ and tag:new notmuch tag -new -- tag:new Both these run as part of my systemd unit file: [Unit] Description=Sync mail with mbsync OnFailure=status-email-user@%n.service [Service] Type=oneshot Nice=10 ExecStartPre=/home/tom/mail/scripts/notmuch-move.sh ExecStart=/usr/bin/mbsync --config /home/tom/.config/mbsync/mbsyncrc all ExecStartPost=/usr/bin/notmuch new ExecStartPost=/home/tom/mail/scripts/notmuch-tag.sh Environment="HOME=/home/tom" Finally, there's the problem of authorization. For Google accounts, I prefer to authorize with OAuth so that I can make use of 2FA and avoid creating insecure app passwords. To do this, I use the oauth2token project, which provides two binaries to create and get an OAuth token from local storage. mbsync and msmtp are then configured using PassCmd and passwordeval respectively to fetch the token using oauth2get, which will automatically refresh the token when required. The XOAUTH2 mechanism requires an implementation of SASL with a XOAUTH2 plugin installed; I use cyrus-sasl and cyrus-sasl-xoauth2. This also requires creating an OAuth2 credential in Google Cloud Console - which is a bit outside the scope of this post, but be aware you'll need to add the https://mail.google.com/ scope to your application, which is a 'restricted scope' and makes it extra awkward to do things: - You can keep the application in testing mode, but your auth tokens will expire after 7 days, so you'll need to re-authorize annoyingly frequently. - In testing mode, you'll get an error if you try to authorize and are logged in with more than one Google account. To fix those issues, you can choose to 'Publishthe application, which will prompt for manual verification by Google, but you don't actually need to do this - you'll only get a scary warning whenever you try to login. --- ## 6. Extra Fun Things Now you have a nice CLI for querying your mail, you can make it work for you. For example, in my personalnotmuch-tagscript, I add thenotifytag to every email that's unread, new and in my inbox. I have the following script running in anExecStartPoston mysystemdunit which notifies me of these new emails, and then removes the notify tag so I don't get duplicate notifications: ``` #!/bin/bash SEARCH="tag:notify" NOTIFY_COUNT=$(notmuch count "$SEARCH") if [ "$NOTIFY_COUNT" -gt 0 ]; then RESULTS=$(notmuch search --format=json --output=summary --limit=3 --sort="newest-first" "$SEARCH" | jq -r '.[] | "\(.authors): \(.subject)"') notify-send "$NOTIFY_COUNT New Emails:" "$RESULTS" fi notmuch tag -notify -- tag:notify ``` Another handy tool is my count of new emails in mypolybar`: #### ~/.config/polybar/scripts/mail.sh #!/bin/bash EMPTY_INBOX_ICON="%{T3}%{T-}" UNREAD_INBOX_ICON="%{T2}%{T-}" UNREAD=$(notmuch count is:inbox and is:unread) if [[ $1 = "count" ]]; then if [ $UNREAD = "0" ]; then echo $EMPTY_INBOX_ICON $UNREAD else echo $UNREAD_INBOX_ICON $UNREAD fi elif [[ $1 = "alot" ]]; then $TERMINAL -e alot & fi #### ~/.config/polybar/config # ... [module/mailcount] type = custom/script interval = 1 format-padding = 0 format = <label> exec = ~/.config/polybar/scripts/mail.sh count click-left = ~/.config/polybar/scripts/mail.sh alot # ... --- ## Summary While this was a fun thing to set up and a great way to play with some more open source goodies, I've found it works well for me for a number of reasons: - Being able to quickly scan and archive all my emails across 4+ email accounts every morning has absolutely saved me time and frustration. - I still get all the benefit of being able to set up mobile email clients with IMAP and everything syncs as you'd expect. - When I'm actually sitting at my desk, this approach lets me deal with emails quickly with minimal distractions without having to flick through various tabs of different webmail providers. - Even though I'm still a little tied to Gmail, it feels liberating to have all my emails locally, backed up and safely stored.

It is 2021, a blog can't exist without an "I trained an AI" post like these trailblazers: - GPT-3 tries pickup lines - Max Woolf's Buzzfeed YouTube video titles So here we are. When we were building out Virtually Expo (UK Games Expo 'gone virtual' for a pandemic-stricken world), we built a Discord bot that let visitors join one of UKGEs famed queues. One of the things this bot did was to give you a random, AI generated board game when you reached the 'front' of the queue. These were generated in advance by training a GPT-2 model (with the aid of Max Woolf's fantastic gpt-2-simple) on the top 2,000 board games from BoardGameGeek. --- First up; if you're not familiar with the board game world - there's a lot of games named after places. From Carcassonne, through Orléans and Mombasa, to Bruges and Snowdonia. The model seemed to pick up on this trend and continued the theme with ideas such as: - France - Monmouth Tycoon - Exeter - High Wycombe - Choose Your Own Adventure: Argentina - Turning Point: Swansea - Crisis in Northumberland --- Then there's the expansion trend. Carcassonne has somewhere between 10 and 134 expansions, depending on how you're counting. If the publisher ever runs out of ideas, this model has them covered: - Carcassonne: Yard Edition - Carcassonne: New York - Carcassonne: Vale of the Fallen Dragons of War - Carcassonne: Abyss – The Lost Chronicles - Carcassonne: Asteroid Derby - Carcassonne: The Cards are on the Rise along with positive vibes from the appropriately titled: - Pandemic: Alive! --- There's the expansions that promise to change the game quite significantly: - Ticket to Ride: The Quest for the Galaxy - Rallyman Gettysburg - Animal Upon Animal: The Camelot Campaign - Unlock! The Impacts of Crime - Rory's Story Cubism - Smash Up: First Person Shooter - Power Grid: The Next Phase of Electronic Warfare - Zooloretto: Run! - Ticket to Ride: The Official Route - Field Commander: Cthulhu - Catan: All the Pretty Things I am tickled by the idea of Zooloretto, a game in which you carefully plan a zoo filled with cute animals to attract visitors, turned on its head by the addition of the 'Run!' expansion. --- There's the philosophical: - Who Will Win? Modern Magic: The Gathering - Where Did All the Money Go? - If There Were No Dragon... - Something Big Is Bluffling the Earth - Mythic Tales: All the President's Cards Are Spell Cards! - Fascism! - Long Story Short: You Can't Always Get What You Want I checked, 'bluffling' isn't a word, but I can definitely agree that something big is indeed bluffling the Earth. --- There's the games I'd play in an instant: - Valley of the Uzis - I Cast a Spell: The Card Game - Accursed Gratuitous Sarcasm - Einstein: Son of a crab and his business partner embark on a daring mission --- and the ones that are perhaps for a more niche audience: - Battle Masters: The Fate of Meddle and Sump-Hole goats - Cottage Cheese - My Wife Is Alive! - Quirky Fate: You Win! War of Entrails - Funky Poop - 10 Days, No Jumpers - Pyromania Board Game - Nuns on the Underground - Rent a Bear In The Night --- The 'creative use of an IP' award goes to: - Warhammer 40,000: The Futon War - Bob Ross: Guitar Hero - Dice Throne: Season Two – Star Wars Battle for Aragorn - Hannibal: Family --- It seems to have a slightly confusing tendency to generate fish-themed games: - Fish Prince - Piranha Detective - My Fish is Fish - Flying Carp - Rolling Fishes - Death to the Lobster of which, 'My Fish is Fish' raises questions I do not have answers for. --- Then there's the titles that cause genuine concern about the future of AI: - EXPLOSION: A New World - Our Souls Are Marketable - Unconditional Surrender-era - No Peace Without Fire Yep. We should probably stop doing this before it's too late. --- Finally, a few I liked: - The Road to Wales - Zombie and Screamin' Zombies - Explosive Substances Control 4: Harry the Accuser - TATOOOMPI - Capital Letters - Get Off My Lawn! Unfairly Impeded - Do Something Crazy - Microsoft Word - Fabio's Garage - There's Something Shifting In Paris - Escape Goat & Cat with Guest Cat - Band-Aid Man - Billionaire Bus Driver - Bulk Spaghetti - Cryptocop - The Captain Is, But He's Bigger - Pie - Gravy Lobby - Whoopi Goldberg - YAM! - Tesla and the Other Robots

A brief summary of what I got up to in April 2021 - 🎮 Finished NieR: Automata - when this first came out I was intrigued but not convinced I'd be able to put the 40-odd hours in to get through it. When it appeared on Game Pass I thought I'd give it a chance, and I'm very glad I did. An excellent combination of thought-provoking story, fun characters, engaging combat and fantastic music, sprinkled with a few tropey JRPG bits and a slight lack of respect for the player's time. - 🎲 We've been playing quite a bit of MicroMacro: Crime City, a game that presents you with a large city map and a set of crimes you need to solve by spotting clues sprinkled throughout the city. A lovely unique concept that I'm excited to see more of. - 📺 Discovered That Van Jolene via GeoWizard. A well-presented and entertaining YouTube series on a couple's adventures in converting a van in to a campervan. Now I'm tempted to buy a van. - 📖 Currently reading Skunk Works: A Personal Memoir of My Years at Lockheed. I'm not usually interested in aerospace or the military, but this book's story of the engineering effort that goes in to building one of these machines is amazing. There's a few interesting lessons on security here too! - 🏠 I bought a Coral Edge TPU and set it up with Frigate on our home server to do real-time object detection on my cameras. I've done a lot of of experimenting with various self-hosted camera monitoring tools (MotionEye, Shinobi, ZoneMinder, Blue Iris) - while they all have their advantages, they all fall short on motion/object detection. Offloading this to an Edge TPU with Frigate is working out great. - 🌻 Our greenhouse has been in a state of disrepair for a while - so we finally got around to replacing all the broken glass panels with corrugated PVC sheets (not quite as pretty, but way more rugged for our windy garden). We also installed fancy automatic window openers to control the temperature in there. These are such clever, simple pieces of engineering made up of a piston filled with wax which expands and contracts based on the temperature. Magic. - 💻 I discovered and switched to asdf which replaces tools like pyenv, nvm and rbenv with a single tool. Very happy with it so far.

I use the wonderful Home Assistant on our home network for a variety of weird and wonderful automations and as a nice dashboard to all the devices in our home. Nothing on my home network can be reached from the outside world without a VPN. Unfortunately, that presents a few issues with Home Assistant: - Accessing the Home Assistant UI from out-and-about is a pain. - The Home Assistant app can't report useful information such as location data unless the device is connected to the VPN. - There are a number of integrations which use webhooks or similar to communicate data to your HA instance. So far, I've been living with these problems. Exposing my entire HA instance to the world isn't something I'm comfortable with. There is a solution for this in the form of Home Assistant Cloud - a paid solution from the creators of Home Assistant. If you're not comfortable with your networking and security knowledge, stop here and go ahead and subscribe to Home Assistant Cloud. It's very good and a great way to support Home Assistant. If you're interested in managing a solution for this yourself, read on. --- Cloudflare's 'Argo Tunnel' product has been around for a while, providing a tool to create a secure tunnel from any network in to the Cloudflare network, but they've recently rebranded it to Cloudflare Tunnel and made it free to everyone. Here's how I set it up to expose my Home Assistant instance. Before you start, you'll need a domain set up with DNS managed by Cloudflare. ## Use cloudflared to configure a tunnel On your home server, use the cloudflared utility to login to Cloudflare and download a certificate. I use the cloudflared docker container, so to do this: 1. Create a folder for your cloudflared configuration to live, I use /etc/cloudflared on the host. 2. Run: docker run --rm -it \ -v "/etc/cloudflared:/home/nonroot/.cloudflared" \ --user 1000:1000 \ cloudflare/cloudflared:2021.4.0 \ tunnel login Replacing --user 1000:1000 with a user/group ID that has access to read and write from your /etc/cloudflared directory. This will provide you with a link to follow to authorise with Cloudflare and to choose a domain to authorise. Once that's done, cloudflared will downloaded the generated certificate and place it in your mounted volume at /etc/cloudflared. 3. Create your tunnel: docker run --rm -it \ -v "/etc/cloudflared:/etc/cloudflared" \ --user 1000:1000 \ cloudflare/cloudflared:2021.4.0 \ tunnel create homeassistant This will create a new tunnel named homeassistant and drop a config file for it in your configuration directory. 4. Create a configuration file to route your tunnel to your Home Assistant instance. In /etc/cloudflared/config.yml: --- url: homeassistant:8123 tunnel: 12345678-9012-3456-7890-123456789012 credentials-file: /etc/cloudflared/12345678-9012-3456-7890-123456789012.json replacing the tunnel ID and credentials-file with a reference to the config file you got from step 3, and replacing the url with the URL for your Home Assistant instance. ## Set up a DNS record for the tunnel In the Cloudflare DNS panel, add a new CNAME from the subdomain you want your instance to be accessible at, to 12345678-9012-3456-7890-123456789012.cfargotunnel.com - where the ID in the target is the same as the tunnel ID you created previously. <Image src={import("./cloudflare-tunnel-ui.png")} alt="Cloudflare DNS UI" /> ## Start the tunnel You'll need some way to start your tunnel and keep it running - I'm doing this using docker-compose, with a docker-compose.yml that looks a bit like: version: '3.9' services: cloudflared: image: cloudflare/cloudflared:2021.4.0 user: 1000:1000 volumes: - "/etc/cloudflared:/etc/cloudflared" command: "tunnel --config /etc/cloudflared/config.yml run homeassistant" Run docker-compose up -d to bring up the tunnel. ## Configuring access The first thing we need to do is give Cloudflare a way to authenticate you so we can make sure access is restricted. 1. Head over to the Cloudflare Teams Dashboard to start configuring access to your tunnel. 2. Start at Configuration -> Authentication. 3. Click '+ Add' next to Login methods to add your first login method. 4. The easiest to get started with here is 'One-time PIN', so choose and enable that. Next up, we need to configure the tunnel to use this login provider: 1. Go to Access -> Applications on the left 2. Click 'Add an application' and choose 'Self-hosted' from the options. 3. Give your application a name and provide the domain you set up previously. 4. In the next step, create a rule for 'Emails' which includes your email address: <Image src={import("./cloudflare-rule-ui.png")} alt="Cloudflare Application Rule UI" /> 5. Leave the 'setup' settings as they are and finalise setup. Once this is done, you should be able to visit the domain you've setup where you'll be prompted to follow the One-time PIN sign in process. If the entered email matches the one you provided in your rule, you'll have remote access to your Home Assistant instance! ## Exposing webhooks Many Home Assistant integrations expose a webhook URL to allow external applications (and mobile apps) to update sensors. These applications won't be able to negotiate through the Cloudflare Access authentication process, so to work around this we'll add a bypass rule specifically for webhooks. 1. Create another application as above, but when prompted for the application domain, enter /api/webhook/* in the path: <Image src={import("./cloudflare-webhook-path.png")} alt="Setting the webhook path in the Cloudflare UI" /> 2. When setting rules, create a rule with the 'Rule action' set to Bypass' and an 'Include' rule set to 'Everyone'. This will allow anonymous users to bypass authentication. <Image src={import("./cloudflare-rule-include-everyone.png")} alt="Setting a rule to bypass everyone in the Cloudflare UI" /> Many webhooks are now configured automatically by Home Assistant. To make sure they point to the tunnel URL rather than your internal URL, head over to Configuration -> General in your Home Assistant UI and set the 'External URL' value to that of the tunnel you've set up. To set up your Home Assistant mobile app to route sensor data through the tunnel, you'll need to set up a separate URL for external and internal use. On Android, this is done by setting the Home Assistant URL setting to the external/tunnel URL, and the Internal Connection URL to the URL you use while connected to the networks listed in Home Network WiFi SSID: <Image src={import("./android-ui.png")} alt="Setting up an external URL in the Home Assistant android app" /> --- I'm still experimenting with this so this solution isn't entirely complete. It works to help limit the exposure of your Home Assistant instance, but it isn't perfect: - Some integrations don't use webbooks as a means to communicate with HA, so you may find you need to expose different URLs - this isn't typically well documented so you'll need to dive in to the code to figure out what you need to configure. - You're still exposing part of your Home Assistant instance to the world - if there's a vulnerability exploitable through the webhook endpoint, this won't help you. - The dashboard in the Home Assistant app won't work with Cloudflare Access in front of it. - Home Assistant Cloud is better, easier, and super cheap. Go and sign up.

Some stuff that happened in July 2021 - 🪶 Wagtail, the open-source CMS on which I've built my livelihood recently started a performance working group. Already some great ideas as to how we can improve the CMS in the long-term - very much looking forward to seeing what comes out of this. - 🎲 Attended the UK Games Expo 2021 show. It's been great to work with the team here building their new site and infrastructure over the past couple of years. Even with the major setback of having to build a 'Virtual' version of the Expo in 2020, I feel like we've made some big improvements. It was great to see this all come together at the show. While definitely scaled back from the past few years, there's still something about being amongst other board gamers that reminds you why the hobby is great. - 🎮 There was a lot of buzz around Final Fantasy XIV (the MMO) recently, initially after the poor reception to the latest WoW patches, and then after the awful stuff that came out of Activision/Blizzard. Despite not having committed to an MMO in many years (only dropping in to the WoW expansions when they release), I still have a soft spot for the genre, so I thought I'd give it a go. Pretty pleased I did right now - very much enjoying the slower pace, the story elements and just exploring a different world. - 🏡 Had the exciting experience of hiring a digger and going to town on our garden with it. Not entirely for entertainment purposes (although my son may have thought otherwise). We're replacing our slowly rotting raised vegetable beds with new, fresh, sleeper-based ones. {% image "src/posts/2021-08-06-july-braindump/digger.jpg", "Building raised beds" %} Despite the painful cost of wood at the moment, I think these will be a huge improvement. Always surprising how much you can learn from a "simple" project like this: * Designing a wood structure in Fusion 360 (I like to be able to visualise these things). * How to drive a digger, and how not to tip it over when working on slopes. * You can buy really long timber screws.