I recently discovered that Apache Airflow allows you to embed markdown documentation directly into the Web UI. This is very neat feature, because it enables you locate your documentation as close as possible to the thing itself, rather than hiding it away in some google doc or confluence wiki. This, in turn, increases the chance it is actually read, rather than being promptly forgotten about and undiscovered by new team members.

I recently came across a neat Chrome extension called SingleFile which saves webpages as HTML files, but first waits for lazy-loading javascript, images and CSS to render. It doesn’t work perfectly—it sometimes includes the blurry version of lazy-loaded photos unless you first scroll to the end of the page—but it works lightyears better than anything else I’ve tried.

The modern knowledge worker works in a highly specialized environment. Specialization improves efficiency, but it comes at a less reactivity and adaptability to change. As units of work grow beyond the span of a single agent, it imposes a trade-off. But we can hack this trade-off. In an organization with multiple actors, the question shouldn’t be Is collaboration worth it? but rather How can we reduce the cost of collaboration? A natural place to start is in the written form of the request/job/project/task.

I fielded an interesting request recently from our PR team, who wanted to generate a creative representation of our data based on the direction and distance of trips booked on our platform. Distance a key attribute of interest for a travel business, so it is naturally easy to retrieve this data. However the direction of a trip is something that had not been previously analyzed, and so it was not available off-the-shelf in our data warehouse.

Last year I built a DIY insulated sous-vide container using $10 of IKEA parts. It worked pretty well, using 60% less electricity than an uninsulated container. But it was a bit of an eye-sore, and I got tired of leaving a mess of towels out on my kitchen counter. Can we do better?

I have previously written about how to use ExternalTaskSensor in Airflow but have since realized that this is not always the best tool for the job. Depending on your specific decision criteria, one of the other approaches may be more suitable to your problem.

Airflow offers rich options for specifying intra-DAG scheduling and dependencies, but it is not immediately obvious how to do so for inter-DAG dependencies. Airflow provides an out-of-the-box sensor called ExternalTaskSensor that we can use to model this one-way dependency between two DAGs.

One textbook which is frequently recommended on Hacker News threads about self-study math material is Blitzstein and Hwang’s An Introduction to Probability. Having just recently finished the book, I realized that this is the first textbook I have truly worked through end-to-end while studying a topic outside of a school course. Here are some thoughts on what the book does well, and my (minor) grievances.

This is Water is a 22-minute commencement speech given by David Foster Wallace at Kenyon College in 2015 which was later adapted into a short book. It is difficult to overstate how powerful it is. Even after listening to this speech countless times, it never fails to send a shiver down my spine. In the interest of periodically forcing myself out of this default mode of thinking, I created a recurring calendar event to re-listen to the speech every sixth months. Here is an abridged audio version for that purpose.

Since acquiring an Anova sous-vide cooker, it has become an essential component of my weekly cooking routine. Their marketing materials show the device being used in any large pot you probably already have. This is fine for occasional use, but since I use the device frequently I started looking for a dedicated vessel. A dedicated vessel also lets you cook a larger quantity of food, or something awkwardly large like a rack of ribs. You can buy a pre-built container but it costs $70 and is not insulated. So I decided to build a simple dedicated container that was semi-insulated, so that it would be energy efficient when cooking ribs for 48 hours.