The result: 8MB down to 53KB. Same visual quality.

One command:

ffmpeg -i input.png -vf scale=1600:-2 -c:v libsvtav1 -crf 35 -b:v 0 output.avif

A few things worth knowing about those args:

• -c:v libsvtav1 — the SVT-AV1 encoder, which is fast and produces excellent results
• -crf 35 — quality level (0–63, lower is better). 30–40 is the web sweet spot
• -b:v 0 — disables the bitrate ceiling so CRF controls quality purely; these two always travel together
• -vf scale=1600:-2 — scales to 1600px wide, auto-calculates height. The -2 (not -1) rounds to an even pixel count, which AV1 requires

AVIF is supported in all modern browsers. WordPress serves it without any configuration.

The Script

Since I’ll want this more than once, I wrapped it in a small bash script at ~/.local/bin/imgopt:

#!/usr/bin/env bash
# Convert an image to AVIF using ffmpeg. Optimized for web delivery.

set -euo pipefail

usage() {
  cat >&2 <<'EOF'
imgopt — Convert images to AVIF for web delivery

Usage:
  imgopt <input> [output] [crf] [max-width]

Arguments:
  input      Source image (PNG, JPG, WebP, etc.)
  output     Output path (default: input basename + .avif)
  crf        Quality level 0-63, lower = better (default: 35)
             28-40 is the web sweet spot
  max-width  Scale width in pixels, preserve aspect ratio (default: no scaling)

Examples:
  imgopt photo.png
  imgopt photo.png photo-web.avif
  imgopt photo.png photo-web.avif 30
  imgopt photo.png photo-web.avif 35 1600
EOF
}

INPUT="${1:-}"
if [[ -z "$INPUT" || "$INPUT" == "--help" || "$INPUT" == "-h" ]]; then
  usage
  [[ -z "$INPUT" ]] && exit 1 || exit 0
fi

BASENAME="${INPUT%.*}"
OUTPUT="${2:-${BASENAME}.avif}"
CRF="${3:-35}"
MAX_WIDTH="${4:-}"

if [[ -n "$MAX_WIDTH" ]]; then
  VF_FILTER="-vf scale=${MAX_WIDTH}:-2"
else
  VF_FILTER="-vf scale=trunc(iw/2)*2:trunc(ih/2)*2"
fi

echo "Converting: $INPUT → $OUTPUT (CRF=$CRF${MAX_WIDTH:+, max-width=${MAX_WIDTH}px})"

ERRLOG=$(mktemp)
if ! ffmpeg -v quiet -i "$INPUT" $VF_FILTER -c:v libsvtav1 -crf "$CRF" -b:v 0 "$OUTPUT" 2>"$ERRLOG"; then
  cat "$ERRLOG" >&2
  rm -f "$ERRLOG"
  exit 1
fi
rm -f "$ERRLOG"

ORIGINAL=$(du -sh "$INPUT" | cut -f1)
RESULT=$(du -sh "$OUTPUT" | cut -f1)
echo "Done: $ORIGINAL → $RESULT"Code language: PHP (php)

Save it, chmod +x ~/.local/bin/imgopt, and make sure ~/.local/bin is on your $PATH. Then it’s just:

imgopt photo.png
imgopt photo.png photo-web.avif 30 1600Code language: CSS (css)

Quieting the Output

The first version of the script was noisier than expected. Adding -v quiet to the ffmpeg command silences ffmpeg’s own output — but libsvtav1, the underlying encoder, writes its info lines directly to stderr and ignores that flag entirely. The result was a wall of Svt[info] lines that -v quiet couldn’t touch.

The fix is to redirect stderr to a temp file and only surface it if the command actually fails:

ERRLOG=$(mktemp)
if ! ffmpeg -v quiet -i "$INPUT" ... 2>"$ERRLOG"; then
  cat "$ERRLOG" >&2
  rm -f "$ERRLOG"
  exit 1
fi
rm -f "$ERRLOG"Code language: JavaScript (javascript)

Silent on success. If something goes wrong, the full error output is still there.

Guarding Against Odd Dimensions

Testing the script on a screenshot surfaced another edge case. AV1 requires even pixel dimensions — width and height must both be divisible by 2. The original command used -2 instead of -1 when scaling to a max width, which handles the height automatically. But when no max-width is set, no scaling happens at all, and an image with an odd width or height goes straight to the encoder and fails:

Svt[error]: Source Width must be even for YUV_420 colorspace
Svt[error]: Source Height must be even for YUV_420 colorspaceCode language: CSS (css)

Screenshots are a common source of this — window sizes tend to be whatever they happen to be. The fix is to always run a scale pass, even when the dimensions aren’t changing. trunc(iw/2)*2:trunc(ih/2)*2 rounds each dimension down to the nearest even number. For images that are already even it’s a no-op; for odd ones it trims a single pixel.

# No max-width specified — still enforce even dimensions
-vf scale=trunc(iw/2)*2:trunc(ih/2)*2Code language: PHP (php)

Week 1 got done. Week 2 got started but never finished. I found I was outgrowing it.

Prompting Is on the List. Other Things Jumped the Queue.

Prompt engineering is a real skill and one I intend to develop. The resolution was a reasonable way to do that — learning to craft better inputs, compare models, understand their strengths. Useful stuff, especially when most of the work happens in a chat window.

What I found was that once I left the chat window, other skills became more pressing. Working with tools like Claude Code, the actual leverage isn’t how cleverly a request is phrased — it’s in how well the context is organized and how clearly tools, like MCP servers, are integrated. A well-structured project beats a well-crafted prompt, pretty much every time. That felt more urgent to get right.

The Moment the Problem Changes Shape

I’m a software engineer at WP Engine. Most of what I do day to day is what software engineers have always done: shipping features, fixing bugs, improving existing systems. What’s changed is the assistance layer — tools like Claude Code mean I’m rarely doing that work alone, and the collaboration is different from anything I’ve used before.

But even there, what I’ve noticed is that the prompting skills the resolution was building aren’t the bottleneck. The bottleneck is context — making sure the agent understands the codebase, the constraints, the goals well enough to be a real collaborator rather than a very fast autocomplete. That’s a different skill entirely. And it’s pointing me toward something I want to build next: custom agent harnesses that can do meaningful automated work on projects like Local. Not just AI-assisted development, but AI-driven workflows.

That distinction — assistance vs. automation — is where things get interesting.

Two Tracks, One Shift

I’ve started thinking about my AI work as two tracks that are developing in parallel.

• Work track: Right now this is growing my skills around agentic tools and AI-assisted development. The near-term direction is building and maintaining custom agent harnesses that can tackle larger, more complex work. The AI isn’t the product; it’s the infrastructure that makes the engineering better.
• Personal track: Building an actual assistant. Not a chatbot. Not a suite of tools. A named, persistent entity that knows my goals, my context, and my preferences — and works on my behalf without me having to think about it much.

Both tracks are moving away from the chat window. Both care a lot more about context architecture than prompting craft. And the personal track in particular led me somewhere that made the resolution feel like a different conversation entirely.

The Assistance Paradigm

Daniel Miessler — who created the Fabric toolkit and has been writing about personal AI infrastructure since 2016 — puts it well in his Personal AI Maturity Model. The model has three tiers: Chatbots, then Agents, then Assistants. Most people are still deep in the agent tier. The interesting work is starting to happen at the assistant tier.

The difference isn’t just complexity. It’s orientation. Chatbots respond to requests. Agents execute assigned tasks. Assistants advocate — understanding goals, monitoring the gap between current state and where things should be, closing it without being asked.

That framing clicked something for me. The AI Daily Brief’s New Year’s Resolution was optimizing me for the chatbot tier. My work track is pushing me into the agent tier. And personally, I’m drawn to the assistant tier — which requires a completely different kind of thinking.

What I’ve Been Actually Building

The tool I’ve started exploring on the personal track is PAI (Personal AI Infrastructure), also by Miessler. It’s open source, it’s opinionated, and it’s built around the idea of a single named assistant — in Daniel’s case, his assistant is called Kai — that acts as a persistent interface to everything else.

The setup involves defining goals, context, and preferences — what Miessler calls an “ideal state” — and giving that to the assistant as the thing it’s constantly working toward. The agents, the skills, the workflows all sit behind that interface. There’s no direct interaction with the infrastructure. There’s just my interaction with my assistant.

I’m early in exploring it. But the mental model it requires is so different from “write better prompts” that I couldn’t keep treating both as the same kind of work. His Personal AI Maturity Model is worth a read.

What the Resolution Actually Gave Me

I didn’t finish the 10 weekends. I also don’t think the resolution failed — it gave me exactly what I needed, just not what I expected.

Week 1 forced me to build something and ship it. That’s always useful. Week 2 pushed me to think systematically about which models are good at what — which turns out to matter when building agent systems, where picking the right model for the right task in a pipeline is a real decision.

But the bigger thing was the clarity. When I stopped trying to complete the resolution and started asking why it didn’t feel urgent anymore, the answer told me a lot about where my head actually was. I wasn’t thinking about chat windows. I was thinking about context, about systems, about what it would look like to have a persistent AI collaborator rather than a tool I pick up and put down.

That’s a different kind of skill to develop. And I think it’s the one worth developing.

It worked. But I’d never really interrogated it.

One of the things I’ve been doing lately is revisiting old tools and workflows with the help of an AI. Not to replace them, but to actually understand them better than I did when I first wrote them. When I walked through my screencast script in a recent session, I came away with a few things I genuinely didn’t know — including one flag that explains why some of my existing videos buffer slightly before playing in a browser.

What My Original Script Did

The original was two lines:

ffmpeg -i "${file}" -vcodec h264 -acodec aac "${slug}.mp4"

That’s a functional command, but it has a few issues worth addressing:

• -vcodec and -acodec are deprecated aliases. The modern flags are -c:v and -c:a
• There’s no explicit quality setting, so ffmpeg uses its default (CRF 23), which is fine but arbitrary
• It doesn’t handle files without audio cleanly
• It’s missing -movflags +faststart, which turns out to matter for web playback

The movflags +faststart Flag

This was the most interesting thing I learned. The ffmpeg formats documentation describes it as:

Run a second pass moving the index (moov atom) to the beginning of the file.

In plain terms: an MP4 file contains a metadata block that tells the browser how to decode and play it. By default, ffmpeg writes that block at the end of the file. That means a browser has to download the entire file before it can start playing it. With -movflags +faststart, that block is moved to the front, and playback can begin immediately.

For short screencasts this isn’t dramatic — but it’s the difference between a video that starts playing and one that sits there for a beat before it does.

CRF for Screencasts

The ffmpeg H.264 encoding guide documents CRF (Constant Rate Factor) as the recommended quality control method for libx264. The range is 0–51, lower is better, default is 23.

Screencasts are mostly flat backgrounds and sharp text — a different compression profile than live video or film. CRF 18–20 produces noticeably crisper text at still-small file sizes for this kind of content. The CRF guide from Sebastian Ebner is a good reference if you want to understand the tradeoffs in more depth.

The Updated Script

#!/usr/bin/env bash
# Convert MOV (or any ffmpeg-compatible video) to MP4 optimized for web delivery.

set -euo pipefail

usage() {
  cat >&2 <<'EOF'
mov2mp4 — Convert video to web-ready MP4

Usage:
  mov2mp4 <input> [crf]

Arguments:
  input   Source video (MOV, MP4, MKV, etc.)
  crf     Quality level 0-51, lower = better (default: 20)
          18-20 is the sweet spot for screencasts
          23 is ffmpeg's general-purpose default

Examples:
  mov2mp4 demo.mov
  mov2mp4 demo.mov 18
EOF
}

INPUT="${1:-}"
if [[ -z "$INPUT" || "$INPUT" == "--help" || "$INPUT" == "-h" ]]; then
  usage
  [[ -z "$INPUT" ]] && exit 1 || exit 0
fi

if ! ffprobe -v error -i "$INPUT" > /dev/null 2>&1; then
  echo "Error: not a valid media file: $INPUT" >&2
  exit 1
fi

CRF="${2:-20}"
SLUG="${INPUT%.*}"
OUTPUT="${SLUG}.mp4"

HAS_AUDIO=$(ffprobe -v error -select_streams a \
  -show_entries stream=codec_type \
  -of default=noprint_wrappers=1 "$INPUT" 2>/dev/null)

AUDIO_FLAGS="-an"
[[ -n "$HAS_AUDIO" ]] && AUDIO_FLAGS="-c:a aac -ar 44100"

echo "Converting: $INPUT → $OUTPUT (CRF=$CRF)"

ffmpeg -i "$INPUT" -c:v libx264 -crf "$CRF" $AUDIO_FLAGS -movflags +faststart "$OUTPUT"

ORIGINAL=$(du -sh "$INPUT" | cut -f1)
RESULT=$(du -sh "$OUTPUT" | cut -f1)
echo "Done: $ORIGINAL → $RESULT"

Save it to ~/.local/bin/mov2mp4, chmod +x ~/.local/bin/mov2mp4, and make sure ~/.local/bin is on your $PATH.

A few things worth noting about how this differs from the original:

• Accepts any ffmpeg-compatible input, not just .mov — so uppercase .MOV from an iPhone works the same as anything from QuickTime or OBS
• Detects whether the input has an audio track and handles it correctly either way: silent screencasts get -an (explicit and clean), files with audio get AAC at 44100Hz
• Input validation via ffprobe before attempting conversion
• Prints file size before and after, which is a small thing but useful for spot-checking output

Week 2 of the AI Daily Brief New Year’s resolution (AIDBNY) sessions is about moving away from anecdotal vibes and toward a repeatable way to measure model performance. I’m starting to build a Model Map—a systematic comparison of how different LLMs handle specific task types.

This is a first pass. The goal isn’t a definitive ranking yet, but rather a baseline to see where different models succeed or fail when given the same set of constraints.

The Workflow: Emacs + gptel

My day-to-day work environment is Emacs, and I’ve been wanting to use gptel in more of my workflows. Gptel’s integration into Emacs is quite good, meaning I can leverage AI within almost any place I can move the cursor.

The Prompt Library

I’m sure there’s an exhaustive set of prompts that can be used. But for this initial evaluation, I’ve settled on six prompts categorized by three task types to begin the evaluation process:

1. Code: Mechanical aspects of software development.
2. Analysis: Use a more advanced level of structural understanding to output insightful and valid diagram syntax.
3. Creative: Test the model’s ability to synthesize unstructured notes into structured strategy or narrative.

P001 (Code: Refactoring)

Identifying dependencies and refactoring snippets into reusable, typed functions with proper documentation.

Refactor this code into a reusable function that accepts all dependencies as parameters: \[CODE\_SNIPPET\]
Requirements:
 \* Identify all variables and functions used in the code block
 \* Create a function signature that accepts these as parameters
 \* Replace hard-coded values with function parameters
 \* Add TypeScript types if applicable
 \* Include JSDoc comments explaining the function's purpose and parameters
 \* Handle edge cases and provide default values where appropriate
 \* Preserve the original functionality exactly

P002 (Code: Debugging)

Finding bugs in JavaScript and providing a root-cause explanation alongside corrected code and test cases.

Debug this JavaScript function and explain the issue: \[CODE\_SNIPPET\]
Requirements:
 \* Identify the bug(s) in the code
 \* Explain why the bug occurs
 \* Provide the corrected code
 \* Include test cases that demonstrate the fix
 \* Explain any potential edge cases

P003 (Analysis: Call Flow)

Analyzing a codebase to generate Mermaid sequence diagrams for specific functions.

Generate a Mermaid sequence diagram for the call flow of \[FUNCTION\_NAME\] using the provided code context.
Requirements:
 \* Accept multiple files as context (codebase)
 \* Locate the specified function \[FUNCTION\_NAME\]
 \* Analyze the function's call graph and dependencies
 \* Generate a Mermaid sequence diagram showing:
   \* Function calls in chronological order
   \* Parameter passing between functions
   \* Return values and data flow
   \* Error handling paths
 \* Include all nested function calls and their relationships
 \* Use proper Mermaid syntax with sequenceDiagram tags
 \* Provide both the diagram code and a visual representation

P004 (Analysis: Architecture)

Using Structurizr DSL to map out system architecture at various C4 levels (Context, Container, Component, or Code).

Generate a C4 diagram using Structurizr syntax for the provided codebase at \[DETAIL\_LEVEL\].
Requirements:
 \* Accept multiple files as context (codebase)
 \* Accept \[DETAIL\_LEVEL\] parameter (C1, C2, C3, or C4)
 \* Generate appropriate C4 diagram based on level:
   \* C1: Context diagram (systems and users)
   \* C2: Container diagram (applications, databases, etc.)
   \* C3: Component diagram (internal components)
   \* C4: Code diagram (classes, interfaces, etc.)
 \* Use proper Structurizr DSL syntax
 \* Include all relevant elements and relationships
 \* Provide both the diagram code and a visual representation
 \* Explain the architectural decisions and patterns identified

P005 (Creative: Strategy Map)

Creating a directed graph of content dependencies and clusters needed to reach a specific topical authority.

Generate a content strategy map and directed graph for achieving \[CONTENT\_DESTINATION\] based on existing content.
Requirements:
 \* Analyze existing content inventory and identify gaps
 \* Define clear \[CONTENT\_DESTINATION\] goal (e.g., "Become recognized expert in AI development")
 \* Create a directed graph showing:
   \* Required content pieces to reach destination
   \* Dependencies between content items
   \* Content hierarchy and relationships
   \* Content clusters and topical authority
 \* Generate a visual representation using Mermaid or Graphviz syntax
 \* Include content type recommendations (blog posts, videos, tutorials, etc.)
 \* Provide timeline and prioritization suggestions
 \* Identify key performance indicators for measuring progress

P006 (Creative: Contextual Drafting)

Synthesizing research files and style guidelines into a cohesive post with proper citations and metadata.

Write a blog post based on provided context notes, source documents, and style guidelines.
Requirements:
 \* Accept multiple context files as research material
 \* Accept \[TARGET\_LENGTH\] parameter (word count)
 \* Accept \[VOICE\_STYLE\_GUIDE\] parameter (brand tone, vocabulary, formatting)
 \* Synthesize information from all sources into a cohesive narrative
 \* Follow the specified voice and style guidelines exactly
 \* Include proper citations and references where appropriate
 \* Generate a compelling headline and meta description
 \* Generate an excerpt of no more than 155 characters
 \* Provide both the full post and an outline structure

Code Tasks and Agents

While I’m including code refactoring and debugging in this initial test set, I suspect these manual prompts may eventually become redundant. As my workflow leans more toward agentic patterns, the need to explicitly prompt for a refactor starts to feel like an intermediate step.

If an agent is operating with full repository context, these individual tasks should ideally happen in the background. For now, however, testing the underlying logic of the models remains the necessary first step.

I am in the process of committing these notes to a Git repository, but not everything is ready to be committed.

I needed a backup solution that was three things: encrypted (because my notes are private), deduplicated (to save space), and automated. I landed on Restic, a fast, secure backup tool that fits perfectly into a CLI-centric workflow.

Lesson Learned: Trust the Tool

When I first sat down to build this, I spent a bit of time over-engineering my folder structure. I was manually injecting shell variables to keep things organized across different machines. It looked something like this:

\# The "Over-Engineered" Way
export backup\_slug="kb"export
restic\_repo="$HOME/.backups/$USER@$(hostname)--${backup\_slug}"

After reviewing Restic’s documentation on listing snapshots, I realized the tool already handles snapshots from multiple hosts natively. Every time you create a snapshot, Restic tags it with the host it came from by default. This means I can keep the repository path simple on my disk (~/.backups/kb) and let Restic’s internal metadata handle the host identification. While the current implementation doesn’t backup multiple hosts to the same remote Restic repo, it positions me to do just that when I eventually save my snapshots to the cloud.

The Manual Foundation

Before scripting everything away, I like to run the commands manually to make sure the “plumbing” works. Now that I’ve ditched the manual namespacing, the setup is much more straightforward.

1. Initializing the Repo

We’ll just use a straightforward kb (knowledge base) directory in our hidden backups folder.

export restic\_repo="$HOME/.backups/kb"mkdir -p "${restic\_repo}"
restic -r "${restic\_repo}" init

A note on security: When you run init, Restic will ask for a password. This password is the only way to decrypt your data. There’s no recovery flow here, so keep it somewhere safe like a password manager.

2. The First Pass

Next, we push the files. I make sure to exclude metadata folders like .git or .stfolder (from Syncthing). There’s no point in backing up my version control history inside a backup.

restic -r "${restic\_repo}" backup \\
    --exclude '.git' \\
    --exclude '.stfolder' \\
    --exclude '.stignore' \\
    ~/kb

Once this finishes, you can run restic snapshots and you’ll see your hostname automatically attached to the entry. Clean and simple.

The “Set It and Forget It” Script

Manual commands are great for learning and one-off execution, but for real-world reliability, we need a script. I’ve wrapped the logic into a Bash script that handles the repo creation automatically.

#!/usr/bin/env bash
# Backup the Org-roam knowledge base
export backup\_slug="kb"
export restic\_repo="${HOME}/.backups/kb"

# Check if repo exists, otherwise, create it. By using this paradigm,
# I can sync this backup script across hosts and backup the same repo:
if \[ ! -f  "${restic\_repo}/config" \]; then
  mkdir -p "${restic\_repo}"
  restic -r "${restic\_repo}" init
fi

# Create the backup
restic -r "${restic\_repo}" backup \\
  --exclude '.git' \\
  --exclude 'node\_modules' \\
  --exclude '.stfolder' \\
  --exclude '.stignore' \\
  ~/kb

What’s Next?

Right now, this is backing up to my local disk. It protects me against accidentally deleting a file or a weird filesystem corruption, but it won’t help if my laptop decides to take a coffee bath.

The next evolution of this is to hook up Rclone. Since Restic supports Rclone as a backend, I can eventually pipe these encrypted snapshots straight to Google Drive, a B2 bucket, or an S3 instance. That’ll get me to that “3-2-1” backup gold standard. But for today? My notes are safe, encrypted, and I can get back to building.

But as any engineer knows, the “how” is just as important as the “what.” I wanted to see if I could find a workflow that felt like actual engineering—not just a glorified chat interface. I tried two different platforms (Replit, Base44) and a CLI agent (OpenCode).

Friction of no-code Builders

I started where most people do: the high-abstraction platforms. I tried using the same prompts across the board to see how they’d handle the logic of a resolution tracker.

Both eventually hit a wall where they wanted me to “pony up” some cash before I had a usable tool. For a hobbyist project where the goal is *play*, hitting a paywall five minutes in is a massive vibe-killer.

Replit

I set up a project at passionsplay/AI-Progress-Tracker, but the honeymoon phase ended fast. It only gave me about two prompts before the output stalled, and the final product wasn’t exactly something I’d want to ship.

Base44

This one was a bit more generous with the prompt count and even dangled a 30% onboarding coupon in front of me. It did a great job of abstracting away the technical decisions, which is cool for a MVP, but as a dev, I kept wanting to peek under the hood.

OpenCode: Where the Real Work (and Play) Happens

I eventually pivoted to OpenCode, and this is where things got interesting. It required more setup, but the environment felt familiar—more like my actual day-to-day workbench.

Instead of just asking for a “tracker app,” I collaborated with an agent to handle the heavy lifting while I made the “dev-y” decisions. We walked through a legitimate engineering workflow:

• Initializing the Git repository.
• Building out the initial prototype.
• Setting up Playwright-cli so the agent could actually see and interact with the project it was building.

You can check out the final Week 1 result here:

• Repo: https://github.com/bgturner/aidb-new-year–wk01-resolution-tracker
• Live MVP: https://bgturner.github.io/aidb-new-year–wk01-resolution-tracker/

The Takeaway

Was OpenCode + Github more work?

Yep.

It’s possible that I still need to learn to let go of caring about the craft of building and just let the AI figure it out.

On the other hand, the output is a real repo that I actually own and can improve upon. Plus, it was effectively free—though I’m sure my interactions are helping train whatever upstream model they’re using.

Week 1 is in the books. I have a tracker, a repository, and a workflow that doesn’t feel like a toy. Onward to Week 2.

But the space is moving so fast that it’s hard to keep current, especially when it comes to other non-engineering tools I don’t use day-to-day.

I stumbled on the AI Daily Brief podcast (YouTube, Spotify), and like the content so far. But my goal isn’t just to consume more content. Instead, I want to integrate AI into my daily routine.

The AI Daily Brief’s site has an interesting UX, and while poking around, I found two guided tracks which help guide you to using and learning AI:

1. Projects to learn and use AI: AI New Years | 10-Week AI Challenge
2. Projects to build you Agentic Swarm: Claw Camp — Build Your Agent Team

So, nothing concretely built yet, but I’m looking forward to seeing where this structured, playful approach takes my technical workflow over the next few months!

A better approach is to keep keys encrypted and only load them into the environment when you are working in a specific project’s directory. This post outlines a straightforward, secure, and directory-aware workflow using GPG and direnv.

The OpenRouter Context

For this example, I’m using OpenRouter to access Claude Code, which offers benefits like provider failover and budget controls. According to their integration guide, the claude CLI requires the following environment variables:

export ANTHROPIC\_BASE\_URL="https://openrouter.ai/api"
export ANTHROPIC\_AUTH\_TOKEN="$OPENROUTER\_API\_KEY"
export ANTHROPIC\_API\_KEY="" # Must be explicitly empty

Our goal is to manage the $OPENROUTER_API_KEY secret securely.

Create and Encrypt the Secrets File

Emacs’s built-in EasyPG integration (epa-mode) makes handling GPG-encrypted files transparent.

1. Create the secret file: In your project, open a new file named .env-secrets.gpg by typing C-x C-f .env-secrets.gpg.
2. Add all required variables: Add the complete configuration, including your secret key.

export OPENROUTER\_API\_KEY='sk-or-v1-...'
export ANTHROPIC\_BASE\_URL="https://openrouter.ai/api"
export ANTHROPIC\_AUTH\_TOKEN="$OPENROUTER\_API\_KEY"
export ANTHROPIC\_API\_KEY=""

3. Save the file: When you save (C-x C-s), Emacs will prompt you to select your GPG public key for encryption.

Your secrets are now encrypted on disk.

Configure direnv to Load Secrets

Next, we use direnv to automatically load the secrets.

1. Create the .envrc file: Create a file named .envrc in your project root with the following content. The 2>/dev/null || true part makes the command more robust by preventing errors if the secrets file doesn’t exist.

\# .envrc
eval $(gpg -dq .env-secrets.gpg 2>/dev/null || true)

2. Allow direnv: In your terminal, run direnv allow.

Now, whenever you cd into this directory, direnv will use GPG to decrypt .env-secrets.gpg in memory and load the variables into your environment. When you cd out, the variables are cleared.

Prevent Committing Secrets

Explicitly ignore the encrypted secrets file in your project’s .gitignore to prevent it from ever being committed.

`echo ".env-secrets.gpg" >> .gitignore`Code language: CSS (css)

Use Claude Code

With the environment variables automatically loaded by direnv, you can now run the client:

claude

A Better Workflow

This setup gives me peace of mind. My secrets are never stored in plaintext, they never pollute my global shell, and they can’t be accidentally committed to Git.

When I enter a project directory, the keys are there for me. When I leave, they’re gone. It’s a simple, secure, and automatic workflow that gets out of the way and lets me focus on the actual work.

I originally came across Dependency Cruiser as a way to help me generate Component diagrams as defined by the C4 Model.

This tool has been great at generating diagrams, but one exciting benefit of using dependency-cruiser is the ability to write architectural rules to enforce boundaries and clean module separation.

Examples of these rules are:

• Restricting imports between the client and server modules
• Flagging unused modules
• Isolating peer folders from seeing each other

By creating custom rules, you can verify changes adhere to architectural decisions for the project at regular intervals, such as during PR reviews or even as part of an automated AI Agent workflow.

Installation

Getting started with Dependency Cruiser involves:

1. Installing the dependency
2. Initializing the configuration file

npm install --save-dev dependency-cruiser
npx depcruise --init
# creates .dependency-cruiser.js

Usage

I’ve found the best DX to be when I pair it with Graphviz’s Dot command to generate SVG and HTML artifacts:

# Scan the app/main folder and only include items in the the app/main
# folder. Save as an HTML page to interact with.
yarn depcruise app/main -v --include-only '^app/main/' --output-type x-dot-webpage -f docs/architecture/dependency-cruiser/dependency-graph.app_main.html
# Scan the app/main folder and only inclue Site.ts and modules that
# directly depend on it.
yarn depcruise app/main --focus '^app/shared/models/Site.ts$' --output-type x-dot-webpage -f docs/architecture/dependency-cruiser/dependency-graph.html

• System Context: Provides a high-level overview of the software system in its environment, showing users and external systems interacting with it.
• Container: Zooms into the system, depicting the major containers (applications, services, databases) that constitute the system and their responsibilities.
• Component: Further decomposes a container into its constituent components, illustrating how the system is internally structured.
• Code: Details the internal structure of components, including classes, interfaces, or functions.

Having this tiered model allows all stakeholders to engage with the architectural diagrams depending on their level of expertise and interest.

Traditional ad-hoc architectural sketches, drawn on whiteboards or with generic drawing tools like LucidChart, Visio, or draw.io don’t enforce naming consistency or relationship correctness. In contrast, the C4 Model provides a framework for creating diagrams that are consistent and precise.

Because the C4 Model is a framework that standardizes the kinds of diagrams to create, there isn’t a formal recommendation for the tooling to use. That being said, the creator of the C4 Model, Simon Brown, also created Structurizr, a collection of tools and a domain-specific-language, to help create C4 Model diagrams.

Over the past few weeks, I’ve taken a closer look at the C4 Model by working with Structurizr to better understand the architecture of Local.

3. Structurizr: Architecture as Code

With Structurizr, you create a workspace.dsl file to contain the complete model of a software system’s elements and their relationships. From this model, multiple C4 diagram views can be generated: System Context, Container, and Component diagrams. Note that the “Code” view defined by the C4 framework isn’t included. The thinking around this is that it’s too specific and not worth keeping up-to-date within the model. Instead, if you need to visualize the code, you should try to generate it using tooling specific to the language or IDE.

One of Structurizr’s primary benefits is its “diagrams as code” approach. Because the architectural model is defined as text, changes can be tracked in version-control. Using version control allows for branch-based experimentation and collaborative editing which reduces the risk of diagrams becoming outdated or inconsistent. Diagrams can be automatically regenerated as the model is updated.

A simplified example for the Local desktop app along with Local Hub (a website to manage a user’s account) might look like:

workspace "Local" "The Local product described on https://localwp.com" {
    !identifiers hierarchical
    model {
        // Entities
        wpdev = person "WordPress Developer"
        lc = softwareSystem "Local Core"
        lh = softwareSystem "Local Hub"
        // Relationships
        wpdev -> lc "Uses"
        wpdev -> lh "Uses"
        lc -> lh "Connects to"
    }
    views {
        // https://docs.structurizr.com/dsl/language#theme
        theme default
        systemLandscape local "Local" {
            include *
            autolayout lr
        }
        systemContext lc "LocalCore" {
            include *
            autolayout lr
        }
        systemContext lh "LocalHub" {
            include *
            autolayout lr
        }
    }
}

Which can be used to generate a diagram like:

A diagram this simple isn’t useful, but by modeling more systems, containers, and components, we can get a much more useful diagram:

Structurizr’s online sandbox is a great place to try things before installing anything locally.

Getting Started Locally

The above sandbox is a decent way to try out the syntax, but eventually, you’ll want to track the model in code. I’ve found that using Structurizr Lite via a Docker container is a great way to get a feel for the language and have the resulting data saved to a file.

docker pull structurizr/lite
docker run -it --rm -p 8080:8080 -v .:/usr/local/structurizr structurizr/lite

From there, just navigate to localhost:8080 to view the compiled diagrams.

Dynamic View

One diagram that I found particularly useful recently was the dynamic view. Similar in usefulness to a sequence diagram, the dynamic view allows you to define a series of steps that are taken over time, through a system. This view shows a series of relationships between entities, an example of which might look like:

workspace "Local" "The Local product described on https://localwp.com" {
    model {
        // Define the system model ...  
    }  
    view {
        // Define the them and what diagrams to export  ...  
        dynamic lc.app {
            title "Create a new WP site"
            wpdev -> lc.app.apptsx "Clicks the 'Add site' button"

            lc.app.apptsx -> lc.app.maints "Calls addSite() via IPC"
            lc.app.maints -> lc.config "Saves site metadata"
            lc.app.maints -> lc.ls "Provisions services"
            lc.app.maints -> lc.app.apptsx "Notifies UI via IPC"
        }
    }  
}

From there, you can compile and step through the flow to get a better idea of how and when various components interact with each other:

The main challenges of documenting these systems are determining who your audience is, knowing when to dive into additional detail, and utilizing consistency across various diagramns.

The C4 Model provides a structured approach to documenting software architecture, helping you communicate effectively with both technical and non-technical stakeholders. By embracing a framework for documenting systems, you can focus on uncovering relationships, letting C4 guide you to the appropriate levels of abstraction and documentation.

C4 Model Summary

The above site does a great job of simply describing the C4 model, but to summarize:

The C4 Model is a set of hierarchical diagrams to describe a system, where each successive diagram type is a more detailed view of the system.

• System Context
• Container
• Component
• Code

System Context diagram

The System Context diagram describes the highest level of abstraction and shows the top-level system, plus any users, and system dependencies.

Container diagram

Not Docker!

The Container diagram gives the overall shape of the architecture and technology choices of a system. As a rough rule of thumb, a C4 container would be any individually deployable unit that needs to be running for a system to work.

For example, if an individual system was a website, the container diagram might include nodes for the database, one or more backend servers, and a front-end client.

Component diagram

The Component diagram shows groupings of related functionality encapsulated behind a well-defined interface. As opposed to a container, a component is not an individually deployable unit.

On the other hand, while a component could be an individual class, it’s more likely a grouping of classes to represent a piece of functionality.

Code diagram

Lastly, a Code diagram shows the atomic building blocks of components and is often generated as UML. In most cases, you wouldn’t want, or need to explicitly create code diagrams, and instead, should rely on tooling to automatically create code diagrams dynamically while working.

Applying the C4 Model

As a concrete example, consider a simple ecommerce web application. Using the C4 Model, you might create:

• A System Context diagram showing the web app, users, and external services, like Stripe.
• A Container diagram including nodes for the front-end, back-end, and database.
• A Component diagram of the back-end container, highlighting things like authentication or payment components.
• Code diagrams wouldn’t be made initially – only dynamically created by IDEs as necessary.

After reading through the Hugo QuickStart guide and adjusting the installed theme, I created a new site that allows me to write the content using Emacs Org-mode.

The process I followed boiled down to:

• Create a sandbox folder and a new Hugo project
• Add a theme as a git submodule
• Add a Hugo Archetype for Org-mode files
• Create content!

I followed the specific process to get a new Hugo site using the Poison theme.

Create the project and add a theme

\# Create a sandbox for this site
mkdir -p ~/Desktop/hugo-sandbox && cd ~/Desktop/hugo-sandbox

# initialize a new hugo site
site\_slug="pp"
hugo new site ${site\_slug}
cd ${site\_slug}
git init --quiet && \\
  git branch -m "main" --quiet && \\
  git add . && \\
  git commit -m "Initial Commit from \\"hugo new site ${site\_slug}\\""

# Add the Poison theme along with minimal config to the config.toml file
git submodule add --quiet https://github.com/lukeorth/poison.git themes/poison
cat << EOF >> config.toml
theme = 'poison'

\[params\]
    brand = "PassionsPlay"
EOF

git add . && \\
  git commit -m "Add Poison Theme"

With that in place, we can start the dev server, but there won’t be much to look at until we get some actual content in place:

hugo server

Creating content

Following the Quickstart guide directs you to create a markdown file for your first piece of content.

hugo new posts/hello-world.md
cat << EOF >> content/posts/hello-world.md
# ~~Old~~ New
It's a new world, \*filled\* with \_markup\_.
EOF
cat content/posts/hello-world.md

Content "/home/benjamin/Desktop/hugo-sandbox/pp/content/posts/hello-world.md" created
---
title: "Hello World"
date: 2024-11-07T21:04:48-08:00
draft: true
---

# ~~Old~~ New
It's a new world, \*filled\* with \_markup\_.

That’s neat, and we can easily fire up our editor to add some actual content. But having the post set to “draft” (the draft: true part), none of that content will show unless you adjust how you launch the dev server. Since we started the dev server with hugo server, those draft posts won’t display. Stop the server with C-c and start a new instance that includes drafts with this command:

hugo server --buildDrafts

Org-mode Archetype

That looks good, but we want to write in Org-mode. We could just try using an org filename:

hugo new posts/hello-org-mode.org
cat content/posts/hello-org-mode.org

Content "/home/benjamin/Desktop/hugo-sandbox/new-blog/content/posts/hello-org-mode.org" created
---
title: "Hello Org Mode"
date: 2024-05-08T21:42:29-07:00
draft: true
---

Running the script created the file in the correct location. However, the Front Matter is using markdown! What we need is a Hugo Archetype for Org-mode files. Adding a file at archetypes/default.org means we can automatically generate the front matter using the same Hugo create command:

cat << EOF > archetypes/default.org
#+TITLE: {{ replace .Name "-" " " | title }}
#+DATE: {{ .Date }}
#+DRAFT: true
#+AUTHOR:
#+CATEGORY:
#+TAGS\[\]:
EOF
cat archetypes/default.org

#+TITLE: {{ replace .Name "-" " " | title }}
#+DATE: {{ .Date }}
#+DRAFT: true
#+AUTHOR:
#+CATEGORY:
#+TAGS\[\]:

With the archetypes/default.org file in place, we can create new posts, in org-mode format, by using the .org suffix:

hugo new posts/hello-org-mode-2.org
cat content/posts/hello-org-mode-2.org

Content "/home/benjamin/Desktop/hugo-sandbox/pp/content/posts/hello-org-mode-2.org" created
#+TITLE: Hello Org Mode 2
#+DATE: 2024-11-07T21:17:25-08:00
#+DRAFT: true
#+AUTHOR:
#+CATEGORY:
#+TAGS\[\]:

For example, here’s a quick script that uses chalk to colorize the args to a index.ts script. With Deno installed, it’s as simple as running with:

deno run --allow-env --allow-read index.ts The quick brown fox jumps over the lazy dog

import { parseArgs } from "jsr:@std/cli/parse-args";
import chalk from "npm:chalk@5.3.0";

const message = parseArgs(Deno.args).\_;

const colorMsg = (words: string\[\]): void => {
  let i = 0;
  // chalk, why you no support the Roy G. Biv!?
  const colors = \[
    "red",
    // "orange",
    "yellow",
    "green",
    "blue",
    // "indigo",
    // "violet",
  \];
  const coloredWords = words.map((word) => {
    const color = chalk\[colors\[i\]\];
    i = (i + 1) % colors.length;
    return color(word);
  });
  console.log(coloredWords.join(" "));
};
colorMsg(message);

Converting existing images to AVIF is easy to script on the command line using the right tool. On Debian-based systems, the process is easy using apt-get.

★  ~ % sudo apt install libavif-bin
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following additional packages will be installed:
  libavif13 libgav1-0 libyuv0
The following NEW packages will be installed:
  libavif-bin libavif13 libgav1-0 libyuv0

With the avifenc command installed, converting images is easy and results in impressive reductions in file size (625k -> 62k 🤯)!

★  Pictures/Screenshots % avifenc avif-image-uploaded-to-wordpress-media-library.png avif-image-uploaded-to-wordpress-media-library.avif
Successfully loaded: avif-image-uploaded-to-wordpress-media-library.png
AVIF to be written: (Lossy)
 * Resolution     : 1600x900
 * Bit Depth      : 8
 * Format         : YUV444
 * Alpha          : Not premultiplied
 * Range          : Full
 * Color Primaries: 1
 * Transfer Char. : 13
 * Matrix Coeffs. : 6
 * ICC Profile    : Absent (0 bytes)
 * XMP Metadata   : Absent (0 bytes)
 * EXIF Metadata  : Absent (0 bytes)
 * Transformations: None
 * Progressive    : Unavailable
Encoding with AV1 codec 'aom' speed [6], color QP [24 (Medium) <-> 26 (Medium)], alpha QP [0 (Lossless) <-> 0 (Lossless)], tileRowsLog2 [0], tileColsLog2 [0], 1 worker thread(s), please wait...
Encoded successfully.
 * Color AV1 total size: 62961 bytes
 * Alpha AV1 total size: 0 bytes
Wrote AVIF: avif-image-uploaded-to-wordpress-media-library.avif
★  Pictures/Screenshots % ls -alh avif*
-rw-rw-r-- 1 benjamin benjamin  62K Jul 27 11:49 avif-image-uploaded-to-wordpress-media-library.avif
-rw-rw-r-- 1 benjamin benjamin 645K Jul 27 11:43 avif-image-uploaded-to-wordpress-media-library.png

What is AVIF?

AVIF (AV1 Image File Format) is a modern image format that offers superior compression and quality compared to traditional formats like JPEG and PNG. It’s based on the AV1 video codec and can significantly reduce image file sizes without compromising quality. This means faster load times and a better user experience on your site.

AVIF is often associated with and compared to HEIF, another high-compression image format. Where AVIF shines is in its open licensing, as opposed to HEIF, which is closed and requires royalty payments to use. This distinction is especially important as it relates to WordPress, which can’t adopt closed solutions into WordPress core.

Benefits of AVIF Support

Here are some compelling reasons to consider AVIF for your WordPress site:

• Better Compression: AVIF images are smaller than JPEG and PNG, which means faster load times.
• Higher Quality: AVIF maintains high image quality despite the smaller file sizes.
• Future-Proofing: As more browsers and devices support AVIF, enabling it ensures your site is ready for the future.

This blog post from the Alliance for Open Media goes into way more detail about the benefits: AVIF: Meet the Next Level Image File Format

AVIF in WordPress 6.5

With WordPress 6.5, AVIF support is baked right into the core. This means you can upload AVIF images directly to your Media Library, and WordPress will handle them just like any other image format.

For server environments with PHP compiled with AVIF support, WordPress can also edit those images to scale, crop, and rotate.

By supporting AVIF, WordPress sites can use smaller, better-quality images, which improve performance and reduce the amount of data that site visitors need to download.

You can read more about AVIF’s inclusion within the 6.5 WordPress release on the WordPress Core blog.

WP’s Use of AVIF

WordPress’s emphasis on backward compatibility means that the features it supports are written using robust and defensive coding practices.

One consequence is that even if your server doesn’t support AVIF, WordPress includes a polyfill to help read AVIF images, allowing you to upload and add AVIF images to your Media Library.

However, this polyfill does come with limitations. While you can upload and insert AVIF images into posts, editing those images is impossible without server support.

For these reasons, ensuring your PHP setup is AVIF-ready is crucial. Most dedicated hosts have already compiled PHP to include support for AVIF. If you’re like me and tasked with getting this working, I’ll go into more detail in upcoming posts about compiling PHP with AVIF support.

Having an actual Typescript buffer with the full power of my editor is the most natural way of turning thought into experiment and feels like such an improvement over tinkering in the Node REPL.

In particular, I have so much muscle memory around Emacs’ compilation workflow that I found myself doing the same few things to get a sandbox up and running for working with Typescript.

So, after creating yet another temporary folder, with a *.ts file, saving the file, and compiling/running this file, I decided to automate this workflow for creating these sandboxes.

My Elisp isn’t great, but for a hacky time-saver, it came together fairly quickly:

(defun bt/create-typescript-sandbox ()
  "Create a temporary Typescript file and set a local variable for
the compile-command."
  (interactive)
  (let ((folder (make-temp-file "typescript-sandbox-" t)))
    (find-file (concat folder "/index.ts"))
    (typescript-mode)
    (set (make-local-variable 'compile-command) (format "deno run %s" (buffer-file-name)))))

The rough outline of what’s going on:

• Create a folder in the system’s temporary folder location, for example /tmp/typescript-sandbox-xxxyyyzz
• Create a file called index.ts within the temporary folder
• Set the major mode to typescript-mode
• Set a local variable for the compile command so that it offers to run deno run on the newly created buffer

From there, it’s just a quick call to the function to get a Typescript file that compiles and runs quickly (thanks Deno!)

Compiling Emacs from scratch on Linux was a new venture, but it was straightforward. One unfamiliar term that popped up was make bootstrap. In Emacs’ case, this essentially wipes the slate clean, deleting any pre-compiled Elisp modules before building from scratch:

make bootstrap – delete all compiled files to force a new bootstrap from a clean slate, then build in the normal way

Here’s a condensed version of the process I followed:

# Setting up Emacs in our source directory
mkdir -p ~/src && cd ~/src
git clone git://git.savannah.gnu.org/emacs.git
git checkout emacs-29

# Enable development libraries and update apt cache
sudo sed -i 's/# deb-src/deb-src/' /etc/apt/sources.list \
  && apt update

# Install necessary dependencies
sudo apt build-dep -y emacs \
  && sudo apt install libtree-sitter-dev

# Generate the configure file
./autogen.sh

# Configure Emacs with desired options
./configure --with-tree-sitter --with-imagemagick --with-json

# Compile with 4 cores
make -j4 bootstrap

# Verify the version
./src/emacs --version

# Optionally, test things by launching Emacs without any configuration
./src/emacs -Q

# When things look good, install Emacs system-wide
sudo make install

There are several ways to install OBS Studio on Linux. One of the easiest methods is through the official OBS Studio PPA (Personal Package Archive) for Ubuntu-based Linux distributions. Here are the general steps to install OBS Studio via the PPA:

1. Open the terminal and add the OBS Studio PPA to your system by running the following command
2. Update your system’s package list by running
3. Finally, install OBS Studio

Those steps look like this within your terminal:

sudo add-apt-repository ppa:obsproject/obs-studio
sudo apt-get update
sudo apt-get install obs-studio

Once the installation is complete, you can launch OBS Studio from the Applications menu or by running the following command in the terminal:

obs

That’s it! You now have OBS Studio installed on your Linux system.

Refer to the OBS Studio documentation or seek help from the OBS Studio community forums if you have any issues.

Why use a PPA?

You may have noticed that OBS Studio is available from the default software repositories for Ubuntu. So why go through the above effort of adding a PPA?

Adding a PPA (Personal Package Archive) is a way to get the latest package version that is not yet available in the official Ubuntu repository. Installing software from a PPA can be a good option for those who want the latest features and improvements before they are officially released.

You can use the apt-cache policy command to view the various software versions associated with different repositories. Here’s how to use to compare the OBS Studio version from the official Ubuntu repositories and the official OBS Studio PPA:

apt-cache policy obs-studio

The above command gives us information that looks like this:

★  Videos/obs % apt-cache policy obs-studio
obs-studio:
  Installed: 29.0.2-0obsproject1~jammy
  Candidate: 29.0.2-0obsproject1~jammy
  Version table:
 \*\*\* 29.0.2-0obsproject1~jammy 500
    500 https://ppa.launchpadcontent.net/obsproject/obs-studio/ubuntu jammy/main amd64 Packages
    100 /var/lib/dpkg/status
     27.2.3+dfsg1-1 500
    500 http://apt.pop-os.org/ubuntu jammy/universe amd64 Packages

Note that as of today (2023-03-26), the version of OBS Studio from the official PPA is two whole versions ahead of what comes from the central Ubuntu repositories (29.0.2 vs. 27.2.3)

• How to share terminal demos as razor-sharp animated SVG — Wasim Lorgat

# install tools
npm install -g svg-term-cli
pipx install asciinema
# start recording
asciinema rec emacs.cast
# do stuff
# stop recording with ctrl-d
# convert to svg
svg-term --in emacs.cast --out emacs.svg --window --width 141 --height 22 --no-optimize

Questions

Did the experience crash at any point?

• No crashes! But a few odd states when navigating via keyboard nav.

Did the saving experience work properly?

• Yes

What did you find particularly confusing or frustrating about the experience?

• While writing posts in Gutenberg, I’ve come to love being able to press enter, then forward slash to choose a new block quickly without leaving the keyboard. Under FSE’s editor, this workflow seems more buggy.
• Performing some of the actions would lose focus. As a concrete example, removing any lock setting via keyboard nav seems to drop the user back into the global context, needing to tab to the location they were previously at. I think that’s likely due to the lock DOM element no longer being present.
• Working with a group’s layout could be improved via the keyboard. I think getting more comfortable with the UI would help, but having additional feedback around what is selected and where I can move things would be helpful.

What did you especially enjoy or appreciate about the experience?

• So cool to see how far FSE has come and all of the improvements related to templates. We’re getting closer to the power of the template hierarchy of traditional themes!

What would have made this experience easier?

• This was great! I love how these FSE experiments are targeted and have clear instructions for me to jump in and do some QA as time permits!

Did you find that what you created matched what you saw on your site?

• Yes

Did it work using Keyboard only?

• Not quite.

Did it work using a screen reader?

• Didn’t test.

Video and Timestamps

• 0:00:00 :: Start recording notes
• 0:01:43 :: Minor thing — when tabbing to the main editor area, the editor area jumped up 10px
• 0:03:56 :: Question: Should I be able to navigate through all of the “Patterns” using the arrow keys when inserting a footer?
• 0:11:34 :: UX consideration: When selecting an option for the Query Loop block, it would be nice to have the left and right arrows (or similar) be able to advance through the various options.
• 0:20:19 :: Usually when pressing “enter” at the end of a block, a new block will be inserted after the current one. For some reason, this “Directions” block won’t insert a new block after it when pressing enter.
• 0:22:53 :: Bug: Tabindex lost when lock icon is removed from the block menu.
  1. With a locked block focused in the editor
  2. Navigate to the lock setting with keyboard nav using “Shift+Tab”
  3. Disable lock settings.
  4. Note that our focus is now on the main window and not somewhere within the block we were editing.
• 0:26:05 :: Re the “enter” for new block issue. As a workaround, I think I can hit enter twice, and then backspace once to get to a new block setup.
• 0:36:15 :: Another instance of not being able to insert a new block by just pressing enter
• 0:40:15 :: Editing the layout of a group is a little tricky. I think getting more comfortable with the UI would help, but having additional feedback around what is selected and where I can move things would be helpful.

Because Local is an Electron app, I pictured myself learning and hacking away at Javascript. While I’ve been learning more about the nuances of Typescript and React in this role, the first couple of sprints were spent investigating an ERR_CERT_INVALID error in Chrome. The specific issue was first reported in the Local Community Forums: SSL Not working on Chrome with NET::ERR_CERT_INVALID

This sort of error was difficult to troubleshoot for a few reasons:

1. Chrome doesn’t tell you what exactly is invalid about a certain certificate.
2. Searching the internet for more info about this error led me to results like the answers in this StackOverflow question, which basically boil down to bypassing HTTPS:
   • No “Proceed Anyway” option on NET::ERR_CERT_INVALID in Chrome on MacOS

Bypassing security isn’t a solution; it’s a temporary workaround and certainly isn’t an option when you’re the dev who’s making a tool that generates SSL certificates.

I learned a lot while working on this issue and thought it best to document some of the tools and resources I found while investigating this error!

An example of the ERR_CERT_INVALID error in Chrome.

Not all ERR_CERT_INVALID errors are the same.

Because there are many potential reasons for a certificate to be invalid, there isn’t an easy-to-find solution for every case. I couldn’t find a definitive list of what Chrome looks for, but the main reason seems to be incorrect dates.

Invalid Cert due to Invalid Dates

When a certificate is generated, it has a Valid Before and a Valid After date:

Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number:
            9b:6b:3d:a3:b9:a3:a4:b4
    Signature Algorithm: sha256WithRSAEncryption
        Issuer: CN=local.mydomain.com
        Validity
            Not Before: Nov 19 13:27:30 2016 GMT
            Not After : Nov 19 13:27:30 2017 GMT
        Subject: CN=local.mydomain.com

The above cert will certainly fail on most systems today because most systems think they are ahead of 2017.

Note that I mentioned “most systems think” — remember that a computer needs to set its notion of time from someplace. If the dates for a certificate seem valid, one possible avenue to explore is if a computer’s Date/Time settings are correct. Pay particular attention to if a system is oddly connecting to the network (like a VPN or a “helpful-IT” work network.)

Something about the chain is invalid.

I know that heading isn’t helpful, but I encourage you to think creatively about what sort of things would make a browser angry. In my case, the issue was caused by a doubling up of the certificate. In particular, Chrome was reporting two of the same PEM blocks.

In my case, I zeroed in on why Chrome was complaining but now needed to determine how the sites came to be in this state.

Note the doubled-up PEM certificate.

Tools for Investigation

Investigating an SSL Certificate using Chrome

I was able to get a general idea of why Chrome didn’t like the site’s certificate by clicking the area to the left of the address bar. You’ll see a lock icon for sites where SSL is working correctly. Sites with miss-configured SSL certificates will be obvious — Chrome won’t let you do anything!

This screencast covers the basics of examining the certificate and drilling down to specific pieces of data. In particular, note that:

• You can click the icon next to the address bar for a window to drill down to specifics.
• For most SSL-related errors, an “Advanced” button will show additional details about what went wrong.
• The actual error (in this case ERR_CERT_INVALID) is clickable and allows you to see the raw PEM-encoded certificate.

Investigating a certificate using OpenSSL

I like polished GUIs like anyone else, but Chrome’s certificate navigator wasn’t doing what I needed to do. At this point, I turned to OpenSSL to query and decode the certificate that the server was giving. Here’s a breakdown of the commands that helped me along the way.

Using OpenSSL as a TLS Client

openssl s_client -showcerts -connect example.local:443  -servername example.local

This command allows you to make and test a connection over HTTPS using OpenSSL. You can read the parts of the above command as:

• openssl s_client — Use s_client, which is OpenSSL’s generic SSL/TLS client for connecting to hosts.
• -showcerts — Include the server’s certificate in the output.
• -connect example.local:443 — Where the connection should be made. In this case, specifying the specific HTTPS port ensures that we are working with a secure connection.
• -servername example.local — This indicates what domain you want the cert for. This is most useful when the server responds to more than one domain.

The above command should give an output similar to this:

depth=0 CN = invalid-cert-c1.local, C = XX, ST = XX, L = Fake Locality, O = "Super Fake Company, Fake.", OU = Fake Organizational Unit
verify error:num=18:self signed certificate
verify return:1
depth=0 CN = invalid-cert-c1.local, C = XX, ST = XX, L = Fake Locality, O = "Super Fake Company, Fake.", OU = Fake Organizational Unit
verify return:1
CONNECTED(00000005)
---
Certificate chain
0 s:/CN=invalid-cert-c1.local/C=XX/ST=XX/L=Fake Locality/O=Super Fake Company, Fake./OU=Fake Organizational Unit
    i:/CN=invalid-cert-c1.local/C=XX/ST=XX/L=Fake Locality/O=Super Fake Company, Fake./OU=Fake Organizational Unit
-----BEGIN CERTIFICATE-----
MIIEfTCCA2WgAwIBAgIJHCV7Ifs31UxeMA0GCSqGSIb3DQEBCwUAMIGZMR4wHAYD
VQQDExVpbnZhbGlkLWNlcnQtYzEubG9jYWwxCzAJBgNVBAYTAlhYMQswCQYDVQQI
EwJYWDEWMBQGA1UEBxMNRmFrZSBMb2NhbGl0eTEiMCAGA1UEChMZU3VwZXIgRmFr
ZSBDb21wYW55LCBGYWtlLjEhMB8GA1UECxMYRmFrZSBPcmdhbml6YXRpb25hbCBV
bml0MB4XDTIyMDIwNDE2MDUxMFoXDTMyMDIwNDE2MDUxMFowgZkxHjAcBgNVBAMT
FWludmFsaWQtY2VydC1jMS5sb2NhbDELMAkGA1UEBhMCWFgxCzAJBgNVBAgTAlhY
MRYwFAYDVQQHEw1GYWtlIExvY2FsaXR5MSIwIAYDVQQKExlTdXBlciBGYWtlIENv
bXBhbnksIEZha2UuMSEwHwYDVQQLExhGYWtlIE9yZ2FuaXphdGlvbmFsIFVuaXQw
ggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQC/BGKuaQTetBNytk54co3u
E7/hbK5b2PnwpWBO7j37+qpqtiyh5iy5RMlXcjEpnTj5oeSGuZppEa0onDO04rFo
Yz9PLawcGlH2NgZqNqAHczVEl5Q5wElqAQNCnXRA3bjvgWzV2VU5bTgVwIMF3ppw
J6eVPfrxT7GBzVzjAXdOMlJVNLETq+yiniEkN1+nEmc2tIq3ypN4YQJlwNryQ6e6
2zEebgnaPDQaQyDkEeOpQM+EYqeqpDjRqR46rG3gGRALlJs1kKigNAI4ozk0eZQK
/bTRNCtIOjcjcTu7wwQNRgqWoeFLR3+d9qA/Sdq0d2xC8tOC/7ua+u1pYYNMIJu1
AgMBAAGjgcUwgcIwCQYDVR0TBAIwADALBgNVHQ8EBAMCAvQwOwYDVR0lBDQwMgYI
KwYBBQUHAwEGCCsGAQUFBwMCBggrBgEFBQcDAwYIKwYBBQUHAwQGCCsGAQUFBwMI
MBEGCWCGSAGG+EIBAQQEAwIA9zAdBgNVHQ4EFgQUssqVADjlsz0/t+bpBK1j2+qy
2hwwOQYDVR0RBDIwMIIVaW52YWxpZC1jZXJ0LWMxLmxvY2FsghcqLmludmFsaWQt
Y2VydC1jMS5sb2NhbDANBgkqhkiG9w0BAQsFAAOCAQEAsjSC8Vlndbz58eZqd/7/
f7j86m3NjZxVzzOKMXWSAnoY+3Uy1z0DT+u4uxDMuY6qx+OskmsUW4nVlidicpye
T2EZUQuykCTJh/aqP6wdPoPuNJ0kYaG+cq/b+T1gDG0eVWT1jER3PmPl1hpOdWpd
kzY1ErUQjN+hQXQ4a31rNUkS7hdMOsu1l+Xwz5n21SkAPc14rIoQfssny0SnCvCQ
kV27Pm5OSAgybmVsD/2WnbIC5aEgcGX2rVgLk4kOB+25REXbwkm1KKggW+1Ae5qN
ptEZ69kacQj2DZxcNyBZEBZbSu9EUFbvrtHkj+boQcZ7AKYdgWGyq8EkuI+okLAl
CA==
-----END CERTIFICATE-----
---
Server certificate
subject=/CN=invalid-cert-c1.local/C=XX/ST=XX/L=Fake Locality/O=Super Fake Company, Fake./OU=Fake Organizational Unit
issuer=/CN=invalid-cert-c1.local/C=XX/ST=XX/L=Fake Locality/O=Super Fake Company, Fake./OU=Fake Organizational Unit
---
No client certificate CA names sent
Server Temp Key: ECDH, P-256, 256 bits
---
SSL handshake has read 1859 bytes and written 352 bytes
---
New, TLSv1/SSLv3, Cipher is ECDHE-RSA-AES128-GCM-SHA256
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : ECDHE-RSA-AES128-GCM-SHA256
    Session-ID: BA84087FF3DEA801A256A4F0B89D30DDA63098C995B3C1D8770215A2B1DC29D7
    Session-ID-ctx: 
    Master-Key: BF6930C37EABB95DFD70A50780956197743BC23BBE6BA3FCAACA8452C36236365734CE4C7244A6110763675CF954CF64
    TLS session ticket lifetime hint: 300 (seconds)
    TLS session ticket:
    0000 - c7 48 17 29 cf 87 1f 51-0c d2 f8 78 cc 64 6e 9e   .H.)...Q...x.dn.
    0010 - 7f b7 c6 52 cb 71 48 aa-5a a0 54 13 44 7c c2 ba   ...R.qH.Z.T.D|..
    0020 - 00 8e 61 d0 d8 42 21 67-4b 3f 0c 87 64 c0 64 48   ..a..B!gK?..d.dH
    0030 - 91 73 90 30 5f 25 4f 89-56 ea a6 55 85 4f c8 3d   .s.0_%O.V..U.O.=
    0040 - 89 18 b2 bd 81 c3 a9 62-12 7d 76 51 6b 5f 33 90   .......b.}vQk_3.
    0050 - 2b 38 45 38 ce d2 af 91-cf 49 4f 99 0a d9 6c 4c   +8E8.....IO...lL
    0060 - 93 8a 79 fb be 77 03 d8-1a b4 08 bf 77 27 36 b5   ..y..w......w'6.
    0070 - 2e 98 bd 2c 95 e9 ba 79-9f 05 95 0b e6 3c e2 43   ...,...y.....<.C
    0080 - f1 64 b5 25 00 8c f8 b4-3d 01 3f 87 5f dd 97 b4   .d.%....=.?._...
    0090 - f7 12 61 ff 61 c4 39 13-5c fc dc 1d fa 37 11 0f   ..a.a.9.\....7..
    00a0 - c5 d4 10 df cf 9a 87 05-c3 eb 6e 19 97 f6 1f d1   ..........n.....
    00b0 - b6 2b 0b 66 f0 a1 f9 fa-7a ce c1 3d 5a 77 80 48   .+.f....z..=Zw.H
    00c0 - c4 f8 a8 5f 65 f2 7e c3-f9 a3 06 e8 e7 a8 87 be   ..._e.~.........

    Start Time: 1644343457
    Timeout   : 7200 (sec)
    Verify return code: 0 (ok)
---
DONE

Verifying a certificate with OpenSSL

openssl verify -purpose any example.local.pem

Making a secure connection using the s_client command is useful, but what if you wanted to verify and work with the certificate from the above output? The above openssl verify command examines a certificate and determines if it is valid.

As a concrete example, you can save the PEM-encoded certificate in the above output to a file and receive output that looks something like this:

★  openssl verify -purpose any daniela.pem
daniela.pem: CN = danielaorg.local, C = XX, ST = XX, L = Fake Locality, O = "Super Fake Company, Fake.", OU = Fake Organizational Unit
error 18 at 0 depth lookup:self signed certificate
OK

For me, on my journey of understanding ERR_CERT_INVALID, verifying the cert ended up not helping. It confirmed what I knew, which was that this certificate was self-signed. But my next question was: “What exactly is in that certificate?”

Viewing the certificate contents with OpenSSL

openssl x509 -text -in example.local.pem

The above command decrypts the PEM-formated cert to get something more human-readable. Calling out a few items specifically:

1. Lines 8-10 — Date validity. Double-check that the certificate is for a date range that makes sense.
2. Lines 7,11 — The cert’s Subject and Issuer fields are usually different “things”. In the output below, they are the same organization, which implies that this is a self-signed certificate.
3. Lines 36-37 — CA:FALSE means that this certificate can’t be considered a Certificate Authority. This ended up being the source of my bug. The certificate wasn’t authorized to be a Certificate Authority, but it was a self-signed cert. Making it so these certificates could be considered CAs made Chrome stop complaining!
4. Lines 39,41 — These lines detail how this certificate can be used. In the previous openssl verify command, we used the -purpose any flag to ask if this certificate was verifiable for ANY use.

Certificate:
    Data:
	Version: 3 (0x2)
	Serial Number:
	    59:c4:b6:1f:0a:8c:81:f6:5b
    Signature Algorithm: sha256WithRSAEncryption
	Issuer: CN=brayanname.local, C=XX, ST=XX, L=Fake Locality, O=Super Fake Company, Fake., OU=Fake Organizational Unit
	Validity
	    Not Before: Feb 15 22:41:37 2022 GMT
	    Not After : Feb 15 22:41:37 2032 GMT
	Subject: CN=brayanname.local, C=XX, ST=XX, L=Fake Locality, O=Super Fake Company, Fake., OU=Fake Organizational Unit
	Subject Public Key Info:
	    Public Key Algorithm: rsaEncryption
		Public-Key: (2048 bit)
		Modulus:
		    00:e8:08:84:51:84:fb:f0:97:4a:c2:4c:d7:87:0e:
		    d5:10:57:41:06:92:08:8a:cd:9e:8e:a1:59:ef:b6:
		    d0:8b:22:a5:1b:ea:4d:77:7d:24:59:d2:1e:2a:54:
		    1d:9a:7f:d2:92:a6:0a:4e:33:63:69:fb:f5:1a:3a:
		    41:0e:11:34:7f:42:62:58:58:ee:a4:f9:88:a1:ed:
		    bf:ba:9e:b3:b9:00:16:77:6c:92:5b:ed:bf:ed:79:
		    89:7b:cd:cc:26:b5:fb:ce:d2:4f:9a:1b:18:01:2b:
		    02:65:f7:f6:90:7e:22:19:ff:f7:d4:3c:8c:09:c8:
		    4f:d1:84:ac:a2:1c:ae:55:1a:87:73:79:63:c2:8d:
		    6b:57:fa:9f:cc:c1:e4:5a:3c:ef:38:17:76:3c:5f:
		    c3:bc:b9:c5:0f:50:dd:4a:7e:bc:00:bc:ca:64:53:
		    af:b1:8d:d2:ad:2e:b6:47:0a:27:a7:03:7b:bb:c0:
		    f4:74:f9:a5:88:9f:ff:59:aa:e1:02:31:ef:db:04:
		    dc:34:a2:49:d9:8f:c8:87:8c:6a:0a:2c:ee:8f:34:
		    b3:7c:d2:98:b5:9e:26:38:8c:33:7c:ab:3b:1f:e7:
		    1d:55:0b:be:76:ac:00:9a:c5:01:f1:1b:47:67:90:
		    d4:37:c2:64:24:38:4e:0c:94:14:ce:3d:a1:77:68:
		    51:d1
		Exponent: 65537 (0x10001)
	X509v3 extensions:
	    X509v3 Basic Constraints: 
		CA:FALSE
	    X509v3 Key Usage: 
		Digital Signature, Non Repudiation, Key Encipherment, Data Encipherment, Certificate Sign
	    X509v3 Extended Key Usage: 
		TLS Web Server Authentication, TLS Web Client Authentication, Code Signing, E-mail Protection, Time Stamping
	    Netscape Cert Type: 
		SSL Client, SSL Server, S/MIME, Object Signing, SSL CA, S/MIME CA, Object Signing CA
	    X509v3 Subject Key Identifier: 
		05:CC:36:C8:0C:42:06:58:5D:DF:DC:47:58:3D:B3:A7:C6:30:75:06
	    X509v3 Subject Alternative Name: 
		DNS:brayanname.local, DNS:*.brayanname.local
    Signature Algorithm: sha256WithRSAEncryption
	 5b:d9:bf:b8:53:7d:2c:48:8c:f4:01:3b:6f:2a:e6:06:c0:07:
	 13:60:82:02:f7:f8:f0:88:4d:c4:20:ca:c3:77:e9:29:21:ea:
	 5b:87:1e:57:12:54:51:c5:ef:32:77:06:3b:78:f8:bd:d7:a9:
	 b4:e2:5c:34:bd:b1:af:b3:ce:54:b3:c1:7a:7e:d9:b6:d4:0a:
	 45:ef:00:6b:78:30:ca:89:f8:f7:d5:91:5c:2d:c6:0f:ce:69:
	 f4:3f:98:17:c2:43:8f:9a:5b:e3:dd:46:81:07:0b:5e:a5:40:
	 fb:81:28:7b:90:20:3a:8d:37:01:d1:93:16:9d:fd:e8:e5:fd:
	 9d:73:b4:5b:73:f7:e6:89:50:ca:74:90:4d:c6:d0:1b:f5:c7:
	 d6:65:12:2c:f2:63:cb:74:34:64:ad:1e:dc:49:9c:c8:74:56:
	 f2:3b:92:df:2f:12:a8:38:5b:22:f0:96:ef:e4:8a:39:b3:64:
	 ed:4f:67:c7:db:38:74:64:58:2c:cc:3b:f9:75:ee:66:81:7c:
	 44:88:07:a9:cf:4c:1c:03:9c:38:90:5e:31:97:ba:2f:5e:0e:
	 30:db:8c:16:5b:01:ec:c3:0d:7d:92:88:da:1a:97:fa:59:d9:
	 b5:9e:4e:b7:5e:7c:20:5d:df:77:6c:8c:73:ff:f4:59:6a:e3:
	 a8:d6:be:02
-----BEGIN CERTIFICATE-----
MIIEaTCCA1GgAwIBAgIJWcS2HwqMgfZbMA0GCSqGSIb3DQEBCwUAMIGUMRkwFwYD
VQQDExBicmF5YW5uYW1lLmxvY2FsMQswCQYDVQQGEwJYWDELMAkGA1UECBMCWFgx
FjAUBgNVBAcTDUZha2UgTG9jYWxpdHkxIjAgBgNVBAoTGVN1cGVyIEZha2UgQ29t
cGFueSwgRmFrZS4xITAfBgNVBAsTGEZha2UgT3JnYW5pemF0aW9uYWwgVW5pdDAe
Fw0yMjAyMTUyMjQxMzdaFw0zMjAyMTUyMjQxMzdaMIGUMRkwFwYDVQQDExBicmF5
YW5uYW1lLmxvY2FsMQswCQYDVQQGEwJYWDELMAkGA1UECBMCWFgxFjAUBgNVBAcT
DUZha2UgTG9jYWxpdHkxIjAgBgNVBAoTGVN1cGVyIEZha2UgQ29tcGFueSwgRmFr
ZS4xITAfBgNVBAsTGEZha2UgT3JnYW5pemF0aW9uYWwgVW5pdDCCASIwDQYJKoZI
hvcNAQEBBQADggEPADCCAQoCggEBAOgIhFGE+/CXSsJM14cO1RBXQQaSCIrNno6h
We+20IsipRvqTXd9JFnSHipUHZp/0pKmCk4zY2n79Ro6QQ4RNH9CYlhY7qT5iKHt
v7qes7kAFndsklvtv+15iXvNzCa1+87ST5obGAErAmX39pB+Ihn/99Q8jAnIT9GE
rKIcrlUah3N5Y8KNa1f6n8zB5Fo87zgXdjxfw7y5xQ9Q3Up+vAC8ymRTr7GN0q0u
tkcKJ6cDe7vA9HT5pYif/1mq4QIx79sE3DSiSdmPyIeMagos7o80s3zSmLWeJjiM
M3yrOx/nHVULvnasAJrFAfEbR2eQ1DfCZCQ4TgyUFM49oXdoUdECAwEAAaOBuzCB
uDAJBgNVHRMEAjAAMAsGA1UdDwQEAwIC9DA7BgNVHSUENDAyBggrBgEFBQcDAQYI
KwYBBQUHAwIGCCsGAQUFBwMDBggrBgEFBQcDBAYIKwYBBQUHAwgwEQYJYIZIAYb4
QgEBBAQDAgD3MB0GA1UdDgQWBBQFzDbIDEIGWF3f3EdYPbOnxjB1BjAvBgNVHREE
KDAmghBicmF5YW5uYW1lLmxvY2FsghIqLmJyYXlhbm5hbWUubG9jYWwwDQYJKoZI
hvcNAQELBQADggEBAFvZv7hTfSxIjPQBO28q5gbABxNgggL3+PCITcQgysN36Skh
6luHHlcSVFHF7zJ3Bjt4+L3XqbTiXDS9sa+zzlSzwXp+2bbUCkXvAGt4MMqJ+PfV
kVwtxg/OafQ/mBfCQ4+aW+PdRoEHC16lQPuBKHuQIDqNNwHRkxad/ejl/Z1ztFtz
9+aJUMp0kE3G0Bv1x9ZlEizyY8t0NGStHtxJnMh0VvI7kt8vEqg4WyLwlu/kijmz
ZO1PZ8fbOHRkWCzMO/l17maBfESIB6nPTBwDnDiQXjGXui9eDjDbjBZbAezDDX2S
iNoal/pZ2bWeTrdefCBd33dsjHP/9Flq46jWvgI=
-----END CERTIFICATE-----

Resources

SSL Terms

When I was first learning about how the various SSL parts fit together, I found this StackOverflow answer to be excellent in unpacking the jargon:

• What is a Pem file and how does it differ from other OpenSSL Generated Key File Formats?

Other Links

• https://prateeknischal.github.io/posts/trust-the-certs/
  • Great post that goes into detail about using openssl and how to understand the output that OpenSSL gives you.
• StackExchange: Does openssl refuse self signed certificates without basic constraints?
  • This was the main resource that keyed me into the idea that a doubled-up certificate chain would require that certificate to be a CA (certificate authority).

Coming into this role is an exciting transition for me. I’ve worked in technology for over nine years, many spent doing “WordPress developer” things. That means wearing many hats, from code-wrangler to content creator to SEO strategist – when you WordPress, you ensure you have a voice on the web.

So what’s different? Well, as a Software Engineer, my day-to-day work is different than the WordPress sites created while working with Peaceful Media. It’s also less focused on the communication and leadership skills I improved while working in the Flywheel support organization. And I’m not doing as much technical writing or presenting like I was as the Local Community Manager.

Building Local means crafting Javascript (it is built on Electron), but it also means being proficient in other things too:

• Compiling and configuring Lightning Services (i.e., native binaries) for various pieces of server software like Apache, Nginx, PHP, and MySQL.
• Working with HTTPS (along with TLS & SSL) and figuring out how those pieces of tech work under the hood.
• Knowing the quirks of WordPress and making the development tool I wish I had when initially doing agency work.

So yeah, many of these topics aren’t entirely new to me, but building with Electron definitely is. Also, building with Javascript is unique, with the frameworks and community constantly changing. And I’ve never done this sort of development in Emacs.

There’s much to learn, but that’s what this new role and the new year are for. Learning and improving my workflow, refining my craft as a developer, and writing about the process!

We found ourselves troubleshooting a potential firewall issue that was affecting only a certain subset of users. There seemed like there was a pattern, but what exactly?

• Was it location-based?
• Was it time-based?
• Internet Provider based?
• Something else?

The only lead we had was that changing to a VPN magically fixed things for the end-users. As a next step, we thought we would examine the metadata associated with the set of IPs that were having problems.

So what do you do when you need to gather lots of info for something you’ve never examined before?

Turn to the terminal and script it!

The basics

There’s probably any number of services we could turn to, but ipinfo.io seemed like a good one. A query to get metadata about an IP address looks like this:

curl http://ipinfo.io/104.57.176.115

{
    "ip": "104.57.176.115",
    "hostname": "104-57-176-115.lightspeed.austtx.sbcglobal.net",
    "city": "Austin",
    "region": "Texas",
    "country": "US",
    "loc": "30.2672,-97.7431",
    "org": "AS7018 AT&T Services, Inc.",
    "postal": "78701",
    "timezone": "America/Chicago",
    "readme": "https://ipinfo.io/missingauth"
  }

Iterating over many IPs

That’s good for one-off requests, and it also means that we can improve on it to handle a collection of IPs. For example, if a file named ip-list has IP addresses on each line, we can use a BASH while loop to fetch more info about each IP address.

\# in a file called ip-list
102.176.65.190
104.219.136.17
104.57.176.115
104.57.29.69
107.190.28.137
109.48.146.95
112.134.164.187
116.97.53.100
12.90.145.30
121.200.4.138

We use a while loop to get more info!

( while read ip;
do curl -s "http://ipinfo.io/${ip}";
done < ips-list ) > ip-info.json

This works and ultimately got me what I needed since I was able to then use jq to investigate various things within the ip-info.json file:

cat ip-info.json | jq -r '\[.ip, .timezone, .country, .org\] | @tsv'

102.176.65.190	Africa/Accra	GH	AS29614 Ghana Telecommunications Company Limited
104.219.136.17	America/Chicago	US	AS14143 Rock Solid Internet & Telephone
104.57.176.115	America/Chicago	US	AS7018 AT&T Services, Inc.
104.57.29.69	America/New\_York	US	AS7018 AT&T Services, Inc.
107.190.28.137	America/Vancouver	CA	AS5645 TekSavvy Solutions, Inc.
109.48.146.95	Europe/Lisbon	PT	AS2860 NOS COMUNICACOES, S.A.
112.134.164.187	Asia/Colombo	LK	AS9329 Sri Lanka Telecom Internet
116.97.53.100	Asia/Ho\_Chi\_Minh	VN	AS7552 Viettel Group
12.90.145.30	America/Chicago	US	AS7018 AT&T Services, Inc.
121.200.4.138	Australia/Melbourne	AU	AS4764 Aussie Broadband
123.243.115.186	Australia/Sydney	AU	AS7545 TPG Telecom Limited
124.186.3.85	Australia/Brisbane	AU	AS1221 Telstra Corporation Ltd
134.56.52.132	America/New\_York	US	AS23089 Hotwire Communications
136.35.164.218	America/Chicago	US	AS16591 Google Fiber Inc.
136.49.143.29	America/Chicago	US	AS16591 Google Fiber Inc.
136.49.143.31	America/Chicago	US	AS16591 Google Fiber Inc.
136.58.116.229	America/Chicago	US	AS16591 Google Fiber Inc.

Make it reusable

The above process was good enough for my needs at that moment, but I wondered if there was a way to make something that was reusable. I pictured the ability to run one command followed by any number of IP addresses which then prints out more information about each IP address in JSON format.

Something like this:

ipinfo 102.176.65.190 104.219.136.17 104.57.176.115

This ended up being fairly easy to create!

Since those arguments are interpreted by Bash as an array of values, we can do a for loop with a quick regex validation to make sure it’s valid IPv4 address before making our request to ipinfo.

#!/bin/bash
# Save and name this script something like ipinfo and put it in your $PATH
for arg; do
    # Only make requests for valid IPv4 addresses
    if \[\[ $arg =~ ^\[0-9\]{1,3}\\.\[0-9\]{1,3}\\.\[0-9\]{1,3}\\.\[0-9\]{1,3}$ \]\]; then
	curl -s "http://ipinfo.io/${arg}";
    fi
done;

The ergonomics of this script mean that it’s pretty composable. We can use something like xargs to mash on any number of IP addresses to be queried!

cat ips-list | xargs ipinfo                   # print to stdout
cat ips-list | xargs ipinfo \\; > ip-info.json # redirect to a file

• Remove language translation check from core update process by bgturner · Pull Request #192 · wp-cli/core-command · GitHub

The main piece of feedback was:

ensure the language installation is reliably disabled across all wp versions.

So, how to do it!?

Learning more about Behat

Using my notes from the last OSS session, I was able to set Behat to use the MySQL instance from a Local site.

The next thing to learn more about is actually writing new functional tests with Behat. I’ve only ever skimmed this tool’s functionality and have never used it to write my own tests. So what do I need to do to start?

From reading the Behat Getting Started guide, the terminology looks like:

• Behat is a tool to do Behavor Driven Design (BDD)
• The format of the tests are written in Gherkin
• Each feature is one file
• Each feature file contains a number of different scenarios

So far so good – this feels like other testing tools! One thing I appreciate with other testing frameworks is to be able to zero in on one specific test to run. So using gherkin’s language: “How do I run specific features, or even specific scenarios?”

Looking at the Behat documentation for Gherkin filters, it looks like we can do both!

./vendor/bin/behat features/core.feature # run a feature
./vendor/bin/behat --name='Update from ZIP file' # run a scenario

Now, how to write a new test?

As I started to investigate how other WPCLI functional tests were written, I realized that I didn’t know what sorts of steps were available to me out of the box.

For example, what exactly am I getting when I write a scenario that starts with Given a WP install?

As it turns out, there’s an existing issue in the WPCLI handbook (ie the documentation repo) that is asking to clarify how to write Behat tests:

Create a dedicated doc for how to write Behat tests · Issue #3 · wp-cli/handbook · GitHub

Four years since that was created, and I gues I’m as good a person as any to start writing that content, so I forked the repo and pushed up a wip commit:

GitHub – bgturner/handbook at issue/3/writing-behat-tests

The quickest way to view the published block-based themes is to browse the “Full site editing” tag of themes on WordPress.org.

But what if we want a way to quickly download and browse the code of those themes on our own machine?

Listing WordPress themes from the CLI

Examining the requests made from the above page, I noticed that the main content is generated from the response to a call to the /themes/info endpoint of api.wordpress.org.

Exploring that endpoint a bit with restclient.el package in Emacs showed that there’s an easy way to get the theme slugs!

Install all Block-themes with curl, jq, and wpcli

So we have a URL, and if we have jq installed, then we can pipe our way to installing all of the block-themes!

curl https://api.wordpress.org/themes/info/1.2/\?action\=query_themes\&request%5Btag%5D%5B%5D\=full-site-editing\&request%5Bper_page%5D\=50 | jq -r '.themes[].slug' | xargs wp theme install

Once those are downloaded, it’s just a matter of listing the themes and activating one!

Trying out the wp-cli-dev repo

As I found in my last Open-source block time, the commands I had installed locally were missing various subcommands that were installed within my shell of a Local site.

Thanks to a pointer by @schlessera in the WordPress #cli slack channel, there’s the wp-cli/wp-cli-dev package that attempts to make it easier to download and scaffold out a full wpcli dev environment.

It wasn’t a seamless path since it seems like Github was throttling me, or in some way rejecting my key. To solve it, I temporarily commented out a call to refresh the various repos right after cloning them. This allowed the composer install command to finish cleanly and put all the various repos into place.

I use Local. How do I run Behat?

Doing the above allowed me to run some of the initial things like PHP_Codesniffer, but when I ran Behat (and probably PHPUnit) I wasn’t able to connect to the DB.

Here’s that specific error:

ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/tmp/mysql.sock' (2)

This basically means that MySQL is having issues connecting to the default socket. But since I use Local to quickly create environments instead of directly installing things like MySQL — what do I need to do?

The general flow was to:

1. Find the configuration file for he running MySQL process.
2. Search that config file for the socket.
3. Export the WP_CLI_TEST_DBHOST environment variable.
4. Run the tests!

For me, under a Local site’s environment, that process looks something like this:

ps aux | grep mysql
grep 'socket' ~/Library/Application\ Support/Local/run/o5ykeOZFY/conf/mysql/my.cnf                             
export WP_CLI_TEST_DBHOST='localhost:/Users/ben.turner/Library/Application Support/Local/run/o5ykeOZFY/mysql/mysqld.sock'
composer run behat

• Unexpected HTML output when updating WordPress #191

Even though the solution was simple and I was able to get a pull request created, I found that there may be some holes in how the WPCLI project onboards new contributors.

Basically, after following the steps in Working on the project as a whole, I noticed that I was missing a number of subcommands:

I’m asking for a bit of direction in the WordPress #cli slack channel, but my hunch is that the main wp-cli repo needs an update to the [composer.json](https://github.com/wp-cli/wp-cli/blob/77e392bd3b38df599dd26dca66e7ee22b5213dba/composer.json#L19) that includes the various subcommands:

"require-dev": {
        "roave/security-advisories": "dev-master",
        "wp-cli/db-command": "^1.3 || ^2",
        "wp-cli/entity-command": "^1.2 || ^2",
        "wp-cli/extension-command": "^1.1 || ^2",
        "wp-cli/package-command": "^1 || ^2",
        "wp-cli/wp-cli-tests": "^3.0.7"
    },

This has been a few weeks in the making, where I basically have:

1. Made the case with my employer/manager that I should give back (it was an easy case to make!)
2. Carve out time on my calendar.
3. Lurk on projects to see where I should help.
4. Start helping!

Since I love the terminal and am using WordPress daily, helping with WPCLI makes sense.

So what did I do today?

Part of me wanting to write these less polished blog posts is to keep me honest about contributing, as well as help establish the habit of working and writing. Ok, so what did I do during today’s open-source time block?

Two main things:

1. My first pull request to WPCLI was accepted!
   • https://github.com/wp-cli/wp-cli/pull/5562
2. Searched through the oldest issues for WPCLI on Github to see if I can replicate, clarify, or possibly close any of them.

Replicating an issue with the core update subcommand

It turns out that the oldest issue seems to be a good-first-issue.

After a little digging in the source code and trying some things in the terminal, I was able to distill down a more direct list of reproduction steps.

My block time is up for today, but leaving myself breadcrumbs in that issue, as well as on this blog, should allow me to dive in soon with a fix!

When I first started to use Emacs, I relied on Spacemacs, which is basically an opinionated configuration that has settings that are useful for someone used to Vim’s way of working with text. (yep, that’s me! Why choose between Emacs and Vim when you can have both!)

At a certain point I wanted to have a better understanding of my editor and in particular, wanted to get a feel for how to approach editing “the Emacs way.” Mostly this means trying new packages or built-in functionality with a minimal amount of cruft.

Since there are useful packages out there, and I don’t want to go and break my existing configuration, I found myself needing a way to quickly create a minimal Emacs sandbox. Something that is quick to scaffold up and just as easy to tear down by deleting a folder.

After a bit of hacking, and the help of this StackOverflow answer, the result is a simple bash script that I’ve pushed to Github:

• https://github.com/bgturner/emacs-sandbox

Usage

The Readme should have more details about using the script, but as a quick intro, here’s what you can expect:

emacs-sandbox.sh -i straight -i evil
# Creates the default 'sandbox' environment with the 'straight'
# package manager and 'evil' (ie, vi-style bindings)

emacs-sandbox.sh -n super-secret -i melpa -i straight -i ivy
# Creates the named 'super-secret' environment with the 'melpa'
# package repository configured along with 'straight' for package
# management  and the 'ivy' package for general completion
# functionality.

You’re right, some (many?) times awk isn’t the right choice — maybe the project has too many edge-cases to consider, or maybe there are too many unknowns that need to be abstracted away. In those cases, turning to a different language with robust libraries makes sense.

On the other hand, the one thing I don’t like about turning to another language is all the scaffolding that’s required to just get working. Usually, to get started, you create a project, initialize a virtual environment, define and install dependencies, mess with configuration because something is updated, and on and on.

Because awk is likely already installed on the system you’re working on, for many problems, you’re just a couple of minutes away from a solution.

CSV of Nexo Transactions

So that’s where I found myself the other night — I needed to convert a CSV file of cryptocurrency transactions from Nexo into a format that Ledger-cli understands.

More specifically, how do I turn something like this:

Transaction,Type,Currency,Amount,USD Equivalent,Details,Outstanding Loan,Date / Time
NXTnxxxxxxxxx,Interest,ADA,0.06513149,$0.14267261305268,approved / ADA Interest Earned,$0.00,2021-08-16 01:00:04
NXTGxxxxxxxxx,Interest,ADA,0.06887155,$0.15001153355925,approved / ADA Interest Earned,$0.00,2021-08-15 01:00:03
NXTcxxxxxxxxx,Interest,ADA,0.06511621,$0.13219163652648,approved / ADA Interest Earned,$0.00,2021-08-14 01:00:03
NXTaxxxxxxxxx,Exchange,USDC/ADA,-1166.227927 / +571.00203394,$1165.3789130691,approved / Exchange USD Coin to Cardano,$0.00,2021-08-13 15:39:29
NXTDxxxxxxxxx,Deposit,USDC,1166.227927,$1187.6814460844,approved / 0xc000000000000000000000000000000000000000000000000000000000000000,$0.00,2021-08-12 20:48:22

Into this:

2021-08-16 01:00:04 Interest -- approved / ADA Interest Earned
	Assets:Current:Nexo:ADA    0.06513149 ADA
	Income:Interest:Nexo:ADA    -$0.14267261305268


2021-08-15 01:00:03 Interest -- approved / ADA Interest Earned
	Assets:Current:Nexo:ADA    0.06887155 ADA
	Income:Interest:Nexo:ADA    -$0.15001153355925


2021-08-14 01:00:03 Interest -- approved / ADA Interest Earned
	Assets:Current:Nexo:ADA    0.06511621 ADA
	Income:Interest:Nexo:ADA    -$0.13219163652648


2021-08-13 15:39:29 Exchange -- approved / Exchange USD Coin to Cardano
	Assets:Current:Nexo:ADA     571.00203394 ADA
	Assets:Current:Nexo:USDC    -1166.227927  USDC


2021-08-12 20:48:22 Deposit -- approved / 0xc000000000000000000000000000000000000000000000000000000000000000
	Assets:Current:Nexo:USDC    1166.227927 USDC
	Equity:Transfers    -$1187.6814460844

Quick Conversion Using Awk

The actual process of writing the script took a few minutes, the result being this bash script:

#!/bin/bash 
awk -F "," '{
if ($2 ~ /Interest/)
   {print $8 " " $2 " -- " $6 "\\n\\tAssets:Current:Nexo:" $3 "    " $4 " " $3 "\\n\\tIncome:Interest:Nexo:" $3 "    -" $5; print "\\n";}
if ($2 ~ /Deposit/)
   {print $8 " " $2 " -- " $6 "\\n\\tAssets:Current:Nexo:" $3 "    " $4 " " $3 "\\n\\tEquity:Transfers    -" $5; print "\\n";}
if ($2 ~ /Exchange/)
   {
   split($3,a,"/");  # split into the two currencies
   split($4,b,"/");  # split into the two amounts
   gsub(/\\+/,"",b\[2\])
   print $8 " " $2 " -- " $6 "\\n\\tAssets:Current:Nexo:" a\[2\] "    " b\[2\] " " a\[2\] "\\n\\tAssets:Current:Nexo:" a\[1\] "    " b\[1\] " " a\[1\];     print "\\n";}
}' nexo\_transactions.csv > nexo.ada.ledger

The few things I’ll call out since it wasn’t completely obvious when I started:

• By default awk’s field separator is a space. Since this is a CSV file, we want to use a comma instead. This is the -F "," part of the above script.
• Because we want slightly different behavior depending on the type of transaction, we need to use if statements that test if a the second (ie the “type”) field is the one we want.
• The various “exchange” transactions had additional info packed into the Currency and Amount fields. Basically there were two values that we needed to extract from the one field. Because of this, we use the split(<field>, <local-variable>, <delimiter>) function.
• Lastly, I didn’t know until working through this exercise, but Ledger-cli doesn’t like to have a plus preceeding positive amounts. Because Nexo is explicit in adding the plus, I added the gsub(<search-regex>, <replacement>, <string>) function to replace the plus in the “split” amount field.

Having been created in the 1970s at Bell Labs, it’s been around for a long time. This is useful because it means that awk is probably available in most terminal sessions you start. It also means that the resources available for learning awk don’t lose their relevance. While you can certainly use a search engine to find tutorials, the best resources I learned from were:

• Awk – A Tutorial and Introduction – by Bruce Barnett
  • Lots of detailed, free information. A somewhat dated site theme, but you’re learning awk, right?
• Sed & Awk – by Dale Dougherty, Tim O’Reilly
  • Great writing and examples. Especially useful if you are wanting to learn about Sed as well.

Ok, so that covers how to learn awk, but maybe you’re not convinced about why you should learn it. The rest of this post will take a deeper look into how I include this tool into my workflow.

Why use awk?

If all you are doing is trying to search for the occurrence of a string within a block of text, then awk is probably overkill. In those situations, I turn to [grep](https://en.wikipedia.org/wiki/Grep), with commands that look something like this:

cat access.log | grep " 500 "
grep " 500 " access.log

The above commands search a file called access.log for any string that has a space, the number 500, and another space. This is a quick way to check for server 500 errors, like those that are reported in an nginx access log that’s exported from Flywheel’s server.

Grep can perform more powerful searches using regex, however, I turn to awk when I find myself starting to ask more advanced questions that require searching for one thing but printing another. Here are some examples of those kinds of questions:

• What is the distribution of HTTP response codes for the server? What about for a specific site?
• What IPs, and how many times have those IPs attempted to access the wp-login.php file for a site?
• What urls are experiencing 50x server errors?

We’ll get into the actual examples for the above questions later, but first let’s get a quick overview of how to use awk so that if you need to tweak things, you know where to start looking.

Basic Usage

Awk works by searching for strings and then doing something when it finds those strings. You can break down an awk command into this simple structure:

pattern { action }

The “pattern” part is a description of what lines awk should act upon. The “action” part of an awk statement is wrapped in curly braces and describes what should be done when a match is found.

Since awk is a command-line tool, we wrap the “awk program” that we want to run with single quotes. Here’s an awk command that does the same thing that the above cat and grep commands do:

awk '$0 ~ /500/ { print $0 }' access.log

To break this down into plain English:

• awk will execute the program within the two single quotes '...' on the access.log file.
• The program will first search each line (the $0 part), and only do something if there is a 500 within it. $0 ~ /500/.
  • Note: This looks more complex than grep because it is a regex statement.
• In this case, the awk program prints the whole line { print $0 }.
  • Note: Printing is the default behavior of awk, so the above could have been simplified to:

awk '$0 ~ /500/' access.log

A little more complex usage

The above doesn’t give us much benefit beyond what we can do with grep, but we can use awk to ask more complex things.

When awk runs, it breaks each line into fields that we can operate on. You can think of these fields like the cells in a spreadsheet. We can reference these fields by using a dollar sign and the number of the field. So $0 is the whole line, while $1 is the first field, $2 is the second field, and so on.

If we take a look at an example line from our access.log file which is basically the the default, nginx “combined” format, awk understand the line like so:

69.162.124.230 - - \[22/Jan/2018:06:25:05 +0000\] "HEAD / HTTP/1.1" 500 0 "http://example.com/" "Mozilla/5.0+(compatible; UptimeRobot/2.0; http://www.example.com/)"

Awk sees the line like this:
\* $01 == 69.162.124.230
\* $02 == -
\* $03 == -
\* $04 == \[22/Jan/2018:06:25:05
\* $05 == +0000\]
\* $06 == "HEAD
\* $07 == /
\* $08 == HTTP/1.1"
\* $09 == 500
\* $10 == 0
\* $11 == "http://example.com/"
\* $12 == "Mozilla/5.0+(compatible;
\* $13 == UptimeRobot/2.0;
\* $14 == http://www.example.com/)"
With some special fields:
\* $0  == The whole line
\* $NF == The last field

Knowing that awk breaks things down this way allows us to start searching exactly through one field and doing something with another field.

For example, we can find all lines that have a 500 HTTP response code (the ninth field), but only print the IP address for that request:

$ awk '$9 ~ /500/{ print $1 }' access.log
69.162.124.230
... lots more IPs ...

Getting more info with additional tools

Since this is the command line, we can use the output of our awk command as the input for other commands. I often will pipe the output of an awk command to both sort and uniq -c to count of the number of times something is printed from awk.

The uniq unix command selects only the unique lines, but requires those lines to be next to each other. Because of this, you will often see me pipe awk to sort and then to uniq. Finally, since there will be a count of the number of things that have happened, I will pipe the output one more time to sort -h, which will give us an ascending list. That general pattern looks something like this:

awk 'pattern{action}' file.log | sort | uniq -c | sort -h

Awk Cheatsheet for working with access.log

Great, you know enough about awk to be dangerous! 🙂 Maybe you need a little inspiration. Here are a few examples of questions I ask myself, and the terminal command that I use to get a better understanding of an nginx access log.

What is the distribution of HTTP response codes for the server?

Because the HTTP response code is the ninth field in the default nginx combined log, we just need to print that field, sort and count the results! In this case, you can see that by far the highest response is a 200 — great! There are a handful of 50x codes and some 40x codes, but overall, this log doesn’t look too bad.

$ awk '{print $9}' access.log | sort | uniq -c | sort -h
      1 499
      4 504
     13 400
     23 503
     30 500
     36 401
     52 403
     88 405
    247 304
   1057 302
   2774 301
   4522 404
  53296 200

What IPs, and how many times have those IPs attempted to access the wp-login.php file?

The seventh field is often then request field and the first field is the IP address. Especially if there’s some sort of bot attack in progress, there might be a large number of different IPs. We can pipe all of our results to the tail command so that only the top ten results are printed.

$ awk '$7 ~ /wp-login.php/ {print $1}' access.log | sort | uniq -c | sort -h | tail
     15 34.209.136.220
     16 85.140.40.253
     18 46.161.9.3
     23 118.190.78.53
     23 185.85.191.196
     24 177.72.199.27
     28 194.6.231.240
     39 85.97.250.114
    118 2002:c1c9:e0d2::c1c9:e0d2
    360 193.201.224.210

What urls are experiencing 50x server errors?

In this case, we’re using a regex to give us all the lines that have a 50x response and print the request url. This can be useful in seeing if there are wide-spread site issues, or only problems on one specific page.

benjamin@fwxxxxxx:~$ awk '($6 ~ /POST/ && $9 ~ /50\[0-9\]/){print $7}' /flywheel/logs/access.log | sort | uniq -c | sort -h
      1 /jquery-file-upload/server/php/
      1 /text.php
      2 /
      2 /product-16/
      2 /wp-content/plugins/apikey/ini.php
      2 /wp-post.php
     13 /product-12/

Learning more about Node.js as well as having a deeper understanding of modern Javascript development is on my list of things to do. For now, all I need is a minimal environment so that I can make use of Microsoft’s Language Server Protocol (LSP). If you haven’t taken a look at LSP, you should!

In my case, as an Emacs user, I’ve been able to simplify and streamline my configuration for numerous programming languages, all by exploring the resources on the Emacs lsp-mode website.

I won’t go into my editor configuration now, instead, I’ll focus on what needs to be done to get the bare minimum Javascript environment configured for Ubuntu.

Minimum Javascript Environment

So what are these core tools to be installed?

• nvm – Installs and manages Node.js versions.
• node.js – A runtime so we can execute Javascript code.
• npm – The original Javascript package manager.
• yarn – Another Javascript package manager because… the community can’t decide on one?

As you can see in this screencast, getting things installed is pretty quick and is done in under a minute. Below you can find more details and specific commands related to each tool!

nvm

Nvm helps to install and manage different versions of Node. The actual installation of nvm is pretty easy and is outlined within the project’s readme on Github. Basically, the steps are to clone the repo and ensure that it’s correctly loaded in your shell.

curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.38.0/install.sh | bash

Once that repo is in place, we can add a block to our .zshrc file:

\# Configure node dev environment
if \[ -d "$HOME/.nvm" \]; then
    export NVM\_DIR="$HOME/.nvm"
    \[ -s "$NVM\_DIR/nvm.sh" \] && \\. "$NVM\_DIR/nvm.sh"  # This loads nvm
fi

node.js

With nvm installed, we’re one command away from having node:

nvm install node

This will download the latest version as well as set that version as the default. There are some crufty projects I still haven’t upgraded, so I’ll list other node versions with nvm ls and install an older LTS release with nvm install lts/fermium:

★  ~ % nvm ls      
->      v16.6.1
default -> node (-> v16.6.1)
iojs -> N/A (default)
unstable -> N/A (default)
node -> stable (-> v16.6.1) (default)
stable -> 16.6 (-> v16.6.1) (default)
lts/\* -> lts/fermium (-> N/A)
lts/argon -> v4.9.1 (-> N/A)
lts/boron -> v6.17.1 (-> N/A)
lts/carbon -> v8.17.0 (-> N/A)
lts/dubnium -> v10.24.1 (-> N/A)
lts/erbium -> v12.22.4 (-> N/A)
lts/fermium -> v14.17.4 (-> N/A)
★  ~ % nvm install lts/fermium
Downloading and installing node v14.17.4...
Downloading https://nodejs.org/dist/v14.17.4/node-v14.17.4-linux-x64.tar.xz...
######################################################################### 100.0%
Computing checksum with sha256sum
Checksums matched!
Now using node v14.17.4 (npm v6.14.14)

npm

Having a section for installing npm is a little misleading since installing node also installs npm. One thing I want to call out which wasn’t obvious to me at first – each version of node comes with its own version of npm:

★  ~ % nvm install lts/fermium
Downloading and installing node v14.17.4...
Downloading https://nodejs.org/dist/v14.17.4/node-v14.17.4-linux-x64.tar.xz...
######################################################################### 100.0%
Computing checksum with sha256sum
Checksums matched!
Now using node v14.17.4 (npm v6.14.14)
★  ~ % npm --version
6.14.14
★  ~ % nvm use stable
Now using node v16.6.1 (npm v7.20.3)
★  ~ % npm --version
7.20.3

yarn

Honestly, I’m a little fuzzy when it comes to knowing when to use npm and when to use yarn. My rule of thumb is to look at the repo and see if it has a yarn.lock file.

Anyway, just so we’re prepared for any project that comes our way, we can use npm to install yarn globally for our user:

npm install -g yarn

Background

Throughout my time in tech, I’ve mostly been a generalist and have employed various languages to just get things done. I’ve never focused solely on learning Python, but I do know enough to wrangle data and create some effective visualizations.

Occasionally I like to completely wipe my personal Linux machine and install things from scratch. There’s something wonderful about nuking the cruft and getting back to the bare essentials for getting things done. So here I am; a mostly blank installation of Ubuntu 21.04, ready to get a Python environment installed and set up. Let’s document this process so that future me won’t have to think about how to get up and running with Python dev!

Since I’m not solely focused on Python projects, it makes sense that I should probably search for how “the pros” configure their environment to be productive. There are a lot of guides out there that say they have the answer, and in sometimes that true. Sometimes you go down a fragile rabbit hole. Many of the tutorials and blog posts covered older versions of Python, or in some way didn’t feel right for how I like to configure my system.

In the end, the best post I found was from Jacob Kaplan-Moss (co-creator of Django). It’s specific, direct, and I get the feeling that like me, a crufty machine won’t be tolerated. Even though the post is from 2019, the general ideas and package recommendations are still good:

1. pyenv
   • Helps to manage multiple versions of Python. Think of nvm, but for Python.
2. pipx
   • Manage your user’s “global” tooling. Something along the lines of installing a Javascript tool for your “global” user. (think npm install <package> --global )
3. poetry
   • Dependency and virtual environment management for specific projects. Think npm or composer, where you declare a project’s dependencies and let the tool build the environment. Additionally, this configuration is “frozen” with a lock file.

So, what are the specifics for getting these things working on a clean install of Ubuntu?

Installing Ubuntu Dependencies

In Jacob’s post above, the installation is for a MacOS system. It took a bit of trial and error, but I was able to zero in on the specific mix of packages that need to be installed. There might be other packages that creep in as I start installing and using various Python tools, but for now, this should get me what I need:

sudo apt-get update;
sudo apt-get install --no-install-recommends \\
     make build-essential libssl-dev zlib1g-dev \\
     libbz2-dev libreadline-dev libsqlite3-dev \\
     wget curl llvm libncurses5-dev xz-utils \\
     tk-dev libxml2-dev libxmlsec1-dev \\
     libffi-dev liblzma-dev

pyenv

Pyenv is pretty neat and does it’s fancy management of Python versions by shimming your $PATH.

Installing is easy and is basically:

1. Clone the repo to your home directory.
2. Update your shell config to reference that repo.

The above process is outlined in more detail within the Basic Github checkout section of the readme. Here’s what that looks like:

git clone https://github.com/pyenv/pyenv.git ~/.pyenv

And this shell configuration added to my ~/.profile file:

if \[ -d "$HOME/.pyenv" \]; then
    export PYENV\_ROOT="$HOME/.pyenv"
    export PATH="$PYENV\_ROOT/bin:$PATH"
    eval "$(pyenv init --path)"
fi

Once Pyenv is configured, we can list and install individual versions of Python from the terminal.

★  src/python-sandbox % pyenv install --list
pyenv install --list
Available versions:
2.1.3
2.2.3
...
★  src/python-sandbox % pyenv install 3.9.1
pyenv install 3.9.1
★  src/python-sandbox % pyenv global 3.9.1
pyenv global 3.9.1
★  src/python-sandbox % which python
which python
/home/benjamin/.pyenv/shims/python
★  src/python-sandbox % python --version
python --version
Python 3.9.1

pipx

As someone who doesn’t live and breath Python, I mostly turn to it for quick and simple scripts as well as a few essential cli apps. Because most systems make use of some sort of system-installed version of Python, I like the idea of having my user-specific tooling constrained to my own user.

Pipx makes having a separate, user-focused python environment easy, and the installation is quick using pip from our Pyenv installation:

★  src/python-sandbox % python -m pip install --user pipx
python -m pip install --user pipx
Collecting pipx
  Downloading pipx-0.15.6.0-py3-none-any.whl (43 kB)
     |████████████████████████████████| 43 kB 1.6 MB/s 
Collecting packaging>=20.0
  Downloading packaging-20.8-py2.py3-none-any.whl (39 kB)
  ...
★  src/python-sandbox % export PATH="$HOME/.local/bin:${PATH}"
export PATH="$HOME/.local/bin:${PATH}"
★  src/python-sandbox % python -m pipx ensurepath
python -m pipx ensurepath
/home/benjamin/.local/bin is already in PATH.
/home/benjamin/.local/bin is already in PATH.

We now have an isolated environment for our user’s Python cli apps, but what now?

One amazing tool that I keep returning to is Visidata, which is like a spreadsheet tool on CLI steroids. It allows you to explore, interact, and clean data from the command line.

In addition to working with csv files, you can extend Visidata by including additional libraries so that it can handle more file formats. For example, installing Pandas will allow more loaders within visidata – basically, any file format that Pandas knows how to open can be opened in Visidata.

So we want Visidata along with the Pandas boost, but how do we do that with Pipx?

By injecting dependencies!

★  src/python-sandbox % pipx install visidata
pipx install visidata
  installed package visidata 2.1.1, Python 3.9.1
  These apps are now globally available
    - vd
    - visidata
done! ✨ 🌟 ✨
★  src/python-sandbox % pipx inject visidata pandas
pipx inject visidata pandas
  injected package pandas into venv visidata
done! ✨ 🌟 ✨

Poetry

The previous tools have let us sandbox our day-to-day Python tools away from a system configuration of Python, but the missing piece is managing dependencies for an actual Python project.

Poetry handles the actual dependency and virtual environment management for specific projects. Think of something like yarn or composer, where you can declare your dependencies as well as freeze the configuration by using a “lock” file.

Using Pipx again, installation is easy, and we can quickly initialize a new project and add our dependencies:

★  src/python-sandbox % pipx install poetry
pipx install poetry
  installed package poetry 1.1.4, Python 3.9.1
  These apps are now globally available
    - poetry
done! ✨ 🌟 ✨
★  src/python-sandbox % poetry init
poetry init

This command will guide you through creating your pyproject.toml config.

Package name \[python-sandbox\]:  

...

★  src/python-sandbox % poetry add pandas numpy bokeh networkx
poetry add pandas numpy bokeh networkx
Creating virtualenv python-sandbox-Z3rSXY9z-py3.9 in /home/benjamin/.cache/pypoetry/virtualenvs
Using version ^1.2.0 for pandas
Using version ^1.19.5 for numpy
Using version ^2.2.3 for bokeh
Using version ^2.5 for networkx

Updating dependencies
...

Nice! Pretty easy!

Wrap-up

I’ve been using this setup for a few weeks now and it feels mostly stable and strikes a good balance of:

1. Being flexible for when I need to switch versions of software.
2. Isolated so that I’m not accidentally breaking other projects.
3. Good-enough defaults for when I just need a quick snippet of Python to be run.

Hope this helps get you up and running quickly and that you can get on with solving interesting problems!

If you view the man page for Vim, there’s an entry for the -c argument that looks like this:

-c {command}
   {command} will be executed after the first file has been read.  
   {command} is interpreted as an Ex command.  If the  {command}  
   contains spaces it must be enclosed in double quotes (this 
   depends on the shell that is used).  Example: Vim "+set si" main.c
   Note: You can use up to 10 "+" or "-c" commands.

That part about Ex command is key — these are the commands that are executed when we type the colon in Vim.

But how do we go from a man page entry to something useful? How about a case study.

Manually Removing Wordfence Tables from a SQL Dump

Let’s say that the Wordfence plugin was exporting data in a binary form which was breaking things when trying to import the database into a new environment. They know about this, and have this helpful guide:

• https://www.wordfence.com/help/advanced/remove-or-reset/

That’s a fine solution if you have the site up and running, but suppose you have a bunch of sql files that need to be cleaned up so that they can be imported cleanly.

After cleaning up those files manually a few times with Vim, you’ll find that you will be doing a number of common motions like searching, deleting, and saving those files. At a certain point, you might even realize that you can us the global ex command to “do stuff” on each line that matches a regex.

For me, when working through the cleanup process of Wordfence from SQL files, that process generally looked like this:

1. Search the SQL file using the “very magic” setting. This might take some fiddling so that the regex gets the lines that you want. The specific keystrokes might look like:
   • **/\vsearch_regex**
2. Using the “global print” ex command with an empty search string (to re-use the last search), do a quick sanity check to make sure the correct lines are being hit. Those keystrokes might look like:
   • :**g//p**
3. Use the “global normal” ex command to execute the normal (as in “normal mode”) keystrokes for every matched line. What this means is that we can do something like “delete the next four paragraphs for each match.” This is helpful in the SQL files that I was working with because each “paragraph” was a block of SQL statements — both definitions as well as data. The keystrokes for that might look like:
   • :**g//normal 4dap**

Those steps are all very abstract, so here’s a screencast working through that process of cleaning up those Wordfence tables:

Automating Vim From a Shell Script

The above steps are great at doing advanced editing during a session, but returning to the goal of actually automating that editing, you’ll want to make use of that -c flag that we mentioned earlier.

As a concrete example, that process of removing Wordfence tables from a SQL file could be abstracted even more. In this case, I created a shell script that:

1. Emptied out a register (in this case the @t register)
2. Do some “normal” keystrokes on lines matching \v_wf.{-}
3. Save the edited file
4. Create a new buffer and paste the contents of the @t buffer into this new buffer

Here’s the actual script that I ended up using to process those SQL files:

#!/bin/bash
file="$1"
vim -c 'let @t=""' \
   -c ':g/\v_wf.{-}`/normal "Tyapdap' \
   -c ":sav $file" \
   -c ':new wf-tables.sql.bak | :normal "tp' \
   -c ':wa | :qa' \
   "$file.original"

Setting up the environment

According to the above call, the main things that need to be done to get testing are:

• Install WordPress
• Install the tt1-blocks theme
• Install the latest version of Gutenberg (v10.7.1 as of now)

I was able to do this quickly within Local by creating a site, opening the Site Shell and running:

wp plugin install gutenberg --activate
wp theme install tt1-blocks --activate

In addition to the code, the testing call recommends importing some demo content from here.

Since the above link is a WordPress xml export, we can use curl to follow the redirect and download the xml for importing into WordPress. For this specific testing call, that command looks like:

wp plugin install wordpress-importer --activate
curl -L https://cloudup.com/files/iEsfmmRZeCg/download > callfortesting.xml
wp import callfortesting.xml --authors=create

Testing the feature

My goal was to create a landing page that has a featured section with the latest post of the portfolio category as well as a secondary section that has two posts and finally one last section to show everything else.

Here’s a recording of my session along with some notes for things I encountered:

https://i.getf.ly/xQu7g0mg

• 4:30 – The spacing looked off to me, but then I realized that the heading was the same color as the background so I wasn’t seeing the heading text. Investigating (in the video) and it looks like a specificity issue at the ~6:40 mark. The anchor link within the post heading doesn’t inherit the text color.
• 14:10 – I tried to adjust the pagination of posts to 10 and the block crashed. I was doing it via keyboard, but the mouse click to increase the number appears to work.
• 16:30 – Some general usability feedback of the column block: I’d love a way to make the vertical margins disappear so that full-width sections that have background colors don’t show any space between them. I was able to hack together a workaround with a custom utility class (see video), but it would be nice to have the ability to control that sort of design pattern within the column block.
• 27:00 – General UX – It appears that I didn’t save the template since it’s showing a 404, even though the title says “portfolio.” I think what confused me was the “Publish” button in the upper right corner. Coming from a WP background I think I understand that “Publish” meant to publish the page template I was editing, but on initial use, I was hesitant to push the button because my context was the original page that I had created, not the page template I was editing. Maybe there’s a way to make that button more explicit. So something along the lines of “Save Page Template.”
• 30:00 – Haha. I broke things. 😀 I tried to move a query to within a column block and hosed this template. Ahh well. It’s time to wrap this up and post my findings!

There’s a great overview within the readme of the Gutenberg repo on Github:

“Gutenberg” is a codename for a whole new paradigm in WordPress site building and publishing, that aims to revolutionize the entire publishing experience as much as Gutenberg did the printed word. Right now, the project is in the second phase of a four-phase process that will touch every piece of WordPress – Editing, Customization (which includes Full Site Editing, Block Patterns, Block Directory and Block based themes), Collaboration, and Multilingual – and is focused on a new editing experience, the block editor.

https://github.com/WordPress/gutenberg

To summarize, there are four main phases of Gutenberg project:

1. Editing (complete, shipped with WP Core in v5.0)
2. Customization (in-progress: full-site-editing, block-patterns, block directory, block-based themes)
3. Collaboration
4. Multilingual

Customization Phase

Becoming Familiar with Full-Site Editing

So we’re in the customization phase of Gutenberg, but from the above outline, there’s a lot of jargon. Where do you go to untangle all of it?

I decided to really focus in on Full-site Editing piece, but searching on Google is hard because of WordPress’ size and the number of entities jumping through the SEO hoops to rank higher than everyone else.

To get oriented quickly, the Full-Site-Editing – the Ultimate Resource List post from Birgit Pauli-Haack on Gutenberg Times seemed to have everything. The post can be informal or rough in places, but that’s more to do with the chaotic nature of open-source communication and having a lot of places where information is changing.

One good way to jump in and start using Full-site editing, is to connect with and contribute to the #fse-outreach-experiment. One interesting thing for me, was to take a look at the tt1-blocks repo to compare the differences between a block-based theme and the default TwentyTwentyOne WordPress theme.

FSE Outreach Experiment

As of June 4th, there are a couple of open experiments that are looking for testers:

• FSE Program Testing Call #7: Polished Portfolios – Make WordPress Test
• Proposal: Test Badges for the FSE Outreach Program – Make WordPress Test

While I don’t have enough time to take these for a spin right now. I think this will be the perfect thing to focus on early next week!

So after bringing it up with my manager, I have the green light to take some time during the week to help out upstream!

Since getting started and finding out where one’s aid can be most beneficial can be challenging, let’s document this process and see where it leads us.

Because my initial inkling is to help out with the db-command, I’ll start by becoming more familiar with the recent messages in the #cli channel in the WP Slack instance.

The first thing I see is that there are weekly office hours on Wednesdays at 15:00 UTC, (or 7:00 am in PDX) in WordPress #cli slack.

WPCLI Contributing Handbook

After reviewing the posts in slack for the last ~3months, I discovered that there is a dedicated contribution page within the WP-CLI handbook:

• https://make.wordpress.org/cli/handbook/contributions/contributing/
• https://github.com/wp-cli/handbook

WPCLI 2021 Roadmap

From one of the weekly office hours, there’s a general roadmap from the maintainer, schlessera.

My goals for 2021:

1. Migrate Behat from v2 to v3 (needs lots of changes in the test syntax)
2. Finish new scaffolding engine and introduce the concept of canonical code generation for WordPress (think artisan make:*)
3. Drastically improve documentation

Most of this was already on the agenda for 2020, but (as @jrf as eloquently put it in her post at https://24daysindecember.net/2020/12/21/a-perfect-storm/), this year has been a perfect storm of compatibility issues.

Hopefully, 2021 will be a bit more cooperative…

To cultivate contributions towards those goals, I work on:

1. Reducing friction for contributions
2. Talking about the goals and their purpose
3. (Going to contributor days to inspire and get people interested) <– maybe this will be possible again in 2021?

— WordPress #cli Slack channel

Documentation, I can help with that!

Additional notes from reading slack

Reading through the posts in slack also gave some good links out to various challenges in the larger PHP community.

PHP 8

• https://24daysindecember.net/2020/12/21/a-perfect-storm/
  • A post outlining why the release of PHP 8 was so hard on open source maintainers.

Migration from Travis CI to Github Actions

Looks like Travis CI changed a things which prevented WP cli from doing automated testing and as a result delayed the release of wp-cli 2.5.0.

Most things appear to have been migrated to Github Actions and there’s just a few remaining pieces before the latest version of WPCLI can be released!

When the zip is quite large, having to extract gigabytes of information just to view the contents of one file is annoying. Because of this I came across this StackExchange link with a few options to extract just one file.

The are the two main ways I’ve been using, but there are others!

Extract the thing, right here!

The typical way which makes sense in most use cases is this:

`unzip -d . site-backup.zip backup.sql`Code language: CSS (css)

This is basically saying is:

Within the site-backkup.zip archive, extract the backup.sql file right here (.)

This is great, and for the most part does what we need it to do.

What About a Deeply Nested File?

The previous example is good, but what if I have a file that is deeply nested, something like:

wp-content/plugins/random-plugin/inc/foo/bar/baz.php

Depending on what I think might be going on, I might just want to take a quick peek. If I used the previous technique, all of those folders would be created and would require me to traverse down to the file before opening. Not too bad, but still a little annoying.

Instead, I can extract that file to stdout and then redirect the output to a file in the current directory. Using the above baz.php file as an example, this would look like:

unzip -p site-backup.zip wp-content/plugins/random-plugin/inc/foo/bar/baz.php > baz.php

Two main downsides:

1. If it’s code, you’ll likely need to other files unless it’s a really trivial issue
2. The fileperms aren’t preseverved

The main issue was that replaceing the search string with a newline character – \n wasn’t visually creating a new line in the Vim buffer.

I found what I was looking for in this StackOverflow question: How to replace a character by a newline in Vim?

The general idea is to replace the search string with a \r as opposed to an \n.

For my use case, I was trying to break apart a really long HTML string along its <br> tags. So something along the lines of this:

:s/\<br>/\<br>\\r/g

Will search for <br> and replace it with a <br> and a visual newline.

Orgmode

The biggest reason I’ve been using Emacs lately has been the amazing Orgmode, and the way in which it can allow me to gather, organize, restructure and refine ideas and tasks as I’m working throughout the day.

If you haven’t taken a look at Orgmode, it’s like markdown, but way more powerful. It includes the basics of being able to markup text with simple conventions but adds to it the power of e-lisp to doing remarkable things like having full-fledged spreadsheet powered by raw text.

Publishing Posts using org2blog

One last piece of this is that the org2blog plugin allows me to write things in org-mode and publish them to a WordPress site using WP’s XML-RPC interface.

If you’re interested in how some of this is setup, I’ve been commiting and pushing changes to the .spacemacs file in my dotfiles repo.

The slides for the talk are publicly available on Google Drive.

Additionally, I created an example repo which outlines a number of steps covered in the presentation. Later commits were not a part of the presentation, and touch upon using Trellis to:

• Provision and deploy to Production
• Encrypt the vault.yml files in order to share the site with others, without allowing them to provision and deploy.

If you have questions, hit me up on Github or Twitter

Additional Clarification

There was one question along the lines of:

How do these projects enforce PHP versions?

My response ok, but let me attempt to improve on it:

Using Bedrock to manage our WordPress code, we are creating a new composer package. By declaring a dependency along the lines of “this site package requires at least this version of php”, we can ensure that our ‘package’ (ie our site) will fail if the dependencies are not met. This helps us quickly determine and meet the dependency that is lacking (ie, update to a more recent version of php).

Remember that Bedrock doesn’t require Trellis, it’s only concerned with the dependencies of our WordPress site. Because of this, our ‘site package’ can run anywhere the dependencies are met — “Nginx, Apache are fine, but we need at least this version of PHP”.

On the Trellis side, you “can” change the version of php. However, running the most recent version of PHP has so many benefits, predominantly the speed boost, that the effort of downgrading Trellis’ version of php may not be worth it.

Still, if you are undeterred, I recommend looking at the Ansible role controlling the installation of php:

• https://github.com/roots/trellis/blob/master/roles/php/tasks/main.yml

The Trellis VM is an Ubuntu machine, so any adjustments you can make normally with the shell can be done with Ansible.

I hope that helps clarify things, and if you are in PDX and want to learn more, let’s setup a co-hacking event to share our knowledge!

This is what I came up with:

$ find ~/ -not \\( -path ~/mail -prune \\) -type f -name '\*\[.\_\]log' -exec du -ah {} + | sort -h

Building Up the Command

I started by doing a simple find, without knowing exactly what I was getting myself into:

$ find ~/ -type f -name '\*\[.\_\]log'

You can read this in simple English as saying “Find in the home directory, any files with a name ending in either ‘.log’ or ‘_log’”. This was a good start, but I had way too many hits and the actual search was taking a couple of seconds to complete — way too long! 😛

Curious about performant ways of excluding directories while using find, I found this answer on Stackoverflow.

Using the above answer as guidance, I pruned the massive amount of email folders that find was traversing:

$ find ~/ -not \\( -path ~/mail -prune \\) -type f -name '\*\[.\_\]log'

At this point, it is a simple matter of executing du to get the file size of each log, and then piping the output to sort the results:

$ find ~/-not \\( -path ~/mail -prune \\) -type f -name '\*\[.\_\]log' -exec du -ah {} + | sort -h

wp db export example.com--$(date +%Y-%m-%d--%H-%M-%S).sql

# gives something like example.com-2017-09-11--14-26-21.sql

Yes, this is a pain to type out — the first time.

But going to the effort of writing this out allows us to quickly recall a command which gives us a filename with a “now” time stamp.

I didn’t learn about using ctrl+r in bash until fairly recently. If you don’t know about it, you can do a reverse search through your bash history. Using the above command substitution technique, I can recall a previous dump with a ‘now’ timestamp with just a few keystrokes:

<ctrl+r> wp db ex\<enter>

See the below screencast for an example.

I find this particularly helpful with database dumps and tar creation commands!

Using Liquid Web’s chat support I discovered that you can clear the cache using your browser by whitelisting your IP to use the PageSpeed Admin.

To replicate, login as root to the server and edit the pagespeed.conf file:

ssh root@xxx.xxx.xxx.xxx
[root@host ~]# vim /etc/httpd/conf/pagespeed.conf

There should be a block that looks like this:

<Location /pagespeed_admin>
Order allow,deny
Allow from localhost
Allow from 127.0.0.1
Allow from xxx.xxx.xxx.xxx # add your IP here
SetHandler pagespeed_admin
</Location>

Save the file and restart Apache. My server was a CentOS one, so:

[root@host ~]# service httpd restart

Doing this should allow you to access the PageSpeed admin by going to the URI at the site’s IP. For example, if the IP for example.com was 73.24.10.10 — you would navigate to http://73.24.10.10/pagespeed_admin

Behold, Web 0.1 in all of it’s glorious styling!

All that’s left is to navigate to the ‘Caches’ page and click ‘Purge Cache’ :

There are a lot of great talks, and I’m glad that I can contribute to the community by sharing my knowledge of managing a WordPress site using the various Roots.io projects.

As of right now, all tickets have been sold out — however you can get up to date info by following @wcpdx on Twitter.

http://blog.endpoint.com/2012/05/vim-working-with-encryption.html

The gist is to add this to your .vimrc file:

" Transparent editing of gpg encrypted files.
" By Wouter Hanegraaff
"    see: http://blog.endpoint.com/2012/05/vim-working-with-encryption.html
"
augroup encrypted
  au!

  " First make sure nothing is written to ~/.viminfo while editing
  " an encrypted file.
  autocmd BufReadPre,FileReadPre \*.gpg set viminfo=
  " We don't want a swap file, as it writes unencrypted data to disk
  autocmd BufReadPre,FileReadPre \*.gpg set noswapfile

  " Switch to binary mode to read the encrypted file
  autocmd BufReadPre,FileReadPre \*.gpg set bin
  autocmd BufReadPre,FileReadPre \*.gpg let ch\_save = &ch|set ch=2
  " (If you use tcsh, you may need to alter this line.)
  autocmd BufReadPost,FileReadPost \*.gpg '\[,'\]!gpg --decrypt 2> /dev/null

  " Switch to normal mode for editing
  autocmd BufReadPost,FileReadPost \*.gpg set nobin
  autocmd BufReadPost,FileReadPost \*.gpg let &ch = ch\_save|unlet ch\_save
  autocmd BufReadPost,FileReadPost \*.gpg execute ":doautocmd BufReadPost " . expand("%:r")

  " Convert all text to encrypted text before writing
  " (If you use tcsh, you may need to alter this line.)
  autocmd BufWritePre,FileWritePre \*.gpg '\[,'\]!gpg --default-recipient-self -ae 2>/dev/null
  " Undo the encryption so we are back in the normal text, directly
  " after the file has been written.
  autocmd BufWritePost,FileWritePost \*.gpg u
augroup END

This is plugin is meant to help self-host asciicasts on your WordPress site. If you are simply wanting to include asciicasts from asciinema.org, then take a look at their embedding guidelines.

Usage

This plugin adds a shortcode to help display the Asciinema player. Once installed, you can add the JSON formated terminal sessions to the ‘asciicast’ directory under the ‘uploads’ folder and reference them using the shortcode. The below terminal session is displayed with:

\[ asciinema src="wp-asciinema-hello-world.json" speed="2" \]

A couple of months ago I started tinkering with bash scripts to automate this process. Since I’ve been using and refining them, I thought I would clean and comment the code as well as provide a detailed readme file.

If you are interested, have a look at the repo:

https://github.com/bgturner/roots_db_scripts

It certainly wasn’t easy, nor do I think that it is the best thing for every developer to take on.

There might even be some truth to the idea that Vim is a dark, elegant weapon, wielded by neckbeards with whom the Unix is strong.

I do have a beard, but my neck is shaved at least some of the time. But using Vim has served as a powerful gateway for me to think about solutions in an even more ‘Unix-y’ way. This includes becoming more familiar with the terminal, but also more general language tools like Regex — a skill which still elicits feelings of dread and awe when I decide to use it.

Since I didn’t get here all on my own, I thought it would be good to share a few of the resources that I’ve found most helpful for really leveling up my Vim-fu:

• Vimcasts.org
• Practical Vim, by Drew Neil
• Vim Mastery, Laracasts

Since Vim has such a strong following in the Ruby on Rails world, I really found the Laracasts series to be more applicable to my day to day use as a PHP developer. All in all though, it’s hard to plow through any of these resources once and be done with it. I found myself biting off a chapter here, or watching a screencast there, looking for a way to utilize what I had just learned.

That’s the thing, the concepts only really started clicking when I applied it to real world tasks.

And there’s still more to learn!

My next bit of Vim learning will be working through Learn Vimscript the Hard Way with an eye towards extending this already powerful tool.

With all of this learning I feel as though I’m closer to having something to actually share back. As such, I’m hoping to post more often here about the tools and tinkering I’ve come across, and how they can be leveraged into useful solutions.

While reading Doug’s answer I found myself identifying with his stories of how he interacts with the world around him — examining, tinkering, understanding.

I remember my time as a visual artist and approaching problems with a balance of learned technique and playful experimentation. Some of the most successful solutions — both in art and now in tech involve toying with the edges of what something is ‘supposed’ to do. As Doug put it:

What does it do?  What is [it] supposed to do?

Those are two entirely different questions!

I’ve always thought that creativity or in this case, hacking — isn’t something that is reserved for “artists” (or whoever your chosen “creative” group is). They are qualities that are intrinsic in all of us. We start our lives wanting to play and interact with the world. The problem is that there are a lot of other things (people, institutions, societal norms, etc) interested in controlling that spark, bending to their interests, or worse extinguishing it when it is inconvenient for them.

Ultimately, I see both creativity and hacking as personal tools for understanding reality. I found the closing paragraph a great summary of what we obtain with the application of these tools by human hands [my emphasis]:

Anything that can do something, or that has complex dynamics, or functions, can be hacked on and the products of that hack are understanding of that thing that goes deeper than the obvious, or official, or expected types of understanding.

https://www.quora.com/How-does-one-become-a-hacker-2/answer/Doug-Dingus

I recently hacked together this single use scraper repo. Fork, extend and tweak for your needs!

Backstory

I’ve recently had the pleasure of launching a new WordPress site for Peaceful Media — Yeah!! — but a cursory walk through of the live site had me stumbling across urls still pointing to the development site.

How could this be? I was sure to use wp-cli and its search and replace function:

wp search-replace '//dev.example.com' '//example.com'

As it turns out, Visual Composer, the page layout plugin that was used to craft the various static pages of the site, was saving some urls in slightly different ways to the database. In a few of those cases it was simply that the urls had been percent encoded. So for example, http://dev.example.com becomes http%3A%2F%2Fdev.example.com. Not too bad. I could have just search and replaced on this new string. But what about the more annoying Raw HTML Element?

The Raw HTML element actually saves content to the database as a base64 encoded string. I actually should give at least a little props to Visual Composer for the clever, albeit hacky way they have managed to ensure that raw HTML markup isn’t “helpfully” sanitized by TinyMCE or WordPress on post save. Since the stuff being saved is just a jumble of characters, there is nothing that either TinyMCE, or WordPress can adjust and break.

That’s all fine and good, however, having the content exist encoded in base64 means I can’t easily interact with the content outside of the GUI admin screens — in this case, perform a simple search and replace on the database. Additionally, should we ever migrate away from using Visual Composer, we now have a more complex migration path ahead of us.

Anyway, back to solving the problem at hand — identifying all of the instances of a Url that match a ‘problematic’ pattern — dev.example.com. What I ended up doing was to create a quick and dirty site scraper to help me scan all of the pages that are easily accessible from the homepage and flag any instances of URLs which match a regex of ‘problem’ urls. Then it’s just a matter of hand updating them. It turned out to be pretty easy using the node-simplescraper module and creating a 75 line nodejs file. One secondary benefit of this is that it is not confined to simply scanning the content of the site database.

Since this is an actual headless browser scraping our site, it scans everything that a visitor would have access to. This includes links, images, styles and more. This means that if there were any problem urls in linked files, for example, external Javascript and Css files — those would get flagged as well. Awesome!

The end result is still a duck-taped, immediate use sort of tool, but I think there is something here that I can come back to and extend. Until then, feel free to poke around and fork the repo on Github!

How to flush mod_pagespeed cache in LiquidWeb‘s new Managed WordPress package:

SSH as root and touch a file:

ssh root@###.###.###.###
touch /var/cache/pagespeed/cache.flush

The mod time of this file is used by mod_pagespeed to determine if its cache needs to be flushed.

Setup

I’ve recently had the opportunity to try out LiquidWeb’s new Managed WordPress package. The initial setup was fast, and simple with a number of really awesome performance settings, such as HHVM, Memcached and PageSpeed setup out of the box.

So the site has been deployed and is noticeably faster. The only issue was finding a way to flush the mod_pagespeed cache.

According to the PageSpeed docs, the most basic method is to

simply touch the file “cache.flush” in the directory specified by `FileCachePath`

My process for finding / doing this followed:

1. Determine where `FileCachePath` was on the server
2. Flush the cache

Finding FileCachePath

This server was a CentOS one, but this should be similar for other systems. First off, we need to login as root and find the apache configuration:

ssh root@###.###.###.###
ps -ef | grep apache
# gives us:
root 23433 1 0 Jan14 ? 00:00:04 /usr/local/apache/bin/httpd -k start
nobody 24069 23433 0 12:04 ? 00:00:00 /usr/local/apache/bin/httpd -k start
nobody 24070 23433 0 12:04 ? 00:00:12 /usr/local/apache/bin/httpd -k start
nobody 24071 23433 0 12:04 ? 00:00:10 /usr/local/apache/bin/httpd -k start
nobody 24072 23433 0 12:04 ? 00:00:09 /usr/local/apache/bin/httpd -k start
nobody 24240 23433 0 12:04 ? 00:00:18 /usr/local/apache/bin/httpd -k start
root 30015 29909 0 13:50 pts/0 00:00:00 grep --color=auto apache

This gives us to the location of the currently running apache process. Next is to use the above path to find the location of the apache configuration file. For me, the apache path was /usr/local/apache/bin/httpd.

/usr/local/apache/bin/httpd -V | grep SERVER\_CONFIG\_FILE
# gives us:
-D SERVER\_CONFIG\_FILE="conf/httpd.conf"

In some systems, that’s the full path, but for me it was a relative path, so I needed to find the apache root:

/usr/local/apache/bin/httpd -V | grep HTTPD\_ROOT
# gives us:
-D HTTPD\_ROOT="/usr/local/apache"

Opening that file I found that the actual pagespeed configuration was included in an additional sibling file called pagespeed.conf.

less /usr/local/apache/conf/httpd.conf
# determine that I need to look in 'pagespeed.conf'
less /usr/local/apache/conf/pagespeed.conf

Being new to PageSpeed, this file was an interesting look into the various options available, however for my purposes, this line was exactly what I needed:

ModPagespeedFileCachePath "/var/cache/pagespeed/"

Flush the Cache

So now we have what we need to flush the cache — simply create a file whose mod time is used by PageSpeed to determine when to flush the cache!

touch /var/cache/pagespeed/cache.flush

The above link to the theme review handbook is quite comprehensive in regards to the steps one needs to take to help out, but this post is meant to be a quick overview of what I did to get up and running:

1. Create a new sandbox site called theme-review.dev to complete the reviews. This is easy if you use vvv, and the extremely helpful scaffolding script vv.
2. Check in with the theme review slack channel. This is a great place to introduce yourself and ask questions as they arise. If you don’t have access to WordPress’s slack channels, make sure that you have a WordPress.org account, and then ask for an invitation.
3. Finally – request a theme to review!

A couple of helpful plugins and notes in regards to setting up the testing environment.

1. Use the Developer plugin. No need to install all of the additional plugins, but a few of the following are useful.
   • Debug Bar
   • Log deprecated notices
   • Monster Widget
   • Regenerate thumbnails
   • RTL tester
   • Theme Check
   • Beta Tester
2. Install the theme meta data.
3. Install the theme check plugin

Next steps — level up theme knowledge!

When I initially conceived of biking the coast I was in a groundless spot in life. This was early in the summer of 2014, and I was in the process of getting a divorce.

A major life changing event can make you realize that many of the assumptions you have been making in your day-to-day living are no longer relevant. For me, I started to examine everything and asking myself questions like:

• “Do you really like the house you are living in? How about the city?”
• “Are you happy with the job you have?”
• “Are you living up to your full potential?”
• “Are you actively pursuing your passions?”

My initial reaction to the divorce was to leave the house, sell all my belongings, and move to a new city. In short, lose myself and find a new me. But as I examined these aspects of my life, and those underlying motives for change, I realized it wasn’t the shedding of old skin that I wanted, rather what I needed was to rediscover old passions and amplify forgotten, or under-used talents.

This means finding and improving one’s core self – and is much different from maintaining the status-quo. Part of this self introspection had now turned into a visceral need to test myself against something much larger than me. I was out to prove something.

So I turned to biking. One night over drinks I voiced a half-formed, half-joking thought of biking the coast, not really knowing what that entailed. The more I thought about it though, the more it felt like a good step.

So I started small. Since I work remotely, I began to force myself to commute. I would throw my laptop into a bag and ride seven to ten miles into Portland and set to work in one of the many coffee shops. This meant I was logging anywhere from 16 to 20 miles on my bike any day that I was commuting.

Repeating this as my new daily rhythm meant I was logging 80 to 100 miles per week on my bike, and so the prospect of a larger tour didn’t seem so impossible.

Granted, there is a sizeable difference between 100 and 380 miles in a week, but when an idea is large and far off, it is easier to squint your eyes and gloss over the differences. So planning was easy, and my routes were mostly the same paths I ended up riding.

So what did you learn?

I like going fast. Those long slow climbs are not as fun as they say, though really I suppose I have never heard anyone say that they enjoy the climbs. Some of my favorite times on the trip included coasting on the long descents with the wind at my back (Top speed of 47mph on Day 4 — Yes!).

I like the Southern Coast. Much of the Oregon Coast is beautiful, but I must say that I really was taken away by the beauty south of Port Orford. Many of the awesome geological formations that I love in the north, like Haystack rock in Cannon Beach could be seen everywhere.

Things are more fun when surrounded by good people. When I initially started to tell people I would be doing this, I was hoping, but not expecting many people to join. While I was the only person in our group to cycle the coast, things were made so much more pleasant by having my parents and girlfriend along to share morning coffee and evening campfires.

But most of all, I’ve learned the importance of aiming high and finding ways to surprise yourself.

Before starting the trip I knew I could do it. I was cocky, and didn’t really know what was to come on my first cycling tour.

On the third day, on a stop for water in Lincoln City my outlook had changed and I knew I certainly couldn’t finish. I had ten more miles for that day, and around 250 for the rest of the trip. My muscles were sore, and my butt was aching. I now had a more realistic understanding of what I had undertaken.

Still, worrying about large challenges isn’t helpful, so I pushed through and finished that day. I slept. And I finished the next day. And slept. I inched to the top of each cape and bombed each descent. By the end of day 7, I was within 20 miles of the border and I got my first flat of the trip. I fixed it and peddled on. I crossed the border and finished something I knew I couldn’t do.

I look back a couple of weeks after finishing, and still can’t quite believe that I finished. That first week back – my butt was still a little sore, but my muscles were mostly better, if just a bit tight. But now, something I thought was impossible, is now just impressive.

This change in perception has secondary benefits as well. My horizons are wider. Instead of a list of impossible tasks, and unrealistic dreams , I now have a much longer list of ‘impressive’ possibilities.

I found that there are lots of places to find information, many of which are free. In particular the Oregon Department of Transportation has this handy pdf. Of particular interest is a table listing all of the state parks and the various amenities that they offer (Hot showers! Yurts!). More detailed information pertaining to the state parks found along the way I found at oregonstateparks.org

While the above information is helpful, the best tool for actually creating my itinerary was Strava’s Route Builder. Dropping waypoints is easy, and a dynamic profile of the elevation changes is provided. After a half hour of tinkering I not only had a destination for each day, but I also have an idea of the kind of training I need to be doing.

Seeing as the worst day has an elevation gain of over 3,600 feet, I foresee laps up and down Mt. Tabor in my future!

The Cycle Schedule

Day 1 : Astoria to Nehalem Bay State Park

Nehalem Bay State Park is located just south of Manzanita, and is sandwiched between the Nehalem River and the Ocean. This day’s ride passes Cannon Beach, whose Haystack Rock is one of my favorite spots along the coast.

• Nehalem Bay State Park Information
• Day 1 Strava Route

Day 2 : Nehalem Bay State Park to Cape Lookout State Park

This stretch of the coast includes Tillamook, which everyone knows has ice cream to pair with your Bourbon. Immediately after Tillamook is the Three Capes Scenic Route, with steep climbs — and amazing views.

• Cape Lookout State Park Information
• Day 2 Strava Route

Day 3 : Cape Lookout State Park to Beverly Beach State Park

A big climb up Cape Lookout first thing in the morning, with another equally steep climb up Cascade Head about half way through the day. Beverly Beach State Park is past Lincoln City and Depoe bay, but not quite to Newport.

• Beverly Beach State Park Information
• Day 3 Strava Route

Day 4 : Beverly Beach State Park to Jessie M. Honeyman Memorial State Park

This day sees us through Yachats and Florence, with the longest day in terms of miles – 60.6.

• Jessie M. Honeyman Memorial State Park Information
• Day 4 Strava Route

Day 5 : Jessie M. Honeyman Memorial State Park to Sunset Bay State Park

And now we’re at the part of the coast that I don’t know much about — anything south of Florence is bright new territory!

• Sunset Bay State Park Information
• Day 5 Strava Route

Day 6 : Sunset Bay State Park to Humbug Mountain State Park

• Humbug Mountain State Park Information
• Day 6 Strava Route

Day 7 : Humbug Mountain State Park to California Border

• Day 7 Strava Route

However, in setting up a bare-bones LEMP server on Digital Ocean I wasn’t installing Apache, so I didn’t have access to the htpasswd utility. So, pulling together information from a number of different sources:

• http://nginxlibrary.com/password-protect-a-directory/
• http://kbeezie.com/protecting-folders-with-nginx/

I came up with this workflow:

Determine the Directory To Protect

Since you are reading this post, you probably already know this. However for the sake of this walk through let’s be thorough.

In a default LEMP stack on Ubuntu 14.04, the NGINX document root is found at:

/usr/share/nginx/html/

You can verify this by taking a look at the sites that are enabled. Open any files found in /etc/nginx/sites-enabled and look for lines that say root /usr/some/path;

For this post I’ll create a new folder and webpage under /usr/share/nginx/html/ to test and make sure things work correctly:

cd /usr/share/nginx/html
mkdir private
echo "A Protected Webpage" > private/index.html

Navigating to the url http://example.com/private/ should give you a white page with the text ‘A Protected Webpage’.

Create The Password

Since this is a LEMP stack we have PHP, so we have an easy way to generate the password to use on our protected directory:

php -r 'echo crypt("password", "salt");echo PHP_EOL;'

The resulting string of text is what we will use in our password file. It should go without saying — don’t use the word password as your password. As for salt part, that helps by giving some random magic to your password. For more info, checkout the manpage for the crypt: man crypt

So, if we did in fact use the above values of ‘password’ and ‘salt’, the resulting string should give us sa3tHJ3/KuYvI

We simply need to store this string in a file so that NGINX can access this. It is important to create this file outside of the server root. I prefer to put this near the HTML root so that it is easily accessible:

cd /usr/share/nginx
nano .passwords

Now add the user and password to that .passwords file. For NGINX the syntax is username:passwordhash:anycomments, and the resulting text to paste is:

benjamin:sa3tHJ3/KuYvI:Protecting the private folder

Save and exit nano.

Update NGINX Server Conf to Actually Protect the Folder

Finally we need to let NGINX know which folder to protect, and what file to check for valid user/password combinations. Edit the site configuration file for our site found in the sites-enabled folder. If it is an out of the box setup, that would be the default file:

nano /etc/nginx/sites-enabled/default

Within the server block, you should see a location block that looks something like:

location ~ \.php$ {
try_files $uri =404;
fastcgi_split_path_info ^(.+\.php)(/.+)$;
fastcgi_pass unix:/var/run/php5-fpm.sock;
fastcgi_index index.php;
include fastcgi_params;
}

Directly below this block (but still within the server block) add another location rule for any URLs that match our private folder, and which points to our passwords file:

location ^~ /private/ {
auth_basic "Restricted";
auth_basic_user_file /usr/share/nginx/.passwords;
}

Save this file and exit nano.

Now restart NGINX:

service nginx restart

If all went well, visiting your protected folder in a browser should prompt you for your password!

http://www.browsersync.io/

This tool is a node module, which synchronizes interactions between multiple browsers and devices, enabling multiple, concurrent views of the same state of a website.

So as an example, we would start browser-sync on the root directory of our site. A new node server listening to a specific port would be created on the local network, allowing any device to access that page. Each device that connects to this temporary site syncs various events such as scrolling, touch, active links, etc.

Additionally, when we start this server, we can set it to watch for changes to the underlying files so that all devices are updated when upon saving.

It’s like having LiveReload on steroids.

See the homepage linked to above for more in depth documentation, but really it’s a node module, so installing and immediately using browser-sync on your local site directory:

npm install -g browser-sync
    browser-sync start --server --files "css/*.css"

Breaking down the second command:

browser-sync start

--server

--files "css/*.css"

Measuring Power Performance

ibam

Working with Power Settings

Tool to help set optimal battery settings:

http://linrunner.de/en/tlp/docs/tlp-linux-advanced-power-management.html

With has a decent intro on makeusof.

To summarize, install using apt-get:

sudo add-apt-repository ppa:linrunner/tlp
sudo apt-get update
sudo apt-get install tlp tlp-rdw

Optionall for ThinkPad:

sudo apt-get install tp-smapi-dkms acpi-call-tools

Start the Service:

sudo tlp start

Vagrant Tools

• VVV (Varying Vagrant Vagrants)
• Alison Barret’s VVV Site Wizard

There are a couple of Stackoverflow discussions of what constitutes PHP shorthand, and why you would possibly not want to use them.

For me, it boils down to portability. It is fairly easy to enable shorttags in the php.ini file by setting

short_open_tag=On

However, I’d just as soon reduce the troubleshooting that I have to do if I ever move to a new environment. This happened recently as I was importing an old WordPress site into my Varying Vagrant Vagrants machine (which has shorttags disabled by default).

Using Sed

I could have used search and replace inside of my editor, however, why not use the power of Sed to quickly replace the tags? Taken even further – why not create a bash script to iterate over all PHP files in a directory?

The Sed command

sed 's:<?=:<?php echo:g' index.php | sed 's:<?:<?php:g' | sed 's:<?phpphp:<?php:g'

At the heart of this is three sed ‘substitution’ commands, piped together.

The first replaces the echo short tag <?= with the verbose <?php echo. The next pipe replaces the basic opening short tag, while the last pipe reverts any tags that originally started out correctly, but were garbled by the second sed statement.

Also note the g at the end of each sed statement. This is the global replacement flag. This is necessary to ensure that sed will operate on all matches within a line of text, not just the first match.

But it Doesn’t Update the File

So the above code is good, there is only one problem, it only pipes the output to stdout. Additionally if you try and use our friend the greater than sign to pipe it into the original file – ‘index.php’ in the above example, it will completely wipe out that file. Yikes!

Of the listed solutions in the above stackoverflow thread, I particularly like writing the contents of the pipe to a temporary file, and then moving the temporary file to the original file’s location. So our new command, which now writes the changes to the original file, looks like:

sed 's:<?=:<?php echo:g' index.php | sed 's:<?:<?php:g' | sed 's:<?phpphp:<?php:g' > index.php.tmp && mv index.php.tmp index.php

Script It!

And finally, because it would be tedious to write out that command for every file that we want to update, lets make a script file that will run our sed command for every php file:

#! /bin/bash
for f in *.php
do
echo "Converting $f"
sed 's:<?=:<?php echo:g' $f | sed 's:<?:<?php:g' | sed 's:<?phpphp:<?php:g' > $f.tmp && mv $f.tmp $f
done

Just create a new file, copy the above code to that file, make it executable, and run it!

Creating WordPress Plugins and Themes is a fairly easy process. However, creating a new WP site on a local machine can be fairly tedious. Add to this the individual quirks of servers, and the setup differences between local and production environments and and you start to appreciate the benefits of a real sand-boxed solution.

The general idea of using Vagrant is the same as the idea and justification for using Virtualbox, or VMWare to run a development server on your local machine.

What is Vagrant?

Vagrant helps to build and automate virtual machines. Where it really shines is through its ability to provision, or configure the software that the server uses through simple configuration files.

Vagrant uses base boxes, common virtual servers as its building blocks for more specific virtual machines. If you are curious about all the different ways that people are using Vagrant, checkout a couple of good introductory articles:

• Why Vagrant? – From Vagrantup.com
• Intro to Vagrant – dzone.com
• Thorough but Older Intro – LinuxJournal.com
• Installation – From Vagrantup.com

Vagrant and WordPress

Because Vagrant can be used to quickly turn a basic virtual server into a specialized WordPress installation, we can use it to easily share the specific settings for our development machine with other developers, as well as mimic the properties of the production server.

Luckily, there are a couple of very good Vagrant projects that have already been setup with WordPress.

Varying Vagrant Vagrants

Jeremy Felt has a good article outlining why he has started to use Vagrant for WordPress Development. Development is still progressing quickly, so to see the work that is currently being applied to the Vagrant Boxes running WordPress, checkout 10up’s Varying Vagrant Vagrants (VVV) repo on Github. The code from these guys is really well documented, and the README.md file has a number of good jumping off points for more learning.

In my mind, there is only one real drawback with VVV — its complexity. Even though the code is well documented, there still is a lot going on. There certainly is a lot of power under the hood, but for smaller, simpler tasks, like a quick test of a new plugin or theme, I found myself wanting a simple, LAMP stack with only one site to choose from.

Vagrantpress

And then I found Vagrantpress. This is a much simpler setup – just a LAMP stack with GIT and WordPress.

Both of these projects are good for different reasons. In general, VVV is large and powerful — almost too much — but really the right tool for most jobs. But there is something powerful in Vagrantpress’ simplicity.

At the end of the day, both projects should be watched. Active development means that both will change and improve, making collabrative development much faster and much easier.

Once downloaded, unzip the file and hack away! See the screenshots for a quick overview, or dive in and read the article.

Importing a New Appliance

1. Select File > Import Appliance

2. Click on ‘Open appliance‘ and select the file with the ‘ovf’ extension.

3. Select Next

4. The next screen is an overview of the settings contained in the ‘ovf’ file. In general the default settings should be good. The only thing I typically change is the first item ‘Name‘, changing it to something a little more meaningful.

   Scroll down to the ‘Virtual Disk Image‘. This is the physical location of the ‘hard drive’ of this appliance and can be located anywhere on your actual hard drive. Simply make sure that the file that is pointed to ends with the ‘vmdk’ file extension.

   This new ‘vmdk’ file will be created as a clone of the original, which is found in the same ‘zip’ archive as our ‘ovf’ file.

5. Click import, and watch as Virtualbox clones the disk and imports the settings!

Configuration and Settings

When we start this appliance, we can think of it as physically turning on another computer. This computer will power on, execute various startup scripts, and most importantly for our purposes, connect to a network.
We can specifically configure this connection, allowing the appliance to be available only to our machine, or appear as a completely separate machine on the local network.
To edit an appliance’s network settings, click Machine > Settings and select the Network item from the list on the left.

Bridged Adapter

The bridged adapter is the default adapter for the WordPress appliance from Turnkey Linux. Using this setting, the WordPress appliance will appear as a separate machine from our own computer on the local area network.
So for situations where the local network is trusted, for example, your home, or work network, this is great! It allows other people to connect to this appliance and use it to test functionality or update content — whatever. To the network, this appliance is just another machine, a server, serving up WordPress content.

Host-only Adapter

So what to do if you don’t trust the local network? Sure, the coffee shop’s network seems safe, but all the same, let’s keep our WordPress development for our eyes only.
For this, Virtualbox uses the ‘Host-only Adapter‘ to create – in essence – a network which exists as software only. The appliance is not connected to any physical hardware, so it cannot talk to the outside world, but neither can the outside world talk to it. Security!!
Setting up our pretend network is easy. From Virtualbox’s main menu, select File > Preferences, and select the Network item from the list on the left. Click the Add host-only network icon on the right.
When we return to the machine’s settings and select ‘Host-only Adapter’, the name should default to the network we just created, which is ‘vboxnet0‘ in the screenshot above.

Setup Complete

There are of course other settings for Virtualbox’s network. Many of these are not necessarily useful to us for WordPress development, however, for more information on the various network settings available, see the Virtualbox manual: Chapter 6: Virtual networking.
From here, our WordPress appliance is setup, but we still need to setup the actual WordPress installation on this appliance. To begin, select the appliance, press Start to turn it on, and follow the prompts to create the default passwords for our WordPress appliance.

As a content management system, WordPress excels at tracking your content. By simply applying a new theme, the feeling and tone of the underlying content can be perceived in new and expressive ways.

So, what do you do when you’re ready to go beyond tweaking a few lines in the CSS stylesheet and really begin developing your own themes and plugins for WordPress?

For starters, you can make the changes to files, live on your production server. This is quick, and works. But how long do you really want to be doing your development out in the open, for everyone to see?

Using XAMMP, you could make your local machine into a small development server. Similarly, if you are on any of the Ubuntu flavors, you could use one line in the terminal to download and install a fully functional LAMP server, complete with phpMyAdmin:

sudo apt-get install lamp-server^ phpmyadmin

This works well, and is a setup I use on my Xubuntu laptop to provide a lightweight sandbox for smaller PHP projects.

But at a certain point, setting up multiple WordPress installations becomes tedious.

The VM Way

A virtual machine (VM) is a simulation of a machine. Because of this, you can essentially start and stop any number of VMs on one physical machine, the only real limit is the performance of the underlying hardware.

At the end of the day, this VM is just software, which means we can do ‘software things’ to it!

• For a backup, copy the machine to an external location.
• To transfer a project to someone else, simply transfer the VM.
• New projects can quickly be cloned from an existing VM.

Doing any of these things using XAMMP, or the localhost of our development machine, would require copying the files, along with extracting content from the database. Do-able. And tedious.

And the drawbacks to VMs? There is one, and it is significant. VMs can be huge, taking up a few hundred megabytes of memory for each WordPress installation.

Still, the gains of a transferable package, and isolated environment, make a strong case for using a VM.

By the way, did I mention that a lot of the grunt work of setting up useful VMs has already been done:

• Turnkey Linux
• BitNami
• Drupal Quickstart
• Oracle Developer VMs
• Microsoft IE Application Compatability VPC Images

If you’re still intrigued: read about Setting Up Virtualbox in Xubuntu for WordPress Development.