The long winding road to…

My website development journey started with free hosting providers like Xoom.
Later I jumped onto the blogging bandwagon. Just because.
Was drawn to create my own digital garden: a wikidot wiki, and eventually a WordPress site to streamline the accumulated mess.

digital garden: a space where you can seed thoughts, sprout them to seedling ideas, transplant ‘em to a dedicated pot page, and nurture them to project plants.
Or in the absence of care, have the seedlings perish untended in the weeds.

Each time, the workflow felt sub-optimal.
Blogspot was simple; wikidot’s in-place editing was convenient. Both lacked a unified online/offline workflow however. Backups would go out of sync pretty quick.
WordPress didn’t offer much relief. An XAMPP (Apache, MySQL, PHP) stack to sync? No thanks.

Jamstack (b. 2014) & SSGs explained

Jamstack was the name that stuck to an alternative way of building static sites with dynamic behaviour.

Static implies no programming in the traditional sense. The infrastructural burden of managing a database and web server that exists in a traditional 3-tier application, is replaced by using REST APIs as the backend. The API provider manages the infra.

The Jam in Jamstack:

  • Javascript to provide dynamic behaviour
  • APIs to access backend services as building blocks
  • Markdown to create content as plain-text

A static site generator (SSG) tool runs the build process, taking the markdown, converting it to a static html site with dynamic behaviour. Jekyll is one such SSG. It’s packaged as a Ruby gem (library).

Jekyll’s power comes from processing directives embedded in the markdown frontmatter (a header section), specified using a templating language called Liquid.

Jekyll is a static site generator written in the Ruby language, packaged as a Ruby gem (library in Ruby parlance). It uses a templating language called Liquid for processing dire

Jamstack website publishing workflow

Jamstack brings a developer-centric workflow to web publishing, bridging the online/offline divide beautifully:

  • Site content is created as markdown, and stored in a Git repository.
  • Git Hooks enable a CDN provider to tap the events in your repository
    • whenever you push a change to the Git server, the SSG supported by the CDN will pull those updates from the repo
  • the SSG will trigger a build and publish your site on the CDN

Git: a distributed code versioning system.
Your code and repository exist on your local machine – so you can work offline.
When ready, push your updates to the remote server (online Git repository).

Content Distribution Network (CDN): a network of edge data-centers that offer low latency distribution of content to end-users. Edge simply means data centers close to dense regions of population, i.e. close to the audience.
Jekyll is supported by both GitHub Pages & Cloudflare Pages. I decided to go with CloudFlare as it offers free SSL and works with private repos. Just in case.
Cloudflare does nudge you to choose their newer Workers Site for serverless capabilities. I didn’t feel like jumping through an additional hoop though, as it looks like Workers doesn’t support Jekyll SSG build process. (Would I need to generate the static assets offline and configure Workers to deploy them?)

Up and running…

This is a long article. It documents my journey to getting my Jamstack site up and running. Jumplinks:

TOC

  • installing Jekyll
  • creating a new site
  • deploying to Cloudflare
  • exporting Wordpress site and import into Jekyll site
  • applying a Jekyll theme

Installing Jekyll

Install rbenv

The Ruby environment manager, rbenv, makes it simpler to get Ruby on your system and switch between multiple installed versions.

installed versions are at ~/.rbenv/versions/
rbenv works by intercepting invocations of ruby executables to check the version of ruby set in your project_dir/.ruby-version and locates it in the installed Ruby versions.

# Install latest rbenv by cloning the repo
mkdir ~/.rbenv
git clone https://github.com/rbenv/rbenv.git ~/.rbenv

# set up shell to load rbenv
~/.rbenv/bin/rbenv init
Install the Ruby pre-requisite dependencies
sudo apt-get install autoconf patch build-essential rustc libssl-dev libyaml-dev libreadline6-dev zlib1g-dev libgmp-dev libncurses5-dev libffi-dev libgdbm6 libgdbm-dev libdb-dev uuid-dev
# consumes ~581 MB
Install Ruby
# first list available ruby versions
rbenv install -L

# If that fails (no such command):
# install the ruby-build plugin by cloning the repo
git clone https://github.com/rbenv/ruby-build.git "$(rbenv root)"/plugins/ruby-build
# placed at ~/.rbenv/plugins/ruby-build

# listing Ruby versions should now succeed. I picked the latest for install:
rbenv install 3.4.4
# tar.gz download: 22 Mb. Install takes: 149 Mb

#set as global:
rbenv global 3.4.4
Follow Jekyll installation guide

Jekyll requires, Ruby, Ruby Gems, gcc / g++ and make.

# verify
ruby -v
gem -v
gcc -v
g++ -v
make -v

Set up GEM_HOME:

# 1. '>>' Appends lines to your ~/.bashrc
echo '# Install Ruby Gems to ~/gems' >> ~/.bashrc
echo 'export GEM_HOME="$HOME/gems"' >> ~/.bashrc
echo 'export PATH="$HOME/gems/bin:$PATH"' >> ~/.bashrc

source ~/.bashrc

To find a particular version directory:
rbenv prefix x.y.z

If you need to uninstall a Ruby version, simply
rm -rf ~/.rbenv/versions/x.y.z
ruby-build also provides an rbenv uninstall to automate the removal process.

(rbenv command reference)

# 2. install jekyll and bundler gems
gem install jekyll bundler
# consumes 46 Mb

At this stage, you’ve successfully installed Jekyll.

Creating a Jekyll site

cd devsrc/ruby/jekyll # location for creating the new site

## alternatively, you could use Bundler to create a new Gemfile for your project's dependencies:
# bundle init
# edit it in a text editor to add the jekyll dependency:
# gem "jekyll"
# or do bundle add jekyll
# and follow along to the [steps listed on the jekyll website](https://jekyllrb.com/docs/step-by-step/01-setup/)

jekyll new silveira_in # choose your site name
cd silveira_in # replace with your site name

# verify the site by serving it locally
bundle exec jekyll serve
# bundle being a Ruby command, rbenv kicks in and brings the appropriate Ruby version into play
# `jekyll build` - builds & outputs static site to '_site' folder  
# `jekyll serve` - builds & serves site on a local web server

At this stage, you will have a default site with boilerplate content.

Before we start customizing it, let’s associate it with a Git repo, so we may use Git’s push workflow to publish changes.

Git for Gits

  • Git: a distributed versioning and change management system used by software developers.
  • GitHub: cloud hosted Git repository server. Microsoft acquired it in a billion-dollar deal. Lets you backup your public/private repos for free.
  • GitLab: another cloud service. It adds CI/CD worfklows for software projects, on top of regular Git repositories. Basically offers an integrated build pipleine, to build, run tests, package, and deploy.
Git worfklow
  1. git commit: saves your modifications in your local repository
  2. git push: takes your locally committed changes, and publishes them to a remote repository for others to fetch at a subsequent time

    the remote repo has an associated name.
    The default remote is usually called origin
    A remote url can be either HTTPS or SSH.

Create a Git repo for the Jekyll site
  1. Begin by initializing a local Git repo
git init

## if you've got a new, unused git install locally
git config --global user.email "<userid@mailhost.com>"
git config --global user.name "<given_name family_name>"
  1. Create a remote repo at github.com and link it to your local repo’s origin

  • Go to github.com and create a repository for your website. This will give you a url like https://github.com/user/repo.git

  • your local repo has a default pointer to the remote repo. It is called origin.
    Link this default remote pointer to the remote url you just created:

git remote add origin https://github.com/user/repo.git

# verify remote
git remote -v

# get in sync with remote (pre-requisite to push)
rm .gitignore # to get rid of conflict
git pull origin main

git add . # stages changes for including in a snapshot

git status # check after add, and after commit

git commit -m "jekyll initial build" # saves the snapshot tothe pojrect history

# push locally committed changes to remote (origin) and main branch
git push origin main # just dont' type master here, or you're in for some more master-main pain, see below:

git fetch # get branch list update just in case
git branch # verify what branch you're on (-r for remote / -a for all)
git checkout main # switch to main branch if you weren't on it. trouble!

A word about master and main: spare yourself some pain
Git uses the concept of branches. A branch is where you do parallel feature development of any kind, before you’re ready to merge it into the trunk or mainline.

Previously this main/trunk line was called master.

In deference to ‘Black Lives Matter’ movement (rambo cops senselessly felled persons of colour in routine checks), the term was changed to main to remove vestiges of racial holdovers (master-slave).

The bulk of QnA on the Web (StackOverflow) referring to master rather than the newer main can be a source of confusion. A caveat to be aware of.

Operations on remote repository failing?

Git switched to remote authentication using Personal Access Tokens (PAT) instead of passwords some time ago. PATs are like an access pass.

  • Login to your Github account and naviate to Setting > Generate a PAT.

    Tokens have a limited validity, and may be fine (can restrict rights) or coarse-grained (easier to provision).

  • At login, when doing a push, use the PAT in place of the password.
Password: <type your password or personal access token (GitHub)
  
# to cache the given record in your computer, so you don't need to enter the token every time:
git config --global credential.helper cache

# Now try to pull with -v to verify
git pull -v

If you need to delete the PAT cache record:

git config --global --unset credential.helper
git config --system --unset credential.helper

Transferring WordPress to Jekyll

  1. Export from WordPress

Open your WordPress Admin console at:

https://YOUR-USER-NAME.wordpress.com/wp-admin/export.php ,or,

https://YOUR-USER-NAME.<your_domain.tld>/wp-admin/export.php

Export your WordPress site content:
Settings > Tools > Export > (*) All Content > Download Export File

The file will be generated with the naming convention, ‘.WordPress..xml'

Rename it to wpexport.xml and copy it over to your Jekyll site root (your project).

  1. Setup Jekyll import gem and dependencies for WordpressDotCom importer
gem install jekyll-import
gem install safe_yaml sequel unidecode # dependencies for jekyll-import
gem install open_uri_redirections # for WordpressDotCom importer
  1. Import the WordPress content dump
# in the project root
ruby -r rubygems -e 'require "jekyll-import";
JekyllImport::Importers::WordPressDotCom.run({
"source" => "wpexport.xml"
})'

# ,or, (any one, not both)
bundle exec jekyll-import WordPressDotCom --source wpexport.xml

This will work, downloading images too:

Downloading: images for An exercise in specifying an MMF for BDD
http://silveira.in/wp-content/uploads/2020/04/Tech-Learning-1024x279.png
OK!
Imported 6 pages
Imported 3 nav_menu_items
Imported 2 attachments
Imported 1 custom_css
Imported 5 posts

** Jekyll importers: WordPress vs. WordpressDotCom**

The naming led me to assume the former is for self-hosted deployments and the latter for the eponymous .com hosting provider service.

I didn’t bother to check the WordpressDotCom importer page at the cost of a couple of hours lost.

Turns out the former is for direct url & db connection to the wordpress website, while the latter is for use with an exported wordpress dump file. Aaargh!

Using the former, results in the error:
LoadError: cannot load such file -- mysql2 (Sequel::AdapterNotFound)
because it depends on MySQL gem for connectivity to the WP database.

Frustrated I tried the latter importer going against my initial assumption, and that worked :P

The importer names are case-sensitive!

Transferring Blogspot to Jekyll

  1. Export from Blogger.com

Login to Blogger.com and navigate to your blog > Settings > Manage Blog > Back up Content > Download

Gets you a file named blog-<MM-DD-YYY>.xml with the exported dump.

Copy this over to the Jekyll project root and rename blogger-export.xml

  1. Download jekyll import dependencies and import
gem install rexml safe_yaml

jekyll-import blogger --source blogger-export.xml --no-blogger-info --replace-internal-link --comments

This works for posts, but embedded images are linked from blogger.com

For instance, https://photos1.blogger.com/img/11/6342/640/d04570502.jpg

Track 2: rupeshtiwari’s gist ruby script:

  • install dependencies: gem install feedjira httparty

  • Copy blogspot_to_jekyll.rb ruby script to the jekyll site root and make the following changes:

  • line 104: xml = HTTParty.get({"http://roopkt.blogspot.com/feeds/posts/default"}).body

change this to xml = HTTParty.get({feed_url}).body

  • lines 39, 40 , ie, the block:

title = post.title

file_name = creation_date.to_s + "-" + title.split(/ */).join("-").delete('\/') + ".html"

with:


title = post.title

safe_title = title.gsub(/[^0-9A-Za-z.\- ]/, '').strip.gsub(/\s+/, '-')

file_name = "#{creation_date}-#{safe_title}.html"

My import was failing due to special characters in the blog titles, like ‘?’. This edit is for sanitizing file names.

  • now run the script from the project root:

ruby blogspot_to_jekyll.rb http://{blog_address}.blogspot.com/feeds/posts/default

You can get the blog_address from Settings > Publishing > Blog Address


ruby blogspot_to_jekyll.rb http://iamthelonewolf.blogspot.com/feeds/posts/default

# Fetching feed http://iamthelonewolf.blogspot.com/feeds/posts/default...

# Parsing feed...

# Writing posts to _posts/...

# Done!

In the end however, I realised that this script does not download images at all. I KNOW. Blind Alley.

I copied images from Media Files in the Blogger console and that was the end of it.

Adding Obsidian to the Workflow

I had thought I’d keep my workflow simple and use Visual Studio Code for editing the markdown. However even with the preview feature editing Markdown in VS Code is not fun. I discovered a blog that referred to Obsidian as a new wave of knowledge management tools. Looked further and it offered a DAG graph concept of linking notes that was intriguing. So decided to give it a shot. Even without having tried those features, the ability to view your markdown in preview mode without switching to another window is a plus. So all you need to do is point your Obsidian vault (create from existing folder) to the _pages folder in your Jekyll site.

Adding PlantUML

  1. Install PlantUML 
    sudo apt update
sudo apt install graphviz
sudo apt install plantuml
  2. Add this line to Gemfile:

gem “jekyll-plantuml”

and the plugin to _config.yml: plugins:

  • jekyll-plantuml

Then Get the plugin through Bundler:

$ bundle install

Now you can use plantuml to generate graphics.

Blogroll