Glenn Jones

Hello 👋 Welcome to my corner of the internet. I write here about the different challenges I encounter, and the projects I work on. Find out more about me.

Autonomous management of static websites with Prismic.io, CircleCI and Github pages

Prismic.io is pretty nice in that it allows you to focus on the things that are relevant to you as a developer / designer. The content is simply not your problem anymore.

At Nebulae2016, we host most of our pages through Github Pages, which not only is free but has many other advantages. This means you need to use static pages. But we prefer to use SLIM for templating and SASS for css - we find this allows us to be more expressive and effective. So we use a static page generator, of which our preference is Middleman.

Like other static page generators, Middleman has a ‘deploy’ command (through the gem middleman-deploy), which builds the project in your local environment and then deploys to the attached github repositories’ gh-pages branch.

The project we developed this system for, needed one source repository through which multiple versions (so, multiple branches in the source and multiple prismic repositories) could be deployed to different deployment repo’s. That looks like this:

‘template’ (middleman source - branch) + ‘content’ (prismic api) => ‘build-result’ (in different repo from source).

To make things easier, we’ve automated the build+deploy process with CircleCI, so that once a content-change gets published in prismic, within 90 seconds, the new version is live. This is what we call ‘autonomous’ management. This post will outline how we did that.

Birds-eye view

Disclaimer: I assume you know how to deal with the Prismic API, and will be going through some steps quite quickly.

When a content-manager updates content on prismic, a webhook can be set to be triggered. You do this in config > webhooks. The trigger is a post to a specified URL, with a configurable ‘secret’ param attached.

From the webhook trigger, CircleCI should start building the build from the appropriate source files (calling the Prismic API in the process). To determine which branch of the source repo and which prismic-api-url the build should use, you send parameters (see Parametrized builds, currently in early access preview). However, prismic only allows a secret parameter while CircleCI uses the specific build_parameters to send parameters to the build. These are as you see, incompatible.

That’s where Google Apps Script (GAS) comes in. The different prismic content source repositories will post to the same GAS script, with a different secret. The GAS script will parse it, determine the source and send to CircleCI a post with the appropriate build_parameters.

Why GAS? It’s easy to write and understand, even easier to deploy and has relatively low latency (certainly better than a free heroku dyno). If anyone has better suggestions, please let me know (@nebulae2016).

So then the build knows which settings to use and effectively how to build the project. It also knows where to deploy to, even if you use different deploy repos, using GH_TOKEN - I will get into this later.

In your project you might want to use a bunch of helpers to ease using the prismic API data, either the official ones (see here and here) or read my earlier blogpost. Mind however that my previous and this post are slightly incompatible.

In detail - walking through the components

Prismic.io repository

Make a repository with the custom types that you plan to use. Fill in some dummy content. Remember the api-endpoint (setting > API & Security).

Github source repo (middleman)

Make a middleman project and include the gems

# Gemfile
gem 'prismic.io', '~> 1.3', '>= 1.3.3', require: 'prismic'
gem 'middleman-deploy'

To build a different project per environment in CircleCI, I use environment variables. Thus, your local build environment also needs that ENV variable as supplied in CircleCI. So go into your project root, touch .env and add a variable. I’ve called the variable location but this could be anything.


# .env

location=paris

Don’t forget to add .env to your .gitignore.

Now in your config.rb, your settings must depend on this env variable. In my case a different location meant a host of different settings: prismic-api-url, deploy-repo owner, deploy-repo url, deploy CNAME. This bloats your config.rb, which I think should only contain functional settings.

So I made a separate yaml that contains all this info per location:


// locations.yml

paris:

  prismic_url: https://lorem-paris.prismic.io/api

  deploy_repo_owner: LoremParis

  deploy_repo_name: sometestpage.loremparis.co

  domain_name: sometestpage.loremparis.co

  tracked_link: http://external-link.com/?utm_source=ipsum-paris

berlin:

  prismic_url: https://lorem-berlin.prismic.io/api

  deploy_repo_owner: LoremBerlin

  deploy_repo_name: othertestpage.loremberlin.co

  domain_name: othertestpage.ipsumberlin.co

  tracked_link: http://external-link.com/?utm_source=ipsum-berlin

Then to the top of the config.rb, add:


require 'yaml'

set :location_settings, YAML.load(File.read("locations.yml"))

For accessing settings in the config.rb, use:


def get_loc_setting(setting_name)

    location = ENV['location']

    config[:location_settings][location.to_s][setting_name.to_s]

end

Now when you want a setting, for example the prismic_url, you simply do get_loc_setting('prismic_url') or get_loc_setting(:prismic_url).

If you also want to deploy to separate repo’s, depending on the ‘location’, you can use the following settings. For this to work you’ll need to get a GH_TOKEN from github (only the ‘repo’ scope is necessary) and add it to your .env file.

def location_based_remote
    repo_owner =    get_loc_setting(:deploy_repo_owner)
    repo_name =     get_loc_setting(:deploy_repo_name)
    return "https://#{ENV['GH_TOKEN']}@github.com/#{repo_owner}/#{repo_name}.git"
end

activate :deploy do |deploy|
    deploy.deploy_method = :git
    deploy.remote = location_based_remote
    deploy.build_before = true
    deploy.commit_message = "Automated deploy at #{Time.now}"
end

Next, don’t get rid of the token just yet. Time to move to the next component. Commit your changes and push to github.

Make sure your .env is not publicly visible. The GH_TOKEN gives anyone with it the right to do changes to your github repositories.

CircleCI

Add the project from your Github list and let it build. It’s likely to fail, but that’s ok. The next step is to configure circleci on how to build your project (official documentation). The steps of your build are defined in a circle.yml file.

Since CircleCI is normally for testing and middleman doesn’t have tests, we have to alter the circle.yml a little.


// circle.yml

general:

  branches:

    ignore:

      - /.*/

machine:

  timezone: UTC

  ruby:

    version: 2.3.0

  environment:

    MM_ENV: production

  pre:

    - "git config --global user.name 'CircleCI'"

    - "git config --global user.email 'lorem@ipsum.com'"

dependencies:

  cache_directories:

    - vendor/bundle

  pre:

    - sudo apt-get update

    - sudo apt-get -yV install mecab

    - sudo apt-get -yV install mecab-ipadic-utf8

test:

  override:

    - echo "No tests are needed"

deployment:

  autodeploy:

    branch: [paris, berlin]

    commands:

      - bundle exec middleman deploy

I’ve set it up so that any pushes to the brances in the source repo get ignored by CircleCI so that builds are only triggered by prismics’ webhooks (through the CircleCI API - that’s coming up next). Tests are overridden and you can see that deployment is only allowed on the branches that are explicitly defined under autodeploy. Autodeploy is just a name I made up because CircleCI required a heading under deployment.

Add, commit and push the changes to your github repo.

Lastly add an api token in your CircleCI account dashboard. You’ll need this token to communicate with the Circle API, which we handle in the next step.

Google Apps Script

The GAS connects the prismic webhook (with a unique secret per repo) to the CircleCI api. The Circle trigger in this case should be a “New build with a branch”.

The following small script makes the connection. I am assuming that the name of the branch that you’ll want to build is the same as the value of the environment variable (in this case, ‘paris’ or ‘berlin’).

function doPost(request) {  
  var jsonString = request.postData.getDataAsString();
  var dataAsJSON = JSON.parse(jsonString);
  var senderIdentity = determinePrismicSender(dataAsJSON)
  triggerBuild(senderIdentity, dataAsJSON)
}

function determinePrismicSender(json){
  if ( json.secret == 'paris-447590409' )       { return 'paris' };
  if ( json.secret == 'berlin-749930999' )      { return 'berlin' }
}

function triggerBuild(sender, data) {

  if ( sender == 'undefined' ) { 
    GmailApp.sendEmail(
            'admin@email.com,
            'trigger attempt without valid secret',
            data);
    return;
  };

  var circle_api_url = buildCircleApiURL(sender)
  var build_params = {"build_parameters" : { "location" : sender }};

  var options = {
    "method" : "post",
    "contentType" : "application/json",
    "payload" : JSON.stringify(build_params)
  }

  UrlFetchApp.fetch(circle_api_url, options);
} 

function buildCircleApiURL(sender) {
  return "https://circleci.com/api/v1/project/repo-owner-name/repo-name/tree/" + sender + "?circle-token=your-circle-token-here";
}

For the secret you can denounce it with the city like I’ve done

You need to deploy that google apps script (click the cloud with the arrow). Important! Execute the app as ‘me’ and allow ‘anyone’ to use the app. This way prismic can access the script.

I’ve found the best way to check if accessibility is ok, is to visit the app url in the browser. Sometimes you need to validate once with your chosen google account. Then do the same in a private window and if you’ve done things right, you should see a Script function not found: doGet. Otherwise, make a change and update to a new version and repeat the above ‘validation’ steps.

Hope it helps!

Links

Previous: Rails 4 has many through source with condition
Next: Visudo: sudoers file busy, try again later