archive/theorangeone.net-legacy
1
Fork 0
Code Activity

The repo has moved to GitLab now

Browse source 
I wouldn't normally update a post like this, but as it was so recent, it makes sense
This commit is contained in:
Jake Howard 2021-06-06 20:59:12 +01:00
parent 6e7c98fe30
commit c7dac93cc6
Signed by: jake
GPG key ID: 57AFB45680EDD477
1 changed files with 8 additions and 8 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

16
content/posts/website-deployment-process.md
View file

@ -16,23 +16,23 @@ I do all my content writing locally, in a variety of different tools depending o
And again, because it's a static site, spinning up the whole thing locally is a breeze. I use the dev server both to check the content is rendering properly and to make any non-content changes like styling. And again, because it's a static site, spinning up the whole thing locally is a breeze. I use the dev server both to check the content is rendering properly and to make any non-content changes like styling.
## Push to GitHub ## Push to GitLab
The source for the site lives in a `git` repository, which makes versioning and syncing incredibly simple. At the moment, the [canonical repository](https://github.com/realorangeone/theorangeone.net) lives on GitHub, so yes you can go see all the source (and judge me all you want). The source for the site lives in a `git` repository, which makes versioning and syncing incredibly simple. At the moment, the [canonical repository](https://git.theorangeone.net/theorangeone/theorangeone.net) lives on [my GitLab server](https://git.theorangeone.net/) which is mirrored to [GitHub](https://github.com/realorangeone/theorangeone.net), so yes you can go see all the source (and judge me all you want).
As a developer, I use `git` quite a lot, and know how to do anything I could realistically need to with it. It can be quite a [complex tool](https://xkcd.com/1597/), but it's incredibly powerful. As a developer, I use `git` quite a lot, and know how to do anything I could realistically need to with it. It can be quite a [complex tool](https://xkcd.com/1597/), but it's incredibly powerful.
## Continuous integration ## Continuous integration / Continuous delivery
Whenever code is pushed to GitHub, the site is automatically run through CI. For this I use [GitHub Actions](https://github.com/RealOrangeOne/theorangeone.net/actions), as it's completely free for open-source projects, and is nicely integrated with the rest of GitHub. During the build, the site is built and formatting [checked](https://github.com/RealOrangeOne/theorangeone.net/blob/master/scripts/test.sh) it meets my OCD nature. This makes sure that the site works perfectly before it's deployed, so what you read is always perfect (ish). Whenever code is pushed, the site is automatically run through CI/CD. For this I use [GitLab CI](https://git.theorangeone.net/theorangeone/theorangeone.net/-/pipelines), as it's nicely integrated with the rest of my GitLab. During [the CI](https://git.theorangeone.net/theorangeone/theorangeone.net/-/blob/master/.gitlab-ci.yml), the site is built and formatting checked it meets my OCD nature. This makes sure that the site works perfectly before it's deployed, so what you read is always perfect (ish).
## Upload to server ## Upload to server
Once the site is built, it's not very useful sat in GitHub actions, it needs deploying to the world. Static sites are by their nature stateless - all you need are the files. Given it's me, the site itself is hosted on my own servers. Once the site is built, it's not very useful sat in the CI artifacts, it needs deploying to the world. Static sites are by their nature stateless - all you need are the files. Given it's me, the site itself is hosted on my own server.
There are quite literally hundreds of ways to move files between servers. A lot of people quite like using SSH and `rsync`, but for me, I'd rather not do things like that. Key management is annoying, and I normally reject all SSH traffic not over a VPN, which I'd have to change. I [previously](https://github.com/RealOrangeOne/theorangeone.net/blob/33258916726b917ed1f673cd3c6b42c452ef00c8/.github/workflows/deploy.yml#L46) used the AWS CLI to upload to [minio](https://min.io/), but found minio far heavier than I really needed, not to mention that the performance really wasn't great ([over a minute](https://github.com/RealOrangeOne/theorangeone.net/runs/2325913989?check_suite_focus=true) to upload my site). There are quite literally hundreds of ways to move files between servers. A lot of people quite like using SSH and `rsync`, but for me, I'd rather not do things like that. Key management is annoying, and I normally reject all SSH traffic not over a VPN, which I'd have to change. I [previously](https://git.theorangeone.net/theorangeone/theorangeone.net/-/blob/33258916726b917ed1f673cd3c6b42c452ef00c8/.github/workflows/deploy.yml#L46) used the AWS CLI to upload to [minio](https://min.io/), but found minio far heavier than I really needed, not to mention that the performance really wasn't great ([over a minute](https://github.com/RealOrangeOne/theorangeone.net/runs/2325913989?check_suite_focus=true) to upload my site).
Once the site is built, I use [`rclone`](https://github.com/RealOrangeOne/theorangeone.net/blob/master/.github/workflows/deploy.yml#L46) to upload it via WebDAV to nginx. WebDAV is a beautifully simple protocol with very minimal overhead, `rclone` is a powerful upload tool and nginx is also incredibly lightweight. The same process is used for my [notes](https://notes.theorangeone.net/), and a couple other sites. The upload process takes just a few seconds, a huge improvement over the previous minio-based approach - I don't know whether this is from minio or `rclone`, but I'm happy with how things work now. Once the site is built, I use [`rclone`](https://git.theorangeone.net/theorangeone/theorangeone.net/-/blob/6e7c98fe304bf53c68b3f1932ecc405f3d4c938e/.gitlab-ci.yml#L61) to upload it via WebDAV to nginx. WebDAV is a beautifully simple protocol with very minimal overhead, `rclone` is a powerful upload tool and nginx is also incredibly lightweight. The same process is used for my [notes](https://notes.theorangeone.net/), and a couple other sites. The upload process takes just a few seconds, a huge improvement over the previous minio-based approach - I don't know whether this is from minio or `rclone`, but I'm happy with how things work now.
Because the files are uploaded in-place, the deployment isn't blue-green, and it's theoretically possible for a race condition in content, but given the number of requests I get, it's unlikely to happen. I've also not had any reports of it, so it's not really worth looking at yet. Because the files are uploaded in-place, the deployment isn't blue-green, and it's theoretically possible for a race condition in content, but given the number of requests I get, it's unlikely to happen. I've also not had any reports of it, so it's not really worth looking at yet.
@ -44,7 +44,7 @@ Whilst the uploads are done to nginx, nginx isn't used to serve it - it's far mo
Requests are served using [`traefik-pages`](https://github.com/realorangeone/traefik-pages), a tool I wrote to serve files from a directory, and [hook](https://github.com/realorangeone/traefik-pages#how-it-works) into Traefik to advertise domains and some powerful middleware. It's a project which is quite complex, and solves likely quite a niche issue, but I think it's super useful - hence writing it. And the icing on the cake: It's written in Rust! :tada: Requests are served using [`traefik-pages`](https://github.com/realorangeone/traefik-pages), a tool I wrote to serve files from a directory, and [hook](https://github.com/realorangeone/traefik-pages#how-it-works) into Traefik to advertise domains and some powerful middleware. It's a project which is quite complex, and solves likely quite a niche issue, but I think it's super useful - hence writing it. And the icing on the cake: It's written in Rust! :tada:
In the past I've also used a [custom nginx container](https://github.com/RealOrangeOne/theorangeone.net/blob/86dff22e02372554806a7dda61a53ec9e0f3ba1c/Dockerfile), my own GitLab pages, and even nginx on the host. In the past I've also used a [custom nginx container](https://git.theorangeone.net/theorangeone/theorangeone.net/-/blob/582425d7033d939180473084624eebe5b8dac220/Dockerfile), my own GitLab pages, and even nginx on the host.
## Future ## Future