wagtail-caching/slides.md

---
title: Wagtail & Caching
class: text-center
highlighter: shiki
transition: slide-left
monaco: false
mdc: true
themeConfig:
  primary: '#2e1f5e'
---

# <logos-wagtail class="fill-white" /> + Caching = ❤️

## Jake Howard{.mt-5}

### Wagtail Space NL 2024{.mt-5}

<!--
- Here to talk about Caching with Wagtail
- Dramatically improve the performance of your site
  - Sometimes, for very little effort
-->

---
layout: full
---

# `$ whoami`

<ul class="list-none! [&>li]:m-0! text-2xl mt-10">
  <li><mdi-fire class="fill-white"/> Senior Systems Engineer @ Torchbox</li>
  <li><logos-wagtail class="fill-white"/> Core, Security & Performance teams @ Wagtail</li>
  <li><mdi-server-plus class="fill-white" /> I'm an avid self-hoster</li>
  <li><mdi-robot-excited class="fill-white" /> I help students build robots in my "spare" time <small>(https://srobo.org)</small></li>
</ul>

<ul class="list-none! text-sm [&>li]:m-0! mt-10 text-xl">
  <li><mdi-earth /> theorangeone.net</li>
  <li><mdi-github /> @RealOrangeOne</li>
  <li><mdi-twitter /> @RealOrangeOne</li>
  <li><mdi-mastodon /> @jake@theorangeone.net</li>
</ul>

<div class="absolute right-3 top-3">
  <img src="/wsnl24-qrcode.png" width="190px" />
</div>

<style>
  .slidev-layout {
    background-color: #e85537;
  }
</style>

<!--
- Hi
- I'm Jake
- Senior Systems Engineer at Torchbox
- I'm also on the security team, performance team, and as of 2 weeks ago the core team for Wagtail
- I exist in many places on the internet
-->

---
layout: fact
---

# Disclaimer{style="color: #fd5765;"}

<!--
- Disclaimer: This isn't a general performance talk
  - Optimising Python, Django and Wagtail are their own topics
    - Big topics!
- Caching can be a part of optimisation
  - But it's not the only part
- It's not a replacement for proper benchmarking and investigations
-->

---
layout: section
---

# What _is_ caching?

<!--
- What is caching?
  - Caching lets your code remember things it's already done
    - Reuse a value you've already done the work to calculate
    - Because it's always slower to do something than nothing
  - Caching comes in many different forms
    - From storing a variable outside a loop
    - To caching functions
    - To caching entire pages
  - Mostly focusing on the higher end today
  - The higher up the list you are, the more smaller processes you're caching
    - And the more performance you can get
-->

---
layout: center
---

# Scale

<style>
@keyframes grow {
  from {
    font-size: 0.5rem;
  }

  to {
    font-size: 30rem;
  }
}

h1 {
  animation-name: grow;
  animation-duration: 30s;
  animation-timing-function: linear;
  animation-fill-mode: forwards;
}

</style>

<!--
- Caching may not seem needed at some scales
  - Small sites will probably be _fine_ without it
  - As your site grows, it becomes more important
    - Otherwise you'll crash under your own weight and success
- But, you may not notice you need it until it's already an issue
  - What runs fine locally may not in production
    - A problem with so many things at scale
- If a page gets popular, it'll soon meet with the hug of death
  - Where it gets overload by its on popularity
  - Or even if it's not that popular, just shared around a bit
    - Like the recently-discovered Mastodon issue
-->

---
layout: section
---

# <logos-wagtail class="fill-white"/> Caching <em>in Wagtail</em>

<!--
- Now, back to Wagtail
  - This is Wagtail Space, after all
- Let's talk about 2 techniques to help you out with caching
- Both are included in all maintained versions of Wagtail
  - So you can use them right now!
-->

---
layout: section
---

## 1:
# Template Fragment Caching

<div class="absolute right-1/2 translate-x-1/2 top-3">
  <img src="/template-fragment-caching-qrcode.png" width="150px" />
</div>

<!--
- Template fragment caching
  - Lots of complex words, but quite a simple concept
  - Introduced ish in 5.2
-->

---
layout: image-right
image: https://d1nvwtjgmbo5v5.cloudfront.net/media/images/Screen_Shot_2015-05-14_at_09.01.5.2e16d0ba.fill-1200x996.png
backgroundSize: cover
class: p-0 m-0 bg-black
---

```html
<div class="one-half--medium one-third--large">
  <div class="card card--link">
    <a href="/developers/springload/made-wagtail/" class="project-image">
      <img src="https://d1nvwtjgmbo5v5.cloudfront.net/media/images/Screen_Shot_2015-05-14_at_09.01.52.2e16d0ba.fill-680x564.png" alt="" title="">
  </a>
  <div class="project__links">
    <a class="project__info" href="/developers/springload/made-wagtail/">
      <span class="u-sr-only">Project info</span>
      <svg class="i i--grey i--hover">
        <use xlink:href="#i-info"></use>
      </svg>
    </a>
    <a class="project__visit" href="http://madewithwagtail.org/" title="Project link" data-analytics="Project|Link click">
      <span class="u-sr-only">Project link</span>
      <svg class="i i--grey i--hover">
        <use xlink:href="#i-visit"></use>
      </svg>
    </a>
  </div>
</div>
<h4 class="project-title">
  Made with Wagtail
</h4>
<p class="project-author">
  <a href="/developers/springload/" title="View the company page of Springload">
    Springload
  </a>
</p>
```

<!--
- Most of what you build in Wagtail will come out as HTML
  - Pages to be shown by the browser
- But probably takes lots of intensive actions to create it
    - Complex lists / filters or suggestions
    - Finding lots of renditions
    - Rendering streamfield blocks
    - etc etc
- These can all be optimised in many different ways
  - The deeper down you go, the more complex and often less impactful those optimisations are
- Why not cache an entire chunk of template?
  - With all of that complexity contained within it
  - Rather than cache the name of a page for the navbar
    - Cache the entire navbar
  - That's the thing people see (sort of)
-->

---
layout: cover
background: /website-homepage.png
---

## Example:
# My website
### https://theorangeone.net

<!--
- An example: My website
  - Subtle self-promotion, I know
- It's a relatively simple blog
  - Currently built with Wagtail
-->

---
layout: image
image: /website-posts.png
---

<!--
- Here's a list of posts
- Each item has a lot to it
  - Title
  - Date
  - An image (renditions or unsplash)
  - Introduction text (automatically generated from page content)
  - Tags
  - Reading time / Word count
- These all come from a few different places in code
  - Database fields
  - Model properties
  - Template tags
- It'd be nice to cache them all together as they are
  - That's fragment caching!
-->

---
layout: none
---

````md magic-move
```html
{% load wagtailcore_tags util_tags %}

<article class="media listing-item">
  <div class="columns">
    <figure class="media-left column is-{{ show_listing_images|yesno:'3,1' }} image-column">
      {% if page.list_image_url %}
        <a href="{% pageurl page %}" class="image" title="{{ page.title }}">
          <img src="{{ page.list_image_url }}" alt="{{ page.hero_image_alt }}" loading="lazy" decoding="async" referrerpolicy="no-referrer" />
        </a>
      {% endif %}
    </figure>
    <div class="media-content column">
      <div>
        {% if breadcrumbs %}
          {% include "common/breadcrumbs.html" with parents=page.get_parent_pages %}
        {% endif %}
        <h2 class="title is-3">
          <a href="{% pageurl page %}">{{ page.title }}</a>
        </h2>
        {% include "common/content-details.html" %}
        <p>{{ page.summary }}</p>
      </div>
    </div>
  </div>
</article>
```

```html {all|1-3,27|4-26|all}
{% load wagtailcore_tags wagtail_cache %}

{% wagtailpagecache FRAGMENT_CACHE_TTL "listing-item" breadcrumbs show_listing_images %}
  <article class="media listing-item">
    <div class="columns">
      <figure class="media-left column is-{{ show_listing_images|yesno:'3,1' }} image-column">
        {% if page.list_image_url %}
          <a href="{% pageurl page %}" class="image" title="{{ page.title }}">
            <img src="{{ page.list_image_url }}" alt="{{ page.hero_image_alt }}" loading="lazy" decoding="async" referrerpolicy="no-referrer" />
          </a>
        {% endif %}
      </figure>
      <div class="media-content column">
        <div>
          {% if breadcrumbs %}
            {% include "common/breadcrumbs.html" with parents=page.get_parent_pages %}
          {% endif %}
          <h2 class="title is-3">
            <a href="{% pageurl page %}">{{ page.title }}</a>
          </h2>
          {% include "common/content-details.html" %}
          <p>{{ page.summary }}</p>
        </div>
      </div>
    </div>
  </article>
{% endwagtailpagecache %}
```
````

<style>
.slidev-page, .slidev-code-wrapper, .slidev-code {
  height: 100%;
}
</style>

<!--
- Let's look at the template
- I like to keep reusable components in separate files
  - Makes them easier to work with
  - And makes fragment caching much easier too
- This has no caching at all
  - Every time I used the template, the template is evaluated
    - All the template tags are run
  - [click]Let's cache it
    - Nice and simple
  - It's just 2 lines of change[click]
- The tag might look a bit strange
  - It's not an inclusion or a filter
    - It's a fully custom template tag
  - Let's look at its structure
    - A name: To uniquely identify this template fragment from others
    - A TTL: To determine how long the cache is valid for
    - Whatever variables the tag needs
      - So if the variables change, the cache is automatically invalidated
      - Don't show listing images? Different cache!
    - [click]Put the HTML you want in the middle
      - It's still a Django template, so calling tags etc is completely fine
        - They'll be cached too
- [click]The template is in charge of caching itself
  - Keep the complexities hidden, and in 1 place
  - Pages using it don't need to worry about it too much
-->

---
layout: cover
background: /website-search.png
class: text-right
---

# What about<br>search?

<!--
- The 'listing item' is a component, it's used in a few different places
  - There's a posts list, and a search
    - And a few others
  - But the UI for each item is the same
- It takes the same context: A page
  - And creates the same HTML
- If it looks the same, just on different pages, can I reuse it?
  - Including its cache?
  - Yes!
-->

---
layout: center
---

```html {3}
<section class="container">
  {% for page in listing_pages %}
    {% include "common/listing-item.html" %}
  {% endfor %}
</section>
```

<style>
code {
  font-size: 1.5rem !important;
}
</style>

<!--
- Include the template somewhere else, and the cache gets reused
  - So long as the right variables are available in context
- Shared components across your site can be accelerated
  - As can entire sections like the navbar
-->

---
layout: image
image: /website-search.png
---

<!--
- Caching HTML blocks has a few nice benefits over caching the whole page
  - For example, a search page
    - Searches are still performed live, but the display of the results are cached
    - Published pages appear immediately, without needing to invalidate a cache
-->

---
layout: full
---

# Updating cached content

<v-click>

```python
>>> BlogPostPage.objects.first().cache_key
```

</v-click>

<br />

<v-click>

```python
@property
def cache_key(self) -> str:
    """
    A generic cache key to identify a page in its current state.
    Should the page change, so will the key.

    Customizations to the cache key should be made in :attr:`get_cache_key_components`.
    """
    ...
```

</v-click>

<style>
code {
  font-size: 1rem !important;
}
</style>

<!--
- Cache invalidation is one of the 2 hardest problems in computer science
  - Alongside naming things and off-by-one errors
- If I change a page, I want the content to update
  - How does it know when to regenerate the HTML?
- [click]`page.cache_key`
  - Custom attribute on all your pages
  - [click]You can use it for other things if you need it
- If a change is published, or the page is moved, the cache key changes, so the existing cache is ignored
  - And the old cached item will eventually expire
- If that's not enough, and you want to manually clear the cache, you can do that too
-->

---
layout: center
---

# What about Django?

````md magic-move
```jinja
{% load cache %}
{% cache 500 sidebar request.user.username %}
    .. sidebar for logged in user ..
{% endcache %}
```

```jinja
{% load wagtail_cache %}
{% wagtailpagecache 500 sidebar request.user.username %}
    .. sidebar for logged in user ..
{% endwagtailpagecache %}
```

```jinja
{% load wagtail_cache %}
{% wagtailcache 500 sidebar request.user.username page.cache_key site.id %}
    .. sidebar for logged in user ..
{% endwagtailcache %}
```
````

<!--
- This might sound familiar to some
  - Django's `{% cache %}` tag
  - This isn't new, Django has had a cache tag for a very long time
    - In fact, 2007
- But it's dangerous to use it with Wagtail unmodified
  - Preview content can get cached
  - Per-site settings cache the wrong value
  - A host of other weird bugs
- [click]It's only a few small changes to use Wagtail's `wagtailpagecache`
  - Which wraps this tag, but is aware of the current page and site automatically
- If you modify the page, the cache is automatically invalidated
  - If you use the on different sites, they're cached separately
- If that's not enough, and you want something more manual
  - [click]There's `wagtailcache`
  - This doesn't automatically use the page and site
  - But _does_ ignore preview requests
  - Making it a closer analogue for Django's `cache` tag
-->

---
layout: section
---

## 2:
# Frontend Caching

<!--
- Part 2
- Frontend caching
- You can spend ages optimising a request
  - Caching common chunks of HTML
  - But not handling the request at all is even better
-->

---
layout: fact
---

```mermaid
flowchart LR
  U[User]
  CDN{Content<br>Delivery Network}
  W[(Wagtail)]

  U--->CDN
  CDN-..->W
```

<!--
- What if we put something in front of Wagtail to serve the requests instead
  - A CDN (eg Cloudflare, CloudFront)
  - They cache the page, and serve it themselves
- A CDN generally helps you in 2 main ways:
-->

---
layout: fact
---

```mermaid
flowchart TD
  U1[🧑‍💻]
  U2[🧑‍💻]
  U3[🧑‍💻]
  U4[🧑‍💻]
  U5[🧑‍💻]
  U6[🧑‍💻]
  U7[🧑‍💻]
  U8[🧑‍💻]
  U9[🧑‍💻]
  U10[🧑‍💻]
  U11[🧑‍💻]
  U12[🧑‍💻]
  U13[🧑‍💻]
  CDN{Content<br>Delivery Network}
  W[(Wagtail)]

  U1--->CDN
  U2--->CDN
  U3--->CDN
  U4--->CDN
  U5--->CDN
  U6--->CDN
  U7--->CDN
  U8--->CDN
  U9--->CDN
  U10--->CDN
  U11--->CDN
  U12--->CDN
  U13--->CDN
  CDN-...->W
```

<!--
- Firstly: Serve a static copy of your website, so your servers aren't processing every request
- Allows you to handle many more requests concurrently
  - Cloudflare handles 50 million requests per second globally (on average)
  - Your servers can't
- Handle more traffic with fewer servers
- Imagine it like converting your site to be static
  - 0 Database queries
  - 0 templates used
  - 0 cache hits
  - 0 lines of code
-->

---
layout: image
image: https://www.cloudflare.com/network-maps/cloudflare-pops-2O04nulSdNrRpJR9fs9OKv.svg
class: bg-white
---

<!--
- Secondly: Store said cached copy closer to your users
  - Reduced latency
  - Makes your site seem faster
  - Even helps on your Google Lighthouse scores
- Both of these are things you'll want as your website scales
-->

---
layout: cover
background: /wagtail-homepage.png
---

## Example
# https://wagtail.org

<!--
- Let's look at an example: Wagtail.org
-->

---
layout: image
image: /map.jpg
transition: fade
---

<!--
- Wagtail.org is hosted in Dublin, Ireland (AWS `eu-west-1`)
- But I'm in Arnhem
  - Quite far away
-->

---
layout: cover
background: /map.jpg
transition: fade
---

## Arnhem &rarr; Ireland
# ~25ms

<style>
h1 {
  font-size: 4rem;
}
</style>

<!--
- 25ms away
- That's just the connection
  - If we tried to load a page from here, it takes closer to 300ms
- So how are page loads taking just 28ms?
  - That's the CDN (Cloudflare, in our case)
-->

---
layout: cover
background: /map.jpg
---

## Arnhem &rarr; Amsterdam
# ~7ms

<style>
h1 {
  font-size: 4rem;
}
</style>

<!--
- This was actually served from Amsterdam
  - Just 7ms from me
- It's closer to us
  - Much faster (28ms)
  - Users get what they want faster
- That 18ms penalty is paid multiple times
  - It's not once per page load
  - It's not even once per request
  - It can be a few times per request
  - So it starts to add up!
-->

---
layout: fact
---

```mermaid
flowchart LR
  U[User]

  subgraph Content Delivery Network
    CDN{Content<br>Delivery Network}
    C[(Cache)]
  end

  W[(Wagtail)]

  U---->CDN
  CDN-.->C
  C---->W
  C-.->CDN
```

<!--
 Let's cache our content!
- Put a caching layer between users and the site
  - Requests are sent via cache
  - If they're cached, return them
    - Otherwise, get the actual content from Wagtail
- Ensure the CDN knows how long to cache pages for
  - Somewhere between "Forever" and "1 second"
    - An hour is probably fine
  - Lots of different options
    - Cache headers
    - Configuration in the CDN itself
    - There's another talk about this
-->

---
layout: fact
---

```mermaid
flowchart LR
  U[User]
  A[Admin]

  subgraph Content Delivery Network
    CDN{Content<br>Delivery Network}
    C[(Cache)]
  end

  W[(Wagtail)]

  U---->CDN
  CDN-.->C
  C---->W
  C-.->CDN

  A---->CDN
  CDN---->W
```

<!--
- Make sure you skip caching authenticated requests (ie the admin)
  - These pages are dynamic based on who is viewing them, so we can't cache them easily
  - Wagtail does its best to make sure the admin is fast anyway
  - Exactly how to do this will again depend on your CDN
-->

---
layout: section
---

# Content is cached!

```mermaid
flowchart LR
  U1[User]
  U2[User]
  U3[User]

  subgraph Content Delivery Network
    CDN{Content<br>Delivery Network}
    C[(Cache)]
  end

  W[(Wagtail)]

  U1===>CDN
  U2===>CDN
  U3===>CDN
  CDN==>C
  C==>CDN
  C-.....->W
```

<!--
- Content is now cached
  - Pages load much faster
  - Server load should drop drastically
    - Might even be able to scale down
  - Wagtail.org serves ~70% of all page loads from the CDN
    - And with some work we could probably get that higher
-->

---
layout: image
image: /wagtail-homepage-typo.png
transition: fade
---

<!--
- Uh oh, there's a typo on one of the pages, better update it
  - Easy, just publish an update to the page
    - A quick visit to the admin makes that a breeze
  - The cache still has the old content
    - Users see the incorrect content
  - But how does the CDN know the content has changed?
-->

---
layout: cover
background: /wagtail-homepage-typo.png
---

# It doesn't*

<!--
- Simple: It doesn't!
  - The whole point of it is it doesn't need to hit Wagtail for every request
  - So it can't know
- The cache will expire _eventually_
  - After an hour or so
  - What if it needs to go out sooner
- What if it's more serious than just a typo
-->

---
layout: section
---

## Solution:
# Frontend Cache Invalidation
### Wagtail secret sauce <logos-wagtail class="fill-white"/>

<div class="absolute right-1/2 translate-x-1/2 top-3">
  <img src="/frontend-caching-qrcode.png" width="150px" />
</div>

<!--
- Frontend cache invalidation!
- Wagtail to the rescue
  - Some magic Wagtail sauce
-->

---
layout: fact
---

```mermaid
flowchart LR
  U[User]

  subgraph Content Delivery Network
    CDN{Content<br>Delivery Network}
    C[(Cache)]
  end

  W[(Wagtail)]

  U--->CDN
  CDN-->C
  C-->CDN
  C--->W

  W-.->|Frontend Cache Invalidation|CDN
```

<!--
- Tighter integrations between Wagtail and the CDN
- Allows Wagtail to instruct the CDN to clear its cache for a given page
  - Purging the cache much quicker
-->

---
layout: center
---

# `settings.py`


<v-click>

### 1.

```python {3}
INSTALLED_APPS = [
    ...
    "wagtail.contrib.frontend_cache"
]
```

</v-click>

<v-click>

#### 2.

```python
WAGTAILFRONTENDCACHE = {
  'cloudflare': {
    'BACKEND': 'wagtail.contrib.frontend_cache.backends.CloudflareBackend',
    'BEARER_TOKEN': os.environ["CF_BEARER_TOKEN"],
    'ZONEID': os.environ["CF_ZONE_ID"],
  },
}
```

</v-click>
<v-click>

#### 3. ???

#### 4. Profit!

</v-click>

<style>
code {
  font-size: 0.9rem !important;
}
</style>

<!--
- Integrating is simple
  - [click]Add entry to `INSTALLED_APPS`
  - [click]Add configuration and credentials
    - Wagtail supports quite a few backends!
- [click]That's it!
  - All the hooks into all the right places are set up for you!
    - You can customize this as you need for listing pages / custom configuration etc
-->

---
layout: fact
---

```mermaid
flowchart LR
  A[Admin]

  subgraph Content Delivery Network
    CDN{Content<br>Delivery Network}
    C[(Cache)]
  end

  W[(Wagtail)]

  A-->|Publish new content to<br>blog post 1|W

  W-.->|<code>PURGE /blog/post-1</code>|CDN

  CDN-.->C
```

<!--
- When you publish a page, it'll automatically be purged
  - It can take a minute or so, depending on your CDN
    - Usually much faster than that
  - Then when a user loads the page from Wagtail, it's cached, and subsequent loads go back to being lightning fast
-->

---
layout: cover
background: https://d1nvwtjgmbo5v5.cloudfront.net/media/images/Screen_Shot_2015-05-14_at_09.01.5.2e16d0ba.fill-1200x996.png
---

# Conclusion

<!--
- When most people think of "performance", they tend to think "database queries"
  - Which is true, that's where a lot of time goes
- But there's more to optimising a site than sprinkling `select_related` / `prefetch_related`
- "Performance" means lots of things to different people
  - Your application is made up of lots of different components
  - So too must your performance investigations
- It's faster to do less than to do more
  - It's faster still to do nothing than something
- The higher up the stack your cache is, the more impactful it can be
  - More of your code is cached
  - But a slow page is still a slow page, even if it's not called as often
-->

---
layout: cover
---

# Performance means:

<Transform :scale="1.5">

<v-clicks>

- Users are happier <mdi-emoticon-happy />
- Servers are happier <mdi-server-network />
- The planet is happier <mdi-earth class="text-green" />

</v-clicks>

</Transform>

<!--
- So why repeat yourself? Make things faster!
  - [click]Users are happier
  - [click]Servers are happier
  - [click]The planet is happier.
-->

---
layout: end
---

END