1
Fork 1

Refine speaker notes

This commit is contained in:
Jake Howard 2024-06-02 22:13:25 +01:00
parent 1d1f846753
commit a3d48c2be9
Signed by: jake
GPG key ID: 57AFB45680EDD477

145
slides.md
View file

@ -20,8 +20,8 @@ themeConfig:
<ul class="list-none! text-xs [&>li]:m-0! mt-5"> <ul class="list-none! text-xs [&>li]:m-0! mt-5">
<li><mdi-earth /> theorangeone.net</li> <li><mdi-earth /> theorangeone.net</li>
<li><mdi-twitter /> @RealOrangeOne</li>
<li><mdi-github /> @RealOrangeOne</li> <li><mdi-github /> @RealOrangeOne</li>
<li><mdi-twitter /> @RealOrangeOne</li>
<li><mdi-mastodon /> @jake@theorangeone.net</li> <li><mdi-mastodon /> @jake@theorangeone.net</li>
</ul> </ul>
@ -157,8 +157,8 @@ flowchart TD
- They keep requests quick - They keep requests quick
- Move the slow bits somewhere else - Move the slow bits somewhere else
- So the user doesn't have to wait - User doesn't have to wait
- This improves throughput and latency - Improves throughput and latency
--> -->
--- ---
@ -197,7 +197,7 @@ layout: section
- But that doesn't mean they're useful for everything, all the time - But that doesn't mean they're useful for everything, all the time
- As with all great things: "It depends" - As with all great things: "It depends"
- The added complexity may not be worth it - Trade-off between complexity and functionality
- A few things to consider - A few things to consider
--> -->
@ -209,11 +209,11 @@ background: https://images.unsplash.com/photo-1518729371765-043e54eb5674?q=80&w=
# Does it take time?{.text-right} # Does it take time?{.text-right}
<!-- <!--
- Something which takes time - Does it take time
- Something which _could_ take time - _Could_ it take time
- Rather than have the user wait - Don't want to make the user wait
- Unable to close the tab or do something else - Unable to close the tab or do something else
- Go off and do it in the background, and let them know whether it's done - Go off and do it in the background, and let them know whether it's done
- Even if that's by polling it in the browser - Even if that's by polling it in the browser
--> -->
@ -247,11 +247,11 @@ flowchart BT
<!-- <!--
- Leaving your infrastructure - Leaving your infrastructure
- The core components (Server, DB, Cache etc) you control and can closely monitor - The core components (Server, DB, Cache etc) you control and can closely monitor
- And are in a good position to fix it if something goes wrong - And are in a good position to fix it if something goes wrong
- That's not true for external APIs - That's not true for external APIs
- It's someone else's SRE team - It's someone else's SRE team
- Their performance regressions shouldn't affect your app - Their performance characteristics shouldn't affect your app
--> -->
--- ---
@ -263,7 +263,7 @@ background: https://images.unsplash.com/photo-1518770660439-4636190af475?q=80&w=
<!-- <!--
- Maybe it's less about when, more about where? - Maybe it's less about when, more about where?
- Maybe it's more about the hardware it runs on - Maybe it's more about the hardware it runs on
- GPUs - GPUs
- Loads of RAM - Loads of RAM
- External hardware - External hardware
@ -281,10 +281,9 @@ background: https://images.unsplash.com/photo-1711606815631-38d32cdaec3e?q=80&w=
<!-- <!--
- An example: Complex reporting - An example: Complex reporting
- Something analytical, crunching lots of data - Something analytical, crunching lots of data
- Initially, it might take a few seconds, which is fine - It might be fast locally
- You build a CSV as part of the request with something like `pandas`
- As your application grows, there'll be more data, so it'll likely take a lot longer - As your application grows, there'll be more data, so it'll likely take a lot longer
- Rather than force the user to wait, let them get the data when it's ready - Rather than force the user to wait, let them get the data when it's ready
- They can get back on with their day - They can get back on with their day
- Web servers can get back to processing other requests - Web servers can get back to processing other requests
--> -->
@ -381,15 +380,14 @@ for user in page.subscribers.iterator():
- [click]This works perfectly fine - [click]This works perfectly fine
- Scales _relatively_ well - Scales _relatively_ well
- But has some issues - But has some issues
- If connecting to the email server takes a while, the user has to wait - If connecting to the email server takes a while, the user has to wait
- Usually only a few ms - Usually only a few ms
- Might take a few seconds - Might take a few seconds
- Subsequent emails are delayed whilst we process the earlier ones
- If something goes wrong with one email, the others won't send - If something goes wrong with one email, the others won't send
- What if your email gateway is down altogether - do your requests start erroring? - What if your email gateway is down altogether - do your requests start erroring?
- How do you handle it if they do? - How do you handle it if they do?
- That web worker (eg gunicorn) can't process any other requests until this is done - That web worker (eg gunicorn) can't process any other requests until this is done
--> -->
--- ---
@ -508,7 +506,7 @@ for user in page.subscribers.iterator():
- Its own features - Its own features
- Its own configuration - Its own configuration
- Its own caveats / implementation details - Its own caveats / implementation details
- What if we wanted to switch to Celery? - What if we wanted to use Celery instead?
- [click]Well, that's easy - [click]Well, that's easy
- [click]Just change a few lines - [click]Just change a few lines
- [click]But there in lies the problem - [click]But there in lies the problem
@ -525,21 +523,20 @@ backgroundSize: 50%
<!-- <!--
- It's hard enough having multiple options - It's hard enough having multiple options
- But how do you choose between them? - But how do you choose between them?
- If you've used multiple tools, you probably know which is best for you - Maybe you have experience with libraries already
- Do you have the time (and patience) to test each one out? - Do you have the time (and patience) to test each one out?
- Maybe you already have a standard you need to work to - Maybe you already have a standard you need to work to
- Do you need a background worker which supports `asyncio`? - Maybe you need specific features
- If you're new to Django, do you really want to spend the time weighing them all up? - If you're new to Django, do you really want to spend the time weighing them all up?
- Knowing it could bite you as you grow or need a specific feature - Knowing it could bite you as you grow or need a specific feature
- Requiring a lot of time refactoring in future - Requiring a lot of time refactoring in future
- What about library maintainers - What about library maintainers
- What if you built a library which has some background task needs
- Like, say, Wagtail - Like, say, Wagtail
- Do you write and maintain integrations for _all_ task libraries - Do you write and maintain integrations for _all_ task libraries
- Do you choose the big one(s) and force your users' hands? - Do you choose the big one(s) and force your users' hands?
- Do you expose a hook and let your users integrate themselves? - Do you expose a hook and let your users integrate themselves?
- It adds a huge maintenance burden, whichever you choose - It adds a huge maintenance burden, whichever you choose
- There isn't really a right answer - There isn't really a right answer
--> -->
@ -551,8 +548,10 @@ backgroundSize: 49%
<!-- <!--
- There _should_ be one universal standard which combines them all - There _should_ be one universal standard which combines them all
- A single API to help developers use a library, without tieing their hands - A single API to help developers use a library
- First-party, allowing library developers to depend on it instead of supporting every separate API - Without tieing their hands
- First-party
- Allowing library developers to depend on it instead of supporting every separate API
- Scale easily as your needs change - Scale easily as your needs change
- Be easy to get started with for small projects - Be easy to get started with for small projects
- But feature-packed for larger deployments - But feature-packed for larger deployments
@ -670,11 +669,11 @@ for user in page.subscribers.iterator():
- Instead, let's rewrite this once to use `django.tasks`[click] - Instead, let's rewrite this once to use `django.tasks`[click]
- Still simple, clear, approachable and easy to use - Still simple, clear, approachable and easy to use
- If I say so myself - If I say so myself
- Now, in future, if we swapped to RQ (for whatever reason), exactly 0 lines need to change - If we swapped to RQ: 0 lines need to change
- If a new library comes out, 0 lines need to change - If a new library comes out, 0 lines need to change
- If this is in a library, not my own code, I can keep using the library no matter what worker I'm using - If this is in a library, not my own code, I'm not constrained by their preferences
- And the maintainer doesn't need to special case - And the maintainer doesn't have extra work to support my preferences
- If I want to test this code, I can swap out the backend to an in-memory one, and interrogate it - For testing, I can use an in-memory backend
- With 0 lines changed - With 0 lines changed
--> -->
@ -735,7 +734,7 @@ class: flex justify-center text-2xl flex-col
- As I've been told whilst working on this DEP - As I've been told whilst working on this DEP
- Why not just vendor something existing? - Why not just vendor something existing?
- If not Celery, then something else - If not Celery, then something else
- That's not really the goal - That's not really the goal
- Shared API contract is - Shared API contract is
- The built-in version will hopefully become great - The built-in version will hopefully become great
- But must be done with careful planning and consideration - But must be done with careful planning and consideration
@ -760,25 +759,25 @@ class: flex justify-center text-xl flex-col
<!-- <!--
- Being built-in reduces the battier to entry - Being built-in reduces the battier to entry
- Integrating becomes much simpler - Integrating becomes much simpler
- There's 1 API to learn, and it will last you a while - There's 1 API to learn
- Much like the ORM has done for different DB engines - It will last you a while
- Scale with your needs
- A developer can join a new project and already be productive - A developer can join a new project and already be productive
- A common API also helps library maintainers - A common API also helps library maintainers
- Maintaining a large library is work enough - Maintaining a large library is work enough
- Without needing to think about how to move code to the background - Without needing to think about how to move code to the background
- If Django can take complexity off you, that's great - If Django can take complexity off you, great
- Currently, it's not really an option - Currently, it's not really an option
- The burden is just too great - The burden is too great
- With this, no additional dependencies - No additional dependencies for your library
- Just import from Django and you're set - Just import from Django and you're set
- The user can use what they want - The user can use what they want
- Or what's suitable for their scale and use case - Or what's suitable for their scale and use case
- Now the barrier is reduced, the ecosystem can flourish - Now the barrier is reduced, the ecosystem can flourish
- Libraries can start assuming background workers, without any additional burden - Libraries can assume background workers, without any additional burden
- The ORM backend should work for the majority of projects - The ORM backend should work for the majority of projects
- If you just want to send emails in the background, you probably don't need Celery - If you just want to send emails in the background, you probably don't need Celery or RQ
- It's overkill - It's overkill
- Even RQ is a bit much
- A vendored solution makes it the easiest to get started with - A vendored solution makes it the easiest to get started with
- Tweak some settings, run an extra process, and you're done. - Tweak some settings, run an extra process, and you're done.
--> -->
@ -836,12 +835,11 @@ layout: center
- And I've had many times - And I've had many times
- ElasticSearch is quite likely better for the ~10% of people who need it - ElasticSearch is quite likely better for the ~10% of people who need it
- But that doesn't mean the other 90% of people won't be happy with PostgreSQL - But that doesn't mean the other 90% of people won't be happy with PostgreSQL
- And probably wouldn't benefit from ElasticSearch anyway - Probably wouldn't benefit from ElasticSearch anyway
- And definitely won't get a return on the extra hosting cost and complexity - Definitely won't get a return on the extra hosting cost and complexity
- They'll be perfectly happy with Postgres FTS - They'll be perfectly happy with Postgres FTS
- Let them get started the easiest way possible - Let them get started the easiest way possible
- We can still invite them into ElasticSearch when they're ready - We can still invite them into ElasticSearch when they're ready
--> -->
--- ---
@ -850,6 +848,10 @@ layout: section
# Where are we now? # Where are we now?
<!--
- I mean, other than Vigo
-->
--- ---
layout: image layout: image
image: /dep.png image: /dep.png
@ -873,7 +875,7 @@ layout: section
- The ORM backend is where the magic happens - The ORM backend is where the magic happens
- Tell me about all the bugs in my code - Tell me about all the bugs in my code
- The more testing we can do now, the better - The more testing we can do now, the better
- There are still features to implement and improve - There's still work to do
--> -->
--- ---
@ -885,15 +887,14 @@ layout: section
<!-- <!--
- More testing - More testing
- Upstreaming - Upstreaming
- The main benefit is in this becoming part of Django - That's the big benefit
- The DEP is approved (ish) - Else it really is just another standard
- Once `django-tasks` is in a better state, it can become `django.tasks` - Once `django-tasks` is in a better state, it can become `django.tasks`
- Hopefully in time for the 5.2 release window - Hopefully in time for the 5.2 release window
- Adoption - Adoption
- The more people know about this, the better it is for everyone - The more people know about this, the better it is for everyone
- Developers can start using `django-tasks` now, and swap for `django.tasks` later - Developers can start working on integrating now
- Or use both side-by-side in older versions of Django / older libraries - Knowing they can trivially upgrade once it's in Django
- The 2 will work correctly together inside the same project
--> -->
--- ---
@ -913,12 +914,13 @@ background: /celery.svg
<!-- <!--
- Is this the end for Celery and alike? - Is this the end for Celery and alike?
- Not at all! - Not at all!
- If you're using Celery, you've not made a mistake - You've not made a mistake
- It's a great choice - It's a great choice
- They have quite a head start - They have quite a head start
- This is much more about usability and flexibility - This is much more about usability and flexibility
- If you need certain features, keep using them! - If you need certain features, keep using them!
- But, now you have the option of a nice, Django-native API - Now you have the option of a Django-native API
- Which could even be Celery under the hood
--> -->
--- ---
@ -941,11 +943,13 @@ class: flex justify-center flex-col text-xl
<!-- <!--
- The world of background workers is huge - The world of background workers is huge
- There are countless nice features
- Not everything is making it into the initial version(s) - Not everything is making it into the initial version(s)
- And that's ok! - And that's ok!
- Existing libraries have a head start - Existing libraries have a head start
- But I hope we can slowly catch them up - But I hope we can slowly catch them up
- Bringing the stability and longevity guarantees that come with Django - Bringing the stability and longevity guarantees that come with Django
- Doesn't mean they'll never come
--> -->
--- ---
@ -958,14 +962,16 @@ background: https://images.unsplash.com/photo-1519187903022-c0055ec4036a?q=80&w=
<!-- <!--
- The future is bright though - The future is bright though
- In time, I see more and more people reaching to `django.tasks` - In time, I see more and more people reaching to `django.tasks`
- And background workers in general
- Moving work to the background will make Django apps _seem_ faster - Moving work to the background will make Django apps _seem_ faster
- Improve throughput - Improve throughput
- Reduce latency - Reduce latency
- Improve reliability - Improve reliability
- Gone are the days of needing additional research and testing to find the tooling you need - Gone are the days of needing additional research and testing to find the tooling you need
- You can use the ones built-in to Django - You can use the ones built-in to Django
- And as you scale, if you find you need to swap something out, you can - And as you scale, it's easy to change
- _without_ rewriting half your application - _without_ rewriting half your application
- With all the knowledge to make an informed decision
--> -->
--- ---
@ -975,12 +981,7 @@ layout: section
# What's next? # What's next?
<div class="absolute right-1/2 translate-x-1/2 mt-12"> <div class="absolute right-1/2 translate-x-1/2 mt-12">
<QRCode <img src="/django-tasks-qrcode.png" width="110px" />
:width="120"
:height="120"
data="https://pypi.org/project/django-tasks/"
:dotsOptions="{ color: 'white' }"
/>
</div> </div>
<!-- <!--
@ -994,16 +995,36 @@ layout: section
- There's plenty of work to do - There's plenty of work to do
- And I can't do it alone! - And I can't do it alone!
- If you maintain a worker library - If you maintain a worker library
- Or have been burned by one - Or have been burned by one...
- let's chat!
--> -->
---
layout: center
class: text-center text-xl
---
# Let's chat!
<ul class="list-none! [&>li]:m-0!">
<li><mdi-earth /> theorangeone.net</li>
<li><mdi-github /> @RealOrangeOne</li>
<li><mdi-twitter /> @RealOrangeOne</li>
<li><mdi-mastodon /> @jake@theorangeone.net</li>
</ul>
<div class="absolute right-1/2 translate-x-1/2 mt-3">
<img src="/dceu24-qrcode.png" width="150px" />
</div>
<style>
.slidev-layout {
background-color: #17181c;
color: #e85537;
}
</style>
--- ---
layout: end layout: end
--- ---
END END
<div class="absolute right-1/2 bottom-4 translate-x-1/2">
<img src="/dceu24-qrcode.png" width="110px" />
</div>