This commit is contained in:
parent
cd626caaba
commit
d751b46308
1 changed files with 39 additions and 24 deletions
63
slides.md
63
slides.md
|
@ -66,7 +66,7 @@ flowchart LR
|
||||||
- Django is a web framework
|
- Django is a web framework
|
||||||
- It's a magic box which turns HTTP requests into HTTP responses
|
- It's a magic box which turns HTTP requests into HTTP responses
|
||||||
- What you do inside that box is up to you
|
- What you do inside that box is up to you
|
||||||
- For something like a blog, that's probably as far as it needs to go
|
- For something like a blog, that's probably as far as it needs to go
|
||||||
-->
|
-->
|
||||||
|
|
||||||
---
|
---
|
||||||
|
@ -150,11 +150,11 @@ flowchart TD
|
||||||
<!--
|
<!--
|
||||||
- You need a background worker
|
- You need a background worker
|
||||||
- But[click]
|
- But[click]
|
||||||
- What are background workers
|
- What _are_ background workers
|
||||||
- Let you offload complexity outside of the request-response cycle
|
- Let you offload complexity outside of the request-response cycle
|
||||||
- To be run somewhere else, potentially at a later date
|
- To be run somewhere else, potentially at a later date
|
||||||
|
|
||||||
- They keep requests quick
|
- They keep requests nice and fast
|
||||||
- Move the slow bits somewhere else
|
- Move the slow bits somewhere else
|
||||||
- User doesn't have to wait
|
- User doesn't have to wait
|
||||||
- Improves throughput and latency
|
- Improves throughput and latency
|
||||||
|
@ -195,9 +195,10 @@ layout: section
|
||||||
- Background workers are very useful tool
|
- Background workers are very useful tool
|
||||||
- But that doesn't mean they're useful for everything, all the time
|
- But that doesn't mean they're useful for everything, all the time
|
||||||
|
|
||||||
- As with all great things: "It depends"
|
- As with all great things: "It depends" when they're useful
|
||||||
- Trade-off between complexity and functionality
|
- Trade-off between complexity and functionality
|
||||||
- A few things to consider
|
- If you're considering whether an action makes sense in the background:
|
||||||
|
- There are a few things to consider...
|
||||||
-->
|
-->
|
||||||
|
|
||||||
---
|
---
|
||||||
|
@ -209,7 +210,7 @@ background: https://images.unsplash.com/photo-1518729371765-043e54eb5674?q=80&w=
|
||||||
|
|
||||||
<!--
|
<!--
|
||||||
- Does it take time
|
- Does it take time
|
||||||
- _Could_ it take time
|
- Or _could_ it take time
|
||||||
- Don't want to make the user wait
|
- Don't want to make the user wait
|
||||||
- Unable to close the tab or do something else
|
- Unable to close the tab or do something else
|
||||||
- Go off and do it in the background, and let them know whether it's done
|
- Go off and do it in the background, and let them know whether it's done
|
||||||
|
@ -245,7 +246,7 @@ flowchart BT
|
||||||
```
|
```
|
||||||
|
|
||||||
<!--
|
<!--
|
||||||
- Leaving your infrastructure
|
- Does control leave your infrastructure?
|
||||||
- The core components (Server, DB, Cache etc) you control and can closely monitor
|
- The core components (Server, DB, Cache etc) you control and can closely monitor
|
||||||
- And are in a good position to fix it if something goes wrong
|
- And are in a good position to fix it if something goes wrong
|
||||||
- That's not true for external APIs
|
- That's not true for external APIs
|
||||||
|
@ -312,8 +313,8 @@ layout: section
|
||||||
|
|
||||||
<!--
|
<!--
|
||||||
- Back to Django
|
- Back to Django
|
||||||
- This is djangocon after all
|
- This is Djangocon after all
|
||||||
- In Python and Django, there are lots of different frameworks to achieve this
|
- In Python and Django, there are lots of different frameworks to achieve background workers
|
||||||
-->
|
-->
|
||||||
|
|
||||||
---
|
---
|
||||||
|
@ -338,7 +339,7 @@ image: https://images.unsplash.com/photo-1444703686981-a3abbc4d4fe3?q=80&w=1740&
|
||||||
|
|
||||||
<!--
|
<!--
|
||||||
- All require an external library
|
- All require an external library
|
||||||
- And possibly some external infrastructure
|
- And possibly some external infrastructure
|
||||||
- Celery is probably the biggest one
|
- Celery is probably the biggest one
|
||||||
- But it's not all that exists
|
- But it's not all that exists
|
||||||
- So many different libraries exist
|
- So many different libraries exist
|
||||||
|
@ -550,6 +551,7 @@ backgroundSize: 50%
|
||||||
- Do you write and maintain integrations for _all_ task libraries
|
- Do you write and maintain integrations for _all_ task libraries
|
||||||
- Do you choose the big one(s) and force your users' hands?
|
- Do you choose the big one(s) and force your users' hands?
|
||||||
- Do you expose a hook and let your users integrate themselves?
|
- Do you expose a hook and let your users integrate themselves?
|
||||||
|
|
||||||
- It adds a huge maintenance burden, whichever you choose
|
- It adds a huge maintenance burden, whichever you choose
|
||||||
- There isn't really a right answer
|
- There isn't really a right answer
|
||||||
-->
|
-->
|
||||||
|
@ -565,7 +567,8 @@ backgroundSize: 49%
|
||||||
- A single API to help developers use a library
|
- A single API to help developers use a library
|
||||||
- Without tieing their hands
|
- Without tieing their hands
|
||||||
- First-party
|
- First-party
|
||||||
- Allowing library developers to depend on it instead of supporting every separate API
|
- Allowing library developers to depend on it
|
||||||
|
- Instead of supporting every separate API
|
||||||
- Scale easily as your needs change
|
- Scale easily as your needs change
|
||||||
- Be easy to get started with for small projects
|
- Be easy to get started with for small projects
|
||||||
- But feature-packed for larger deployments
|
- But feature-packed for larger deployments
|
||||||
|
@ -597,7 +600,7 @@ class: flex items-center text-xl
|
||||||
|
|
||||||
- API contract between library and application developers
|
- API contract between library and application developers
|
||||||
- Swappable backends through `settings.py`
|
- Swappable backends through `settings.py`
|
||||||
- Built in backends:
|
- Built in implementations:
|
||||||
- ORM
|
- ORM
|
||||||
- "Immediate"
|
- "Immediate"
|
||||||
- "Dummy"
|
- "Dummy"
|
||||||
|
@ -608,8 +611,8 @@ class: flex items-center text-xl
|
||||||
- An API contract between worker library maintainers and application developers
|
- An API contract between worker library maintainers and application developers
|
||||||
- Compatibility layer between Django and their native APIs
|
- Compatibility layer between Django and their native APIs
|
||||||
- Hopefully the promise of "Write once, run anywhere"
|
- Hopefully the promise of "Write once, run anywhere"
|
||||||
- Built-in worker queues
|
- Built-in implementations
|
||||||
- ORM based (production grade)
|
- ORM based (production grade, building on the power of the ORM)
|
||||||
- "Immediate" (ie doesn't background anything) loaded by default
|
- "Immediate" (ie doesn't background anything) loaded by default
|
||||||
- Dummy (for testing)
|
- Dummy (for testing)
|
||||||
- Hopefully landing in Django 5.2
|
- Hopefully landing in Django 5.2
|
||||||
|
@ -680,13 +683,14 @@ for user in page.subscribers.iterator():
|
||||||
- Let's look at the same code example as before
|
- Let's look at the same code example as before
|
||||||
- This is tied to Celery
|
- This is tied to Celery
|
||||||
- If want to support RQ too, I'd have to duplicate some parts
|
- If want to support RQ too, I'd have to duplicate some parts
|
||||||
- Instead, let's rewrite this once to use `django.tasks`[click]
|
- Instead, let's write this once to use `django.tasks`[click]
|
||||||
- Still simple, clear, approachable and easy to use
|
- Still simple, clear, approachable and easy to use
|
||||||
- If I say so myself
|
- If I say so myself
|
||||||
- If we swapped to RQ: 0 lines need to change
|
- If we swapped to RQ: 0 lines need to change
|
||||||
- If a new library comes out, 0 lines need to change
|
- If a new library comes out, 0 lines need to change
|
||||||
- If this is in a library, not my own code, I'm not constrained by their preferences
|
- If this is in a library, not my own code, I'm not constrained by their preferences
|
||||||
- And the maintainer doesn't have extra work to support my preferences
|
- And the maintainer doesn't have extra work to support my preferences
|
||||||
|
- They can use what they like, I can use what I like
|
||||||
- For testing, I can use an in-memory backend
|
- For testing, I can use an in-memory backend
|
||||||
- With 0 lines changed
|
- With 0 lines changed
|
||||||
-->
|
-->
|
||||||
|
@ -726,9 +730,9 @@ EMAIL_BACKEND = "django.core.mail.backends.tasks.SMTPEmailBackend"
|
||||||
<!--
|
<!--
|
||||||
- In this case, we can actually make it even easier
|
- In this case, we can actually make it even easier
|
||||||
- Because email is such a common use case, and so easy to extract
|
- Because email is such a common use case, and so easy to extract
|
||||||
- Go back to the simple implementation
|
- Let's go back to the simple implementation
|
||||||
- No background workers in sight
|
- No background workers in sight
|
||||||
- [click]Use the built-in task email backend
|
- [click]Instead, we can change the email backend
|
||||||
- Emails are magically sent in the background automatically
|
- Emails are magically sent in the background automatically
|
||||||
- Without additional work
|
- Without additional work
|
||||||
-->
|
-->
|
||||||
|
@ -753,6 +757,7 @@ class: flex justify-center text-2xl flex-col
|
||||||
- The built-in version will hopefully become great
|
- The built-in version will hopefully become great
|
||||||
- But must be done with careful planning and consideration
|
- But must be done with careful planning and consideration
|
||||||
- Django needs to remain the stable and reliable base it always has been
|
- Django needs to remain the stable and reliable base it always has been
|
||||||
|
- Don't want to burn-out the Django maintainers
|
||||||
-->
|
-->
|
||||||
|
|
||||||
---
|
---
|
||||||
|
@ -778,9 +783,10 @@ class: flex justify-center text-xl flex-col
|
||||||
- Scale with your needs
|
- Scale with your needs
|
||||||
- A developer can join a new project and already be productive
|
- A developer can join a new project and already be productive
|
||||||
- A common API also helps library maintainers
|
- A common API also helps library maintainers
|
||||||
- Maintaining a large library is work enough
|
- Maintaining a library is work enough
|
||||||
- Without needing to think about how to move code to the background
|
- Without needing to think about how to move code to the background
|
||||||
- If Django can take complexity off you, great
|
- If Django can take complexity off you, great
|
||||||
|
- Use the tools provided to you
|
||||||
- Currently, it's not really an option
|
- Currently, it's not really an option
|
||||||
- The burden is too great
|
- The burden is too great
|
||||||
- No additional dependencies for your library
|
- No additional dependencies for your library
|
||||||
|
@ -890,6 +896,11 @@ layout: section
|
||||||
- Tell me about all the bugs in my code
|
- Tell me about all the bugs in my code
|
||||||
- The more testing we can do now, the better
|
- The more testing we can do now, the better
|
||||||
- There's still work to do
|
- There's still work to do
|
||||||
|
- Features
|
||||||
|
- Improvements
|
||||||
|
- Performance
|
||||||
|
- Scalability
|
||||||
|
- etc etc
|
||||||
-->
|
-->
|
||||||
|
|
||||||
---
|
---
|
||||||
|
@ -943,7 +954,7 @@ image: https://images.unsplash.com/photo-1451187580459-43490279c0fa?q=80&w=1744&
|
||||||
class: flex justify-center flex-col text-xl
|
class: flex justify-center flex-col text-xl
|
||||||
---
|
---
|
||||||
|
|
||||||
# Out of scope
|
# Out of scope (_for now_)
|
||||||
|
|
||||||
- Completion / failed hooks
|
- Completion / failed hooks
|
||||||
- Bulk queueing
|
- Bulk queueing
|
||||||
|
@ -964,6 +975,7 @@ class: flex justify-center flex-col text-xl
|
||||||
- But I hope we can slowly catch them up
|
- But I hope we can slowly catch them up
|
||||||
- Bringing the stability and longevity guarantees that come with Django
|
- Bringing the stability and longevity guarantees that come with Django
|
||||||
- Doesn't mean they'll never come
|
- Doesn't mean they'll never come
|
||||||
|
- With your help, we can make these happen
|
||||||
-->
|
-->
|
||||||
|
|
||||||
---
|
---
|
||||||
|
@ -981,11 +993,12 @@ background: https://images.unsplash.com/photo-1519187903022-c0055ec4036a?q=80&w=
|
||||||
- Improve throughput
|
- Improve throughput
|
||||||
- Reduce latency
|
- Reduce latency
|
||||||
- Improve reliability
|
- Improve reliability
|
||||||
- Gone are the days of needing additional research and testing to find the tooling you need
|
- Gone are the days of needing additional research and testing to find the tooling you need
|
||||||
- You can use the ones built-in to Django
|
- You can use the ones built-in to Django
|
||||||
- And as you scale, it's easy to change
|
- And as you scale, it's easy to change
|
||||||
- _without_ rewriting half your application
|
- _without_ rewriting half your application
|
||||||
- With all the knowledge to make an informed decision
|
- With all the knowledge to make an informed decision
|
||||||
|
- A lower barrier helps everyone!
|
||||||
-->
|
-->
|
||||||
|
|
||||||
---
|
---
|
||||||
|
@ -1005,10 +1018,12 @@ layout: section
|
||||||
- Test it out
|
- Test it out
|
||||||
- Report back your issues
|
- Report back your issues
|
||||||
- Suggest improvements
|
- Suggest improvements
|
||||||
|
- Issues / Discussions are open
|
||||||
- If you want to get involved, please do!
|
- If you want to get involved, please do!
|
||||||
- There's plenty of work to do
|
- There's plenty of work to do
|
||||||
- And I can't do it alone!
|
- And I can't do it alone!
|
||||||
- If you maintain a worker library
|
- If you maintain a worker library
|
||||||
|
- Have interesting use cases
|
||||||
- Or have been burned by one...
|
- Or have been burned by one...
|
||||||
-->
|
-->
|
||||||
|
|
||||||
|
|
Loading…
Reference in a new issue