forked from slides/empowering-django-with-background-workers
Bootstrap some initial slides
This commit is contained in:
parent
2688dc9dda
commit
63022eee1a
3 changed files with 460 additions and 0 deletions
BIN
public/ridiculous.png
Normal file
BIN
public/ridiculous.png
Normal file
Binary file not shown.
After Width: | Height: | Size: 23 KiB |
BIN
public/soon.png
Normal file
BIN
public/soon.png
Normal file
Binary file not shown.
After Width: | Height: | Size: 10 KiB |
460
slides.md
460
slides.md
|
@ -24,3 +24,463 @@ themeConfig:
|
||||||
<li><mdi-github /> @RealOrangeOne</li>
|
<li><mdi-github /> @RealOrangeOne</li>
|
||||||
<li><mdi-mastodon /> @jake@theorangeone.net</li>
|
<li><mdi-mastodon /> @jake@theorangeone.net</li>
|
||||||
</ul>
|
</ul>
|
||||||
|
|
||||||
|
---
|
||||||
|
layout: center
|
||||||
|
---
|
||||||
|
|
||||||
|
# Django isn't _just_ for websites
|
||||||
|
|
||||||
|
|
||||||
|
```mermaid
|
||||||
|
flowchart LR
|
||||||
|
U(User 🧑💻)
|
||||||
|
D[\Django/]
|
||||||
|
|
||||||
|
U---->|Request|D
|
||||||
|
D---->|Response|U
|
||||||
|
```
|
||||||
|
|
||||||
|
<style>
|
||||||
|
.mermaid {
|
||||||
|
margin-top: 2rem;
|
||||||
|
text-align: center;
|
||||||
|
transform: scale(1.5);
|
||||||
|
}
|
||||||
|
</style>
|
||||||
|
|
||||||
|
---
|
||||||
|
layout: full
|
||||||
|
---
|
||||||
|
|
||||||
|
```mermaid
|
||||||
|
flowchart BT
|
||||||
|
U[User 🧑💻]
|
||||||
|
D[\Django/]
|
||||||
|
DB[(Database)]
|
||||||
|
C[(Cache)]
|
||||||
|
E>Email]
|
||||||
|
EA[External API]
|
||||||
|
V[[Video Transcode]]
|
||||||
|
ML((Machine Learning))
|
||||||
|
|
||||||
|
U---->|Request|D
|
||||||
|
D---->|Response|U
|
||||||
|
|
||||||
|
D-.-DB & E & EA & V & ML & C
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
layout: full
|
||||||
|
---
|
||||||
|
|
||||||
|
```mermaid
|
||||||
|
flowchart BT
|
||||||
|
U[User 🧑💻]
|
||||||
|
D[\Django/]
|
||||||
|
DB[(Database)]
|
||||||
|
C[(Cache)]
|
||||||
|
E>Email]
|
||||||
|
EA[External API]
|
||||||
|
V[[Video Transcode]]
|
||||||
|
ML((Machine Learning))
|
||||||
|
B{{Background Processing}}
|
||||||
|
|
||||||
|
U--->|Request|D
|
||||||
|
D--->|Response|U
|
||||||
|
|
||||||
|
D---B
|
||||||
|
|
||||||
|
D-.-C & DB
|
||||||
|
|
||||||
|
B---E & V & ML & EA
|
||||||
|
B---C & DB
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
layout: cover
|
||||||
|
---
|
||||||
|
|
||||||
|
# Background Workers?
|
||||||
|
|
||||||
|
---
|
||||||
|
layout: fact
|
||||||
|
---
|
||||||
|
|
||||||
|
```mermaid
|
||||||
|
flowchart LR
|
||||||
|
D[\Django/]
|
||||||
|
S[(Queue Store)]
|
||||||
|
R1{Runner}
|
||||||
|
R2{Runner}
|
||||||
|
R3{Runner}
|
||||||
|
|
||||||
|
D<--->S<-..->R1 & R2 & R3
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
layout: cover
|
||||||
|
---
|
||||||
|
|
||||||
|
# When?
|
||||||
|
|
||||||
|
---
|
||||||
|
layout: fact
|
||||||
|
---
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
```mermaid
|
||||||
|
flowchart BT
|
||||||
|
D[\Django/]
|
||||||
|
|
||||||
|
subgraph Fast & Reliable
|
||||||
|
DB[(Database)]
|
||||||
|
C[(Cache)]
|
||||||
|
end
|
||||||
|
|
||||||
|
subgraph Slow / Unreliable
|
||||||
|
E>Email]
|
||||||
|
EA[External API]
|
||||||
|
V[[Video Transcode]]
|
||||||
|
ML((Machine Learning))
|
||||||
|
end
|
||||||
|
|
||||||
|
D---DB & C
|
||||||
|
|
||||||
|
D-.-E & EA & V & ML
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
layout: cover
|
||||||
|
---
|
||||||
|
|
||||||
|
# Background Workers _in Django_
|
||||||
|
|
||||||
|
---
|
||||||
|
layout: cover
|
||||||
|
background: https://docs.celeryq.dev/en/stable/_static/celery_512.png
|
||||||
|
---
|
||||||
|
|
||||||
|
# Celery!
|
||||||
|
|
||||||
|
<style>
|
||||||
|
.slidev-layout {
|
||||||
|
background: white;
|
||||||
|
}
|
||||||
|
</style>
|
||||||
|
|
||||||
|
---
|
||||||
|
layout: image-right
|
||||||
|
image: https://images.unsplash.com/photo-1444703686981-a3abbc4d4fe3?q=80&w=1740&auto=format&fit=crop&ixlib=rb-4.0.3&ixid=M3wxMjA3fDB8MHxwaG90by1wYWdlfHx8fGVufDB8fHx8fA%3D%3D
|
||||||
|
---
|
||||||
|
|
||||||
|
# Others...
|
||||||
|
|
||||||
|
<Transform :scale="1.05">
|
||||||
|
|
||||||
|
- ~~Celery~~
|
||||||
|
- arq
|
||||||
|
- Django DB Queue
|
||||||
|
- Django Lightweight Queue
|
||||||
|
- Django Too Simple Q
|
||||||
|
- Django-Q
|
||||||
|
- Django-Q2
|
||||||
|
- Dramatiq
|
||||||
|
- Huey
|
||||||
|
- RQ
|
||||||
|
- Taskiq
|
||||||
|
- ...
|
||||||
|
|
||||||
|
</Transform>
|
||||||
|
|
||||||
|
---
|
||||||
|
layout: cover
|
||||||
|
background: https://images.unsplash.com/photo-1522096823084-2d1aa8411c13?q=80&w=1740&auto=format&fit=crop&ixlib=rb-4.0.3&ixid=M3wxMjA3fDB8MHxwaG90by1wYWdlfHx8fGVufDB8fHx8fA%3D%3D
|
||||||
|
---
|
||||||
|
|
||||||
|
# Email <mdi-email-fast-outline />
|
||||||
|
|
||||||
|
---
|
||||||
|
layout: none
|
||||||
|
---
|
||||||
|
|
||||||
|
```python {all|8|9-14|16-19|all}
|
||||||
|
from django.contrib.auth.models import User
|
||||||
|
from django.core.mail import send_mail
|
||||||
|
from django.template.loader import render_to_string
|
||||||
|
|
||||||
|
import django_rq
|
||||||
|
|
||||||
|
def send_email_to_user(user: User):
|
||||||
|
email_content = render_to_string("email-template.html", {"user": user})
|
||||||
|
send_mail(
|
||||||
|
subject="Here is your important customized message",
|
||||||
|
message=email_content
|
||||||
|
from_email=None, # Use the default sender email
|
||||||
|
recipient_list=[user.email]
|
||||||
|
)
|
||||||
|
|
||||||
|
users_to_email = User.objects.all()
|
||||||
|
|
||||||
|
for user in users_to_email.iterator():
|
||||||
|
django_rq.enqueue(send_email_to_user, user)
|
||||||
|
```
|
||||||
|
|
||||||
|
<style>
|
||||||
|
.slidev-page, .slidev-code-wrapper, .slidev-code {
|
||||||
|
height: 100%;
|
||||||
|
}
|
||||||
|
</style>
|
||||||
|
|
||||||
|
---
|
||||||
|
layout: center
|
||||||
|
---
|
||||||
|
|
||||||
|
# A problem
|
||||||
|
|
||||||
|
````md magic-move
|
||||||
|
```python
|
||||||
|
from django.contrib.auth.models import User
|
||||||
|
from django.core.mail import send_mail
|
||||||
|
from django.template.loader import render_to_string
|
||||||
|
|
||||||
|
import django_rq
|
||||||
|
|
||||||
|
def send_email_to_user(user: User):
|
||||||
|
email_content = render_to_string("email-template.html", {"user": user})
|
||||||
|
send_mail(
|
||||||
|
subject="Here is your important customized message",
|
||||||
|
message=email_content
|
||||||
|
from_email=None, # Use the default sender email
|
||||||
|
recipient_list=[user.email]
|
||||||
|
)
|
||||||
|
|
||||||
|
users_to_email = User.objects.all()
|
||||||
|
|
||||||
|
for user in users_to_email.iterator():
|
||||||
|
django_rq.enqueue(send_email_to_user, user)
|
||||||
|
```
|
||||||
|
|
||||||
|
```python {all|5-7,20|all}
|
||||||
|
from django.contrib.auth.models import User
|
||||||
|
from django.core.mail import send_mail
|
||||||
|
from django.template.loader import render_to_string
|
||||||
|
|
||||||
|
from my_celery_config import app
|
||||||
|
|
||||||
|
@app.task
|
||||||
|
def send_email_to_user(user: User):
|
||||||
|
email_content = render_to_string("email-template.html", {"user": user})
|
||||||
|
send_mail(
|
||||||
|
subject="Here is your important customized message",
|
||||||
|
message=email_content
|
||||||
|
from_email=None, # Use the default sender email
|
||||||
|
recipient_list=[user.email]
|
||||||
|
)
|
||||||
|
|
||||||
|
users_to_email = User.objects.all()
|
||||||
|
|
||||||
|
for user in users_to_email.iterator():
|
||||||
|
send_email_to_user.delay(user)
|
||||||
|
```
|
||||||
|
````
|
||||||
|
|
||||||
|
---
|
||||||
|
layout: fact
|
||||||
|
---
|
||||||
|
|
||||||
|
# Situation:
|
||||||
|
## There are _14_ competing standards.
|
||||||
|
|
||||||
|
<style>
|
||||||
|
.slidev-layout {
|
||||||
|
background: white;
|
||||||
|
color: black;
|
||||||
|
}
|
||||||
|
</style>
|
||||||
|
|
||||||
|
---
|
||||||
|
layout: image-right
|
||||||
|
image: https://images.unsplash.com/photo-1674027444485-cec3da58eef4?q=80&w=1932&auto=format&fit=crop&ixlib=rb-4.0.3&ixid=M3wxMjA3fDB8MHxwaG90by1wYWdlfHx8fGVufDB8fHx8fA%3D%3D
|
||||||
|
class: flex items-center text-2xl
|
||||||
|
---
|
||||||
|
|
||||||
|
- Application developers
|
||||||
|
- Library maintainers
|
||||||
|
- System Engineers
|
||||||
|
|
||||||
|
---
|
||||||
|
layout: image
|
||||||
|
image: /ridiculous.png
|
||||||
|
class: bg-top!
|
||||||
|
---
|
||||||
|
|
||||||
|
# Ridiculous!
|
||||||
|
|
||||||
|
---
|
||||||
|
layout: fact
|
||||||
|
---
|
||||||
|
|
||||||
|
## Introducing:{.mb-5}
|
||||||
|
|
||||||
|
# `django.tasks`
|
||||||
|
|
||||||
|
---
|
||||||
|
layout: center
|
||||||
|
---
|
||||||
|
|
||||||
|
````md magic-move
|
||||||
|
```python
|
||||||
|
from django.contrib.auth.models import User
|
||||||
|
from django.core.mail import send_mail
|
||||||
|
from django.template.loader import render_to_string
|
||||||
|
|
||||||
|
from my_celery_config import app
|
||||||
|
|
||||||
|
@app.task
|
||||||
|
def send_email_to_user(user: User):
|
||||||
|
email_content = render_to_string("email-template.html", {"user": user})
|
||||||
|
send_mail(
|
||||||
|
subject="Here is your important customized message",
|
||||||
|
message=email_content
|
||||||
|
from_email=None, # Use the default sender email
|
||||||
|
recipient_list=[user.email]
|
||||||
|
)
|
||||||
|
|
||||||
|
users_to_email = User.objects.all()
|
||||||
|
|
||||||
|
for user in users_to_email.iterator():
|
||||||
|
send_email_to_user.delay(user)
|
||||||
|
```
|
||||||
|
|
||||||
|
```python {all|5-7,20|all}
|
||||||
|
from django.contrib.auth.models import User
|
||||||
|
from django.core.mail import send_mail
|
||||||
|
from django.template.loader import render_to_string
|
||||||
|
|
||||||
|
from django.tasks import task
|
||||||
|
|
||||||
|
@task()
|
||||||
|
def send_email_to_user(user: User):
|
||||||
|
email_content = render_to_string("email-template.html", {"user": user})
|
||||||
|
send_mail(
|
||||||
|
subject="Here is your important customized message",
|
||||||
|
message=email_content
|
||||||
|
from_email=None, # Use the default sender email
|
||||||
|
recipient_list=[user.email]
|
||||||
|
)
|
||||||
|
|
||||||
|
users_to_email = User.objects.all()
|
||||||
|
|
||||||
|
for user in users_to_email.iterator():
|
||||||
|
send_email_to_user.enqueue(user)
|
||||||
|
```
|
||||||
|
````
|
||||||
|
|
||||||
|
---
|
||||||
|
layout: center
|
||||||
|
---
|
||||||
|
|
||||||
|
```python
|
||||||
|
from django.contrib.auth.models import User
|
||||||
|
from django.core.mail import send_mail
|
||||||
|
from django.template.loader import render_to_string
|
||||||
|
|
||||||
|
users_to_email = User.objects.all()
|
||||||
|
|
||||||
|
email_content = render_to_string("email-template.html")
|
||||||
|
|
||||||
|
for user in users_to_email.iterator():
|
||||||
|
send_mail(
|
||||||
|
subject="Here is your important message",
|
||||||
|
message=email_content
|
||||||
|
from_email=None, # Use the default sender email
|
||||||
|
recipient_list=[user.email]
|
||||||
|
)
|
||||||
|
```
|
||||||
|
|
||||||
|
<v-click>
|
||||||
|
|
||||||
|
```python
|
||||||
|
# settings.py
|
||||||
|
EMAIL_BACKEND = "django.core.mail.backends.tasks.SMTPEmailBackend"
|
||||||
|
```
|
||||||
|
|
||||||
|
</v-click>
|
||||||
|
|
||||||
|
|
||||||
|
---
|
||||||
|
layout: image-right
|
||||||
|
image: /soon.png
|
||||||
|
class: flex justify-center text-2xl flex-col
|
||||||
|
---
|
||||||
|
|
||||||
|
# Q: Why something new?
|
||||||
|
|
||||||
|
<v-click>
|
||||||
|
|
||||||
|
### A: We're not
|
||||||
|
|
||||||
|
</v-click>
|
||||||
|
|
||||||
|
---
|
||||||
|
layout: center
|
||||||
|
---
|
||||||
|
|
||||||
|
# Why something built-in?
|
||||||
|
|
||||||
|
<Transform :scale="1.3">
|
||||||
|
<v-clicks>
|
||||||
|
|
||||||
|
- Reduces barrier to entry
|
||||||
|
- Improve interoperability
|
||||||
|
- Reduce cognitive load
|
||||||
|
|
||||||
|
</v-clicks>
|
||||||
|
</Transform>
|
||||||
|
|
||||||
|
---
|
||||||
|
layout: section
|
||||||
|
---
|
||||||
|
|
||||||
|
# Where are we now?
|
||||||
|
|
||||||
|
<v-click>
|
||||||
|
|
||||||
|
## `pip install django-tasks`
|
||||||
|
|
||||||
|
</v-click>
|
||||||
|
|
||||||
|
---
|
||||||
|
layout: section
|
||||||
|
---
|
||||||
|
|
||||||
|
# Where are we going?
|
||||||
|
|
||||||
|
---
|
||||||
|
layout: image-right
|
||||||
|
image: https://images.unsplash.com/photo-1451187580459-43490279c0fa?q=80&w=1744&auto=format&fit=crop&ixlib=rb-4.0.3&ixid=M3wxMjA3fDB8MHxwaG90by1wYWdlfHx8fGVufDB8fHx8fA%3D%3D
|
||||||
|
class: flex justify-center flex-col text-xl
|
||||||
|
---
|
||||||
|
|
||||||
|
# Out of scope
|
||||||
|
|
||||||
|
- Completion / failed hooks
|
||||||
|
- Bulk queueing
|
||||||
|
- Automated task retrying
|
||||||
|
- Task runner API
|
||||||
|
- Unified observability
|
||||||
|
- Cron-based scheduling
|
||||||
|
- Task timeouts
|
||||||
|
- Swappable argument serialization
|
||||||
|
|
||||||
|
---
|
||||||
|
layout: section
|
||||||
|
---
|
||||||
|
|
||||||
|
# What's next?
|
||||||
|
|
||||||
|
---
|
||||||
|
layout: end
|
||||||
|
---
|
||||||
|
|
Loading…
Reference in a new issue