archive/theorangeone.net-legacy
1
Fork 0
Code Activity

Update clickhouse config example to fully disable log tables

Browse source 
Also write a better query to truncate existing log tables

367de37fab
This commit is contained in:
Jake Howard 2020-11-12 23:16:22 +00:00
parent ebf15376f4
commit b58455d2c1
Signed by: jake
GPG key ID: 57AFB45680EDD477
1 changed files with 14 additions and 13 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

27
content/posts/calming-down-clickhouse.md
View file

@ -50,7 +50,7 @@ The first step, and likely the one which makes the most difference, is to flat o
### Reduce logging ### Reduce logging
Step 2 is to reduce the log level on both the file logger, and any other table logging we can't disable entirely. This change is done in [`config.xml`](https://github.com/RealOrangeOne/infrastructure/blob/master/ansible/roles/plausible/files/clickhouse-config.xml). I set the level to `warning`, so it's still obvious to see when something is wrong, but it won't bombard me with information. Step 2 is to reduce the log level on both the file logger, and any other table logging we can't disable entirely. This change is done in [`config.xml`](https://github.com/RealOrangeOne/infrastructure/blob/master/ansible/roles/plausible/files/clickhouse-config.xml). I set the level to `warning`, so it's still obvious to see when something is wrong, but these all go into the shell, so it's fine. All the table logs are completely disabled.
```xml ```xml
<yandex> <yandex>
@ -58,13 +58,12 @@ Step 2 is to reduce the log level on both the file logger, and any other table l
<level>warning</level> <level>warning</level>
<console>true</console> <console>true</console>
</logger> </logger>
<text_log> <query_thread_log remove="remove"/>
<level>warning</level> <query_log remove="remove"/>
</text_log> <text_log remove="remove"/>
<metric_log> <trace_log remove="remove"/>
<level>warning</level> <metric_log remove="remove"/>
</metric_log> <asynchronous_metric_log remove="remove"/>
<trace_log></trace_log>
</yandex> </yandex>
``` ```
@ -76,11 +75,13 @@ In addition to reducing the log level, I moved the log files to a`tmpfs` mount.
The final step is to reclaim the disk space we lost to the overly verbose logs. Because everything was logged to tables, it's all still around. This probably won't impact runtime performance much, but disk usage went from 8GB to 200MB, which is quite nice! The final step is to reclaim the disk space we lost to the overly verbose logs. Because everything was logged to tables, it's all still around. This probably won't impact runtime performance much, but disk usage went from 8GB to 200MB, which is quite nice!
1. Log in to the Clickhouse shell: `docker-compose exec clickhouse clickhouse-client` 1. Log in to the Clickhouse shell: `docker-compose exec clickhouse bash`
2. Truncate some tables: `truncate table <table_name>;` 2. Truncate the existing logs:
- `system.query_log`
- `system.query_thread_log` ```bash
- `system.metric_log` clickhouse-client -q "SELECT name FROM system.tables WHERE name LIKE '%log%';" | xargs -I{} clickhouse-client -q "TRUNCATE TABLE system.{};"
```
3. Quit: `\q` 3. Quit: `\q`
## Review ## Review