[RFC PATCH 0/2] logging: add service to cleanup internal log files

Nikolay Shirokovskiy posted 2 patches 2 years, 2 months ago
Test syntax-check failed
Patches applied successfully (tree, apply log)
git fetch https://github.com/patchew-project/libvirt tags/patchew/20220224150618.47404-1-nshirokovskiy@virtuozzo.com
There is a newer version of this series
src/logging/log_handler.c             | 113 ++++++++++++++++++++++----
src/logging/meson.build               |  15 ++++
src/logging/virtlogcleaner.service.in |   7 ++
src/logging/virtlogcleaner.sh         |   9 ++
src/logging/virtlogcleaner.timer      |   8 ++
src/logging/virtlogd.service.in       |   1 +
6 files changed, 138 insertions(+), 15 deletions(-)
create mode 100644 src/logging/virtlogcleaner.service.in
create mode 100755 src/logging/virtlogcleaner.sh
create mode 100644 src/logging/virtlogcleaner.timer
[RFC PATCH 0/2] logging: add service to cleanup internal log files
Posted by Nikolay Shirokovskiy 2 years, 2 months ago
Hi, all.

This is a RFC for cleanup service suggested in [1]. I argumented there
that logrotate is not suitable in current form to cooperate with
virtlogd and only peform cleanup function.

There I thought we need to have file locking for cleanup service to
cooperate nicely with virtlogd. In this patch series I used timestamps
for that purpuse which seems to be more simple/reliable technique. The
idea is to keep logs for active VMs fresh by touching them periodically
for the case when log is not actually written for a long time.

This is an RFC and so it misses a lot of pieces:
- we only need to touch internal log files (and not log files of serial
  devices for example)
- openrc support is missing
- spec bits are missing

I guess service itself need to be written in C. For a couple of
reasons:

- not sure if we have a policy not to use shell scripts in deployments 
  (we still have libvirt-guests.sh)
- cleaner time complexity in this version is O(N^2). It is not a big
  deal to write a O(N) version but looks like it will be ugly looking
  in shell script
- I'd like to make at least max age to be configurable parameter
  in virtlogd.conf. And we have conf parser in C already. As a side 
  note I'm not sure if we should make cleanup invocation period and touching
  period to be configurable (both are 1 day currently)

There is also a need for drivers to add their cleanup paths to cleanup
service configuration. I guess we can

- add "cleanup_paths = []" to virtlogd.conf
- drivers can add conf files to virtlogd.d/. For example qemu drivers
  will have in virtlogd.d/qemu.conf:

  cleanup_paths += "/var/log/libvirt/qemu/"

On this way we need to support configure directories and += syntax for
lists.

[1] Re: removing VMs logs
https://listman.redhat.com/archives/libvir-list/2022-February/msg00425.html

Nikolay Shirokovskiy (2):
  logging: touch opened files periodically
  logging: add virtlogcleaner service

 src/logging/log_handler.c             | 113 ++++++++++++++++++++++----
 src/logging/meson.build               |  15 ++++
 src/logging/virtlogcleaner.service.in |   7 ++
 src/logging/virtlogcleaner.sh         |   9 ++
 src/logging/virtlogcleaner.timer      |   8 ++
 src/logging/virtlogd.service.in       |   1 +
 6 files changed, 138 insertions(+), 15 deletions(-)
 create mode 100644 src/logging/virtlogcleaner.service.in
 create mode 100755 src/logging/virtlogcleaner.sh
 create mode 100644 src/logging/virtlogcleaner.timer

-- 
2.31.1