Ping, i'd like to get this merged before we get close to the
freeze so we have debugging time if something unexpected
happens...
On Thu, Apr 12, 2018 at 02:28:17PM +0100, Daniel P. Berrangé wrote:
> This series aims to improve the way we handle translations.
>
> The key problems with our current approach
>
> - The pot & po files stored in GIT contain huge set of
> annotations about source file names & line numbers.
> These are out of date as soon as a change is commited
> to git following a translation refresh. This makes
> diffs impossible to meaningfully review, as they are
> 98% noise, 2% signal.
>
> - The po file messages are sorted by source location,
> so when we move code between files, or rename files,
> the po file message order changes for no good reason.
> This makes diffs even more impossible to review.
>
> - The po files contain entries for all messages even
> if most have no translation, bloating size of po/
> data stored in git
>
> - Whenever 'make dist' is run, it alters all the pot
> and po files, so developers need to then reset their
> content to match git HEAD manually. This is caused
> by having auto-generated content (source file locations)
> mixed in with the static content (the actual translated
> strings)
>
> Overall the *.po files we store consume 100MB, and when
> I refresh content from Zanata, it results in git commits
> that are many 10's of MB in size and impossible todo any
> meaningful review on.
>
> After this series, we only minimized po files in git, with
> the redundated & outdated source locations info stripped.
> This stripped info is re-added automatically during build
> to create the real .po files, that we distribute, and/or
> upload to translators in Zanata.
>
> As a result the po directory is only 19MB in size, and
> when refreshing from Zanata, we have git commits that
> clearly show *only* the altered translations, nothing
> else. The importance of this cannot be overstated - by
> having these clear diffs I discovered a serious bug in
> the Zanata client that has been screwing up translations
> in every project that uses Zanata by adding bogus "fuzzy"
> annotations.
>
> Note that what I've posted here is a cut-down version of
> what will actually be pushed. I have cut out the 99 other
> patches that actally change the .po files. There's no
> meaningful way to review those commits because of the
> noise in the original .po files we're replacing. I have
> gone through manual steps, however, to validate that the
> .po files we generate from the new .mini.po files are
> identical to the original .po files we used to store,
> aside from the bogus fuzzy annotations due to broken
> Zanata.
>
> The diffstat shown below covers the full set of patches
> on the branch at:
>
> https://github.com/berrange/libvirt/commits/i18n-4
>
> The killer statistic is:
>
> 418787 insertions(+), 4388281 deletions(-)
>
> Daniel P. Berrangé (5):
> po: provide custom make rules for po file management
> po: remove language list from zanata configuration
> po: add rules for integration with zanata
> po: stop storing libvirt.pot in git
> po: minimize & canonicalize translations stored in git
>
> .gitignore | 9 +-
> ABOUT-NLS | 1 +
> autogen.sh | 3 +-
> bootstrap.conf | 38 -
> build-aux/minimize-po.pl | 37 +
> configure.ac | 33 +-
> m4/virt-nls.m4 | 70 +
> po/Makefile.am | 107 +
> po/{POTFILES.in => POTFILES} | 0
> po/README.md | 75 +
> po/af.mini.po | 20 +
> po/af.po | 46395 ------------------------------
> po/am.mini.po | 20 +
> po/am.po | 46395 ------------------------------
> po/anp.mini.po | 20 +
> po/anp.po | 46395 ------------------------------
> po/ar.mini.po | 762 +
> po/ar.po | 46685 ------------------------------
> po/{as.po => as.mini.po} | 54246 ++++++++++-------------------------
> po/ast.mini.po | 20 +
> po/ast.po | 46395 ------------------------------
> po/bal.mini.po | 20 +
> po/bal.po | 46395 ------------------------------
> po/be.mini.po | 21 +
> po/be.po | 46396 ------------------------------
> po/bg.mini.po | 1204 +
> po/bg.po | 46733 ------------------------------
> po/bn.mini.po | 761 +
> po/bn.po | 46684 ------------------------------
> po/bn_IN.mini.po | 9131 ++++++
> po/bn_IN.po | 46729 ------------------------------
> po/bo.mini.po | 20 +
> po/bo.po | 46395 ------------------------------
> po/br.mini.po | 20 +
> po/br.po | 46395 ------------------------------
> po/brx.mini.po | 20 +
> po/brx.po | 46395 ------------------------------
> po/bs.mini.po | 767 +
> po/bs.po | 46693 ------------------------------
> po/ca.mini.po | 1618 ++
> po/ca.po | 46555 ------------------------------
> po/cs.mini.po | 4146 +++
> po/cs.po | 46751 ------------------------------
> po/cy.mini.po | 762 +
> po/cy.po | 46701 ------------------------------
> po/da.mini.po | 982 +
> po/da.po | 46704 ------------------------------
> po/de.mini.po | 22654 +++++++++++++++
> po/de.po | 47656 -------------------------------
> po/de_CH.mini.po | 20 +
> po/de_CH.po | 46395 ------------------------------
> po/el.mini.po | 724 +
> po/el.po | 46857 ------------------------------
> po/en_GB.mini.po | 22271 +++++++++++++++
> po/en_GB.po | 47573 -------------------------------
> po/eo.mini.po | 20 +
> po/eo.po | 46395 ------------------------------
> po/es.mini.po | 22622 +++++++++++++++
> po/es.po | 47748 -------------------------------
> po/et.mini.po | 761 +
> po/et.po | 46684 ------------------------------
> po/eu.mini.po | 761 +
> po/eu.po | 46684 ------------------------------
> po/fa.mini.po | 20 +
> po/fa.po | 46395 ------------------------------
> po/fi.mini.po | 1281 +
> po/fi.po | 46744 ------------------------------
> po/fil.mini.po | 20 +
> po/fr.mini.po | 4258 +++
> po/fr.po | 46907 ------------------------------
> po/fur.mini.po | 20 +
> po/ga.mini.po | 21 +
> po/gl.mini.po | 761 +
> po/gl.po | 46700 ------------------------------
> po/{gu.po => gu.mini.po} | 54191 ++++++++++-------------------------
> po/he.mini.po | 761 +
> po/he.po | 46684 ------------------------------
> po/hi.mini.po | 13772 +++++++++
> po/hi.po | 47013 -------------------------------
> po/hr.mini.po | 21 +
> po/hr.po | 46396 ------------------------------
> po/hu.mini.po | 987 +
> po/hu.po | 46709 ------------------------------
> po/ia.mini.po | 20 +
> po/ia.po | 46395 ------------------------------
> po/id.mini.po | 728 +
> po/id.po | 46691 ------------------------------
> po/ilo.mini.po | 20 +
> po/ilo.po | 46395 ------------------------------
> po/is.mini.po | 761 +
> po/is.po | 46684 ------------------------------
> po/it.mini.po | 6588 +++++
> po/it.po | 46663 ------------------------------
> po/{ja.po => ja.mini.po} | 54737 ++++++++++-------------------------
> po/ka.mini.po | 761 +
> po/ka.po | 46684 ------------------------------
> po/kk.mini.po | 20 +
> po/kk.po | 46395 ------------------------------
> po/km.mini.po | 20 +
> po/km.po | 46395 ------------------------------
> po/{kn.po => kn.mini.po} | 54156 ++++++++++-------------------------
> po/ko.mini.po | 12838 +++++++++
> po/ko.po | 46723 ------------------------------
> po/kw.mini.po | 21 +
> po/kw.po | 46396 ------------------------------
> po/kw@kkcor.mini.po | 21 +
> po/kw@kkcor.po | 46396 ------------------------------
> po/kw@uccor.mini.po | 21 +
> po/kw@uccor.po | 46396 ------------------------------
> po/kw_GB.mini.po | 21 +
> po/kw_GB.po | 46396 ------------------------------
> po/ky.mini.po | 20 +
> po/ky.po | 46395 ------------------------------
> po/libvirt.pot | 46394 ------------------------------
> po/lt.mini.po | 762 +
> po/lt.po | 46685 ------------------------------
> po/lv.mini.po | 762 +
> po/lv.po | 46685 ------------------------------
> po/mai.mini.po | 20 +
> po/mai.po | 46395 ------------------------------
> po/mk.mini.po | 1138 +
> po/mk.po | 46724 ------------------------------
> po/{ml.po => ml.mini.po} | 53963 ++++++++++-------------------------
> po/mn.mini.po | 20 +
> po/mn.po | 46395 ------------------------------
> po/{mr.po => mr.mini.po} | 55484 +++++++++++-------------------------
> po/ms.mini.po | 189 +
> po/ms.po | 46580 ------------------------------
> po/my.mini.po | 20 +
> po/nb.mini.po | 492 +
> po/nb.po | 46667 ------------------------------
> po/nds.mini.po | 20 +
> po/nds.po | 46395 ------------------------------
> po/ne.mini.po | 20 +
> po/ne.po | 46395 ------------------------------
> po/nl.mini.po | 9868 +++++++
> po/nl.po | 47062 -------------------------------
> po/nn.mini.po | 761 +
> po/nn.po | 46684 ------------------------------
> po/nso.mini.po | 761 +
> po/nso.po | 46684 ------------------------------
> po/{or.po => or.mini.po} | 53161 +++++++++-------------------------
> po/{pa.po => pa.mini.po} | 53711 ++++++++++-------------------------
> po/pl.mini.po | 10274 +++++++
> po/pl.po | 47166 -------------------------------
> po/pt.mini.po | 1243 +
> po/pt.po | 46587 ------------------------------
> po/pt_BR.mini.po | 22821 +++++++++++++++
> po/pt_BR.po | 47565 -------------------------------
> po/ro.mini.po | 762 +
> po/ro.po | 46685 ------------------------------
> po/{ru.po => ru.mini.po} | 54065 ++++++++++-------------------------
> po/si.mini.po | 761 +
> po/si.po | 46684 ------------------------------
> po/sk.mini.po | 761 +
> po/sk.po | 46684 ------------------------------
> po/sl.mini.po | 762 +
> po/sl.po | 46685 ------------------------------
> po/sq.mini.po | 24 +
> po/sq.po | 46399 ------------------------------
> po/sr.mini.po | 2203 ++
> po/sr.po | 46910 ------------------------------
> po/sr@latin.mini.po | 2204 ++
> po/sr@latin.po | 46911 ------------------------------
> po/sv.mini.po | 2559 ++
> po/sv.po | 46696 ------------------------------
> po/{ta.po => ta.mini.po} | 54968 +++++++++++-------------------------
> po/{te.po => te.mini.po} | 53427 ++++++++++-------------------------
> po/tg.mini.po | 20 +
> po/tg.po | 46395 ------------------------------
> po/th.mini.po | 761 +
> po/th.po | 46684 ------------------------------
> po/tr.mini.po | 761 +
> po/tr.po | 46684 ------------------------------
> po/tw.mini.po | 20 +
> po/tw.po | 46395 ------------------------------
> po/{uk.po => uk.mini.po} | 62575 +++++++++++++++++------------------------
> po/ur.mini.po | 761 +
> po/ur.po | 46684 ------------------------------
> po/vi.mini.po | 8674 ++++++
> po/vi.po | 47171 -------------------------------
> po/wba.mini.po | 20 +
> po/wba.po | 46395 ------------------------------
> po/yo.mini.po | 20 +
> po/yo.po | 46395 ------------------------------
> po/zanata.xml | 99 -
> po/zh_CN.mini.po | 23613 ++++++++++++++++
> po/zh_CN.po | 46701 ------------------------------
> po/zh_HK.mini.po | 20 +
> po/zh_HK.po | 46395 ------------------------------
> po/zh_TW.mini.po | 958 +
> po/zh_TW.po | 46725 ------------------------------
> po/zu.mini.po | 761 +
> po/zu.po | 46684 ------------------------------
> src/internal.h | 16 +-
> src/libvirt-admin.c | 2 +
> src/libvirt.c | 2 +
> src/util/virgettext.c | 3 +-
> tools/virt-host-validate.c | 4 +-
> 199 files changed, 418787 insertions(+), 4388281 deletions(-)
>
> --
> 2.14.3
>
Regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list