Forgejo monthly update - November 2024
The monthly report is meant to provide a good overview of what has changed in Forgejo in the past month. If you would like to help, please get in touch in the chatroom or participate in the ongoing discussions.
Forgejo releases
The Forgejo v9.0.2 and v7.0.11 security releases were published. They have an unprecedented number of security fixes, some of which took months to mature. The difficulty in working on multiple security fixes is that they need to be combined and verified before the release date, to minimize the chances that issues are discovered at the last minute, on the day of the release. The release team has set up the tools to work with the security team in these preparations. In a nutshell it duplicates the Forgejo development setup but strips out the workflows that are not useful in this context.
The Forgejo release process grew and transformed significantly and frequently to adapt to a moving landscape. The two most significant events were the switch from Woodpecker CI to Forgejo Actions as soon as it was released early 2023. And earlier this year, Forgejo became independent of Gitea, which led to the definition of a new release cycle and the first instance of a long term support release.
It stabilized over the past six months and that allowed for the emergence of tools with the goal automate what is still currently a largely manual and long checklist. The checklist for the upcoming v10.0 release tracks their progress. The first two developments are:
- A machine readable release schedule.
- An action to cut branches, set branch protections and manage backport labels to be used by the documentation repository and later by the Forgejo repository itself.
Forgejo Actions
The security audit began mid November and some of its findings were fixed. It led to the publication of multiple releases from 4.0.1 to 5.0.2. They also include better control for verbosity and graceful handling of some corner cases that would crash the runner.
Codeberg started to offer hosted Forgejo Actions, the service is considered open alpha. If you didn’t get to trying Forgejo Actions with a self-hosted runner yet, you can now give it a try with the hosted runner, in case your projects meet the requirements of Codeberg.org.
The extensive usage of Forgejo Actions for the development of Forgejo itself reveals bugs in edge cases from time to time. A workflow which checks labels in a pull request as a merge condition was added, temporarily reverted, and re-added after relevant bugs were addressed. Several related issues were discovered and fixed, and working with label events in pull requests can now be considered more mature in Forgejo Actions.
Helm chart
Some OpenShift compabillity on chart version v10.1 and bug fix releases for v7 and v10 for the Forgejo security releases.
Accessibility
Improvements to colorblind themes were implemented. There is an ongoing discussion about this topic. If you rely on Forgejo’s colorblind themes, or would like to but can’t use the themes, consider getting involved in the discussion post.
Localization
Two new members have joined the localization team with the intention of maintaining Latvian and Low German translations.
The work on the new translation for Low German started in October and the translation is now completed, only proofreading remains. This language will be available to users in Forgejo v10 and can already be tested out on the dev instance.
A new script was created to process translation files. It allows to perform backporting of translations safely which was done for v9 as well as other maintenance chores such as removal of orphan strings and duplicates.
Infrastructure
The k8s cluster bootstrapped in October has matured and is now in production, hosting all services that previously were using ad-hoc scripts. It was initially motivated to improve the availability of the Forgejo resources in the wake of a downtime that disrupted https://code.forgejo.org during 10 hours in September. For this migration two new EX44 machines (one in Germany, the other in Finland) were setup and replace a larger EX101 machine.
Another benefit of the k8s cluster is that it does not require manual intervention, it is driven by the repository that defines it. This will allow, for instance, for a workflow to dynamically and automatically provision and modify test instances (https://v7.next.forgejo.org, https://v10.next.forgejo.org, etc.) based on the machine readable release schedule.
Renovate
Some more repos adopted to use automated Renovate updates.
Testing efficiency
Careful testing is an important goal within the Forgejo community and contributions to Forgejo need reasonable test coverage in order to be accepted.
Migration tests for Gitea and GitHub existed, but were not yet run. They have now been enabled, improving test coverage for migrations across forges.
Running test pipelines for a project at Forgejo’s scale has significant costs, not only financially but also ecologically. Reducing energy consumption as well as feedback time is important.
Several contributors picked up the task and made the journey to improve the situation. From improvements increasing quality and speed to refactors to improve reliability and avoid unnecessary retries, this month was very active with CI/CD and test optimizations.
Several PRs like this one successively moved a lot of test data into memory, not only speeding up the tests but also reducing writes and thus disk wear on the hardware. Finally, even some database parameters were tuned for MySQL and PostgreSQL.
There are more ideas on how to continue towards sustainable and efficient CI/CD.
Performance at scale
During moments of extreme load at Codeberg, some insights about slow database queries have been shared to the Forgejo community. Improvements were quickly integrated in the next version of Forgejo, scheduled for publication in January 2025.
Especially subqueries that use WHERE ... IN instead of JOIN
can lead to poor performance with MariaDB,
as can be seen on Codeberg.
Small contributions in this area,
such as this simple change can bring significant improvement.
After deploying this change to Codeberg, the “New repository” page now loads significantly faster
(down from up to 3 seconds back into a millisecond range).
More contributions in this area are more than welcome.
Stability and database corruption at Codeberg
In some cases, Forgejo’s database can get inconsistent, and there is a command-line tool (the “doctor”) to perform checks and fix the database. Especially on busy instances, doing this regularly is recommended to clean stale data and discover issues.
However, while running this on Codeberg, a corruption issue was discovered and fixed. The consistency checker incorrectly deleted global OAuth2 applications, because they do not have a user assigned.
Users of global OAuth2 applications are advised not to run the doctor until the fix is released in Forgejo v9.0.3.
We Forge
Forgejo is a community of people who contribute in an inclusive environment. We forge on an equal footing, by reporting a bug, voicing an idea in the chatroom or implementing a new feature. The following list of contributors is meant to reflect this diversity and acknowledge all contributions since the last monthly report was published. If you are missing, please ask for an update.
- https://codeberg.org/0ko
- https://codeberg.org/0n1cOn3
- https://codeberg.org/71rd
- https://codeberg.org/achyrva
- https://codeberg.org/algernon
- https://codeberg.org/aljazerzen
- https://codeberg.org/anbraten
- https://codeberg.org/andar1an
- https://codeberg.org/Andre601
- https://codeberg.org/angelnu
- https://codeberg.org/Arsen
- https://codeberg.org/ashimokawa
- https://codeberg.org/atimy
- https://codeberg.org/avobs
- https://codeberg.org/awiteb
- https://codeberg.org/AYM1607
- https://codeberg.org/bachorp
- https://codeberg.org/baltazar
- https://codeberg.org/bartvdbraak
- https://codeberg.org/becm
- https://codeberg.org/benniekiss
- https://codeberg.org/Beowulf
- https://codeberg.org/billynoah
- https://codeberg.org/blaise
- https://codeberg.org/bmcclure
- https://codeberg.org/c8h4
- https://codeberg.org/caesar
- https://codeberg.org/cfebs
- https://codeberg.org/cobak78
- https://codeberg.org/Crown0815
- https://codeberg.org/danielbaumann
- https://codeberg.org/DanielGibson
- https://codeberg.org/David-Guillot
- https://codeberg.org/daylien
- https://codeberg.org/dcz_pf
- https://codeberg.org/Dirk
- https://codeberg.org/d-k-bo
- https://codeberg.org/dmowitz
- https://codeberg.org/ds-cbo
- https://codeberg.org/earl-warren
- https://codeberg.org/el0n
- https://codeberg.org/ell1e
- https://codeberg.org/Ember
- https://codeberg.org/ERROR404
- https://codeberg.org/famfo
- https://codeberg.org/Felitendo
- https://codeberg.org/FermeLeLundi
- https://codeberg.org/fina
- https://codeberg.org/fjordo
- https://codeberg.org/flexstrongo
- https://codeberg.org/floss4good
- https://codeberg.org/fnetX
- https://codeberg.org/GamePlayer-8
- https://codeberg.org/GDWR
- https://codeberg.org/gming
- https://codeberg.org/Gnu1
- https://codeberg.org/Gravy59
- https://codeberg.org/gregdechene
- https://codeberg.org/gtsiam
- https://codeberg.org/Gusted
- https://codeberg.org/hacknorris
- https://codeberg.org/halibut
- https://codeberg.org/io7m
- https://codeberg.org/Ironfractal
- https://codeberg.org/itsTurnip
- https://codeberg.org/iustin
- https://codeberg.org/JakobDev
- https://codeberg.org/jean-daricade
- https://codeberg.org/jerger
- https://codeberg.org/jinn
- https://codeberg.org/jmakov
- https://codeberg.org/jornfranke
- https://codeberg.org/jpkhawam
- https://codeberg.org/julianfoad
- https://codeberg.org/jwildeboer
- https://codeberg.org/k8ie
- https://codeberg.org/KaKi87
- https://codeberg.org/kassuro
- https://codeberg.org/kita
- https://codeberg.org/Kokomo
- https://codeberg.org/kurets
- https://codeberg.org/Kwonunn
- https://codeberg.org/lweller
- https://codeberg.org/mafen
- https://codeberg.org/mahlzahn
- https://codeberg.org/Maniues
- https://codeberg.org/marcellmars
- https://codeberg.org/marcoaraujojunior
- https://codeberg.org/Mark3xtrm
- https://codeberg.org/martinwguy
- https://codeberg.org/matrss
- https://codeberg.org/mehrad
- https://codeberg.org/Merith-TK
- https://codeberg.org/mfenniak
- https://codeberg.org/MichaelAgarkov
- https://codeberg.org/michael-sparrow
- https://codeberg.org/mjesusdev
- https://codeberg.org/mlucas
- https://codeberg.org/morvy
- https://codeberg.org/mvdkleijn
- https://codeberg.org/mysticmode
- https://codeberg.org/n0toose
- https://codeberg.org/natct
- https://codeberg.org/ng-dm
- https://codeberg.org/Nordfriese
- https://codeberg.org/nostar
- https://codeberg.org/oliverpool
- https://codeberg.org/onerandomusername
- https://codeberg.org/paspflue
- https://codeberg.org/patrickuhlmann
- https://codeberg.org/Peaksol
- https://codeberg.org/Piturnah
- https://codeberg.org/pointlessone
- https://codeberg.org/programmerjake
- https://codeberg.org/psentee
- https://codeberg.org/PurpleBooth
- https://codeberg.org/RagnarGrootKoerkamp
- https://codeberg.org/ReptoxX
- https://codeberg.org/rfc2549
- https://codeberg.org/shaowenwan
- https://codeberg.org/shinebayar-g
- https://codeberg.org/Silejonu
- https://codeberg.org/snematoda
- https://codeberg.org/solonovamax
- https://codeberg.org/SpareJoe
- https://codeberg.org/spiffyk
- https://codeberg.org/Squel
- https://codeberg.org/stb
- https://codeberg.org/thefinn93
- https://codeberg.org/thilinajayanath
- https://codeberg.org/timotheyca
- https://codeberg.org/Tom3201
- https://codeberg.org/tseeker
- https://codeberg.org/VA1DER
- https://codeberg.org/VadZ
- https://codeberg.org/viceice
- https://codeberg.org/voltagex
- https://codeberg.org/wetneb
- https://codeberg.org/wolftune
- https://codeberg.org/xenrox
- https://codeberg.org/xtex
- https://codeberg.org/Zip
- https://translate.codeberg.org/user/antaanimosity
- https://translate.codeberg.org/user/artnay
- https://translate.codeberg.org/user/Atalanttore
- https://translate.codeberg.org/user/Atul_Eterno
- https://translate.codeberg.org/user/Baempaieo
- https://translate.codeberg.org/user/CN-P5
- https://translate.codeberg.org/user/dobrvlskyi
- https://translate.codeberg.org/user/Edgarsons
- https://translate.codeberg.org/user/emansije
- https://translate.codeberg.org/user/EssGeeEich
- https://translate.codeberg.org/user/faoquad
- https://translate.codeberg.org/user/Fjuro
- https://translate.codeberg.org/user/jutty
- https://translate.codeberg.org/user/kwoot
- https://translate.codeberg.org/user/Lzebulon
- https://translate.codeberg.org/user/Outbreak2096
- https://translate.codeberg.org/user/raspher
- https://translate.codeberg.org/user/SomeTr
- https://translate.codeberg.org/user/WithLithum
- https://translate.codeberg.org/user/Wuzzy
- https://translate.codeberg.org/user/yumechi
- https://translate.codeberg.org/user/Zughy
A minority of Forgejo contributors earn a living by implementing the roadmap co-created by the Forgejo community, see the sustainability repository for the details.