Forgejo monthly update - January 2025

The monthly report is meant to provide a good overview of what has changed in Forgejo in the past month. If you would like to help, please get in touch in the chatroom or participate in the ongoing discussions.

Releases

Forgejo v10.0.0 was published on 16 January, one day after the planned release date because a regression was discovered at the last minute. Another issue surfaced a few days later, when migrating Forgejo instances that contained orphaned records in a database table. Such records are not found in any test environment and do not prevent Forgejo from running, but they interfered with the upgrade of two instances. Fortunately, there is a rather simple workaround and the doctor is now able to dispose of them. Codeberg was able to upgrade without incident a week later.

The Forgejo release manager was improved on this occasion. The tasks that are common between patch releases and major releases (release-tasks, release-announce, etc…) were re-used as-is. There are still a number of items that need to be done manually and they can gradually be automated within this framework.

The Forgejo runner v6 major release was published continuing the effort to fix security issues and to bring it closer to a beta stage. It also contains support for ephemeral runners and force rebuilds.

A new major release of the Forgejo helm chart was published 15 January. It contained a regression that was fixed the next day, with 11.0.1. It has a few breaking changes and defaults to using Forgejo v10.0.0. The Forgejo helm v7.1.7 patch release to be used with Forgejo v7 was also published.

Translations

A PR initiating the switch to a more flexible translation format has been merged, however, a bug was discovered in Weblate that prevented a new component from being used right away, seemingly because Forgejo is pioneering usage of go-i18-v2 format in Weblate. @Gusted figured it out and submitted a fix to upstream. It has been merged, but we’re waiting for a Weblate update which will include that fix.

Filipino translations were completed (thanks to the effort of @kita) in December 2024 and have been released for the first time in the v10 release in January. Similarly, Lower German is now available as a new language. Especially since there are comparatively few speakers, we are proud to be among the software projects available in these locales.

FOSDEM

Forgejo and Codeberg had a stand during FOSDEM February 1 and 2 in Brussels. We might go into details about this in the February update (because, well, apart from travelling there most of it happened in February anyway and we still need to wrap things up on our end). You can read and participate in our discussion about this year’s FOSDEM, feel free to share your impressions.

It was a pleasure talking to so many of you, and the first time many of our team members have seen each other in person. Thank you for your trust and support, and for distributing our stickers!

Federation

If you follow the Forgejo monthly updates, you might have noticed that we haven’t written about federation recently, and some of you asked about it. We apologize for the omission, but rest assured: Federation is still a high priority and we haven’t given up on it.

Due to personal constraints, things have slowed down a little bit by the end of the year. With federated star, a first user-visible feature has been implemented and some follow-up PRs have been in development, but not yet merged. Federated unstar (which complements the starring of repos) will be the next task, but due to it being a little destructive, more backend work is happening. The developers briefly talked about this during their FOSDEM talk and first code was submitted by the end of January.

A federation roadmap was developed and discussed, which provides the next steps and their requirements to make federation useful in some capacity. There is commitment to complete some of these milestones in 2025.

To ensure the federation project can move on regardless of limited funding capabilities, it was agreed to compensate the current federation maintainers for maintenance jobs such as code review and onboarding with up to € 500 / month. This ensures that community contributions to federation can be reviewed and merged quickly. The funding was provided to Forgejo by Codeberg e.V..

Website and documentation

Forgejo actions were implemented to check for spelling mistakes upon pull requests, both for documentation and website using the source code checker typos. To our positive surprise, there weren’t many spelling mistakes on the website, which means that Forgejo’s contributors already did a good job in detecting those during blog post draft approval.

Broken links are an annoyance to readers of the website. The amount of those in the documentation was significantly reduced thanks to the scheduled link check action. It uses the lychee binary and was added to the website pages, too. While implementing, too many requests were sent to codeberg.org and mistaken for abuse, we’re sorry for that! Fortunately, the server is set up with automatic mitigation of such events by blocking the concerned IP address for half an hour. The reason was the massive amount of links to contributors, which are now excluded from the daily link checks.

Apart from the usual documentation improvements, the documentation footer on the website has been improved to properly attribute the authors of derived CC-licensed work and the edit link has been replaced to point always to the related document in the next branch.

To further simplify the release process, when building the website, latest and LTS releases are now automatically retrieved using the recently introduced release schedule.

Work has started to improve the accessibility of the Forgejo’s website, such as adding a global search and documentation version switches. Contributions are always welcome to both the website source and the documentation.

Infrastructure

On the day of the Forgejo v10.0.0 release, the IP of the machine running the workflows responsible for the release was rate limited by https://hub.docker.com/ which host most of the container images used in the CI. It disrupted a number of workflows and had to be dealt with immediately. All workflows were modified months ago to use Forgejo hosted mirrors updated daily to prevent that kind of disruption but some references were overlooked. After they were fixed, the workflow responsible for the release was no longer rate limited and the release could proceed.

code.forgejo.org went down during 15 minutes on 21 January 2025. It was the first unscheduled downtime since 9 September which lasted about 10 hours. Read more in the post-mortem.

About two years ago, Forgejo action runners were installed manually on a machine that is still in use today. It grew over time and around twenty were also installed manually. Ideally they could all be migrated to the new k8s cluster but there is no LXC support in k8s and it is required for a number of workflows that cannot run out of application containers (podman or docker).

A script managing a systemd unit dedicated to the runner was created and used to manage all runners with familiar commands such as systemctl start forgejo-runner@42.

Dependency management

The forgejo dependency configuration and the renovate organization (formerly in forgejo-contrib) are the central point from which the dozens of repositories involved in the making of Forgejo (CI, tools or releases) are watched. It automates the obvious but tedious task of ensuring, for instance, that the go release found in go.mod has an up-to-date version with regard to security vulnerabilities.

Doing so for an individual repository (which was the starting point of that effort with the main Forgejo repository in 2024) is convenient but it does not scale well. To help with that, shared configurations (see the one used by Forgejo for instance) have been created and underwent significant improvements this month.

In some cases, such as dependencies that are only used in the CI, it is OK for a pull request containing an upgrade to be merged unattended as soon as the workflows pass. This is what the automerge feature is for but it left a branch behind because it was not supported prior to Forgejo v10. The Forgejo driver for renovate was patched and there are no longer lingering branches after an automerge.

A number of additional repositories were added to the pool of repositories that renovate watches and it is no longer necessary to individually approve them, as long as they belong to a known organization.

Because renovate was not in use in 2023 (or in some cases where scripting is involved), some dependencies are not specified in a way that renovate can natively discover. For those, dedicated rules are being developed which run the equivalent of sed -i -e 's/old version/new version/' on the designated file (e.g upgrading the RUNNER_VERSION file in setup-forgejo).

Now that dependency management has mature tooling in all Forgejo spaces, it is easier for both volunteers or paid staff to review each individual upgrade. There are just a few every day but most of them require human scrutiny. A call for participation was posted and will hopefully be attractive to people willing to contribute to Forgejo.

Moderation incidents

January was a little noisy regarding moderation actions. There have been cases of community members that created a lot of noisy issues, presumably with the help of AI tools. While probably in good faith, throwing walls of text by increasing the amount of open issues in forgejo/discussions by 10% in only a few days as well as responding with huge amounts of text to even simple questions was exceeding the capacity of our volunteers to deal with.

Additionally, some new contributors required a lot of help, started heated discussions, or obstinately pushed their own ideas even if they faced disagreement by several other community members. These incidents have created a slightly tense atmosphere for a few days.

Luckily, the community has openly talked about the incidents, both in issues and in the Matrix channel, and most of the problems could be resolved quickly. There have been cases where the noise has led to premature reactions by some maintainers, for example when a lengthy issue has been reported as “spam” or “trolling”, even if they might have been created without bad intents.

We apologize if you have been impacted by any of these incidents, either because an issue you have reported has been flooded with heated discussions, or because a contribution from you has been flagged too quickly. If you think one of your contributions did not receive the attention it deserved, feel free to carefully reopen or recreate it, respecting the pace at which the community can possibly deal with them.

The exceptional level of tensions lasted for a few days and calmed down again. We expect that this was mainly bad timing to have multiple such cases at the same time and there are no signs this would repeat in the new future. Nevertheless, a new moderator was proposed to increase the human resources within the Forgejo moderation team.

Forgejo strives to be an inclusive project where everyone can participate in a safe environment. If you, as a community member, feel unsafe for any reason, feel free to reach out to the moderation team.

We Forge

Forgejo is a community of people who contribute in an inclusive environment. We forge on an equal footing, by reporting a bug, voicing an idea in the chatroom or implementing a new feature. The following list of contributors is meant to reflect this diversity and acknowledge all contributions since the last monthly report was published. If you are missing, please ask for an update.