My Forgejo monthly update – February 2023

This is my personal view of what happened during February in Forgejo. It took me about five to six hours to write and I find it very useful, to reflect on what happened and to take the time to think about what is ahead. In January it took me another eight hours over a few days to reach out to community members, collect their feedback, learn about what they have done and update the report accordingly. That is the same kind of effort it would take for this blog post to be published in the Forgejo blog. It is unfortunately more than I can sustainably do on a monthly basis but someone else could. I wrote the content of this post in the same style as the January update so that it can be conveniently copy/pasted, should a Forgejo community member be motivated to.

Forgejo, as a software product, is showing signs of doing well. There was a steady stream of stable releases and Forgejo v1.19 is almost ready with an experimental integrated CI, a new documentation and more.

The decision making process was finalized and put to use. For technical decisions such as defining the criterion for pull requests to be merged and appointing the security team.

Forgejo experimental integrated CI

Prior to Forgejo v1.19 running CI jobs required a third party software such as Woodpecker CI. It has its own web interface, relies on webhooks to be notified something changed in a repository and relies on the Forgejo API to figure out user permissions or access to private repositories. When it is finished, the error or success is also sent back to Forgejo via the API, with a link the user needs to click to get more details. It works well enough and Forgejo has been using Woodpecker CI from day one, for testing pull requests or publishing releases.

With v1.19 comes an experimental CI (not ready for production just yet) integrated in Forgejo. The CI jobs are configured with a syntax that is similar to GitHub Actions.

The Forgejo runner is a new addition to the Forgejo dependencies (git & ssh) and is not yet packaged, it must be built from sources. It is a thin layer on top of ACT which implements the core of the logic to interpret and run the jobs. A soft fork of ACT is used and contains commits that are not yet submitted or accepted upstream.

Read more about Forgejo Actions.

Admin and user documentation

The first version of the Forgejo documentation is available and includes:

  • An administrator guide targeted to people who run Forgejo on their machines.
  • A user guide targeted to people who are using Forgejo but are not involved in running it on their machines.

Content was copied from the Codeberg and Gitea documentations and improved while writing the release notes for Forgejo v1.19. Links to similar documentation maintained elsewhere are included in each page and will be used to keep the documentation up to date.

There is a versioned link for the v1.19 documentation at https://forgejo.org/docs/v1.19/. As soon as it is released, a new version will be created at https://forgejo.org/docs/v1.20/ to receive updates about new features. https://forgejo.org/docs/latest/ will always be an alias to the latest stable version.

This documentation has useful content that did not exist at all before but is still lacking in terms of look & feel, maintainability, accessibility and internationalization.

Privacy

A new method was implemented in Forgejo v1.18.3-1 for Forgejo admins to display a reminder of the availability of new release in the administration panel.

If activated, it will query the DNS using a request similar to:

$ dig +short release.forgejo.org TXT
"forgejo_versions=1.18.5-0"

It offers better privacy protection than the existing method which relies on a HTTP request to a single URL that leaks the IP address of the Forgejo instance to whoever has access to the web server logs.

Accessibility

With Forgejo v1.18.3-1 came accessibility bug fixes. They will also be in the upcoming Forgejo v1.19 releases.

Forgejo 1.19 release candidates

The first release candidate for the upcoming Forgejo v1.19 release was published. To avoid any confusion with stable releases, it is located in the forgejo-experimental organization.

Forgejo v1.19 is the second major release of Forgejo and was specially challenging because of the integrated CI.

After some discussion it was decided that:

In the interest of dogfooding, the CI of the runner uses Forgejo Actions and runs on a temporary Forgejo instance. Ideally the runner would be hosted on Codeberg but its experimental nature is incompatible with the stability requirements of a Forgejo instance hosting over 50,000 users.

The release notes are written and organized to help the Forgejo admin:

  • Each breaking change includes an explanation of why it is breaking in plain English together with the recommended actions
  • Links to the relevant page of the Forgejo documentation are included where appropriate
  • Instead of adding a list of all commits that clutter the release notes, a link to the git history is provided and contains the same information

The drafting of the release notes started when the first release candidate was published and will be improved until the stable release is published.

A solid release team

The stable releases team is in the process of being agreed upon by the Forgejo community, with three candidates so far.

The security team was the first to be officially appointed in accordance to the decision making process. It participated in two stable releases (Forgejo v1.18.3-2 and Forgejo v1.18.5-0) and improved its internal organization and tooling.

To effectively reduce the bus factor, Forgejo v1.18.3-1, Forgejo v1.18.3-2 and Forgejo v1.18.5-0 were each taken care of by different members of the team. It also helped identify pain points that need work.

A significant bug was discovered in the release process. It was quickly fixed and a detailed blog post was published to provide details information about it and included recommended actions. There is no such thing as a bug free release process and communicating properly when a problem happens is what Forgejo users expect and deserve.

Soft fork tools

Forgejo follows a workflow to organize the development and the releases that unusual because it is a soft fork. It was documented early on and carried out manually until recently.

Over a period of months, the rebasing on top of Gitea was done weekly and documented in an issue. It evolved and matured to the point where it became a kind of specification for a process that can be mostly automated. The only parts requiring a human brain are:

  • conflict resolution
  • maintaining a clean commit history for each feature branch

A tool is being developed and although still very experimental, it has been used successfully in the past month. It is shell based and includes thorough testing.

Each rebase is now a milestone (see the 25 February rebase for an example) that includes a pull request for each rebase of a feature branch, with comments regarding how conflicts needed to be resolved and how commits were squashed, if any. In the simpler case, when a feature branch had no activity and does not conflict, the rebase is fully automated.

Governance

The work to define the governance made good progress. The decision making process was finalized and documented. Next in line is the design of an election mechanism and it was discussed during the 20 February online meeting. There is no explicit process for assigning roles to people in the project. The general decision-making process is a good basis, but it doesn’t tackle all the questions and challenges related to election (e.g. who facilitates the election process; how to determine (and who determines) who’s a relevant candidate for a given role, etc.

Community challenges

A racist comment was published early February in a Forgejo space and the author continued with a threat of physical violence and numerous aggressive comments disrupting communications and creating tensions. It went on during weeks and the interim moderation team was not prepared to handle the challenge.

The unfortunate side effect of the actions of this single individual was to push some community members over the edge and turn against each other. Fortunately the Forgejo community is still young and solutions are actively worked on to heal and rebuild.

If anything, this sad episode shows how important it is to preserve a safe space and organize moderation to efficiently and quickly put an end to the most toxic behaviors.