Switch to Waf build system

[Vojtech Horky]

Rewrite the current make-based build scripts into ​Waf-based ones.

Details

The build process of HelenOS is currently controlled by make. The build system works well but there is space for improvement that is not possible with Makefiles. The task is to convert currently existing Makefiles into Waf scripts.

Compiling an operating system usually requires a non-trivial build scripts and, unfortunately, HelenOS is not an exception. Thus, the conversion would not be a trivial search-and-replace exercise. HelenOS has a lot of configuration options that are reflected in the build scripts where they control what has to be built and how. All this would have to be converted and for many of the tasks there are not built-in equivalents in Waf: new tasks would have to be defined and probably a lot of existing ones would have to be changed.

The effort to switch to Waf started some time ago but due to lack of time was abandoned. It is possible to get inspiration from the obsolete branch located at ​lp:~vojtech-horky/helenos/waf and a somewhat more recent branch at ​lp:~jakub/helenos/waf-revival

There are a lot of helper tools that are used during the build process and most of them are written in Python. Some changes to them might be necessary as well to better integrate them with Waf.

What Gains and Benefits will this bring?

Although it is possible to build out of the source tree with make, it would require extensive patching of the current Makefiles. Waf offers this automatically. Furthermore, it shall be possible to have different build configurations at the same time in the same source tree. That would allow to incrementally build for different architectures at the same time. This would significantly reduce the time needed to check whether the functionality works okay across different platforms.

Make supports parallel build but this does not work well across directories while Waf has a "global" knowledge of what has to be built and shall provide better parallelization (more evenly balanced).

Specifying which library to link the application with currently means adding both compiler and linker flags to the respective Makefile. With Waf it is possible to specify exported headers for each library and then reference them just by library name. That would simplify writing the application scripts and would also make them more robust.

Waf scripts are actually snippets of Python code which could be useful in some special situation where writing the logic in an imperative language simply makes more sense.

Last year(s) a lot of effort was invested into making HelenOS self-hosting. Porting Python is a necessary prerequisite because of the tools used during the build. Having also the build process controlled by a Python script would reduce the number of other applications that would have to be ported (namely Unix shell, make and makedepend).

Difficulty

Easy to Medium

Required skills

A successful applicant will have good skills of programming in the Python language and good understanding of the compile/link process used when building C programs. Good knowledge of make and related tools is necessary. Prior knowledge of Waf is clearly an advantage.

Documentation

• ​Waf homepage
• ​The old HelenOS/Waf branch
• ​The unfinished HelenOS/Waf revival branch
• ​Mailing-list discussion on out-of-source build in HelenOS

Possible mentors

HelenOS Core Team, Vojtech Horky, Jakub Jermar

[Jakub Jermář]

Commit ​32b4302cc4be7f8c85a3318ccb131beffe806690 switched the build system to Meson, so deferring this.