Rakudo on OpenBSD

EDIT: After writing this post, tyil, the maintainer of rakudo star, reached out to me and added proper OpenBSD compatibility. The portion of this post dedicated to working around the failing downloads can now be ignored, and rstar now includes a warning if the user doesn't have a login class set.

EDIT: The development version of Rakudo Star targets Rakudo 2020.07 instead of 2020.02, which has uses even more memory than the staff login class is allowed. You may need to edit /etc/login.conf and change this line.

--- login.conf	Wed Jul 29 17:54:15 2020
+++ login.conf.new	Wed Jul 29 17:54:38 2020
@@ -71,7 +71,7 @@
 # Staff have fewer restrictions and can login even when nologins are set.
-	:datasize-cur=1536M:\
+	:datasize-cur=2536M:\

Or you can set a higher memory limit if you're using the staff class using ulimit -m.

I really enjoy using raku to write small scripts for system maintenance and text parsing. Its regex and grammar engine are next level! The problem with using it on OpenBSD is that the packaged version is a couple years out of date. The version in ports is from 2018, which contains a bug regarding NativeCall on OpenBSD. Not to mention it's missing a lot of performance gains and patches.

Instead of just compiling everything from source and installing them myself as I did on my last system, I installed it using the rakudo star distribution and its rstar command. Rakudo Star is raku plus some community modules and the zef package manager. It also comes with the rstar command, which helps you in the build process.

Unfortunately, the first thing I had to do was install bash, as that is what the rstar command is written in.

doas pkg_add bash

After installing bash, I came across a problem. Running ./bin/rstar fetch fetches all of the materials required to compile and assemble the star distribution, and while all of the community modules were successfully retrieved using git, it failed to pull in all 3 major source components. MoarVM, nqp, and rakudo, are pulled in as gzipped tarballs, each of them failing to download. I suspect it's an issue with something in the bash scripts. Tar also throws a warning about one of its arguments, which has to do with the script using a GNU tar specific command.

[2020-07-25T05:25:33] [NOTIC] Downloading https://www.moarvm.org/releases/MoarVM-2020.02.1.tar.gz to /home/dante/star/tmp/tmp.IMPt8mIFoC
tar: WARNING! These patterns were not matched:
[2020-07-25T05:25:34] [CRIT]  Failed to download /home/dante/star/src/moarvm-2020.02.1
[2020-07-25T05:25:34] [NOTIC] Downloading https://github.com/perl6/nqp/releases/download/2020.02.1/nqp-2020.02.1.tar.gz to /home/dante/star/tmp/tmp.zDxiGW2Sxq
tar: WARNING! These patterns were not matched:
[2020-07-25T05:25:40] [CRIT]  Failed to download /home/dante/star/src/nqp-2020.02.1
[2020-07-25T05:25:40] [NOTIC] Downloading https://github.com/rakudo/rakudo/releases/download/2020.02.1/rakudo-2020.02.1.tar.gz to /home/dante/star/tmp/tmp.W1mRYHVj1C
tar: WARNING! These patterns were not matched:
[2020-07-25T05:25:46] [CRIT]  Failed to download /home/dante/star/src/rakudo-2020.02.1

To work around this I downloaded the required files manually using OpenBSD's ftp command and extracted them into the src directory that had been created by the script. A small caveat is that MoarVM automatically extracts into a directory called MoarVM-2020.02.1, which needs to be completely lower-cased for the rstar to work.

cd src
ftp https://www.moarvm.org/releases/MoarVM-2020.02.1.tar.gz
ftp https://github.com/perl6/nqp/releases/download/2020.02.1/nqp-2020.02.1.tar.gz
ftp https://github.com/rakudo/rakudo/releases/download/2020.02.1/rakudo-2020.02.1.tar.gz
tar -xzf MoarVM-2020.02.1.tar.gz
tar -xzf nqp-2020.02.1.tar.gz
tar -xzf rakudo-2020.02.1.tar.gz
mv MoarVM-2020.02.1 moarvm-2020.02.1
rm *.tar.gz
cd ..

Running rstar install then began compiling things. rstar install will install rakudo star into the build directory by default, but you can change that with -p to specify a prefix location.

MoarVM and nqp both compiled and installed fine, but when it came to compiling Rakudo, it failed with a memory allocation error message.

+++ Generating	gen/moar/Compiler.nqp
+++ Generating	gen/moar/Optimizer.nqp
+++ Compiling	blib/Perl6/Optimizer.moarvm
+++ Compiling	blib/Perl6/Compiler.moarvm
+++ Compiling	rakudo.moarvm
+++ Generating	gen/moar/BOOTSTRAP/v6c.nqp
+++ Generating	gen/moar/Metamodel.nqp
+++ Compiling	blib/Perl6/Metamodel.moarvm
+++ Compiling	blib/Perl6/BOOTSTRAP/v6c.moarvm
+++ Compiling	blib/CORE.c.setting.moarvm
The following step can take a long time, please be patient.
Stage start      :   0.000
MoarVM panic: Memory allocation failed; could not allocate 84800 bytes
*** Error 1 in /home/dante/star/tmp/tmp.gqTyPvsgV1 (Makefile:800 'blib/CORE.c.setting.moarvm': @'/home/dante/star/bin/moar' --libpath='/home...)
[2020-07-25T05:38:43] [ALERT] Build failed!

I tried then manually building rakudo to see if I could figure out what the problem was.

cd src/rakudo-2020.02.1
./Configure.pl --prefix /home/dante/star/

While rakudo was compiling, I monitored memory usage in a separate tmux pane, and noticed that the moarvm process was using around 760 MB of RAM before it crashed. 768 MB is the maximum amount of ram a process can use in OpenBSD when run by a user with the default login class, as specified in /etc/login.conf.

To remedy the problem, I changed my user's class to staff, which grants it a much higher datasize limit, among a couple other things. If you're interested about how this works, you can check out login.conf(5) and the /etc/login.conf file on your system.

doas usermod -L staff dante

After setting my login class, and logging out and back in, I restarted the build and install using the rstar install command as before, just to make sure it sets everything up how it wants.

After about half an hour of compilation, I was finally presented the following message.

[2020-07-25T06:34:06] [INFO]  Rakudo Star has been installed into /home/dante/star!
[2020-07-25T06:34:06] [INFO]  The installation took 0h 25m 02s.
[2020-07-25T06:34:06] [INFO]
[2020-07-25T06:34:06] [INFO]  You may need to add the following paths to your $PATH:
[2020-07-25T06:34:06] [INFO]    /home/dante/star/bin
[2020-07-25T06:34:06] [INFO]    /home/dante/star/share/perl6/site/bin
[2020-07-25T06:34:06] [INFO]    /home/dante/star/share/perl6/vendor/bin
[2020-07-25T06:34:06] [INFO]    /home/dante/star/share/perl6/core/bin


I added the specified directories to my path by editing ~/.profile and inserting the following line.

--- .profile	Sat Jul 25 06:42:48 2020
+++ .profile	Sat Jul 25 06:42:31 2020
@@ -3,4 +3,5 @@
 # sh/ksh initialization


I chose to keep it in the /home/dante/star directory instead of installing it to /usr/local in case the version of rakudo in ports gets updated some time.

I now have an up to date version of raku running on my OpenBSD machine!