Category Archives: Linux

Software Build

 

For those who are following this blog, you are aware that I am attempting to apply some of the knowledge I have recently gained in a software portability and optimization class. For those of you who are not aware you can check my previous posts, specifically:

https://penguinpro.ca/category/school-work/project/

And for those interested in reading my entries on some of the material we worked on in the course:

https://penguinpro.ca/category/school-work/

 

Briefing:

Before I get into this I am going to make a clear note here that as mentioned in the previous posts about the course having a time restriction I am “concerned” in regards to the complexity of the chosen software. This Build has not been the easiest software to build, in regards to the required runtime software and kernel modules; user permissions and even source modifications, as you will soon see/read.

What does this mean? Well I may have chosen another, potentially more appropriate for the course, piece of software. Regardless, this will not stop me from chipping away at this project too, even if just for personal interest.

 

Before Building:

The first thing we do is download the source, which comes as a tar file:

https://download.virtualbox.org/virtualbox/6.0.14/VirtualBox-6.0.14.tar.bz2

This file is then extracted with:

tar –xvf VirtualBox-6.0.14.tar.bz2

Running this with the time command shows that it took about 57.98 Seconds to decompress, and after which we are given a directory with our source. I am going to provide this specific link just in case you are attempting this build yourself. Even just glancing this will save you save you some headaches:

https://www.virtualbox.org/wiki/Linux%20build%20instructions

There is even a section showing what args to pass the packages manager for many Linux distributions specifically. Now let’s move into that directory it’s time to build.

 

Building:

Within this directory we can see that there is a configure file that needs to be run, but let’s first take a look at some of the files with “config” in the name.

The first is CONFIG.kmk, after some google searching we can see that kmk is part of “KBUILD” a type of makefile:

https://stackoverflow.com/questions/8016678/understand-what-is-kbuild

From this point you can read some of the README’s and configurations yourself as this post does have to be completed at some point today and this information is pretty straight forward for anyone who is interested enough.

 

Configuration:

At this point we are going to run the configure command:

                ./configure

Which will run the script with the file of the same name. The point of this is to make sure that we have all the requirements to properly build this software, let’s run it with the time command. Not that during this write up I had already previously run through the dependencies to make sure there would not be any issues during my explanation.

After we run this, (which took about 15 seconds), we get out output stating that we have “Sucessfully generated autoconfig.kmk and env.sh”, more about this a moment because we have some interesting output:

 

  +++ WARNING +++ WARNING +++ WARNING +++ WARNING +++ WARNING +++ WARNING +++

  Hardening is enabled which means that the VBox binaries will not run from

  the binary directory. The binaries have to be installed suid root and some

  more prerequisites have to be fulfilled which is normally done by installing

  the final package. For development, the hardening feature can be disabled

  by specifying the –disable-hardening parameter. Please never disable that

  feature for the final distribution!

  +++ WARNING +++ WARNING +++ WARNING +++ WARNING +++ WARNING +++ WARNING +++

 

It seems that we are required to run configure with “—disable-hardening –disable-docs” to get a build that would be appropriate for our purposes. “—disable-docs” being included to avoid the docbooks dependency.

Running it again we are given a visually similar warning with output stating:

 

  +++ WARNING +++ WARNING +++ WARNING +++ WARNING +++ WARNING +++ WARNING +++

  Hardening is disabled. Please do NOT build packages for distribution with

  disabled hardening!

  +++ WARNING +++ WARNING +++ WARNING +++ WARNING +++ WARNING +++ WARNING +++

 

Compiling:

Looking at the Oracle build instructions, from this point it seems that we are supposed to run env.sh and then a command called “kmk all”, so we will need to first run:

chmod +x env.sh

after running this command and executing ./env.sh we have no output and there is no “kmk” command. After further analysis we can see that in the kBuild directory there is another env.sh. Also I should note that it is already set to be executable, lets run it.

We get the output:         “Spawning work shell…”

 

So that’s interesting, lets go back to our main working directory where the source was decompressed to, and try running “kmk all” again, which runs until:

                Src/VBox/Devices/PC/vboxssdt-cpuhotplug.dsl

With a repeating error of: “_UID inside processor declaration must be an integer ^ (found a string)”

Using google to find the error message led me to this fix:

https://forums.virtualbox.org/viewtopic.php?f=10&t=94467

Within the link, there is a comment from someone with similar issue. The solution as you will see, involved changing the last argument to an incrementing number of int constants. At this point being left with minimal options in this, I am going to attempt the fix on the source code

 

Source Modification:

In src/VBox/Devices/PC/vboxssdt-cpuhotplug.dsl between lines 97 and 130 (as of this writing,) is this block of code:

 

        GENERATE_CPU_OBJECT(0x00, SCK0, “SCKCPU0”, CPU0, “SCK0-CPU0”)

        GENERATE_CPU_OBJECT(0x01, SCK1, “SCKCPU1”, CPU1, “SCK1-CPU0”)

        GENERATE_CPU_OBJECT(0x02, SCK2, “SCKCPU2”, CPU2, “SCK2-CPU0”)

                […snip…]

        GENERATE_CPU_OBJECT(0x1e, SCKU, “SCKCPUU”, CPUU, “SCKU-CPU0”)

        GENERATE_CPU_OBJECT(0x1f, SCKV, “SCKCPUV”, CPUV, “SCKV-CPU0”)

 

We need to go through and change the source to be:

        GENERATE_CPU_OBJECT(0x00, SCK0, “SCKCPU0”, CPU0, 0)

        GENERATE_CPU_OBJECT(0x01, SCK1, “SCKCPU1”, CPU1, 1)

        GENERATE_CPU_OBJECT(0x02, SCK2, “SCKCPU2”, CPU2, 2)

                […SNIP…]

 

All the way through the list from 0 – 31

The exact reasoning for this, I will have to look into deeper, but in the case that you are the curious type and want to research more yourself. I think it (might) relate to these links:

https://en.wikipedia.org/wiki/Advanced_Configuration_and_Power_Interface#SSDT

https://en.wikipedia.org/wiki/System_Service_Descriptor_Table

 

JAVA:

Carrying on with the build where the compilation stops again to complain that:

                src/libs/xpcom18a4/java/src/nsAppFileLocProviderProxy.h

Cannot find the included jni.h, and digging deeper into the error message output from the compiler, it is requesting a directory of:

/usr/lib/jvm/java-6-sun/include

Again another link for the curious:

https://en.wikipedia.org/wiki/Java_Native_Interface

 

At this point I am assuming this is looking for java version 6 but unfortunately in Arch Linux that means we are going to need to build a package. You can use google to look for “Arch Linux Java 6” or I can just provide you the link here:

https://aur.archlinux.org/jdk6.git

 

Building Dependencies:

Specifically we can use git like this:

Git clone https://aur.archlinux.org/jdk6.git

Of course being aware of what directory you are in; basically, don’t add it to the VirtualBox source. It may be a good idea to have a separate directory in your home for this stuff, but let’s not get too off topic. Once you have that run the next set of commands, it may warn you about dependencies as you go and you will be required to install them as well.

                makepkg

                pacman –U jdk6-[version]-pkg.tar.xz

Remember to change [version] to the version info in the built filename.

 

After building and installing this version of the JVM it failed at the same spot again, it seems that it may also require some further modifications.

Modifying directories:

Looking at the error message produced during compilation. It seems that the build is looking for /usr/lib/jvm/java-6-sun, where as we have /usr/lib/jvm/java-6-jdk.

The files are all the same internally so I am going to attempt a symlink:

ln –s /usr/lib/jvm/java-6-jdk /usr/lib/jvm/java-6-sun

I don’t know if this is a good idea but it seems to work.

 

Building Other Requirements:

There we have it, our build is complete (kind of,) still more that needs to be done. Our next step is to add the kernel modules. To get there; from our VirtualBox source directory let’s type the command:

                cd out/<your platform>/release/bin

Where we need to go into the directory labeled src where the source files are for creating our kernel modules. In this directory we can simply type:

make

Once that completes and we have the .ko files, do not attempt to add them yourself with insmod, there is a script in the bin directory we were previously in.

Back in bin there is are shell scripts called loadall.sh and load.sh, these scripts will handle adding those built modules in whatever order they are required.

Finally we are ready to run:

                ./VirtualBox

Where we are provided with:

Image_1

As you can see, this was not the easiest thing I have ever tried to build from source; and with that being noted I unfortunately do not feel that it is appropriate for the context of this class.

I am however hopeful that I can continue on with this technology by myself. Learning more about virtualization and how all of it works is a bit of a personal fascination. One that I will most likely pursue on my own and although I will be building another piece of software for my course. You are more than welcome to stick around and see how “playing” with the VirtualBox source works out.

As for now, how well does this thing work? Well in user space? It doesn’t. It does start and I can create VMs but they crash with permission errors upon loading the .iso file. When running as root however, (which is another thing that complicates things in terms of attempting optimization.) I am able to create VMs, install O.S’s and even import appliances, check it out:

image2

 

Thanks for reading, and I hope to have the my next post about the built process of my actual chosen software build up soon.

 

With love,

ElliePenguins

VirtualBox pfSense

Configuring a virtual router for clutter free development.

 

When needing access to a physical arm based platform but unable to have all of the external physical components, (monitor, mouse, keyboard, etc.)  It can be really useful to just attach the raspberry pi directly to a laptop Ethernet port and then ssh via. powershell or whatever your preferred client or shell may be.

This will be a quick guide on how to install pfSense on virtualbox and set it up to create a dhcp enabled network on a machines Ethernet port. Of course this set up can not only be used for just raspberry pi but for any physical machine that needs to be connected without a physical router. The benefits of which can range from personal experimentation to creating lab environments for those involved in remote training workshops.

Requirements:

  • Computer capable of running virtualbox, preferably with 2 interfaces at least one Ethernet, usb Ethernet nics can work well too.

  • Understanding of how to create a virtual box vm and install an OS onto it. (There are lots of excellent tutorials with a simple google search, and overall does not take long to get a good enough understanding to continue with this guide.)

  • pfSense installation, can be acquired here: https://www.pfsense.org/download/

  • Ethernet cable

  • External machine (eg. Raspberry Pi)

 

Before Starting:

Be sure that your raspberry pi (or whatever machine you are connecting to your host machine) has it’s ssh server enabled as the reason in our case for doing this is to avoiding the need of physical hardware (monitors, keyboards etc.) If the ssh server is not enabled we will not have a means of accessing the device remotely.

 

Setup and Installation:

Create a new virtual machine and set the OS type to: BSD, and software version to: FreeBSD. The default settings beyond this point should be fine, use your personal preference for storage types.

The next step is to go into the new vm’s settings in VirtualBox and click on network, from there we need to enable 2 network interfaces, the first will be bridged to our wireless nic. and will be used for our wan interface, the second will be bridged to our Ethernet port which will become the lan nic. that our lab machine will be connected to.

At this point we can fire up the VM and attach the installation media (in our case an iso image)

install_prompt

The basic installation is fine enough, there is no need to try and get fancy with the installation for our purposes. Once the installation finishes follow the prompts and you will be asked to reboot the machine, at this point remember to remove the installation media:

( devices -> optical drives -> {uncheck} pfsense.iso )

 

Configuration:

Note that during the first boot, because the interfaces are not set up properly it may take some time on some boot options, be patient. Then once rebooting has completed you will be presented with:

prompt_one

Now we are actually configuring the device, first we set up the lan interface and depending on your reasons for following this guide you may or may not need the wan interface but I will go through that process as well.

At this point, at least theoretically the Ethernet interface is ready to start dishing out dhcp addresses; although, as you can see in the image there may be a problem with the default subnet if we choose to set up a wan interface. Our next step then will be to assign the interfaces:

The cli interface is not particularly helpful but it is what we have to work with at this point.

Press 1 then enter

We are asked if we need vlans, answer no then press enter.

Next we are asked to enter the wan interface name: note from the above picture we have em1 and em0.

For our setup we type em0 and hit enter, however depending on how you configured your interfaces in the VirtualBox vm’s settings menu, it could be different.

We are then prompted to enter the lan interface, Now we enter em1 and press enter.

Then, if everything looks right we can confirm our settings.

We are back at the main menu, we can now set up our interface IP address:

Press 2 and hit enter.

Select the lan interface, which in my case is 2 and hit enter again.

Enter the preferred lan ip address and press enter:

I will choose: 192.168.25.1

Then we are prompted for our subnet mask, which will be in CIDR notation.

For my purposes I will type 24 and press enter.

Next you may be asked for a gateway address, just press enter.

Same with the prompt for ipv6 (which you can configure if you wanted to, I am choosing not to) so just press enter again.

This is important, you will next be prompted if you want to “enable the DHCP server on LAN”, type y and press enter. (We need to be able to assign addresses to our connected devices.)

Next enter the start of the ipv4 client address range, personally for this type of learning environment I like to start at 100 as it makes client addresses simple and visible. Although As long as the subnet is the same, you can set this to whatever you prefer.

Type 192.168.25.100 and press enter.

The next prompt is for the end address. For my purposes I choose 199, although if you plan to have more than 99 hosts in your lab setup you can increase this number to 250, as well as lowering the start address range.

Type 192.168.25.199 and press enter.

Finally we are prompted with “Do you want to revert to http for the webConfigurator protocol:”

I will select yes as this is a personal practice environment, if you need a more secure option then you can select no.

A prompt will show you how to access the web interface if you wanted a more complex setup, although this guide is only for a simple installation. You can play with it more on your own, learn and configure things to your preference. (virtualbox snapshots are helpful for this, I suggest learning a bit about them.)

prompt_two

At this point your “virtual router” is now configured and you can boot it up whenever you want to play with some external physical hardware via. your Ethernet port.

One final note: on the main menu selecting pfTop (option 9 as of this writing) will provide you live data about the hosts connected, this can be useful when trying to find the ip address of a device you are trying to connect to.

Have fun!