Building Linux kernel

https://github.com/MiSTer-devel/Main_MiSTer/wiki

Moderators: Mug UK, Zorro 2, spiny, Greenious, Sorgelig, Moderator Team

bernouilli
Obsessive compulsive Atari behavior
Obsessive compulsive Atari behavior
Posts: 100
Joined: Fri Jan 06, 2017 5:02 pm
Location: Paris/France

Building Linux kernel

Postby bernouilli » Tue Dec 12, 2017 1:51 pm

I'm trying to compile linux kernel without success.
The kernel just doesn't boot.

Here's what I've done :

I cloned the folowing github :
https://github.com/MiSTer-devel/Linux-K ... 5.0_MiSTer
I also tried by downloading the zip file which by the way is named Linux-Kernel_4.5.0_MiSTer-socfpga-4.5.zip but I suppose the content is the same.
I've installed the linaro arm toolchain (arm-linux-gnueabihf).
I also had to install lz4c and dtc :

Code: Select all

sudo apt install liblz4-util
sudp apt install device-tree-compiler

Then I enter the Linux-Kernel_4.5.0_MiSTer directory and do :

Code: Select all

export ARCH=arm
export CROSS_COMPILE=arm-linux-gnueabihf-
make mrproper
make de10-nano_minimal_defconfig
make zImage

I starts compiling, which takes about 10-15 minutes and produces the zImage (in arch/arm/boot).
Then I copy the zImage to the /boot directory of the second partition on the sdcard, replacing the zImage that was there.
When I boot the board, I get the uboot-spl and u-boot traces and it stops after loading the zImage.
I kept the original dtb file (socfpga.dtb) but when I compile a new dtb (make socfpga_cyclone5_de10_sockit.dtb and rename the socfpga_cyclone5_de10_sockit.dtb to socfpga.dtb and put it in /boot), ther issue is the same.

Has anyone tried to compile the kernel ? What am I missing ?

I checked that the zImage is really an arm elf compressed executable with file, just in case it was producing a x86 elf executable.

With the same environment, I managed to compile the main MiSTer and the u-boot and they work.
Last edited by bernouilli on Sun Dec 17, 2017 12:34 am, edited 2 times in total.

ex68k
Atari freak
Atari freak
Posts: 53
Joined: Sat Oct 26, 2013 11:13 am

Re: Building Linux kernel

Postby ex68k » Fri Dec 15, 2017 2:32 pm

bernouilli wrote:I'm trying to compile linux kernel without success.

Code: Select all

sudo apt install liblz4-util
sudp apt install device-tree-compiler

Then I enter the Linux-Kernel_4.5.0_MiSTer directory and do :

Code: Select all

export ARCH=arm
export CROSS_COMPILE=arm-linux-gnueabihf-
make mrproper
make de10-nano_minimal_defconfig
make zImage



I actually tried to set up a system here too.
Made a clean install of xubuntu-16.04.2-desktop-amd64,

installed the "apt install device-tree-compiler liblz4-tool" (you had a typo here)

And ended up with a non-working kernel like you ...

Sorgelig
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 2358
Joined: Mon Dec 14, 2015 10:51 am
Location: Russia/Taiwan

Re: Building Linux kernel

Postby Sorgelig » Fri Dec 15, 2017 3:51 pm

Make sure you're using branch socfpga-4.5
I've pushed commits i've made long time ago but forgot to push. Although they shouldn't affect the booting process.

I'm using arm-linux-gnueabihf-gcc-6.2.1 to build the kernel.

bernouilli
Obsessive compulsive Atari behavior
Obsessive compulsive Atari behavior
Posts: 100
Joined: Fri Jan 06, 2017 5:02 pm
Location: Paris/France

Re: Building Linux kernel

Postby bernouilli » Sat Dec 16, 2017 6:44 pm

I tried the de10_nano_defconfig and managed to build a working kernel.
Then I started to look at the differences between the two configs.
de10_nano_defconfig contains :

Code: Select all

CONFIG_KERNEL_LZO=y
# CONFIG_KERNEL_LZ4 is not set


I first tried setting CONFIG_KERNEL_LZO instead of CONFIG_KERNEL_LZ4 but this didn't work.

I continued looking and I found the issue.

I commented the two lines in de10_nano_minimal_defconfig :

Code: Select all

CONFIG_THUMB2_KERNEL=y
CONFIG_THUMB2_AVOID_R_ARM_THM_JUMP11=y


And now the kernel boots. The zImage is however a bit bigger (4686552 bytes).

I didn't try with gcc 6.2, maybe gcc 5.4 has a problem with thumb2.

bernouilli
Obsessive compulsive Atari behavior
Obsessive compulsive Atari behavior
Posts: 100
Joined: Fri Jan 06, 2017 5:02 pm
Location: Paris/France

Re: Building Linux kernel

Postby bernouilli » Sun Dec 17, 2017 10:45 am

I tried with linaro gcc 7.2.1 (latest) and with the original defconfig and the kernel boots.
So thumb2 option is a problem for gcc 5.4

However, the compiled kernel with gcc5.4 and no thumb2 option does bring up the two cpus whereas with the original (the official that is on github mister sd installer) and the one I compiled with gcc7.2.1, the second cpu fails to come up.

Maybe it's related to the thumb2 option.

And I noticed that the heat produced is higher with the official kernel than with the one I compiled with gcc5.4 and no thumb2 options. I doubt that it's activating the second cpu that makes less heat.

I'll try a kernel with only 1 cpu activated to see. Meanwhile I'll do some thermal testings.

Sorgelig
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 2358
Joined: Mon Dec 14, 2015 10:51 am
Location: Russia/Taiwan

Re: Building Linux kernel

Postby Sorgelig » Sun Dec 17, 2017 11:40 am

Using only 1 core is intended option. If your kernel uses 2 cores, then it means it doesn't understand option string passed to kernel upon booting.

1 core is more than enough for current MiSTer version. Second core is reserved for special features like hybrid emulation, currently is not used.

bernouilli
Obsessive compulsive Atari behavior
Obsessive compulsive Atari behavior
Posts: 100
Joined: Fri Jan 06, 2017 5:02 pm
Location: Paris/France

Re: Building Linux kernel

Postby bernouilli » Sun Dec 17, 2017 8:59 pm

I'm sure this is how it is intended to work but it's not actually how it works.

The boot arguments sent to kernel are in the file include/configs/socfpga_de10_nano.h from the u-boot_MiSTer project :

Code: Select all

#define CONFIG_BOOTARGS      "console=ttyS0," __stringify(CONFIG_BAUDRATE) " $v mem=511M memmap=513M$511M"

The parameter maxcpus=1 is missing.

You can also check it on the running linux with the command :

Code: Select all

# cat /proc/cmdline
console=ttyS0,115200 loglevel=4 mem=511M memmap=513M$511M root=/dev/mmcblk0p2 ro rootwait


The second cpu doesn't come up (for some reason maybe related to the THUMB2 option), you can see it in the /tmp/message file :

Code: Select all

Jan  1 00:00:02 de10-nano user.info kernel: [    0.000939] CPU: Testing write buffer coherency: ok
Jan  1 00:00:02 de10-nano user.info kernel: [    0.000965] ftrace: allocating 19285 entries in 38 pages
Jan  1 00:00:02 de10-nano user.info kernel: [    0.030842] CPU0: thread -1, cpu 0, socket 0, mpidr 80000000
Jan  1 00:00:02 de10-nano user.info kernel: [    0.031058] Setting up static identity map for 0x8240 - 0x828c
Jan  1 00:00:02 de10-nano user.crit kernel: [    1.029735] CPU1: failed to come online
Jan  1 00:00:02 de10-nano user.info kernel: [    1.029764] Brought up 1 CPUs
Jan  1 00:00:02 de10-nano user.info kernel: [    1.029773] SMP: Total of 1 processors activated (200.00 BogoMIPS).
Jan  1 00:00:02 de10-nano user.info kernel: [    1.029779] CPU: All CPU(s) started in SVC mode.


With the kernel compiled wthout the THUMB2 option, this is what I get in /tmp/message regarding the cpus with the official u-boot :

Code: Select all

Jan  1 00:00:02 de10-nano user.info kernel: [    0.002969] CPU: Testing write buffer coherency: ok
Jan  1 00:00:02 de10-nano user.info kernel: [    0.003071] ftrace: allocating 19294 entries in 57 pages
Jan  1 00:00:02 de10-nano user.info kernel: [    0.134976] CPU0: thread -1, cpu 0, socket 0, mpidr 80000000
Jan  1 00:00:02 de10-nano user.info kernel: [    0.135599] Setting up static identity map for 0x8280 - 0x82d8
Jan  1 00:00:02 de10-nano user.info kernel: [    0.140146] CPU1: thread -1, cpu 1, socket 0, mpidr 80000001
Jan  1 00:00:02 de10-nano user.info kernel: [    0.140385] Brought up 2 CPUs
Jan  1 00:00:02 de10-nano user.info kernel: [    0.140441] SMP: Total of 2 processors activated (400.00 BogoMIPS).
Jan  1 00:00:02 de10-nano user.info kernel: [    0.140466] CPU: All CPU(s) started in SVC mode.


and with bootargs containing maxcpus=1 :

Code: Select all

Jan  1 01:00:02 de10-nano user.info kernel: [    0.003652] CPU: Testing write buffer coherency: ok
Jan  1 01:00:02 de10-nano user.info kernel: [    0.003752] ftrace: allocating 19315 entries in 57 pages
Jan  1 01:00:02 de10-nano user.info kernel: [    0.133615] CPU0: thread -1, cpu 0, socket 0, mpidr 80000000
Jan  1 01:00:02 de10-nano user.info kernel: [    0.134230] Setting up static identity map for 0x8280 - 0x82d8
Jan  1 01:00:02 de10-nano user.info kernel: [    0.137180] Brought up 1 CPUs
Jan  1 01:00:02 de10-nano user.info kernel: [    0.137225] SMP: Total of 1 processors activated (200.00 BogoMIPS).
Jan  1 01:00:02 de10-nano user.info kernel: [    0.137253] CPU: All CPU(s) started in SVC mode.
Last edited by bernouilli on Sun Dec 17, 2017 9:16 pm, edited 1 time in total.

Sorgelig
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 2358
Joined: Mon Dec 14, 2015 10:51 am
Location: Russia/Taiwan

Re: Building Linux kernel

Postby Sorgelig » Sun Dec 17, 2017 9:14 pm

Actually i've forgot how i've limited the number of CPU :)
May be through kernel config options.

ex68k
Atari freak
Atari freak
Posts: 53
Joined: Sat Oct 26, 2013 11:13 am

Re: Building Linux kernel

Postby ex68k » Wed Dec 20, 2017 7:49 pm

bernouilli wrote:I tried with linaro gcc 7.2.1 (latest) and with the original defconfig and the kernel boots.
So thumb2 option is a problem for gcc 5.4


Yup, works for me too ...

Sorgelig
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 2358
Joined: Mon Dec 14, 2015 10:51 am
Location: Russia/Taiwan

Re: Building Linux kernel

Postby Sorgelig » Tue Dec 26, 2017 11:55 am

By the way, i would like to simplify the SD card structure.

Probably i cannot avoid boot partition and it should remain type 0xA2, but at least it would be good to eliminate Linux partition and move it into image file laying on exFAT partition.
It will allow to update Linux/Kernel more easy by just copying the file.

Would be good if someone would help to create such layout. It's pure linux and traditional programming part. Thus anyone is welcome.

kolla
Captain Atari
Captain Atari
Posts: 163
Joined: Thu Sep 17, 2015 11:39 pm
Contact:

Re: Building Linux kernel

Postby kolla » Wed Jan 10, 2018 12:33 am

A simple request, I hope - could you add max_part=8 (or other suitable number) to boot command for the loop kernel module, so that partitions on disk images can easily be mounted via loop? It looks like cmdline is hard coded into uboot sfp or/or the kernel, as I don't find it in any text file.

For example, if you create a disk image with MBR partitions (or RDB, if kernel supports that), and use "losetup -f disk.img", partitions will show up as /dev/loop0p1, /dev/loop0p2 etc. and can then easily be to mounted.

Another request I want to ask about, is to include wpa_supplicant on the rootfs, and at have the kernel support at least a small handfull of usb wifi sticks :)
-- kolla

kolla
Captain Atari
Captain Atari
Posts: 163
Joined: Thu Sep 17, 2015 11:39 pm
Contact:

Re: Building Linux kernel

Postby kolla » Wed Jan 10, 2018 12:48 am

Sorgelig wrote:Would be good if someone would help to create such layout. It's pure linux and traditional programming part. Thus anyone is welcome.


I can do this, I agree it would simplify updating the Linux side of the system, you could then even have a backup of the previous kernels and filesystems.

Normally this would be done using a small initramfs, where the image containing the linux filesystem is mounted as new root before exec to init there. Not sure if any initamfs is used when booting the MiSTer? I suppose uboot has no trouble reading kernel and initrd from FAT32? I guess the answers are in the uboot docs :)
-- kolla

alfishe
Atari freak
Atari freak
Posts: 63
Joined: Wed Jul 19, 2017 8:24 pm

Re: Building Linux kernel

Postby alfishe » Wed Jan 10, 2018 12:58 am

kolla wrote:I suppose uboot has no trouble reading kernel and initrd from FAT32? I guess the answers are in the uboot docs :)


U-Boot can read from most of widespread filesystems. But Linux image stored on FAT partition has higher chances to be corrupted than sitting hidden in dedicated partition.
I'm personally against such "improvement". Engineering principle - works? don't touch! =)

Sorgelig
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 2358
Joined: Mon Dec 14, 2015 10:51 am
Location: Russia/Taiwan

Re: Building Linux kernel

Postby Sorgelig » Wed Jan 10, 2018 2:36 am

kernel boot options is set by U-boot. U-boot uses its environment variables to construct the command line for kernel. You can modify it from U-boot. You can stop boot in u-boot by holding ESC key in console while booting.

bernouilli
Obsessive compulsive Atari behavior
Obsessive compulsive Atari behavior
Posts: 100
Joined: Fri Jan 06, 2017 5:02 pm
Location: Paris/France

Re: Building Linux kernel

Postby bernouilli » Wed Jan 10, 2018 9:17 am

I agree with alfishe, having the linux rootfs on a ext4 partition makes it hidden in windows and a separate partition on linux. This way the chances of removing files from the rootfs partition is limited. It's a good thing to leave it in a dedicated partition.
Plus how does a linux behave with a rootfs on a FAT32 partition ? I don't think file permissions can be used on FAT32.

Sorgelig
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 2358
Joined: Mon Dec 14, 2015 10:51 am
Location: Russia/Taiwan

Re: Building Linux kernel

Postby Sorgelig » Wed Jan 10, 2018 5:49 pm

1) it won't be FAT32 definitely. It will be exFAT.
2) nothing to do with FAT permissions as it will be a linux image, not the Linux FS over FAT.
3) corruption in linux image is not a problem at all. You can replace the image by new one if it gets corrupted. Linux image is always the same (till next Linux update) and not being altered on user device unless user altered it on purpose.
During the more than half year of extensive SD card usage in MiSTer, i never encountered a corrupted exFAT partition, although i haven't been careful. Device always powered off or reset suddenly. No single corruption.
It doesn't matter if linux is on its own partition or as a image on exFAT - Linux partition is normally mounted as read-only. Thus exFAT won't get additional source of corruption. It may become corrupted either way and Linux won't add any instability being an image on exFAT.

Sorgelig
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 2358
Joined: Mon Dec 14, 2015 10:51 am
Location: Russia/Taiwan

Re: Building Linux kernel

Postby Sorgelig » Wed Jan 10, 2018 5:54 pm

Having Linux image on exFAT is very convenient. You won't need any special tool for updating the linux. You even can update the linux on working MiSTer by just FTP connection! Then simply reboot and you are in new version already. So you won't need even to remove SD card.

Sorgelig
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 2358
Joined: Mon Dec 14, 2015 10:51 am
Location: Russia/Taiwan

Re: Building Linux kernel

Postby Sorgelig » Wed Jan 10, 2018 5:58 pm

I don't remember if U-boot supports loop devices itself or not. If yes, then i even can embed kernel and initramfs inside the linux image. U-boot will mount the linux image to read the kernel and initramfs from there and then pass control to kernel which will re-mount the linux image again.
Well.. i just need to understand how to accomplish all this..

bernouilli
Obsessive compulsive Atari behavior
Obsessive compulsive Atari behavior
Posts: 100
Joined: Fri Jan 06, 2017 5:02 pm
Location: Paris/France

Re: Building Linux kernel

Postby bernouilli » Wed Jan 10, 2018 7:45 pm

Ok. If I understand correctly the Linux rootfs which is now on its own partition will be replaced by an image that would be on that exFAT partition. Why not ? This way you can update the Linux part by just replacing the image on the exFAT partition.
A simple way to avoid corruption would be to mount the Linux partition as read only. Unless there are reasons not to do so.
I didn't understand your point previously.

Sorgelig
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 2358
Joined: Mon Dec 14, 2015 10:51 am
Location: Russia/Taiwan

Re: Building Linux kernel

Postby Sorgelig » Wed Jan 10, 2018 7:49 pm

bernouilli wrote:A simple way to avoid corruption would be to mount the Linux partition as read only. Unless there are reasons not to do so.

If you don't login to Linux then it's mounted as read-only. It's already done from first release.
When linux image will be on exFAT it will be possible to corrupt it even if it's mounted as read-only because exFAT is mounted as read/write and can be corrupted as well. But as i mentioned, Linux is not the crucial part of MiSTer and can be recovered by simple replacing the corrupted image.
If Linux will be as an image then i won't need to keep it with 512MB size.

Sorgelig
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 2358
Joined: Mon Dec 14, 2015 10:51 am
Location: Russia/Taiwan

Re: Building Linux kernel

Postby Sorgelig » Thu Jan 11, 2018 3:21 am

I need help!
i want to load initramfs or initrd but no matter what i try kernel doesn't recognize initrd...

actually i'm tired to do EVRYTHING by myself. My brain is boiling from so many tasks and many things i need to dig and understand.. Is there anyone is willing to help or everyone is prefer to sit and wait??

Or may be i will go to closed source - anyway it's useless if no one is willing to help...

User avatar
cvenditti
Atari maniac
Atari maniac
Posts: 83
Joined: Sun Nov 19, 2017 12:45 am

Re: Building Linux kernel

Postby cvenditti » Thu Jan 11, 2018 4:17 pm

Sorgelig wrote:I need help!
i want to load initramfs or initrd but no matter what i try kernel doesn't recognize initrd...

actually i'm tired to do EVRYTHING by myself. My brain is boiling from so many tasks and many things i need to dig and understand.. Is there anyone is willing to help or everyone is prefer to sit and wait??

Or may be i will go to closed source - anyway it's useless if no one is willing to help...


I can understand your frustration. Sorry I can help you with this one. But if you can make a list of outstanding task that you have in mind and if it is in my reach (I'm also willing to learn) I will take it.

kolla
Captain Atari
Captain Atari
Posts: 163
Joined: Thu Sep 17, 2015 11:39 pm
Contact:

Re: Building Linux kernel

Postby kolla » Thu Jan 11, 2018 7:23 pm

Sorgelig wrote:I need help!
i want to load initramfs or initrd but no matter what i try kernel doesn't recognize initrd.


That's sounds strange, have you tried building the initramfs into the kernel image itself? I want to help, I just need to get a working build environment, really thought I would have this done during xmas/newyear week, but instead I spent that time fighting Windows10 on mom's laptop. I will see how far I get tonight.

EDIT: I don't know what you have done to kernel config, but at least the one running on my MiSTer...

Code: Select all

Linux version 4.5.0-socfpga-r1 (user@kubuntu16) (gcc version 6.2.1 20161016 (Linaro GCC 6.2-2016.11) ) #4 SMP Tue Jan 2 19:16:37 CST 2018
does not support initrd at all (# CONFIG_BLK_DEV_INITRD is not set).

I am building my own kernels now, so I will look into getting initramfs working too. :)
Last edited by kolla on Thu Jan 11, 2018 9:44 pm, edited 3 times in total.
-- kolla

bernouilli
Obsessive compulsive Atari behavior
Obsessive compulsive Atari behavior
Posts: 100
Joined: Fri Jan 06, 2017 5:02 pm
Location: Paris/France

Re: Building Linux kernel

Postby bernouilli » Thu Jan 11, 2018 7:53 pm

How is the initrd file built? Is it an image of a fs or a cpio file?
I think that nowadays it's cpio format that is used (cpio archive compressed with gzip). I don't know if the format can be chooses when building the kernel.
And then, you pass it to the kernel using initrd= ?

I'm willing to help for this but I don't have much time. I can try to do that on Saturday.

kolla
Captain Atari
Captain Atari
Posts: 163
Joined: Thu Sep 17, 2015 11:39 pm
Contact:

Re: Building Linux kernel

Postby kolla » Thu Jan 11, 2018 9:43 pm

I now have a kernel with...

CONFIG_BLK_DEV_INITRD=y
CONFIG_INITRAMFS_SOURCE=""
# CONFIG_RD_GZIP is not set
# CONFIG_RD_BZIP2 is not set
# CONFIG_RD_LZMA is not set
# CONFIG_RD_XZ is not set
# CONFIG_RD_LZO is not set
CONFIG_RD_LZ4=y

which I _suppose_ should be able to boot with lz4 compressed cpio archived initrd. Now it is time to learn uboot again :p
-- kolla


Return to “MiSTer”

Who is online

Users browsing this forum: No registered users and 3 guests