Building Linux kernel

https://github.com/MiSTer-devel/Main_MiSTer/wiki

Moderators: Mug UK, Zorro 2, spiny, Greenious, Sorgelig, Moderator Team

Sorgelig
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 2906
Joined: Mon Dec 14, 2015 10:51 am
Location: Russia/Taiwan

Re: Building Linux kernel

Postby Sorgelig » Thu Jan 11, 2018 11:17 pm

I went by other way which should be faster and less resource consuming than initrd/initramfs

kolla
Captain Atari
Captain Atari
Posts: 163
Joined: Thu Sep 17, 2015 11:39 pm
Contact:

Re: Building Linux kernel

Postby kolla » Thu Jan 11, 2018 11:30 pm

Sorgelig wrote:I went by other way which should be faster and less resource consuming than initrd/initramfs


OK - still may be useful.
With a filesystem small enough, entire linux filesystem could be the initrd, built into the kernel image :)

Anyways, found my way around uboot, added "loop.max_part=8" to boot command line, and now I can mount partitions from MBR and RDB image files:

Code: Select all

# cd /media/fat/Amiga
# losetup -f mister.hdf
# cat /proc/partitions
major minor  #blocks  name

   1        0       8192 ram0
   1        1       8192 ram1
   7        0     655360 loop0
   7        1     104580 loop0p1
   7        2     218106 loop0p2
   7        3     200970 loop0p3
   7        4     131418 loop0p4
 179        0    7716864 mmcblk0
 179        1    7202816 mmcblk0p1
 179        2     512000 mmcblk0p2
 179        3       1023 mmcblk0p3
#
# mount -t affs /dev/loop0p4 /mnt/amiga/dh3
# df -T /mnt/amiga/dh3
Filesystem     Type 1K-blocks   Used Available Use% Mounted on
/dev/loop0p4   affs    131417 116626     14792  89% /mnt/amiga/dh3


Yay :)

Next mission is adding some usb wifi support and add wpa_supplicant to userland.
-- kolla

Sorgelig
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 2906
Joined: Mon Dec 14, 2015 10:51 am
Location: Russia/Taiwan

Re: Building Linux kernel

Postby Sorgelig » Fri Jan 12, 2018 12:06 am

kolla wrote:Anyways, found my way around uboot, added "loop.max_part=8" to boot command line, and now I can mount partitions from MBR and RDB image files:

do you mean you couldn't mount partitions on loop devices before?

kolla wrote:With a filesystem small enough, entire linux filesystem could be the initrd, built into the kernel image

it's easy in theory, but not so easy in practice, especially if you use u-boot which has slightly different view on "right" kernel image.

kolla wrote:Next mission is adding some usb wifi support and add wpa_supplicant to userland.

don't forget that Linux part is supposed to be a read-only the whole time. Unless you use a special key combination or command line command to reboot the MiSTer, every reboot and poweroff is unsafe. You can easily corrupt your linux partition if it will be required to be mounted as read-write.

kolla
Captain Atari
Captain Atari
Posts: 163
Joined: Thu Sep 17, 2015 11:39 pm
Contact:

Re: Building Linux kernel

Postby kolla » Fri Jan 12, 2018 12:20 am

Sorgelig wrote:
kolla wrote:Anyways, found my way around uboot, added "loop.max_part=8" to boot command line, and now I can mount partitions from MBR and RDB image files:

do you mean you couldn't mount partitions on loop devices before?


Correct - could only mount "filesystem" images.

kolla wrote:With a filesystem small enough, entire linux filesystem could be the initrd, built into the kernel image

it's easy in theory, but not so easy in practice, especially if you use u-boot which has slightly different view on "right" kernel image.

Yes, I can imagine - one can always cheat using a "dummy kernel" that just kexec to the actual kernel, but it quickly gets a little complicated. :)

kolla wrote:Next mission is adding some usb wifi support and add wpa_supplicant to userland.

don't forget that Linux part is supposed to be a read-only the whole time. Unless you use a special key combination or command line command to reboot the MiSTer, every reboot and poweroff is unsafe. You can easily corrupt your linux partition if it will be required to be mounted as read-write.


Yes, I am aware. My idea is just add wpa_supplicant, and configure it to use a wpa_supplicant.conf located on the fat partition.
-- kolla

kolla
Captain Atari
Captain Atari
Posts: 163
Joined: Thu Sep 17, 2015 11:39 pm
Contact:

Re: Building Linux kernel

Postby kolla » Fri Jan 12, 2018 12:46 am

OK, I could need a little hint on how one build userland, rootfs and the stuff in addon.tar - is this based on the de10-nano distro that came with the box?
-- kolla

Sorgelig
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 2906
Joined: Mon Dec 14, 2015 10:51 am
Location: Russia/Taiwan

Re: Building Linux kernel

Postby Sorgelig » Fri Jan 12, 2018 12:52 am

kolla wrote:Correct - could only mount "filesystem" images.

it seems there is no kernel config option for this. I will add this option to command line then.

kolla wrote:OK, I could need a little hint on how one build userland, rootfs and the stuff in addon.tar - is this based on the de10-nano distro that came with the box?

it's based on buildroot with my manual/scripting modifications.

kolla
Captain Atari
Captain Atari
Posts: 163
Joined: Thu Sep 17, 2015 11:39 pm
Contact:

Re: Building Linux kernel

Postby kolla » Sat Jan 13, 2018 12:26 am

Sorgelig wrote:it's based on buildroot with my manual/scripting modifications.


Would be nice to see how you do it...

Anyways, I cheated - I copied over binaries from a raspberry pi, wpa_supplicant with friends, a few libs and config, and presto - the mister is now on wifi! :D

Would be nice to do it "properly" though. :)

I put wpa_supplicant.conf in /media/fat/config, though I suspect that dir is intended solely for soft cores, but it just seems "messy" to have it on /media/fat :p
-- kolla

Sorgelig
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 2906
Joined: Mon Dec 14, 2015 10:51 am
Location: Russia/Taiwan

Re: Building Linux kernel

Postby Sorgelig » Sat Jan 13, 2018 1:44 am

give me files you've used from RPi. I will see what can i do. May be i will be able to compile same files from buildroot.
In upcoming linux version, there will be new layout where linux folder on fat partition will be used to hold linux image and kernel. Probably same folder can be used for linux related configs.

kolla
Captain Atari
Captain Atari
Posts: 163
Joined: Thu Sep 17, 2015 11:39 pm
Contact:

Re: Building Linux kernel

Postby kolla » Sat Jan 13, 2018 2:11 am

Sorgelig wrote:give me files you've used from RPi. I will see what can i do. May be i will be able to compile same files from buildroot.
In upcoming linux version, there will be new layout where linux folder on fat partition will be used to hold linux image and kernel. Probably same folder can be used for linux related configs.


That would be cool - you have private message with link etc :)
-- kolla

Sorgelig
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 2906
Joined: Mon Dec 14, 2015 10:51 am
Location: Russia/Taiwan

Re: Building Linux kernel

Postby Sorgelig » Mon Jan 15, 2018 3:59 am

New Linux release has WiFi support.
Put your SSID/Pass to _wpa_supplicant.conf and rename it to wpa_supplicant.conf

kolla
Captain Atari
Captain Atari
Posts: 163
Joined: Thu Sep 17, 2015 11:39 pm
Contact:

Re: Building Linux kernel

Postby kolla » Mon Jan 15, 2018 9:43 am

Awesome, looking forward to try it out tonight!
-- kolla

kolla
Captain Atari
Captain Atari
Posts: 163
Joined: Thu Sep 17, 2015 11:39 pm
Contact:

Re: Building Linux kernel

Postby kolla » Mon Jan 15, 2018 11:06 pm

Hehe, by luck the exact kernel modules that I need for my USB wifi sticks were not enabled :)

For sake of (my) convenience, maybe you could add for next release...

Code: Select all

-# CONFIG_RT2800USB_RT53XX is not set
-# CONFIG_RT2800USB_RT55XX is not set
+CONFIG_RT2800USB_RT53XX=y
+CONFIG_RT2800USB_RT55XX=y


So, I built my own kernel, but then realized that I need a zImage_dtb and not just a zImage - I have looked at various online resources, but not really found a how-to for going from having a zImage to having a zImage_dtb, seems to vary a bit from device to device?

Thanks again :)
-- kolla

Sorgelig
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 2906
Joined: Mon Dec 14, 2015 10:51 am
Location: Russia/Taiwan

Re: Building Linux kernel

Postby Sorgelig » Tue Jan 16, 2018 12:25 am

zImage_dtb is a simple concatenation of zImage and socfpga.dtb

I've updated the kernel. Get it from here: https://github.com/MiSTer-devel/Linux_I ... tor_MiSTer

kolla
Captain Atari
Captain Atari
Posts: 163
Joined: Thu Sep 17, 2015 11:39 pm
Contact:

Re: Building Linux kernel

Postby kolla » Tue Jan 16, 2018 2:10 am

Brilliant! I needed a piece of firmware too (rt2870.bin - forgot to mention it), just put it in the firmware.tar.gz file, ran the create_img.sh, copied over the resulting image, and presto - the mister is online with wifi symbol in menu and all :)

So, with my own kernel I can just do a

Code: Select all

cat zImage socfpga.dtb > zImage_dtb

or vice versa? :)

Thanks a lot!
-- kolla

Sorgelig
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 2906
Joined: Mon Dec 14, 2015 10:51 am
Location: Russia/Taiwan

Re: Building Linux kernel

Postby Sorgelig » Tue Jan 16, 2018 2:20 am

kolla wrote:I needed a piece of firmware too (rt2870.bin - forgot to mention it), just put it in the firmware.tar.gz file, ran the create_img.sh, copied over the resulting image, and presto - the mister is online with wifi symbol in menu and all :)

Kernel source doesn't include firmwares for all devices due to license. Thus some wifi dongles will require additional steps to make them work. My ASUS dongle with realtek chip works without any additions.

kolla wrote:So, with my own kernel I can just do a
cat zImage socfpga.dtb > zImage_dtb


right.

Doozer
Atariator
Atariator
Posts: 21
Joined: Fri Apr 22, 2016 12:26 pm

Re: Major update for MiSTer

Postby Doozer » Fri Mar 09, 2018 8:59 am

Sorgelig wrote:I update the sources regularly. You can see the change log in kernel repository.


Hi Sorgelig,

I need a clarification to the process to get a kernel able to launch MiSTer. I am stuck at bad init sequence because no mtd is discovered when the kernel is launched.

I can compile the kernel (with duplicate config from your 4.15 running kernel) without issue. After the compilation, I am adding the dtd at the end and I am able to boot on the new kernel. Nevertheless the system refuses to get the MiSTer binary from the root filesystem. The rootfs is normally passed to the kernel by uboot but it seems that the new kernel does not see it. You help would be greatly appreciated.

Code: Select all

reading /linux/zImage_dtb
5627599 bytes read in 556 ms (9.7 MiB/s)
## Flattened Device Tree blob at 01556670
   Booting using the fdt blob at 0x1556670
   reserving fdt memory region: addr=0 size=1000
   Loading Device Tree to 03ff5000, end 03fff85e ... OK

Starting kernel ...

[    0.000000] L2C: platform provided aux values permit register corruption.
[    0.689483] rtc-m41t80 1-0068: read error
[    0.693484] rtc-m41t80 1-0068: hctosys: unable to read the hardware clock
[    0.845211] Kernel panic - not syncing: No working init found.  Try passing init= option to kernel. See Linux Documentation/init.txt for guidance.
[    0.858287] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 4.5.0+ #0
[    0.864180] Hardware name: Altera SOCFPGA
[    0.868208] [<c0018af8>] (unwind_backtrace) from [<c0014494>] (show_stack+0x20/0x24)
[    0.875929] [<c0014494>] (show_stack) from [<c02a1b5c>] (dump_stack+0x90/0xa4)
[    0.883133] [<c02a1b5c>] (dump_stack) from [<c00e3f58>] (panic+0xe8/0x23c)
[    0.889999] [<c00e3f58>] (panic) from [<c065d974>] (__irq_alloc_descs+0x0/0x1e4)
[    0.897367] CPU1: stopping
[    0.900069] CPU: 1 PID: 0 Comm: swapper/1 Not tainted 4.5.0+ #0
[    0.905963] Hardware name: Altera SOCFPGA
[    0.909971] [<c0018af8>] (unwind_backtrace) from [<c0014494>] (show_stack+0x20/0x24)
[    0.917687] [<c0014494>] (show_stack) from [<c02a1b5c>] (dump_stack+0x90/0xa4)
[    0.924884] [<c02a1b5c>] (dump_stack) from [<c0017124>] (handle_IPI+0x2d8/0x2f4)
[    0.932252] [<c0017124>] (handle_IPI) from [<c0009528>] (gic_handle_irq+0x94/0x98)
[    0.939792] [<c0009528>] (gic_handle_irq) from [<c0015014>] (__irq_svc+0x54/0x70)
[    0.947241] Exception stack(0xdf50bf48 to 0xdf50bf90)
[    0.952273] bf40:                   00000001 00000000 00000000 c00217c0 ffffe000 c097c4a0
[    0.960418] bf60: 00000000 c09df034 c097c454 00000000 c097c4a8 df50bfa4 df50bfa8 df50bf98
[    0.968558] bf80: c00107e4 c00107e8 60000013 ffffffff
[    0.973594] [<c0015014>] (__irq_svc) from [<c00107e8>] (arch_cpu_idle+0x48/0x4c)
[    0.980972] [<c00107e8>] (arch_cpu_idle) from [<c00651e4>] (default_idle_call+0x30/0x3c)
[    0.989034] [<c00651e4>] (default_idle_call) from [<c0065440>] (cpu_startup_entry+0x250/0x358)
[    0.997613] [<c0065440>] (cpu_startup_entry) from [<c0016bd0>] (secondary_start_kernel+0x16c/0x174)
[    1.006622] [<c0016bd0>] (secondary_start_kernel) from [<000095cc>] (0x95cc)
[    1.013645] ---[ end Kernel panic - not syncing: No working init found.  Try passing init= option to kernel. See Linux Documentation/init.txt for guidance.


Please move the discussion to a separate thread if necessary.

Sorgelig
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 2906
Joined: Mon Dec 14, 2015 10:51 am
Location: Russia/Taiwan

Re: Major update for MiSTer

Postby Sorgelig » Fri Mar 09, 2018 9:13 am

1) make sure you've update uboot,kernel and linux image with latest version using SD installer.
2) you may enable more debug messages by removing "v" variable in uboot environment
3) you need to build the kernel from my repository as it has important patches to be able to boot from loop device.

Doozer wrote:with duplicate config from your 4.15 running kernel

probably you've mistyped it but correct kernel version if 4.5.x

From your message it's unclear where is the problem.

Doozer
Atariator
Atariator
Posts: 21
Joined: Fri Apr 22, 2016 12:26 pm

Re: Major update for MiSTer

Postby Doozer » Fri Mar 09, 2018 9:51 am

Yes, I am speaking about kernel 4.5 from your repository. The issue was related to the .config file not being the latest one. I was using de10-nano_defconfig from this forum. Now I use an extract from your kernel dated Feb 15 (in attachment).

Here are the steps I use for the compilation and dtb concatenation.

Code: Select all

make ARCH=arm CROSS_COMPILE=/usr/bin/arm-linux-gnueabi- oldconfig
make ARCH=arm CROSS_COMPILE=/usr/bin/arm-linux-gnueabi- -k zImage dtbs
cat arch/arm/boot/zImage arch/arm/boot/dts/socfpga_cyclone5_de10_sockit.dtb > zImage_dtb


The kernel is now booting and launching MiSTer properly. Thanks.
You do not have the required permissions to view the files attached to this post.

Sorgelig
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 2906
Joined: Mon Dec 14, 2015 10:51 am
Location: Russia/Taiwan

Re: Building Linux kernel

Postby Sorgelig » Fri Mar 09, 2018 12:22 pm

You don't need extract anything. Current config is always in repository: https://github.com/MiSTer-devel/Linux-K ... _defconfig

Locutus73
Captain Atari
Captain Atari
Posts: 170
Joined: Wed Feb 07, 2018 6:13 pm

Re: Building Linux kernel

Postby Locutus73 » Sat Mar 17, 2018 12:13 am

This evening I compiled the Kernel for the first time (it was the first time I crosscompiled anything too), so I want to share my recipe.
I started from a brand new minimal Debian 9.4.0 amd64, just the basic system, no gui, nothing, just the terminal (and the ssh server for convenience), so here's the steps I followed.

Code: Select all

cd ~

sudo apt-get install build-essential
sudo apt-get install bc
sudo apt-get install liblz4-tool
sudo apt-get install device-tree-compiler
sudo apt-get install git
sudo apt-get install libncurses5-dev
sudo apt-get install libncursesw5-dev

wget https://releases.linaro.org/components/toolchain/binaries/7.2-2017.11/arm-linux-gnueabihf/gcc-linaro-7.2.1-2017.11-x86_64_arm-linux-gnueabihf.tar.xz
tar -xvf gcc-linaro-7.2.1-2017.11-x86_64_arm-linux-gnueabihf.tar.xz

git clone -b socfpga-4.5 https://github.com/MiSTer-devel/Linux-Kernel_4.5.0_MiSTer

export ARCH=arm
export CROSS_COMPILE=$PWD/gcc-linaro-7.2.1-2017.11-x86_64_arm-linux-gnueabihf/bin/arm-linux-gnueabihf-

cd Linux-Kernel_4.5.0_MiSTer

make clean
make mrproper
make MiSTer_defconfig
make zImage modules dtbs

cd ~
cat Linux-Kernel_4.5.0_MiSTer/arch/arm/boot/zImage Linux-Kernel_4.5.0_MiSTer/arch/arm/boot/dts/socfpga_cyclone5_de10_sockit.dtb > zImage_dtb


In order to compile the last xpad driver, just before the make steps, I performed

Code: Select all

cd ~
cd Linux-Kernel_4.5.0_MiSTer/drivers/input/joystick
mv xpad.c xpad.c.old
wget https://raw.githubusercontent.com/paroj/xpad/master/xpad.c
cd ~

But I think Sorgelig is going to merge the last driver on github very soon.

[EDIT]
added
sudo apt-get install libncurses5-dev
sudo apt-get install libncursesw5-dev
in order to make make menuconfig work

Locutus73

kolla
Captain Atari
Captain Atari
Posts: 163
Joined: Thu Sep 17, 2015 11:39 pm
Contact:

Re: Building Linux kernel

Postby kolla » Sun May 13, 2018 4:40 pm

Looks like the socfpga_cyclone5_de10_sockit.dtb is now socfpga_cyclone5_de10_nano.dtb? At least that is what I use.

Anyways, since I am doing small changes to the kernel config and need to recompile it every now and then (and because I tend to use macs as build systems these days), I just as well made a docker container for building the MiSTer kernel. It is quite rudimentary and "hardcoded", but it works, for now... https://github.com/kowoba/Linux-Kernel_MiSTer_build :)
-- kolla

kolla
Captain Atari
Captain Atari
Posts: 163
Joined: Thu Sep 17, 2015 11:39 pm
Contact:

Re: Building Linux kernel

Postby kolla » Wed May 23, 2018 10:06 am

https://github.com/kowoba/Linux-Kernel_MiSTer_build

I don't think anyone else has used my docker yet, but nevertheless, I have fixed on it a bit, and updated the README accordingly.

Next project is to make a similar docker for building custom user-land images (I have needs and wishes that are outside the scope of the default build) :)
-- kolla

alfishe
Atari freak
Atari freak
Posts: 64
Joined: Wed Jul 19, 2017 8:24 pm

Re: Building Linux kernel

Postby alfishe » Fri May 25, 2018 3:15 am

kolla wrote:https://github.com/kowoba/Linux-Kernel_MiSTer_build


Thank you for you efforts. Basically it works. Probably few things to polish:
1) Exclude kernel compilation into Docker image build phase
2) Put several shell scripts to have compilation in different modes:
- With default MiSTer kernel config (no interactive prompts)
- Load MiSTer config but allow to customize before compilation
3) Publish to dockerhub

kolla
Captain Atari
Captain Atari
Posts: 163
Joined: Thu Sep 17, 2015 11:39 pm
Contact:

Re: Building Linux kernel

Postby kolla » Wed May 30, 2018 8:37 pm

alfishe wrote:
kolla wrote:https://github.com/kowoba/Linux-Kernel_MiSTer_build


Thank you for you efforts. Basically it works. Probably few things to polish:
1) Exclude kernel compilation into Docker image build phase


And build the entire kernel everything from scratch every time, for minor kernel changes? Kinda what I want to avoid :)

2) Put several shell scripts to have compilation in different modes:
- With default MiSTer kernel config (no interactive prompts)
- Load MiSTer config but allow to customize before compilation


I have no need for for this - if I want default config, it is already there, I just exit the menu, and since it is already prebuilt, it will be done quickly.
Custom config, if present, is already loaded before presenting the menu config.

As I mentioned in the readme, the purpose of this docker, for me, is not to build default kernels, but to easily build custom kernels with the features I want :)

3) Publish to dockerhub
[/quote][/quote]

Thanks for feedback - feel free to fork, adjust and carry on with this :)
-- kolla


Return to “MiSTer”

Who is online

Users browsing this forum: No registered users and 11 guests