EihiS

March 24, 2016

Real Time kernel patching and cross compilation

Filed under: Raspberry 3.14, linux — Tags: , , , , , , — admin @ 4:03 pm

This is a log of the recompilation of the latest available branch 4.1 for the raspberry pi, with the RT patch applied.

The target machine is a raspberry pi, with actual kernel : 4.1.13+ #826 PREEMPT.

I assume the  ‘raspberrypi_target’ directory exists. (see an old post about kernel cross compilation..)

So , you have to update the kernel sources before doing the patching.

to avoid errors while fetching the new sources, from the raspberrypi_target/linux directory, you can split the fetching of the new files like this:
for m in `seq 1 100`;do git fetch –depth=$m;done

once done,switch to the latest stable branch 'rpi-4.1.y' (at the time i write this post) :

git checkout -b rpi-4.1.y

Now its time to get the patch.

I assume you have a complete linux, branch 4.1.y … at the time i’m writing, the working patch for this version is located at :

https://www.kernel.org/pub/linux/kernel/projects/rt/4.1/patch-4.1.19-rt22.patch.gz

perform a save of your original,clean directory :

from the “raspberrypi_target”  dir :

cp -a linux linux_before_patching

so, let’s get the patch:

wget https://www.kernel.org/pub/linux/kernel/projects/rt/4.1/patch-4.1.19-rt22.patch.gz

..And check for any error (do it!) before patching (option –dry-run performs a ‘blank’ patching with no file modifications ) :

from the linux dir where the patch .gz is :

zcat patch-4.1.19-rt22.patch.gz | patch -p1 –dry-run | grep FAIL

if the command returns nothing, everything is OK, now effectively apply the patch :

zcat patch-4.1.19-rt22.patch.gz | patch -p1

okay, now you have to get some base config for the configuration menu.

The simple way : use the raspberry pi connected to your network, and issue a : sudo modprobe configs

then download the config.gz file from the raspberry to your cross-compiler’s machine :

scp <username>@<ip.ip.ip.ip>:/proc/config.gz ./

now you should have the config.gz file into the /linux/ folder where you issued this command (on the compiling machine )

now deflate it :

zcat config.gz > .config

And be a little curious ! (ie :   cat .config )

Okay, time to launch the menuconfig :

make ARCH=arm CROSS_COMPILE=/usr/bin/arm-linux-gnueabi- menuconfig

Now, the menu is on the screen. have to activate your options and customize, Then

make ARCH=arm CROSS_COMPILE=/usr/bin/arm-linux-gnueabihf- -k -j5

and voila(oups ..no ..), an error throws :

HOSTCC  scripts/genksyms/lex.lex.o
arch/arm/kernel/asm-offsets.c:54:2: error: #error Your compiler is too buggy; it is known to miscompile kernels

#error Your compiler is too buggy; it is known to miscompile kernels

arch/arm/kernel/asm-offsets.c:55:2: error: #error and result in filesystem corruption and oopses.
#error and result in filesystem corruption and oopses.
^
make[1]: *** [arch/arm/kernel/asm-offsets.s] Error 1
make[1]: Target `__build’ not remade because of errors.
make: *** [prepare0] Error 2
HOSTCC  scripts/dtc/data.o
HOSTCC  scripts/kallsyms

Damn…

Okay, lets get another cross compiler, form the github’s “raspberrypi/tools” dir :

cd to the home for your user name ( cd ~ )

then git clone https://github.com/raspberrypi/tools.git

… time to drink a cofee…

once done, the ‘tools’ folder is there.

okay, lets try rebuilding the menuconfig, this time using linaro gnueabi , from the tools.
the kernel modules will go to the ‘rtkern’ folder ..(create it : “mkdir rtkern” from your home folder )

make ARCH=arm CROSS_COMPILE=~/tools/arm-bcm2708/gcc-linaro-arm-linux-gnueabihf-raspbian/bin/arm-linux-gnueabihf- INSTALL_MOD_PATH=~/rtkern

Once again,errors, on my machine : missing some “libstdc++.so.6″ libraries, fix it ( we need the 6th version)

Okay, got it.. i used the 32bit version but i’m on a 64bit machine with ubuntu , so :

first, lets add the path, arch and destination for modules to the system :

export ARCH=arm
export INSTALL_MOD_PATH=~/rtkern
export CROSS_COMPILE=~/tools/arm-bcm2708/gcc-linaro-arm-linux-gnueabihf-raspbian-x64/bin/arm-linux-gnueabihf-

notice i specified the -x64 version of the crosscompiler.
now, from the ‘linux’ directory, try again the menuconfig, it should work fine now :

make menuconfig

>> it’s time to activate the options we want, in the menu. for the RealTime aspects, you’ll have to activate ‘KERNEL PREEMPT FULL’ , and also the HIgh-Resolution timers .

then after customisation and save+exit , in the ‘linux’ folder, execute :

make

make modules

make modules_install” <- the module are installed in the folder specified by the ‘INSTALL_MOD_PATH’ variable, obviously

now “cd ~/tools/mkimage

check out your built image , it should be listed when doing :

ls ~/raspberrypi_target/linux/arch/arm/boot -al

notice there is Image and zImage that have been created.

first method :

  • type "./imagetool-uncompressed.py ../../raspberrypi_target/linux/arch/arm/boot/Image"
  • the created boot image is in the current folder, named ‘kernel.img
  • now you have to replace the files into the target pi ,  from ~/raspberrypi_target/linux/arch/arm/boot/kernel.img into the /boot/ folder
    ( NOTICE : the /boot/ folder depends on the image you installed on your SD card. beware in case of a ‘noobs’ image’ , because in this case , the right ‘/boot’ folder is not at the regular place )
  • and rename/erase the target pi’s /lib/modules and /lib/firmware with the one freshly compiled,
Second method :
  • Use the mkknimg utility , located in the same folder. this one generates a kernel image from the zImage source, and the generated kernel.img file is configured to make use the of the device tree overlay (DTS) , then copy the created file to the /boot folder.
  • Remove / rename the target pi’s /boot/overlays folder by the one contained into ~/raspberrypi_target/linux/arch/arm/boot/dts/overlays

Note : more infos about the DTS , DTB files, syntax and examples here :

https://www.raspberrypi.org/documentation/configuration/device-tree.md

and for the tool to convert from .DTS to .DTB (bin) :

sudo apt-get install device-tree-compiler

.. the source of mkknlimg tool here :

https://github.com/raspberrypi/tools/blob/master/mkimage/mkknlimg

once done, do not forget to “sync” to ensure the data are written onto the SD.

then plug the fresh thing into the raspberry en enjoy.

uname -a gives a “Linux raspberrypi 4.1.20-rt22 #1 PREEMPT RT

314159265358979323846264338327950288
419716939937510582097494459230781640
628620899862803482534211706798214808

November 19, 2015

Change the raspberry pi swap file size without reboot

Filed under: Raspberry 3.14 — Tags: , , , , — admin @ 10:21 am
sudo nano /etc/dphys-swapfile

change the size ( default : CONF_SWAPSIZE=100 ) , is 100MB,to the desired one , then ctrl+O (save) , ctrl+X (exit nano editor)

now, stop the swap system, and restart :

/etc/init.d/dphys-swapfile stop

then :

/etc/init.d/dphys-swapfile start

314159265358979323846264338327950288
419716939937510582097494459230781640
628620899862803482534211706798214808

September 3, 2015

RPi VFPU : the denormal numbers terminator code

Filed under: Raspberry 3.14, linux — Tags: , , , , , , , , — admin @ 9:19 am
static void runfast(void) {
    uint32_t fpscr;
    __asm__ __volatile__ ("vmrs %0,fpscr" : "=r" (fpscr));
    fpscr |= 0x03000000;
    __asm__ __volatile__ ("vmsr fpscr,%0" : :"ri" (fpscr));
}

Raspberry Pi model B+ , impacts on all the program code at runtime, (and all #included libraries )

314159265358979323846264338327950288
419716939937510582097494459230781640
628620899862803482534211706798214808

July 3, 2015

rPI B+ : dispmanx resources , VC GPU, and mailboxes hacks

Filed under: Raspberry 3.14 — Tags: , , , , , , , , , , — admin @ 3:03 pm

This post is provided ‘as is’ etc.
Primary need : get access to GPU allocated memory blocks to be able to share between 2 different apps ( namely SDL2 / dispman_x resource based app / and openGLES2.0 based.

program VCDBG can be found at /opt/vc/bin

(sudo) vcdbg –help for a list of commands.

//

Let’s assume a vc_dispmanx_resource was created by an app ( still runing OR NOT - app stopped, and resource not cleaned up  )

#vcdbg reloc | grep dispmanx_resource

could give result like :

[  30] 0x10a71a20: used 240K (refcount 2 lock count 8, size   245760, align   32
, data 0x10a71a40, d1rual) 'dispmanx_resource'

for this particular result, the format of the resource used for the text is known to be : 512×160 pixels, RGB888 surface..

The size matches ( 512×160x3 = 245760 ).This gpu memory bloc is the one of the dispmanx resource.

NOTE : ** the [ 30] at the start of the line is the VC_MEM_HANDLE number **

using vcdbg, one can dump and create a file from this GPU memory area , by doing :

#vcdbg save /path_to_folder/filename.raw  0x10a71a40 245760
** raw notes following **

and from : https://github.com/raspberrypi/userland/blob/master/interface/vmcs_host/vc_vchi_dispmanx.c

#ifdef SELF_HOSTED
VCHPRE_ int VCHPOST_ vc_dispmanx_resource_write_data_handle( DISPMANX_RESOURCE_HANDLE_T handle, VC_IMAGE_TYPE_T src_type /* not used */,
int src_pitch, VCHI_MEM_HANDLE_T mem_handle, uint32_t offset,
const VC_RECT_T * rect ) {
int32_t bulk_len;
uint32_t param[3];
uint32_t success = 0;
//Note that x coordinate of the rect is NOT used
//Address of data in host
   offset += src_pitch * rect->y;
   bulk_len = src_pitch * rect->height;
//Now send the bulk transfer across
//command parameters: resource handle, destination y, bulk length
   param[0] = VC_HTOV32(handle);
   param[1] = VC_HTOV32(rect->y);
   param[2] = VC_HTOV32(bulk_len);
   success = dispmanx_send_command(  EDispmanBulkWrite | DISPMANX_NO_REPLY_MASK, param, sizeof(param));
if(success == 0)
   {
lock_obtain();
      success = vchi_bulk_queue_transmit_reloc( dispmanx_client.client_handle[0],
                                                mem_handle, offset,
                                                bulk_len,
                                                VCHI_FLAGS_BLOCK_UNTIL_DATA_READ,
NULL );
lock_release();
   }
return (int) success;
}
#endif

looks possible to move GPU to GPU memory blocs. the idea here is to use the function to switch gpu bitmaps from an app to another without the extra move from GPU to the arm.
Moving gpu to gpu *should* be faster .

– and raw notes –

19-09-2015 update:

After many try and fails to recompile the libraries with “SELF_HOSTED” on, i have succesfully tested a different approach for the problem :

Successfull test were made with switching a dispmanx’s element mem_handle reference to 2 different gpu memory areas.

Successfull tests were also made with direct access to the GPU’s bitmap memory blocs.
It implicates many manipulations ,including Build , send and receive mailboxes (”get_mem_handle, mem_lock/mem_unlock ..”) , memmap’s open/close to access the GPU datas from the arm side (read/write or modify ) .

From the SDL2.0 point of view, a SDL texture created with RGBA8888 properties, is stored into the GPU side with format VC_IMAGE_TF_RGBA32 so, if one needs to modify a dispmanx resource’s memory handle value, the dispmanx resource has to be created with this type, and the things will be easier (correct displayed datas without extra bytes conversion )

Also, due to the 32bits align into the GPU memory, it’s easier to choose powers of two for the bitmaps sizes , and 4 bytes for colors (RGBA) because the width * height * bytesperpixel final result will generate 32-ready-aligned total sizes, easy to found into the “vcdbg reloc” results list.

– popcornmix wrote :

“vc_dispmanx_resource_get_image_handle(res)” returns a (GPU) physical address containing a VC_IMAGE_T structure (the third and fourth shorts contain width and height).
There is a mem_handle in this structure that can be passed to the mailbox lock function to get a physical address to the memory.

this is the struct the function ’s gives a pointer to :

VC_IMAGE_T , from vc_image.h

struct VC_IMAGE_T {
#ifdef __HIGHC__
VC_IMAGE_TYPE_T type; /* Metaware will use 16 bits for this enum
so use the correct type for debug info */
#else
unsigned short type; /* should restrict to 16 bits */
#endif
VC_IMAGE_INFO_T info; /* format-specific info; zero for VC02 behaviour */
unsigned short width; /* width in pixels */
unsigned short height; /* height in pixels */
int pitch; /* pitch of image_data array in bytes */
int size; /* number of bytes available in image_data array */
void *image_data; /* pixel data */
VC_IMAGE_EXTRA_T extra; /* extra data like palette pointer */
VC_METADATA_HEADER_T *metadata; /* metadata header for the image */
struct opaque_vc_pool_object_s *pool_object; /* nonNULL if image was allocated from a vc_pool */
MEM_HANDLE_T mem_handle; /* the mem handle for relocatable memory storage */
int metadata_size; /* size of metadata of each channel in bytes */
int channel_offset; /* offset of consecutive channels in bytes */
uint32_t video_timestamp;/* 90000 Hz RTP times domain - derived from audio timestamp */
uint8_t num_channels; /* number of channels (2 for stereo) */
uint8_t current_channel;/* the channel this header is currently pointing to */
uint8_t linked_multichann_flag;/* Indicate the header has the linked-multichannel structure*/
uint8_t is_channel_linked; /* Track if the above structure is been used to link the header
into a linked-mulitchannel image */
uint8_t channel_index; /* index of the channel this header represents while
it is being linked. */
uint8_t _dummy[3]; /* pad struct to 64 bytes */
};

To modify the data inside it, one needs to memmap the returned pointer to an arm-side pointer.
Then the datas can be modifed/read etc.
From my tests, i’ve found that the modifications to this struct are not taken into account ready when done.
This structs can be listed using “vcdbg malloc” , sized 64 bytes like the definition listed before is.

For the dispmanx element to be updated once some modifications to the strcuture datas has been made, the only way i’ve found is to call a ‘dispmanx_resource_change’ function ( to a ‘fake’ other resource), then recall the same function to make it use the same, original resource. that way, the datas of the structure are reloaded, and the changes you made are applied.
//

One can also need to access to the GPU’s side bitmap datas.
To do this, get the VC_MEM_HANDLE value from the structure,

Then use the mailbox 0×30014 message to get the GPU’s pointer for this MEM_HANDLE. once the mailbox sends back the pointer,  (you can compare it with the (”vcdbg reloc” results))

You have to lock the GPU’s memory ( mailbox : memory_lock (handle) )

Then modify/read/write the datas thru a memmap’d pointer to the GPU side bitmap.

Then mailbox a ‘mem_unlock(handle)’

..and you’re done.

// and the discussion ’s related function added ( TAG 0×00030014 , Get dispmanx resource mem handle ) at  https://github.com/raspberrypi/firmware/wiki/Mailbox-property-interface

//

//// hot-note, SDL related, SDL_Texture structure


50 {
51  const void *magic; 4 = 15 d1 f4 b6
52  Uint32 format; // 4 = 04 18 16 16
53  int access; // 4 = 00 00 00 00
54  int w; // 4 = 00 00 00 c8
55  int h; // 4 = 00 00 00 c8
56  int modMode; // 4 = 00 00 00 00
57  SDL_BlendMode blendMode; // 6 = 00 00 00 00 00 00
58  Uint8 r, g, b, a; // 4 = ff ff ff ff
60  SDL_Renderer *renderer; // 32bit pointer 4 = 18 3b a0 b5
61
62  /* Support for formats not supported directly by the renderer */
63  SDL_Texture *native; // 32bit arm pointer 4 = 00 00 00 00
64  SDL_SW_YUVTexture *yuv; // 4 = 00 00 00 00
65  void *pixels;
66  int pitch;
68
69  void *driverdata;
73 };
// hotnote , SDL_Texture* datas
15 D1 F4 B6 04 18 16 16 00 00 00 00 C8 00 00 00 // c8 = 200 = width
C8 00 00 00 00 00 00 00 00 00 00 00 FF FF FF FF // c8 = 200 = height
18 3B A0 B5 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 28 42 A0 B5 00 00 00 00 00 00 00 00
00 00 00 00 25 00 00 00 01 00 00 00 E1 0D 00 00
08 19 00 00 01 14 00 00 00 00 00 00 00 00 00 00
00 00 00 00 35 00 00 00 00 00 00 00 00 00 00 00
20 04 00 00 00 00 FF 00 00 FF 00 00 FF 00 00 00
00 00 00 00 00 00 00 08 10 08 00 00 00 00 00 00
50 1F A0 B5 45 00 00 00 00 00 00 00 00 00 00 00
C8 00 00 00 C8 00 00 00 20 03 00 00 08 E0 B5 B5
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 C8 00 00 00 C8 00 00 00 00 00 00 00
00 00 00 00 89 00 00 00 40 00 A0 B5 40 00 A0 B5
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
// texture_magic macro :
#define CHECK_TEXTURE_MAGIC(texture, retval) \
    if (!texture || texture->magic != &texture_magic) { \
        SDL_SetError("Invalid texture"); \
        return retval; \
    }

// mailboxes and co , relevant cut’n'pastes
//
// github, hello_pi hello_fft : mailbox.h , mailbox.c

opt/vc/src/hello_pi/hello_fft/mailbox.c
opt/vc/src/hello_pi/hello_fft/mailbox.h
//

With the exception of the property tags mailbox channel, when passing memory addresses as the data part of a mailbox message, the addresses should be bus addresses as seen from the VC. These vary depending on whether the L2 cache is enabled. If it is, physical memory is mapped to start at 0×40000000 by the VC MMU; if L2 caching is disabled, physical memory is mapped to start at 0xC0000000 by the VC MMU. Returned addresses (both those returned in the data part of the mailbox response and any written into the buffer you passed) will also be as mapped by the VC MMU. In the exceptional case when you are using the property tags mailbox channel you should send and receive physical addresses (the same as you’d see from the ARM before enabling the MMU).

For example, if you have created a framebuffer description structure in memory (without having enabled the ARM MMU) at 0×00010000 and you have not changed config.txt to disable the L2 cache, to send it to channel 1 you would send 0×40010001 (0×40000000 | 0×00010000 | 0×1) to the mailbox. Your structure would be updated to include a framebuffer address starting from 0×40000000 (e.g. 0×4D385000) and you would write to it using the corresponding ARM physical address (e.g. 0×0D385000).

From the above :

volatile unsigned int mailbox[100] __attribute__ ((aligned(16))); // align16, 100 uint32_t for tags etc

then calling a func like :

mailbox_write((uint32_t) (mailbox+0x40000000), channel); // L2 cache on, else +0xC0000000 for L2 disabled
314159265358979323846264338327950288
419716939937510582097494459230781640
628620899862803482534211706798214808
Older Posts »

cat{ 50 } { post_696 } { } 2009-2015 EIhIS Powered by WordPress