Wednesday, August 13, 2014

Onboard Camera - V4L wrapper with Yocto Project

Based on my last post, there are a lot of people having problems to use this wrapper in Yocto, since it uses a kernel header mxcfb.h and it differs a bit from LTIB, so this post is to help solving this issue.

We have a nice tool on linux that most people don't use it (myself included) and can do the job of dealing with dependencies like a charm ! This tool is called AUTOTOOLS, yes, that one which create the pratical and easy ./configure ./make and ./make install system. Below I will demonstrate how we can use that to solve our issue:


STEP#1 - YOCTO BUILD CHANGES

We now need to add the kernel-dev to our local.conf (in build/conf) file and it will look like:

     BB_NUMBER_THREADS ?= "${@oe.utils.cpu_count()}"
     PARALLEL_MAKE ?= "-j ${@oe.utils.cpu_count()}"
     MACHINE ??= 'imx6qsabresd'
     DISTRO ?= 'poky'
     PACKAGE_CLASSES ?= "package_rpm"
     EXTRA_IMAGE_FEATURES = "debug-tweaks"
     USER_CLASSES ?= "buildstats image-mklibs image-prelink"
     PATCHRESOLVE = "noop"
     BB_DISKMON_DIRS = "\
         STOPTASKS,${TMPDIR},1G,100K \
         STOPTASKS,${DL_DIR},1G,100K \
         STOPTASKS,${SSTATE_DIR},1G,100K \
         ABORT,${TMPDIR},100M,1K \
         ABORT,${DL_DIR},100M,1K \
         ABORT,${SSTATE_DIR},100M,1K" 
     PACKAGECONFIG_pn-qemu-native = "sdl"
     ASSUME_PROVIDED += "libsdl-native"
     CONF_VERSION = "1"

     BB_NUMBER_THREADS = '4'
     PARALLEL_MAKE = '-j 4'

     DL_DIR ?= "${BSPDIR}/downloads/"
    ACCEPT_FSL_EULA = ""

     DISTRO_FEATURES_remove = "x11 wayland"

     CORE_IMAGE_EXTRA_INSTALL += "gpu-viv-bin-mx6q gpu-viv-bin-mx6q-dev kernel-dev"


STEP #2 - REBUILDING YOCTO

     $ bitbake core-image-base


STEP#3 - POSSIBLE ISSUE

     if the mxcfb.h file doens't appear in your sysroot (/opt/poky/1.6.1/sysroots/cortexa9hf-vfp-neon-poky-linux-gnueabi/) then you need to build your toolchain and then reinstall it as well:

     rebuild with -c populate:

          $ cd fsl-community-bsp/build
          $ bitbake core-image-base -c populate_sdk

     and then install:

          $ cd /fsl-community-bsp/build/tmp/deploy/sdk
          $ ./poky-eglibc-x86_64-core-image-base-cortexa9hf-vfp-neon-toolchain-1.6.1.sh


STEP #4 - MODIFYING THE APP TO SUPPORT AUTOTOOLS

     Now is the interesting part, we will need 2 different files (pretty simple ones actually), they are: Makefile.am and configure.ac.

     Based on my projects configuration, I like to have bin, include and src folders, so the sample below will be on those:


     Project Tree:
    
             Project folder
                        |
a)                     ----- Makefile.am
                        |
b)                     ----- configure.ac
                        |
                        ----- bin /
                        |
                        ----- src /
                        |         |
c)                     |         ------ Makefile.am
                        |         |
                        |         ------ camera_test.c
                        |         |
                        |         ------ v4l_wrapper.c
                        |
                        ----- include /
                                  |
                                  ------ v4l_wrapper.h


    a) Makefile.am

             AUTOMAKE_OPTIONS = foreign
        SUBDIRS = src          # add just those ones that will have a Makefile inside.

    b) configure.ac
         
             AC_INIT([camera_test], [0.1], [andre.silva@freescale.com])
             AM_INIT_AUTOMAKE([-Wall -Werror foreign])
             AC_PROG_CC
             AC_CONFIG_HEADERS([config.h])
             AC_CHECK_HEADERS([fcntl.h stdint.h stdlib.h string.h sys/ioctl.h unistd.h])

             # Checks for library functions.
             AC_FUNC_MMAP
             AC_CHECK_FUNCS([gettimeofday memset munmap])

             ##########################################################################
             # Checks for programs needed for tests
             ##########################################################################

             AC_CONFIG_FILES([Makefile
             src/Makefile])

             AC_OUTPUT


    c) Makefile.am

             bin_PROGRAMS = camera_test

             camera_test_SOURCES = camera_test.c ../include/v4l_wrapper.h
             nodist_camera_test_SOURCES = ../src/v4l_wrapper.c

             ROOTFS_DIR = $(SDKTARGETSYSROOT)

             TARGET_PATH_LIB = $(ROOTFS_DIR)/usr/lib
             TARGET_PATH_INCLUDE = $(ROOTFS_DIR)/usr/include

             AM_CPPFLAGS = -I $(prefix)/usr/src/kernel/include/uapi -I $(prefix)/usr/src/kernel/include/ -I $(TARGET_PATH_INCLUDE) -I ../include

             AM_LDFLAGS = -Wl,--library-path=$(TARGET_PATH_LIB),-rpath-link=$(TARGET_PATH_LIB) -lm  -L $(prefix)/usr/lib 
     
             # workaround to get the binaries copied to local /bin folder
             all:
                       mv ../src/$(bin_PROGRAMS) ../bin

             clean-local: 
                        rm -rf ../bin/*
                  rm -rf ../src/*.o


     * if you are not going to use any additional source you can only let the camera_test_SOURCES = camera_test.c


STEP #5 - GENERATING THE BUILD FILES

     In a clean terminal enter the following commands to generate the files:

          $ autoheader
          $ autoreconf --install

     it will create all the necessary files to build your application, including dealing with dependencies, even if they are from kernel.


STEP #6 - BUILDING THE APPLICATION

     For building any application using yocto (unless you have a recipe for it), you have to export the toolchain environment variables, and once it is done, the terminal you have used is now dirty and if you want to do all the steps above again, you will need another one (cleaned).

          $ cd /opt/poky/1.6.1/
          $ .  ./environment-setup-cortexa9hf-vfp-neon-poky-linux-gnueabi
          $ cd /home/usr/v4l_wrapper_yocto/
          $ ./configure --host=arm-poky-linux-gnueabi --prefix=$SDKTARGETSYSROOT
          $ make

     At this point you will have your binary placed in bin/ folder, if you enter the make install command it will install (copy) the application binary into your rootfs/bin, where the rootfs is defined by the --ṕrefix in ./configure line.


THE RESULT





The sample code can be found here.

Thanks to Rogério Pimentel who helped me with the autotools and how to solve the mxcfb.h dependency issue and Daiane Angolini with YOCTO.

EOF !




15 comments:

  1. This comment has been removed by the author.

    ReplyDelete
  2. Hi Andre! thanks for your updated post! I met exactly the same problem here, so I tried to follow your new post just now. everything works smoothly until the step 5:
    I met:

    bingqian@bingqian-ThinkPad-X61:~/Downloads/v4l_opencv_wrapper_yocto-master$ autoheader
    bingqian@bingqian-ThinkPad-X61:~/Downloads/v4l_opencv_wrapper_yocto-master$ autoreconf --install
    automake: warnings are treated as errors
    src/Makefile.am:4: warning: source file '../v4l_wrapper.c' is in a subdirectory,
    src/Makefile.am:4: but option 'subdir-objects' is disabled
    automake: warning: possible forward-incompatibility.
    automake: At least a source file is in a subdirectory, but the 'subdir-objects'
    automake: automake option hasn't been enabled. For now, the corresponding output
    automake: object file(s) will be placed in the top-level directory. However,
    automake: this behaviour will change in future Automake versions: they will
    automake: unconditionally cause object files to be placed in the same subdirectory
    automake: of the corresponding sources.
    automake: You are advised to start using 'subdir-objects' option throughout your
    automake: project, to avoid future incompatibilities.
    autoreconf: automake failed with exit status: 1


    any thoughts?
    Thanks a lot!!
    Best,
    Zoe

    ReplyDelete
  3. Hi there mate,

    how is your project tree ? the subdirectory where is placed your source has the makefile ?

    regards,
    Andre

    ReplyDelete
  4. Hi Andre

    I followed your tutorial building image by enabling kernel-dev.
    After all I copied camera_test from bin directory to Sabrelite rootfs directory.
    I run the command"DISPLAY=:0 ./camera_test

    I got following error with USB as well as mipi camera shipped with board
    Initializing Camera Device: Video0 (640x480)
    unable to open /dev/video0

    device not found, make sure the driver are properly installed.
    ov5642_camera.ko, ov5640_camera_mipi.ko and mxc_v4l2_capture.ko

    ReplyDelete
    Replies
    1. Andre

      I couldn't find any search result relevant to mipi camera driver..
      ov5642_camera.ko, ov5640_camera_mipi.ko and mxc_v4l2_capture.ko
      How to check If installed or not?
      If not installed How to install then?

      please help..

      Delete
    2. during the Linux boot (or just enter the dmesg command) see if you find the information there, also please, past your lsmod command log here.

      regards,
      Andre

      Delete
    3. Please find the logs of both commands dmesg and lsmod..
      http://pastebin.com/DpHFyPd9
      http://pastebin.com/RzS5P55E

      Delete
  5. Andre,
    Thanks for the example. I built it as per your instructions, had to make one minor change (src/Makefile.am: changed ../src/v4l_wrapper.c to v4l_wrapper.c).

    Anyway, it built and ran fine aside from that.
    I'd like to do some processing on the incoming image, so how do I tell the IPU to convert the incoming image to RGB? Ideally, I'd like it to be compatible with openCV, but any RGB would be a start.

    ReplyDelete
    Replies
    1. Hi Ed, sorry for this late response.

      Please enter this question on the freescale community (community.freescale.com) an IPU expert will help you out on this question.

      regards,
      Andre

      Delete
  6. Andre (more),
    Hopefully I don't need to do this in C, right? I know how to do it in C, I just want the accellerator to do it for me.
    Tx
    Ed

    ReplyDelete
    Replies
    1. Hi Ed,

      yes, you can set IPU to perform color conversion for you. But you will need to use the library functions to do that, which means that at least for setting the IPU you will need to use C =)

      Delete
  7. Hi,
    I'm trying to follow your steps but I'm getting the below error when I try to bitbake core-image-base.

    ERROR: Nothing RPROVIDES 'gpu-viv-bin-mx6q' (but /home/bosch/fsl-bosch-bsp/sources/poky/meta/recipes-core/images/core-image-base.bb RDEPENDS on or otherwise requires it)
    NOTE: Runtime target 'gpu-viv-bin-mx6q' is unbuildable, removing...
    Missing or unbuildable dependency chain was: ['gpu-viv-bin-mx6q']
    ERROR: Required build target 'core-image-base' has no buildable providers.
    Missing or unbuildable dependency chain was: ['core-image-base', 'gpu-viv-bin-mx6q']


    Can you advise me in how to correct this error ?

    Thank you in advance,
    Pedro X. Matos

    ReplyDelete
    Replies
    1. Hi Pedro,

      Please use the bsp provided by nxp(ex-freescale). git.freescale.com (get the latest ga release)

      regards,
      Andre

      Delete
  8. when I run, autoreconf --install, I got
    perl: warning: Falling back to the standard locale ("C").
    perl: warning: Setting locale failed.
    perl: warning: Please check that your locale settings:
    LANGUAGE = "en_IN:en",
    LC_ALL = (unset),
    LANG = "en_IN"
    are supported and installed on your system.
    perl: warning: Falling back to the standard locale ("C").
    configure.ac:3: installing './compile'
    configure.ac:2: installing './install-sh'
    configure.ac:2: installing './missing'
    automake: warnings are treated as errors
    src/Makefile.am:4: warning: source file '../src/v4l_wrapper.c' is in a subdirectory,
    src/Makefile.am:4: but option 'subdir-objects' is disabled
    automake: warning: possible forward-incompatibility.
    automake: At least a source file is in a subdirectory, but the 'subdir-objects'
    automake: automake option hasn't been enabled. For now, the corresponding output
    automake: object file(s) will be placed in the top-level directory. However,
    automake: this behaviour will change in future Automake versions: they will
    automake: unconditionally cause object files to be placed in the same subdirectory
    automake: of the corresponding sources.
    automake: You are advised to start using 'subdir-objects' option throughout your
    automake: project, to avoid future incompatibilities.
    src/Makefile.am: installing './depcomp'
    autoreconf: automake failed with exit status: 1

    ReplyDelete
  9. Can you provide any good resources for understanding makefile.am. I have to write it..I have little knowledge of linux..

    ReplyDelete