\input texinfo    @c -*-texinfo-*-
%
% wrs-build.in - main file for the documentation
%
%%%%

%------------------------------------------------------------------------------
%
%                         NOTE FOR THE UNAWARE USER
%                         =========================
%
%    This file is a texinfo source. It isn't the binary file of some strange
%    editor of mine. If you want ASCII, you should "make wrs-build.txt".
%
%------------------------------------------------------------------------------

%
% This is not a conventional info file...
% I use three extra features:
%   - The '%' as a comment marker, if at beginning of line ("\%" -> "%")
%   - leading blanks are allowed (this is something I cannot live without)
%   - braces are automatically escaped when they appear in example blocks
%

@comment %**start of header
@documentlanguage en
@documentencoding ISO-8859-1
@setfilename wrs-build.info
@settitle wrs-build
@iftex
@afourpaper
@end iftex
@paragraphindent none
@comment %**end of header

@setchapternewpage off

@set update-month March 2012

@finalout

@titlepage
@title White Rabbit Switch: software build scripts
@subtitle @value{update-month}
@subtitle How to rebuild the whole software package from sources
@author Alessandro Rubini (@code{rubini@@gnudd.com})
@end titlepage
@headings single

@c ##########################################################################
@iftex
@contents
@end iftex

@c ##########################################################################
@node Top
@top Introduction

This document describes the software build procedures for the White
Rabbit Switch.  The procedure described is for version 3 of the
hardware project, which  has been successfully booted for the first
time on Sep 13th 2011.

If you have an older Version-2 switch, please refer to the v2 branch
of the repository at
@code{git://ohwr.org/white-rabbit/wr-switch-sw.git}
(or @code{git@@ohwr.org:white-rabbit/wr-switch-sw.git} if you are
granted write access).

If you do not want to get your hands dirty, just skip to @ref{Quick
Build}.

@menu
* Overview::                    
* Quick Build::                 
* The Compiler::                
* Buildroot::                   
* The IPL::                     
* The Boot Loader::             
* The Linux Kernel::            
* Kernel Modules::              
* Initial tools for the FPGA::  
* PTPd::                        
* User Space Applications::     
* VHDL Binaries::               
* The Complete Filesystem::     
* Installation of WRS-3::       
* Code layout in a production switch::  
* Troubleshooting::             
@end menu

@c ##########################################################################
@node Overview
@chapter Overview

The scripts build over previous work by Tomasz Wlostowski, who first
made the whole thing work and stick together -- a serious result from
serious efforts, I am really amazed by his achievements.

The rewriting described in this document (and embodied in the @code{build/}
subdirectory of the @code{wr-switch-sw} package) is meant to make the stuff
more maintainable over time.  I am an old-fashioned follower of the rule
``first make it work then make it work well'', and this step is expected
to make it work well for everybody.

The purpose of the build-script rewrite is achieving the following targets:

@itemize @bullet
@item One-command build.
	The non-technical user should be able to rebuild the whole software
        package with a single command. This includes the IPL and boot-loader
        even though they are expected to be pre-installed in the switch with
        no real need for upgrading.
@item Sub-package separation.
	Users and developers should be able to rebuild each
        sub-package by itself. Sub-packages are the kernel,
	@i{buildroot}, libraries and so on. If you have a problem (or a
        customization you need on one sub-package), you should be able to work
        on the specific part ignoring the whole as much as possible.
@item Documentation.
	The steps are documented as much as possible, because mishaps
        do happen, and you should easily understand where the problem is.
@item Avoid redundant downloads.
	People with non-mainstream network connections would rather
	avoid downloading the same package over and over. Thus, a centralized
	download directory is defined where all external packages are
        retrieved. Even if you ``make distclean'' in the build scripts you
        will not need re-get everything from the network.  In a similar
        mood, people who already have a local copy of the big packages
        (kernel, u-boot, white-rabbit svn) will not need to re-download not
        even the first time they build the WRS software.
@end itemize

The build system is set up as a mix of scripts and makefiles. Every
sub-package is built by its own script and/or Makefile, and configuration is
passed over through environment variables. The top-level build script
sets all environment variables, while keeping defaults from your
preexisting environment -- so you can override anything even when
rebuilding it all from scratch.

@menu
* Portability::                 
* Environment Variables::       
* Downloading Files::           
* Rebuilding Parts::            
@end menu

@c ==========================================================================
@node Portability
@section Portability

The scripts in their current status are not expected to be very
portable. I am sure a number of @i{bashisms} exist, and I did no effort
to even identify them.  To relieve the user from possible pain,
internally the name @i{bash} is used instead of @i{sh}, so things
should work even if your default shell is @i{ash} or @i{dash};
actually the scripts have been tested in one system where the
default @i{sh} is @i{dash}.

Similarly, the scripts are likely to fail if you use spaces in directory
names; that is because not all
uses of shell variables are properly quoted. I urge you to use directory names
with no spaces in them, or to submit a patch to fix the scripts.

It should go without saying that the build environment is expected to
be a native GNU/Linux system; success reports about other environments
(e.g. cygwin) are welcome, possibly with associated patches.

@c ==========================================================================
@node Environment Variables
@section Environment Variables

The scripts use a number of environment variables; you can pre-set
them as you wish. If they are not pre-set, defaults apply as
described below.

When rebuilding everything, the defaults are applied for each unset
variable; but when rebuilding a sub-package you will need to set the
variables beforehand.  Each sub-package complains if it needs variables
that are not set in your environment.

The following variables are used in one or more parts of the scripts;
let me restate, though, that sensible default values apply by default,
so this list is mainly for your curiosity unless you are a developer.

They are listed in an order that seems logical to me, but may sound
random to a different person, please forgive this.  Most of the
variables are prefixed with @code{WRS_} to make them easily identified
in the overall mess of variables and command names (all scripts used here
have a similar prefix for the same reason).

@table @code
@item WRS_BASE_DIR
	The pathname of the build directory (i.e., the @code{build/}
        subdirectory of @code{wr-switch-sw}).
        The variable is internally set to the directory
        name of the main script. Note that the script cannot
        be run from the same directory (i.e.: @code{./wrs_build-all}
        is not allowed), you must call it from your output directory
        using a pathname to  invoke it.  This variable cannot be
        overridden in the main script, but must be pre-set if you run
        a sub-script to rebuild only part of the software suite.

@item WRS_OUTPUT_DIR
	The pathname of the directory where output is placed. It defaults
        to the current directory whence you invoke the script (i.e., you
        can invoke @code{/path/to/wrs_build-all} to have all output
        in the current directory).  Compilation
        happens in a @i{build} subdirectory of @code{WRS_OUTPUT_DIR},
        done-markers are placed in a @i{_done} subdirectory
        and final images are placed in a @i{images} subdirectory.

@item WRS_DOWNLOAD_DIR
	The directory where downloaded files are placed. If unset
        it defaults to @code{$WRS_OUTPUT_DIR/downloads}, which is
        created if needed.  By pre-setting this variable you can
        simply recursively delete
        the output directory to force a full rebuild, without
        the overhead of re-downloading everything.

@item CROSS_COMPILE
	The variable is the usual cross-compilation prefix. For example,
        @code{arm-linux-} if you have @i{arm-linux-gcc} in your path,
        or a full pathname without the trailing @code{gcc}. If unset,
        it defaults to the compiler that @i{buildroot} self-builds.
        See @ref{The Compiler} for some more details.

@end table

Other variables are used internally in the script; since they are only
useful to people working on the script itself, they are documented in
place.

If you compiled for version 2 you will notice we do not have
@code{WRS_WR_REPOSITORY} any more. We used to rely on upstream
archives to be part of the @i{White Rabbit} subversion tree, but I
did not add the new archives in there because of their size.

@c ==========================================================================
@node Downloading Files
@section Downloading Files

Every downloaded file is saved to the @code{download} directory, you should
arrange not to remove that directory when you recompile over and over
during development.  I chose to make the first
script download everything, because I think this eases development in a
way: you first wait a while but can tell download errors from other
issues, and then you can build it all even without a network connection.

For each upstream archive needed, the following steps are performed:

@itemize @bullet
@item If the file exists in the download directory, the @i{md5sum} is
      checked; on success, nothing else is done.
@item If the previous step fails, the file is retrieved from upstream.
@item If the previous step fails, the file is downloaded from the
      buildroot web site.
@end itemize

The policy just described is implemented in @i{wrs_download}, in the file
@code{scripts/wrs_functions}, based on @code{download-info} in the
main build directory.

The messages of a download run are like the following ones:

@smallexample
   2012-01-12 18:30:46: --- Downloading all files
   2012-01-12 18:30:53: Retrieved at91bootstrap-3-3.0.tar.gz from upstream
   2012-01-12 18:31:00: Retrieved buildroot-2011.11.tar.bz2 from upstream
   [...]
   2012-01-12 18:37:53: Retrieved uClibc-0.9.32.tar.bz2 from upstream
   2012-01-12 18:37:56: Retrieved zlib-1.2.5.tar.bz2 from upstream
@end smallexample

@c ==========================================================================
@node Rebuilding Parts
@section Rebuilding Parts

When the main script succeeds in building one part (sub-package),
it creates a file in the @code{build/_done} directory.

When you rebuild everything, steps for which the marker file exists
are not rebuilt. To force rebuilding of one specific part, just remove
the marker.  Markers are numbered, reflecting the order of compilation
steps, but they also have a name: names like @code{04-kernel} should be
self-explicative.

As an alternative, you can run the individual script from within
@i{scripts/}, after setting the proper environment variables.

@c ##########################################################################
@node Quick Build
@chapter Quick Build

If you just want to build stuff, with no concern about network
downloads and without even knowing what is happening, just create a
directory where you want the output to be generated and start
compilation. Note that it takes around 3GB of storage.

Then run this (but please read more for a better command):

@example
   /path/to/wr-switch-sw/build/wrs_build-all
@end example

Note that progress messages are sent to @i{stderr}, so you may want to
save @i{stdout} to a file, like this (again, it is recommended you read
further for a better command):

@example
   /path/to/wr-switch-sw/build/wrs_build-all > logfile
@end example

Please note that there are also a number of warning messages being
printed to @i{stderr}. It is a few hundred lines over the many
minutes it takes to build @i{buildroot}, but you can safely ignore them,
trusting the build process will complete successfully.

The progress messages look like what is shown here below.  The log
file will be rather big (6 or 7MB or so), as all the compilation steps are
quite verbose.

The following example shows a run on a quad core system (18k
bogoMips in total). If files had already been downloaded, the first
step takes only a pair of seconds to verify the checksums:

@c FIXME: example
@smallexample
2012-01-12 18:59:04: --- Downloading all files
2012-01-12 18:59:06: --- Buildroot compiler and filesystem
2012-01-12 18:59:06: Uncompressing buildroot
2012-01-12 18:59:06: Patching buildroot
2012-01-12 18:59:06: Reconfiguring buildroot
2012-01-12 18:59:07: Compiling buildroot
2012-01-12 19:14:16: --- AT91Boot
2012-01-12 19:14:16: Patching AT91Boot
2012-01-12 19:14:16: Building AT91Boot
2012-01-12 19:14:17: --- Barebox
2012-01-12 19:14:17: Patching Barebox
2012-01-12 19:14:17: Building Barebox
2012-01-12 19:14:25: --- Linux kernel for switch
2012-01-12 19:15:43: --- Kernel modules from this package
[...]
@end smallexample

You may prefer to save @i{stderr} with @i{stdout} to the log file
but still see the time-stamped messages from the scripts. In this
case you can issue the following command -- which is what I used
to generate the terse output shown above:

@example
   /path/to/wr-switch-sw/build/wrs_build-all 2>&1 | tee logfile \
        | grep '^20..-..-.. ..:'
@end example

If you are lucky, everything completes by itself. The time taken
depends on you CPU, disk and network speed.
At the end you will find your final files in the @code{images}
subdirectory.  If you are not too lucky, the build stops because
you have found a bug in the build scripts; most likely because your
setup differs from the ones we have been testing on.

In order to re-run the build from the beginning, please remove (or
rename) the output directory and reissue the command.  To only
redo one step, you can remove the marker in @code{build/_done}
(the markers are empty files with name like @code{01-buildroot}
and @code{04-kernel}).

@c ##########################################################################
@node The Compiler
@chapter The Compiler

The predefined compiler used here is the one built by @i{buildroot}.
The default configuration selects this choice.  If you pre-set a
different @code{CROSS_COMPILE} prefix in your environment, your own
choice will be used by modifying the @i{buildroot} configuration file.
Note, however, that not all cross-compilers will work (@i{buildroot}
wants one that has been configured with @code{--sysroot} and it is
quite unlikely yours has been).

In practice, you may want to set @code{CROSS_COMPILE} when you compile
the boot loader and kernel by themselves, and avoid it when compiling
the complete package.

@c ##########################################################################
@node Buildroot
@chapter Buildroot

The distribution being used here is @i{buildroot}. It is the first
step being built, because it creates the cross-compiler it will use. This
compiler is later used to compile all other software for the White
Rabbit Switch.

The configuration for @i{buildroot} comes from
@code{patches/buildroot/buildroot-config-wrswitch}. The configuration
is then changed only if you pre-set your own @code{CROSS_COMPILE}
variable.

@c The following can be shown in a set of commands instead of this hard
@c to read explanation.

If you want to change the configuration, you can do so after the first
build iteration: change directory to @code{build/buildroot-2011.11}
and run @code{make menuconfig}. After making your choices, copy back
the file @code{.config} to
@code{patches/buildroot/buildroot-config-wrswitch} in this package.

You can also set @code{WRS_BUILDROOT_CONFIG} to the full pathname of
your configuration file of choice. The file must be a copy of the
@code{.config} after the @code{make menuconfig} step described above.
Note that if the variable is not pointing to a regular file it is
ignored with a simple warning -- rather than stopping the build procedure.

@c ##########################################################################
@node The IPL
@chapter The IPL

The version of @i{at91bootstrap} being used in the switch as
@i{Initial Program Loader} is found at @code{http://repository.timesys.com/}
The patches we applied are in the directory @i{patches/at91boot/v3.3},
and currently it is only a single patch (we are piggy-backing on the Atmel
evaluation board without even changing the board name):

@example
   0001-board-9g45ek-fix-ddr-config-for-WRS-V3.patch
@end example

The script uncompresses, patches and builds, leaving
@code{images/at91bootstrap.bin} after it is over. This file is
the one to be loaded in the hardware.

For simplicity, a known-working binary is part of the @i{binaries}
@c The 'sane' name?? do you mean 'same'?
directory of this package as @code{at91bootstrap.bin}, the sane name used
later in the installation instructions.

@b{Warning}: with some distributions, this compilation step will print
a scary message about memory corruption. The message is reporting a bug in
the configuration program which has no actual effects and can be ignored.
Maybe we will switch to another version in the future.

@c ##########################################################################
@node The Boot Loader
@chapter The Boot Loader

The switch uses @i{barebox} as a boot loader. We are running version
2011-09, with one simple local patch and the chosen configuration
file. More patches will be needed to customize board names (we are
piggy-backing on the Ronetix PM9G45 board).

The patches are part of this package in @i{patches/barebox} and
are currently the following ones:

@example
   0001-91samg45-removed-two-clock-that-failed-compilation.patch
   0002-sam945-include-mtd-nand.h-in-device-file.patch
   0003-arm-config-added-wrs3_defconfig-and-fixed-default-at.patch
   0004-new-config-for-barebox-env.patch
   0005-nand-wrs-it-s-16-bits-not-8.patch
@end example

The build scripts compile with the following commands, where the
cross-compiler is the one built by @i{buildroot}. If you run these
by hand you can use a different compiler (as shown):

@example
   export CROSS_COMPILE=/opt/arm-2010q1/bin/arm-none-eabi-
   export ARCH=arm
   make wrs3_defconfig
   make
   cp barebox.bin /tftpboot/bb.bin
@end example

A pre-built binary is available as @code{binaries/barebox.bin}.

The ELF version is copied to @i{images} as well, as
@code{images/barebox}; this file includes the symbol table and may (or may not) be
useful.

@c ##########################################################################
@node The Linux Kernel
@chapter The Linux Kernel

The kernel is currently version 2.6.39, compiled from an uncompressed
tar file (so not within a @i{git} repository).  The upstream
vanilla kernel is downloaded, then
local patches are applied (they come from a @i{git}
repository, but they are currently applied with a simple @i{patch}
command).

The relevant patches are available in @i{patches/kernel/v2.6.39},
and are currently the following ones:

@example
   0001-wrs3-changes-to-g45ek.patch
   0002-initramfs-stop-after-one-cpio-archive.patch
   0003-at91-NR_IRQS-increase-by-64-to-fit-custom-muxes.patch
   0004-irq-export-symbols-for-external-irq-controller.patch
@end example

The configuration being used is copied from this package, so it
is easier to change it if needed.

@c again, maybe simplify this to a set of commands?
If you want to change the configuration, you can do so after the first
build iteration: change directory to @code{build/kernel}
and run @code{make menuconfig}. After making your choices, copy back
the file @code{.config} to
@code{patches/kernel/v2.6.39/linux-config-wrswitch} in this package.

You can also set @code{WRS_KERNEL_CONFIG} to the full pathname of
your configuration file of choice. The file must be a copy of the
@code{.config} found in the main kernel directory,
(for example the one left after the @code{make menuconfig} step).
Note that if the @code{WRS_KERNEL_CONFIG}
variable is not pointing to a regular file it is
ignored with a simple warning, without stopping the build procedure.

The build scripts copy both @i{zImage} and all compiled kernel
modules to the @i{images/} directory of the build place. This currently
includes modules 

@c ##########################################################################
@node Kernel Modules
@chapter Kernel Modules

In the next step the scripts compile modules that are part of this
package.  The step depends on the kernel being available in the
build directory. The modules are then copied into the @code{images}
subdirectory of the main build directory.

Please note that modules (and later user-space) are compiled in-place;
ie. not in the output directory.  The disadvantage is that your repository
becomes dirty with output and intermediate files. The advantage is that
any modification you make to the code is already in the repository
for your to commit.

Currently, the modules build built are @i{wr_vic.ko} (the interrupt
controller) and @i{wr-nic.ko} (the network ``card'' driver).  The
@i{wr_rtu.ko} driver is compiled too, but it is not currently used nor
has it been tested.

@b{Warning}: I plan to soon rename all modules to have a hyphen
instead of an underscore in the name.

@c ##########################################################################
@node Initial tools for the FPGA
@chapter Initial tools for the FPGA

In order to make some tests with your board and be able to develop
further, the directory @i{tools} includes the following programs:

@table @i
@item mapper
@itemx wmapper
	The programs read from @i{/dev/mem} writing to @i{stdout} and
        read from @i{stdin} writing to @i{/dev/mem}, respectively.
        They are classic tools distributed in the @i{Linux Device Drivers}
        examples since 1998.

@item load-virtex
	Loads a bitstream to the Virtex6 device of the switch. The biststream
        file name must be passed in the command line (@i{/dev/stdin} allowed).
        The is a limit of 10MB (me lazy!), but the program complains if it
        detects the limit is reached.

@item load-lm32
	Loads a program to the internal LM32 soft-core.

@item spi-pll
	Uses the @i{opencores_spi} device part of the FPGA bitstream to
        talk with the AD9516 device that is mounted on the board. It is
        only of historical relevance, as the internal LM32 is not dealing
        with the PLL device.
@end table

The list above used to include @i{devmem2}, but now @i{devmem} is part
of the installed @i{busybox} and has the same command-line interface.

The functions to load the FPGA and the LM32 have been turned into a
library, so external programs can link @code{libtools.a}.

Please note that to compiler you need to specify both the kernel
directory (@code{LINUX=}) and the cross-compiler to use
(@code{CROSS_COMPILE=}). Moreover, the @i{tools} subdir is not
integrated in the build scripts.
@c FIXME: tools

@c ##########################################################################
@node PTPd
@chapter PTPd

@c FIXME: the new PTP
@b{Warning}: This part of the document, about PTP,  needs to be updated, what
follows is the old text, kept here for reference.

The Precision Time Protocol Daemon being used is host in a different
repository. It is the code base that has been ported to compile in a
freestanding environment, downloaded from
@code{git://gnudd.com/ptp-noposix.git} .

We are working on a better code base for a portable PTP, but it is
not yet ready as of this writing.

A plain @i{make} in the @i{ptp-noposix} directory will fail,
because after building
@i{ptpd} other steps are tried, but they are only needed for
the freestanding environment (i.e., LM32 with supporting code) and will
fail for this @i{arm-linux} hosted build.

Additionally, the script installs the header and library for
@i{wr_ipc}, which are needed for later steps and are now part of the
@c hal or HAL?
@i{ptp} repository. Similarly, the script installs headers for the hal
and @i{libptpnetif}.

The script is known to work with version @code{fea09f7} of the @i{git}
repository, but the script checks out the @i{master} branch, being
confident newer versions will be at least as good as the current one
(for example, the current version as I write this has some concerning
warning messages).

@c ##########################################################################
@node User Space Applications
@chapter User Space Applications

@c FIXME: user space apps
@b{Warning}: This part of the document, about user space, needs to be
updated, what follows is the old text, kept here for reference.

The build of user space is concerned about the following steps:

@table @i

@item libswitchhw 
	This is a library of functions needed by the next
        step. @code{libswitchhw.a} is thus installed in @code{images/wr/lib}.

@item wrsw_hal
	This is the main application program for the White Rabbit Switch
        operation. The script installs the executable in @code{images/wr/bin}.

@item wrsw_rtud
	This is the daemon for the routing table unit, installed in
        @code{images/wr/bin}.

@end table

@c ##########################################################################
@node VHDL Binaries
@chapter VHDL and LM32 Binaries

The binaries are currently missing from the filesystem-making procedures.
You will need to get the most recent binaries and copy them in the filesystem
by yourself. This part will be filled last in this document, when the
switches are distributed for production.

@c FIXME: the binaries.


@c ##########################################################################
@node The Complete Filesystem
@chapter The Complete Filesystem

The final step in compiling the filesystem is making the CPIO archive
with the overall filesystem contents.  This archive can be used as an
@i{initramfs} or uncompressed to some directory.  The build procedure
does not leave a directory tree on disk because that would require
administrator privileges. I think it is best not to call @i{sudo} from
within build scripts, to respect our users' security concerns.

The output file is called @i{images/wrs-image.cpio.gz} in the build
directory.  You can uncompress it, for example to run NFS-root,
with the following command in a newly-created empty directory:

@example
   zcat $WRS_OUTPUT_DIR/images/wrs-image.cpio.gz | sudo cpio --extract
@end example

The @i{cpio} archive includes a number of device special files in
@i{dev}, even though the configured @i{buildroot} uses @code{mdev -s}
to build all needed devices. The pre-created devices come from
@i{userspace/devices.tar.gz}. Note that the buildroot output
directory, @i{build/buildroot-2011.11/output/target} does not
include any target (and no white-rabbit specific files), so it
cannot be used as a root filesystem by itself.

The @i{cpio} filesystem is built by copying the @i{buildroot} file
tree and overlaying it with @code{userspace/rootfs_override/}; device
files, as said,  are extracted from @code{userspace/devices.tar.gz}.

The @i{root} user has an empty password (set forth in
@code{userspace/rootfs_override/}) and authorizes a few @i{ssh} keys,
to allow @i{ssh} login (and @i{scp}) without the need to provide a
password -- though empty.  The script authorizes the key of the user
running the script and all keys listed in the file @code{authorized_keys}
in the main directory of this package, if present.

@c ##########################################################################
@node Installation of WRS-3
@chapter Installation of WRS-3

A switch with no code has three communication channels available: a
serial port, on 2.54mm pins, a JTAG connector with standard ARM-20
pinout and a USB port driven by the internal CPU ROM if no boot code
is found.  Please note that if you write a faulty @i{at91boot} binary
the internal ROM will jump into it without activating the USB port.
The solution here is either the @i{USB loader} or the JTAG, as described
later
and the JTAG will be your only way out

If you have a @i{mini-backplane}, you can connect the 6 pins of P2 to
the similar connector on the backplane labeled as @code{Connect to
MCH/P2}. The backplane hosts a CP2102 USB adapter and a mini-USB
socket.  Note that you may connect all 6 pins, but only TX and RX
signals are needed.

You can work without a serial port, but it is strongly suggested to
connect it.  With a UART you are able to see diagnostics and interact
with the boot loader or the operating system even if it fails to
configure the network.  The port uses 3.3V signals, so you will most
likely need a level converter; the figure below shows the connection
(black is GND, orange is RX and white is TX).
@sp 1
@center @image{wrs-v3-uart, 5cm}
@sp 1

@menu
* Installing from Jtag::        
* Installing with the USB Flasher::  
* Booting the Kernel::          
@end menu

@c ==========================================================================
@node Installing from Jtag
@section Installing from Jtag

You can boot and install the system using a JTAG debugger, although
the @i{USB Flasher} is currently the preferred technique. Each debugger
has its own command language, so you will need to adapt to yours. What is
shown here refers to the @i{peedi} tool.

As a first step, you will need to ensure the JTAG clock is slow enough.
The clock can be no faster than 1/6th of the CPU clock, so you need
3kHz at most (the G45 starts up with an internal oscillator, which has
an unpredictable value between 20kHz and 40kHz).  Then, I would verify
that the internal SRAM is working; I do that with cool food and bad
coffee instead of the usual smelly dead beef.

@example
   clock init
   mem write 0x300000 0xc001f00d
   mem write 0x300004 0xbadc0ffe
   mem read  0x300000 2
   ==> 0x300000: 0xC001F00D 0xBADC0FFE
@end example

Now, you can load your @i{at91bootstrap} to the internal SRAM,
retrieving it from the network or your host filesystem.  Since no boot
loader is there, you should place a breakpoint after @i{at91bootstrap}
initialized SDRAM and the PLL. Finally you can load @i{barebox} and jump
to it. Such step is better performed with the full JTAG clock, or
it would take several dozen minutes.

@example
  mem load at91bootstrap.bin 0x300000
  break add hard 0x300088
  go
  ## wait for the breakpoint to happen
  break del all
  clock normal
  mem load barebox.bin 0x73f00000
  go
@end example

On the serial port you will see the following messages at 115200,8N1.
The first 4 lines are printed by @i{at91bootstrap}, the rest by @i{barebox}.

@c FIXME: the messages and prompts
@smallexample
   Start AT91Bootstrap...
   Begin to load image...
   ++++++++
   Loading image done.


   barebox 2011.09.0-dirty (Sep 12 2011 - 17:09:12)

   Board: Ronetix PM9G45
   Clocks: CPU 400 MHz, master 133 MHz, main 12.000 MHz
   registered netconsole as cs1
   Malloc space: 0x73b00000 -> 0x73f00000 (size  4 MB)
   Stack space : 0x73af8000 -> 0x73b00000 (size 32 kB)
   Open /dev/env0 No such file or directory
   no valid environment found on /dev/env0. Using default environment
   running /env/bin/init...

   Hit any key to stop autoboot:  2
   barebox@@Ronetix PM9G45:/
@end smallexample

When the boot loader is running, you can boot a kernel and use its
own @i{/dev/mtd} devices to write to the DataFlash and NAND memories.

According to the partition table you have in your kernel sources, you will
see a different set of @i{mtd} files, but you can identify them by looking
at @code{/proc/mtd}:

@example
   # cat /proc/mtd
   dev:    size   erasesize  name
   mtd0: 04000000 00020000 "Partition 1"
   mtd1: 1c000000 00020000 "Partition 2"
   mtd2: 00840000 00000420 "spi0.0-AT45DB642x"
@end example

Here above, the DataFlash is @i{/dev/mtd2}, whereas the former partitions
refer to NAND memory.  You should then write @i{at91boot} to offset 0
of the DataFlash and @i{barebox} to offset 0x8400 (33792):

@example
   cat at91bootstrap.bin > /dev/mtd2
   dd bs=33792 seek=1 if=barebox.bin of=/dev/mtd2
@end example

Now you can detach the debugger, press reset and see @i{barebox} starting.

@c ==========================================================================
@node Installing with the USB Flasher
@section Installing with the USB Flasher

You can install the IPL and boot loader using the @i{USB-loader} program,
put together by Tomasz based on Atmel sources available to the public
with a permissive Free Software license.

In order to force the ROM to run the boot protocol, you must prevent
it from finding code in the @i{data-flash}. To do so you can short
pins 1 and 4 of the chip -- this shorts CS* and 5V to the ROM
can access to the memory's contents. I used two wires
to be shorted together as needed.  Then, you need to connect a USB
cable to the device socket near the JTAG connector. The next figure
shows both the shorted pins (the @i{dataflash} is on the left and there
are two arrowheads pointing to them) and the
USB cable (on the right).
@sp 1
@center @image{flasher, 14cm}
@sp 1

After shorting the pins you can press reset (the button near the USB
connector). If things go well, the device is enumerated as shown;
you must un-short the wires at this point or you will not be able
to write the new information to @i{dataflash}

@smallexample
   brezza% lsusb | grep Atmel
   Bus 001 Device 025: ID 03eb:6124 Atmel Corp. at91sam SAMBA bootloader
@end smallexample

The device should also appear as @code{/dev/ttyACM0} or equivalent.

Now you can un-short the @i{data flash} ping and program the IPL and
boot loader using the script @i{build/flash-wrs} provided in this
package.

The script is invoked as superuser in the following way, from the
toplevel directory of @code{wr-switch-sw}:

@example
   ./build/flash-wrs [uart-device] [xx:yy:zz:aa:bb:cc]
@end example

The @i{uart device} is optional. It defaults to @code{/dev/ttyACM0}.
If your system maps the Atmel ROM to a different device name, please
pass the name on the command line. The script wants a full pathname:
the leading @code{/} is used to tell the device argument from the
mac-address one).

The MAC address is optional; if not specified, the default
@code{02:0b:ad:c0:ff:ee} applies.  The script checks that the
address is properly formed before applying it to the @i{barebox}.

If you want to flash your own @i{at91boot.bin} or @i{barebox.bin} you
can just place them in the @i{binaries} subdirectory before calling
the script.

The script performs the following steps:

@itemize @bullet
@item It compiles the loader (@i{usb-loader/} subdir).
@item It picks @i{at91boot} and @i{barebox} from the @i{binaries} subdirectory.
@item It packs them together in a single binary image, with proper offsets,
@item Optionally, it changes the default MAC address in @i{barebox}
      default environment, so you can make all switches different.
@item It uses the @i{sam-ba} protocol to write to @i{dataflash}.
@end itemize

If all goes well, at the next reset you will find @i{barebox} accessing
the network with your preferred MAC address.

The output you must expect from the flasher is like the following, and
the process takes half a minute more or less:

@example
   Initializing SAM-BA: CPU ID: 0x819b05a2
   loading applet isp-extram-at91sam9g45 at 0x00300000
   Initializing DataFlash...
   loading applet isp-dataflash-at91sam9g45 at 0x00300000
   Programming DataFlash...
   Dataflash: Writing 279896 bytes at offset 0x0 buffer 70000000....ABCDE
   Fdone.
   Programming done!
@end example

If you look at the serial port, during programming, you will see
messages like these:

@example
   -I- Statup: PMC_MCKR 1202 MCK = 100000000 command = 0
   -I- -- EXTRAM ISP Applet 2.9 --
   -I- -- AT91SAM9G45-EK
   [...]
   -I- VERIFY at offset: 0x0 buffer at : 0x70055d28 of: 0x45d28 Bytes
   -I-     End of applet (command : 2 --- status : 0)
@end example

@c ==========================================================================
@node Booting the Kernel
@section Booting the Kernel

After initial installation, the DataFlash only includes up to @i{barebox},
while the rest must still be loaded.

Assuming you have a known-working NFS-Root installation and a TFTP
server, but you do not want to use DHCP because you already have one
in your network, the following commands will load a kernel and
boot it as NFS-Root. You will need to change IP addresses and names
to match your personal situation.

@c FIXME: kernel boot procedure
@example
   eth0.ipaddr=192.168.16.9
   eth0.serverip=192.168.16.1
   addpart /dev/mem  0x400000@@0x71000000(kernel)
   tftp zImage /dev/mem.kernel
   bootargs="$bootargs mem=64m root=nfs nfsroot=/opt/root/wrs,tcp "
   bootargs="$bootargs ip=192.168.16.9:192.168.16.1:255.255.255.0"
   bootz /dev/mem.kernel
@end example

Please note that the default boot command in the barebox binary with the
provided patches runs the following three commands:

@example
   dhcp
   tftp wrboot
   ./wrboot
@end example

This means that if you have both DHCP and TFTP, you can just write
your @i{barebox} commands in a file called @i{wrboot} in your TFTP
public location, and the commands will automatically executed at boot.
Actually, I am currently using the stanza shown above to boot my
switch over the network with no interaction at all, while being able
to change the boot procedure on the server while the switch is off.

To have a self-hosting switch, you should copy your
filesystem to NAND flash. See @ref{Installing the filesystem to NAND},
for a quick primer (the production procedure is not finalized at this
point).

@c ##########################################################################
@node Code layout in a production switch
@chapter Code layout in a production switch

This is the suggested arrangement of the various binary blobs in the
final switch. Any comment and suggestion is welcome, as nothing
is cast in stone at this point.

The basic idea is to have two systems installed: one is the production
system, that boots by default, and the other is the fail-safe system that
users will be able to boot if they messed up badly with the production
code base.  There will be a button to select the fail-safe system when
needed.

The two boot procedures will be like this:

@table @i

@item Fail-safe boot

	This is using the binaries that are already built by the
        package in its current status.  The IPL lives in @i{dataflash};
        if loads @i{barebox} from @i{dataflash} and the configuration
        fetches the boot procedure from the network.  This gives
        technicians complete freedom to boot whatever they want, either
        fore development or recovery purposes

@item Production boot

	During normal work the switch would boot from NAND flash. The
        IPL lives in NAND memory (the manufactures certifies that the
        first block can never be bad) and it loads a @i{barebox} binary
        from @i{dataflash}. This is a different @i{barebox} than the
        one used in fail-safe boot.  This @i{barebox} loads a kernel
        from NAND memory and runs it, mounting the root filesystem
        from another NAND partition.

@end table

With this setup, the @i{dataflash} can be made read-only when accessed
by Linux, for better safety.  Normal use cases will be able to change
either the kernel or the filesystem (or both) without touching
@i{dataflash}. On the other hand, @i{at91boot} is unable to deal with
bad blocks in NAND, or with correction of single-bit errors; that is
why it will load its @i{barebox} from @i{dataflash}.

The internal CPU ROM will boot from NAND if it finds the magic
signature in there. If no signature is there, it will bot from
@i{dataflash}.  Thus, the production boot will be entered by default;
it will boot the fail-safe sequence only if the NAND flash is erased
(or has never been programmed) or somebody pressed the spacial
@i{fail-safe} at power-up.  The button simply disables the
@i{chip-select} pin of NAND memory; the ROM will fail reading NAND and
will go on booting from @i{dataflash}.

We foresee no issues in making the @i{dataflash} read-only, because
user customization is expected to be limited to the kernel and the
filesystem; we therefor think that preventing accidental erasure of
@i{dataflash} is good policy.  Those users who want to replace their
@i{at91boot} or @i{barebox} will be able to do that anyways, either
from @i{barebox} itself or from a custom build of the kernel.

Later releases of this document will include the complete recovery
procedures, as well as building rules for the production versions of
@i{at91boot} and @i{barebox}.

@menu
* Installing the filesystem to NAND::  
@end menu

@c ==========================================================================
@node Installing the filesystem to NAND
@section Installing the filesystem to NAND

This is an initial procedure to install the filesystem to NAND memory
and set up a self-hosted switch to keep in your pocket for
demonstration purposes.  Pleae note this is not the production release,
as some shortcuts have been taken.

After a successful NFS-Root boot, you'll notice the system identified
two flash devices, for a total of 3 partitions. Note however that NAND
memory may have bad blocks:

@example
   # dmesg | grep -i mtd
   Creating 2 MTD partitions on "atmel_nand":
   mtd_dataflash spi0.0: AT45DB642x (8448 KBytes) pagesize 1056 bytes (OTP)

   # cat /proc/mtd
   dev:    size   erasesize  name
   mtd0: 04000000 00020000 "Partition 1"
   mtd1: 1c000000 00020000 "Partition 2"
   mtd2: 00840000 00000420 "spi0.0-AT45DB642x"

   # dmesg | grep -i bad
   Scanning device for bad blocks
   Bad eraseblock 2 at 0x000000040000
   Bad eraseblock 914 at 0x000007240000
   Bad eraseblock 2649 at 0x000014b20000
   Bad eraseblock 2944 at 0x000017000000

@end example

To set up a boot procedure that doesn't depend on the network,
we need to choose where the place the kernel and filesystem. To
avoid repartitioning the NAND memory let's use the two
partitions we have: mtd0 to fit the kernel and mtd1 for the filesystem.

Please note that the environment of @i{barebox} is stored in NAND
memory from 256k to 512k. The size is so big because @i{barebox}
will skip badblocks within that are.

The suggested setup, thus, is the following:

@example
   0x0000.0000 - 0x0004.0000    Empty (space for a barebox)
   0x0004.0000 - 0x0008.0000    Barebox environment
   0x0008.0000 - 0x0010.0000    Empty (up to 1MB)
   0x0010.0000 - 0x0090.0000    Kernel (plenty of space)
   0x0090.0000 - 0x0400.0000    Empty (up to the end of /dev/mtd0)
   0x0400.0000 - 0x2000.0000    Filesystem space, jffs2
@end example

With this in mind, the installation procedure is as follow:

@table @i

@item  Erase the filesystem partion if needed

	@code{flash_eraseall -j /dev/mtd1}

@item Mount as @i{jffs2}

        @code{mount -t jffs2 /dev/mtdblock1 /mnt}

@item Copy your filesystem data, possibly from the cpio archive

	@code{cd /mnt; zcat /tmp2/wrs-image.cpio.gz | cpio --extract}

	Here, you may want to copy the current NFS-Root filesystem,
        but the @i{cp} command we have on the target lacks the
	@code{-x} (@code{--one-file-system}) option.

@item Umount the filesystem

	@code{cd /; umount /mnt}

@item Erase space for the kernel

	@code{flash_erase /dev/mtd0 0x100000 0x40}

	Note that the offset is in bytes (1MB) and the lenght is in
        number of blocks (each block is 128kB

@item Copy the kernel to nand

	@code{cat /path/to/zImage /dev/zero | cat | dd bs=1k count=2048 > /tmp/k}
        @code{dd if=/tmp/k bs=128k of=/dev/mtd0 seek=8}

	The former command is needed to pad the kernel to a multiple
        of 128kB, as the flash has 128k-sized blocks. The size here
        is 2MB, which is more than enough for our current kernel.

	The latter command copies to @code{/dev/mtd0} as offset 1MB.

@end table

The following repeats the commands in a format that is simpler to cut
and paste to your serial port:

@smallexample
    flash_eraseall -j /dev/mtd1
    mount -t jffs2 /dev/mtdblock1 /mnt
    cd /mnt; zcat /tmp2/wrs-image.cpio.gz | cpio --extract
    cd /; umount /mnt
    flash_erase /dev/mtd0 0x100000 0x40
    cat /path/to/zImage /dev/zero | cat | dd bs=1k count=2048 > /tmp/k
    dd if=/tmp/k bs=128k of=/dev/mtd0 seek=8
@end smallexample

Now, we need to go to @i{barebox} and change the default boot command.
To do it, you need to call @code{edit /env/config} and later
@code{saveenv}.  The internal editor of @i{barebox} is pretty basic, but
it use the arrow keys (it is not like @i{vi}, for your pleasure but
not for mine). To save and exit use @i{ctrl-D}.


@c ##########################################################################
@node Troubleshooting
@chapter Troubleshooting

This chapter includes notes about issues we found and how to address them,
or extra tools 

@menu
* Testing SDRAM::               
@end menu

@c ==========================================================================
@node Testing SDRAM
@section Testing SDRAM

In case you suspect to have RAM issues and are not even able to boot
@i{barebox}, this package offers the program @i{g45memtest}. It is a
merge of @i{at91boot} and @code{commands/memtest.c} from barebox, with
the small @i{printf} I use in other projects (e.g. @i{pptp}).

The binary is included in the package as @code{binaries/g45memtest}.
It should be loaded like @code{at91boot}: it configures the RAM and
the PLL, and then runs the test.

The source is distributed as a patch-set over @i{at91boot} (the same
version we use to boot the switch), in @code{patches/g45memtest}.  To
compile, untar the source, apply the patches and compile.  The
following commands are what I used to compile it (you will need to set
@code{CROSS_COMPILE} first, though).

@example
   tar xzf /opt/wrs-build/downloads/at91bootstrap-3-3.0.tar.gz
   git am $WRS_BASE_DIR/../patches/g45memtest/00*
   make at91sam9g45ek_defconfig
   make
@end example

The output binary is the only file called ``@code{binaries/*.bin}'',
that should be renamed to @code{g45memtest}.


@c ##########################################################################
@bye


@c  LocalWords:  gnudd titlepage iftex texinfo CERN timestamping smallexample
@c  LocalWords:  LocalWords ietf timestamp misc timestamps ttstamp onestamp
@c  LocalWords:  Tomasz Wlostowski buildroot distclean defconfig wrswitch REPO
@c  LocalWords:  menuconfig config dataflash whiterabbit stdout stderr svnsync
@c  LocalWords:  filesystem diff ohwr http mkdir linux rubini itemize PTPd VHDL
@c  LocalWords:  noposix ptpd userspace libswitchhw DataFlash NAND barebox FPGA
@c  LocalWords:  Atmel
