Gadgetron Users Guide

The Gadgetron depends on a number of libraries that can either be downloaded for free or that may already be part of the installation on your workstation. If you are working on a Linux platform you should be able to install all dependencies without compiling anything. The following is a list of the components that you will need. Some are optional.

To install these components please follow the platform specific installation instructions provided below (Section 1.3, “Compiling and Installing Gadgetron”).

Linux is preferred operating system to get started with the Gadgetron. All of the required dependencies are included in most major Linux distributions and can be installed easily and without having to compile anything. In the following sections we walk you through the required steps to set up a full Gadgetron installation. We assume that you are starting with a freshly installed Ubuntu 11.10 available from the Ubuntu website (http://www.ubuntu.com). If you don't have a machine available for installing Ubuntu, you can always try it out in a virtual machine using virtualization software such as VirtualBox (https://www.virtualbox.org/).

If you would like to use the GPU components included in the Gadgetron and you have a GPU available on your system, please complete the CUDA/CULA installations as described in Section 1.3.1.1, “Installing GPU components (CUDA and CULA) on Linux”.

First install all dependencies for Gadgetron. The following will install everything you need:

user@mycomputer:~$ sudo apt-get install doxygen cmake \
 libqt4-dev libglew1.6-dev \
 docbook5-xml docbook-xsl-doc-pdf \
 docbook-xsl-doc-html docbook-xsl-ns xsltproc \
 fop git-core libboost-dev libboost-python-dev \
 libfftw3-dev libace-dev python-dev python-numpy \
 freeglut3-dev libxi-dev liblapack-dev build-essential   

Now download the Gadgetron archive and unpack it somewhere. If you have access to a git repository, you can get the code with:

user@mycomputer:~$ git clone ssh://user@hostname/path/gadgetron.git

Configure and build the Gadgetron:

user@mycomputer:~$ cd gadgetron
user@mycomputer:~/gadgetron$ mkdir build
user@mycomputer:~/gadgetron$ cd build
user@mycomputer:~/gadgetron/build$ cmake ../
user@mycomputer:~/gadgetron/build$ make  

Install (default location is /usr/local/gadgetron):

user@mycomputer:~/gadgetron/build$ sudo make install 

The final step is to add/modify a few environment variables in your ~/.bashrc file.

export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/gadgetron/lib
export PATH=$PATH:/usr/local/gadgetron/bin
export GADGETRON_HOME=/usr/local/gadgetron     

You are now set up to run a simple example reconstruction as outlined in Section 1.4, “Hello Gadgetron: Your First Image Reconstruction”.

user@mycomputer:~$ sudo apt-get install nvidia-current \
  nvidia-current-dev nvidia-current-updates \
  nvidia-current-updates-dev    

user@mycomputer:~$ sudo apt-get install \
  gcc-4.4 g++-4.4 build-essential   

user@mycomputer:~$ sudo update-alternatives \
 --install /usr/bin/gcc gcc /usr/bin/gcc-4.6 40 \
 --slave /usr/bin/g++ g++ /usr/bin/g++-4.6

user@mycomputer:~$ sudo update-alternatives \
 --install /usr/bin/gcc gcc /usr/bin/gcc-4.4 60 \
 --slave /usr/bin/g++ g++ /usr/bin/g++-4.4    

user@mycomputer:~$ gcc -v      

user@mycomputer:~$ sudo update-alternatives --config gcc      

user@c:~$ chmod +x cudatoolkit_4.0.17_linux_64_ubuntu10.10.run
user@c:~$ sudo ./cudatoolkit_4.0.17_linux_64_ubuntu10.10.run
user@c:~$ chmod +x cula_dense_free_R13a-linux64.run
user@c:~$ sudo ./cula_dense_free_R13a-linux64.run

export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/cuda/lib64
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/cuda/lib
export CULA_ROOT="/usr/local/cula"
export CULA_INC_PATH="$CULA_ROOT/include"
export CULA_BIN_PATH_32="$CULA_ROOT/bin"
export CULA_BIN_PATH_64="$CULA_ROOT/bin64"
export CULA_LIB_PATH_32="$CULA_ROOT/lib"
export CULA_LIB_PATH_64="$CULA_ROOT/lib64"
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$CULA_LIB_PATH_64    

The following instructions assume that you are starting on a Mac with OS X 10.6.8 (Snow Leopard) Installed. Additionally it assumes that you have Xcode (3.2.6) installed. If you have upgraded to Lion or are on an older release, you should still be able to make it all compile, but you may have to make some adjustments.

We will use MacPorts (http://www.macports.org/) to install the required dependencies. You may use a different package management system or prefer to install packages manually. In that case, please look at the list of dependencies (Section 1.2.1, “Dependencies”) and install the required dependencies for the components you would like to use.

MacPorts is not the fastest way to install packages. We use this methods here to make it easier to follow the instructions. Please be patient when running the port commands.

It is probably appropriate to start this section with a warning: Windows is not the easiest environment in which to work with the Gadgetron. As indicated in Section 1.2.1, “Dependencies”, the Gadgetron relies on multiple external libraries. Many of those libraries are not available as easy install packages and must be compiled separately. If you are uncomfortable setting up development tools on Windows, or if you are just looking for a fast and easy way to get started with the Gadgetron, we recommend installing on Ubuntu Linux - possibly using a virtual machine inside Windows (see Section 1.3.1, “Linux Installation Instructions”).

The following is a list of steps we have used to install the Gadgetron on a clean Windows 7 (64-bit) machine:

You now have a working installation of the Gadgetron in Windows. Follow the instructions below to run a simple reconstruction example (Section 1.4, “Hello Gadgetron: Your First Image Reconstruction”).

Before attempting to run any reconstructions, please set the environment variable GADGETRON_HOME to point to the installation folder of your Gadgetron installation and make sure that the paths of all dependencies are in your PATH environment variable.

A Gadget is the functional unit of the Gadgetron. You can think of the Gadget as a device with an input and output. Data passes through the device and is modified and/or transformed between input and output. By wiring multiple Gadgets together you create a reconstruction program. A schematic outline of a Gadget is seen in Figure 2.2, “Gadget”

Figure 2.2. Gadget

The Gadget is an active object based on the ACE_Task from the ACE library. It has its own thread (or threads) of execution and an input queue where data is placed for processing by either the Gadgetron framework or an upstream Gadget.

The active thread(s) in the Gadget will pick up a data package from the queue, and then pass it on to a virtual process. An abbreviated version of the header Gadget.h is seen below:

class Gadget : public ACE_Task<ACE_MT_SYNCH>
{

public:
   virtual int svc(void)
   {
      //Pick up package from queue
     
      //Call process
      if (this->process(m) == -1) {
         //Handle error
      }
      return 0;
   }

   //More function (left out for simplicity)

protected:
   virtual int process(ACE_Message_Block * m) = 0;

   virtual int process_config(ACE_Message_Block * m) {
      return 0;
   }

};

The data package used by the ACE_Task is the ACE_Message_Block, which is a very basic block of data (essentially just a byte array). To allow the Gadgets to check if the data blocks on the message queue are of the expected type, the Gadgetron uses a modified ACE_Message_Block called GadgetContainerMessage, which can contain any class with a no-argument constructor. It is possible to check if the GadgetContainerMessage contains a specific type of data, and if so, access that object. Suppose we want to store a class named MyClass:

GadgetContainerMessage<MyClass>* m = 
  new GadgetContainerMessage<MyClass>();

MyClass* mc = m->getObjectPtr();

//Do something with mc

m->release(); //Delete the message block and containing data

When a function receives an ACE_Message_Block it is possible to check if it is of a certain type:

int process(ACE_Message_Block* mb)
{
  
  GadgetContainerMessage<MyClass>* m = 
    AsContainerMessage<MyClass>(mb);

  if (m) {
    MyClass* mc = m->getObjectPtr();
    
    //Do something with mc

  } else {
    //Something went wrong, deal with error
    return -1;
  }

  mb->release();

  return 0;
}

It is possible to chain more than one ACE_Message_Block together using the cont function. This effectively provides a way to pass multiple arguments into a Gadget and checking if they have the appropriate types:

int process(ACE_Message_Block* mb)
{
  
  GadgetContainerMessage<MyClass>* m1 = 
    AsContainerMessage<MyClass>(mb);

  GadgetContainerMessage<MyOtherClass>* m2 = 
    AsContainerMessage<MyOtherClass>(mb->cont());

  if (m1 && m2) {
    MyClass* mc = m1->getObjectPtr();
    MyOtherClass* moc = m2->getObjectPtr();
    
    //Do something with mc

  } else {
    //Something went wrong, deal with error
    return -1;
  }

  mb->release(); //This deletes both message blocks

  return 0;
}

It gets a bit tedious and error prone to repeat code like the above in every Gadget. To overcome this, the Gadgetron comes with a set of templated classes to automate the steps. Say we would like to make a Gadget which takes a single input argument, we would inherit from Gadget1. If you need two arguments, you inherit from Gadget2:

template <class P1, class P2> class Gadget2 : public Gadget
{
protected:
   int process(ACE_Message_Block* mb)
   {
     //Do type checking 
   }

   virtual int process(GadgetContainerMessage<P1>* m1, 
     GadgetContainerMessage<P2>* m2) = 0;
};

The base class performs the type checking for you and only when the arguments have been verified, it will call the virtual process above. So, all you need to do in order to implement a Gadget that takes two arguments is to implement this function. As an example, let's look at a very simple Gadget, which receives an image header and some image data and does a Fourier transform of the first 3 dimensions. First the header file FFTGadget.h

#include "gadgetroncore_export.h"
#include "Gadget.h"
#include "GadgetMRIHeaders.h"
#include "hoNDArray.h"
#include <complex>

class EXPORTGADGETSCORE FFTGadget : 
public Gadget2<GadgetMessageImage, hoNDArray< std::complex<float> > >
{
 public:
  GADGET_DECLARE(FFTGadget)

 protected:
  virtual int process( 
     GadgetContainerMessage< GadgetMessageImage>* m1,
     GadgetContainerMessage< hoNDArray< std::complex<float> > >* m2);

};

Let us walk through the code step by step. The Gadget takes two arguments: 1) GadgetMessageImage, which is just a struct with some image header information (it is defined in GadgetMRIHeaders.h), and 2) a hoNDArray, which is a multidimensional array (see Section 2.2.1, “NDArray”) storage container. In this case the hoNDArray contains complex floating point data.

There are a couple of other things to notice. One is the EXPORTGADGETSCORE macro in the class definition. This is needed to make things work properly on Windows. It is defined in gadgetroncore_export.h and is used on Windows to indicate if the class is being imported or exported from a DLL. It translates into __declspec(dllexport) or __declspec(dllimport) in Windows and is empty in Linux/OSX. It is beyond the scope of this manual to go into why such a declaration is needed, but keep this in mind when you start creating your own Gadgets. Each shared library (DLL) has its own export declaration macro.

The other thing to notice is the GADGET_DECLARE(FFTGadget) macro. This macro is required for Windows to correctly handle shared libraries and is needed whenever you create a new Gadget to make things work properly on Windows.

The actual implementation looks like this:

#include "FFTGadget.h"
#include "FFT.h"

int FFTGadget::process( 
   GadgetContainerMessage< GadgetMessageImage>* m1,
   GadgetContainerMessage< hoNDArray< std::complex<float> > >* m2)
{
  FFT<float>::instance()->ifft(m2->getObjectPtr(),0);
  FFT<float>::instance()->ifft(m2->getObjectPtr(),1);
  FFT<float>::instance()->ifft(m2->getObjectPtr(),2);

  if (this->next()->putq(m1) < 0) {
     return GADGET_FAIL;
  }

  return GADGET_OK;
}

GADGET_FACTORY_DECLARE(FFTGadget)

Once we are inside the process function, the data has already been converted to the appropriate container messages and we can start processing the data. This function uses an FFT toolbox (more on toolboxes in Section 2.2, “Gadgetron Toolboxes”). After the data has been Fourier transformed along the first 3 dimensions it is placed on the next Gadgets queue. Remember the two GadgetContainerMessage objects were originally picked up from the message queue as a chain of ACE_Message_Block objects. They are still chained together, i.e. when passing m1 on to the next Gadget we are effectively passing on both arguments.

Another couple of macros to notice are the GADGET_OK and GADGET_FAIL. They are defined as 0 and -1 respectively. The convention in the Gadgetron is to return 0 when a function succeeds and < 0 when it fails - unless the function returns a pointer.

Last thing to notice is the GADGET_FACTORY_DECLARE(FFTGadget) statement. This is a macro which declares functions for loading a Gadget of this type out of a shared library (DLL) and destroying it again when we are done. It ensures that we can load the Gadget on all platforms. When you create your own gadgets you should use this macro to declare the factory function for the Gadget.

For a tutorial on how to make your own Gadget library see Section 2.3.3, “Making a new Gadget Library”.

<?xml version="1.0" ?>
<gadgetron>
    <encoding>
        <kspace>
            <base_resolution>
                <value>128</value>
            </base_resolution>
            <readout_length>
                <value>256</value>
            </readout_length>
            <phase_encoding_lines>
                <value>128</value>
            </phase_encoding_lines>
            <partitions>
                <value>64</value>
            </partitions>
            <phase_resolution>
                <value>1</value>
            </phase_resolution>
            <slice_resolution>
                <value>1</value>
            </slice_resolution>
            <matrix_size>
                <value>128</value>
                <value>128</value>
            </matrix_size>
        </kspace>
        <trajectory>
            <value>1</value>
        </trajectory>
        <slices>
            <value>1</value>
        </slices>
        <acquisition_dwell_time_ns>
            <value>7800</value>
        </acquisition_dwell_time_ns>
        <channels>
            <value>32</value>
        </channels>
    </encoding>
    <timing>
        <TR>
            <value>5860</value>
        </TR>
        <TE>
            <value>2960</value>
        </TE>
    </timing>
</gadgetron>

class GadgetXMLNode
{
 public:
  GadgetXMLNode(TiXmlNode* anchor);

  //Other functions
 
  template<typename T> std::vector<T> get(std::string name);
}

int MyGadget::process_config(ACE_Message_Block* mb)
{

  //Use TinyXML to parse the doucment
  TiXmlDocument doc;
  doc.Parse(mb->rd_ptr());
 
  std::string path("gadgetron.encoding.kspace.matrix_size.value");

  //Get a vector with all the values 
  std::vector<long> dims = 
    n.get<long>(path);

  //Do something with the dimensions


  return GADGET_OK;
}

As illustrated in Figure 2.1, “Gadgetron Architecture” the Gadgetron uses a set of Readers and Writers to deal with the incoming communication on the TCP/IP socket. Readers are responsible for deserialization of packages and Writers are responsible for serialization of packages. All packages that arrive on the socket will start with a message ID. Based on this ID, the Gadgetron delegates the responsibility of reading the package of the socket to a particular instance of a GadgetMessageReader defined by the following abstract class:

class GadgetMessageReader
{
 public:
  virtual ACE_Message_Block* read(ACE_SOCK_Stream* stream) = 0;
};

In order to be able to read a specific type of data, the read function must be implemented for that data type. As an example here is the GadgetMessageReader, which reads an MRI data acquisition from the socket.

class EXPORTGADGETSCORE MRIAcquisitionReader 
: public GadgetMessageReader
{
 public:
  GADGETRON_READER_DECLARE(MRIAcquisitionReader);
  virtual ACE_Message_Block* read(ACE_SOCK_Stream* socket);
};

Note the GADGETRON_READER_DECLARE(MRIAcquisitionReader) declaration. This is equivalent to the declaration needed for the Gadgets (see Section 2.1.1, “Gadgets”) in order to make them load properly from shared libraries.

The implementation of this particular reader is as follows (this is an abbreviated version without error checking, etc.):

ACE_Message_Block* MRIAcquisitionReader::read(ACE_SOCK_Stream* sock)
{

   GadgetContainerMessage<GadgetMessageAcquisition>* m1 = 
      new GadgetContainerMessage<GadgetMessageAcquisition>();
  
   GadgetContainerMessage<hoNDArray< std::complex<float> > >* m2 = 
      new GadgetContainerMessage< hoNDArray< std::complex<float> > >();
  
   m1->cont(m2);

   sock->recv_n(m1->getObjectPtr(), sizeof(GadgetMessageAcquisition);

   std::vector<unsigned int> adims;
   adims.push_back(m1->getObjectPtr()->samples);
   adims.push_back(m1->getObjectPtr()->channels);

   m2->getObjectPtr()->create(&adims)
  
   long data_length = sizeof(std::complex<float>)*
      m1->getObjectPtr()->samples*m1->getObjectPtr()->channels;
      
   sock->recv_n (m2->getObjectPtr()->get_data_ptr(), data_length);
   
   return m1;
}

GADGETRON_READER_FACTORY_DECLARE(MRIAcquisitionReader)

The Reader allocates two GadgetContainerMessage data blocks to contain the incoming data. First an MRI acquisition header (defined in GadgetMRIHeaders.h) is read. Based hereon the length of each acquisition (number of samples) and the number of acquisition channels are determined. A hoNDArray is allocated to store the data read from the socket. Notice that the two GadgetContainerMessage are chained together using the cont function.

A final important statement to notice is:

GADGETRON_READER_FACTORY_DECLARE(MRIAcquisitionReader)

This macro declares create and destroy functions to load the reader from a shared library on all platforms supported.

Whereas the Readers are responsible for deserialization, The GadgetMessageWriter is responsible for the opposite operation (serialization). In practice, Gadgets that produce an output for the client application can hand that data back to the Gadgetron framework where it is placed on the output queue along with a message ID. This is for instance done in this (abbreviated) code from an ImageFinishGadget:

int ImageFinishGadget
::process(GadgetContainerMessage<GadgetMessageImage>* m1,
  GadgetContainerMessage< hoNDArray< float > >* m2)
{
   GadgetContainerMessage<GadgetMessageIdentifier>* mb =
      new GadgetContainerMessage<GadgetMessageIdentifier>();

   mb->getObjectPtr()->id = GADGET_MESSAGE_IMAGE_REAL_FLOAT;

   mb->cont(m1);

   int ret =  this->controller_->output_ready(mb);

   if ( (ret < 0) ) {
      return GADGET_FAIL;
   }

  return GADGET_OK;
}

Notice that the Gadget has a reference to the Gadgetron framework through the controller_ member variable, which is set during initialization.

In the framework (more specifically in the GadgetStreamController) there is an active thread responsible for writing messages that are put on to the output queue. This is done by investigating the message ID and then picking the GadgetMessageWriter associated with this ID. A Writer must implement the following abstract class:

class GadgetMessageWriter
{
 public:
  virtual int write(ACE_SOCK_Stream* stream, 
                    ACE_Message_Block* mb) = 0;
};

The Writer is handed control of the socket along with the message block. A Writer declaration could look like:

class MRIImageWriter : public GadgetMessageWriter
{

public:
   GADGETRON_WRITER_DECLARE(GadgetMessageWriter);
   virtual int write(ACE_SOCK_Stream* sock, 
                     ACE_Message_Block* mb);
};

Notice again the GADGETRON_WRITER_DECLARE(GadgetMessageWriterFLOAT) which ensures proper run-time linking behavior. The implementation could look like (abbreviated with no error checking, etc.):

int MRIImageWriter::write(ACE_SOCK_Stream* sock, 
                          ACE_Message_Block* mb)
{

   GadgetContainerMessage<GadgetMessageImage>* imagemb = 
      AsContainerMessage<GadgetMessageImage>(mb);
  
   GadgetContainerMessage< hoNDArray< float > >* datamb =
      AsContainerMessage< hoNDArray< float > >(imagemb->cont());
  
   if (!datamb || !imagemb) {
      //Deal with errors
   }
   
   GadgetMessageIdentifier id;
   id.id = GADGET_MESSAGE_IMAGE_REAL_FLOAT;
 
   sock->send_n (&id, sizeof(GadgetMessageIdentifier));

   sock->send_n (imagemb->getObjectPtr(), sizeof(GadgetMessageImage));

   sock->send_n (datamb->getObjectPtr()->get_data_ptr(), 
      sizeof(float)*datamb->getObjectPtr()->get_number_of_elements());

   return 0;
}

GADGETRON_WRITER_FACTORY_DECLARE(MRIImageWriter)

Once again notice the required GADGETRON_WRITER_FACTORY_DECLARE(MRIImageWriter) macro. Also notice that the message ID is transmitted to the client. The client is expected to follow the same communication model as the Reader, but it is determined entirely by the Writer implementation how the message is transmitted.

Readers and Writers are loaded dynamically at run-time along with the Gadgets (see Section 2.1.3, “Stream Configuration”). The input and output behaviour can be adapted by manipulating which Readers and Writers are associated with which message IDs.

A Gadgetron reconstruction is made up of modules, i.e. Readers, Writers, and Gadgets. New reconstruction programs can be created by simply assembling existing components in a new way. The configuration of the Gadgetron stream is done at run-time and new configuration chains can be created without recompiling any of the underlying Gadgets. More specifically, the configuration is specified in an XML file that the Gadgetron will read before receiving data. The best way to explain the format is by looking at a (simplified) example:

<?xml version="1.0" ?>  
<gadgetron>
 
 <readers>
   <reader>
    <slot>1001</slot>
    <dll>gadgetroncore</dll>
    <class>MRIAcquisitionReader</class>
   </reader>
 </readers>
  
 <writers>
  <writer>
   <slot>1005</slot>
   <dll>gadgetroncore</dll>
   <class>MRIImageWriterFLOAT</class>
  </writer>
  <writer>
   <slot>1006</slot>
   <dll>gadgetroncore</dll>
   <class>MRIImageWriterUSHORT</class>
  </writer>
 </writers>

 <stream>
  <gadget>
   <name>Accumulator</name>
   <dll>gadgetroncore</dll>
   <class>AccumulatorGadget</class>
  </gadget>

  <gadget>
   <name>FFT</name>
   <dll>gadgetroncore</dll>
   <class>FFTGadget</class>
  </gadget>

  <gadget>
    <name>ExtractMAG</name>
    <dll>gadgetroncore</dll>
    <class>ExtractGadget</class>
  </gadget>

  <gadget>
   <name>ImageFinishFLOAT</name>
   <dll>gadgetroncore</dll>
   <class>ImageFinishGadgetFLOAT</class>
  </gadget>
 </stream>

</gadgetron>

The configuration file format contains 3 sections: 1) Readers, 2) Writers, 3) Stream (with Gadgets) corresponding to the 3 different types of components that can be assembled in the Gadgetron.

In the example above, the Readers section contains only one reader, which is the MRIAcquisitionReader mentioned previously. The message ID associated with this Reader is 1001. Every time a message with ID 1001 arrives on the socket, responsibility for reading the message will be delegated to the MRIAcquisitionReader. When the Gadgetron configuration is loaded, the framework will load the MRIAcquisitionReader from the DLL (shared library) gadgetroncore. On the Linux platform this would be a shared library called libgadgetroncore.so and on the Windows platform it would be called gadgetroncore.dll.

The Gadgetron framework knows how to load the components from the DLLs assuming that they have been declared properly as described in Section 2.1.2, “Readers and Writers” and Section 2.1.1, “Gadgets”.

The example Gadgetron configuration has two Writers, i.e. it is capable of outputting two different types of data. Again the declarations cause the Gadgetron framework to load specific instances of GadgetMessageWriter and associate them with specific ID numbers.

There are certain built-in Readers and Writers in addition to those specified in the configuration file. As an example, there are Readers for receiving configurations to be used by the Gadgetron and for receiving the parameters that will be passed to all Gadgets (see Section 2.1.4, “Communication Sequence”). If the Gadgetron receives a message with an ID for which there is no associated Reader or encounters a message on the output queue for which there is no associated Writer an error will be generated, the Gadgetron stream shuts down, and the connection to the client will be closed.

The <stream> section of the configuration specifies which Gadgets to load. In the example above, we have 4 Gadgets. In this specific case the reconstruction is an MRI reconstruction and the first Gadget is an AccumulatorGadget, which collects individual lines and inserts them in k-space. When the k-space image is complete it is sent to the next Gadget in the chain, the FFTGadget, which is responsible for Fourier transforming the data into image space. The next Gadget (ExtractGadget) will extract the magnitude of the complex image. Finally the last Gadget in the chain (ImageFinishGadgetFLOAT) sends the reconstructed image back to the Gadgetron framework where it is added to the output queue.

It is also possible to send configuration parameters to Gadgets using the XML file. For example, to set a parameter in a Gadget, one could write:

  <gadget>
   <name>Accumulator</name>
   <dll>gadgetroncore</dll>
   <class>AccumulatorGadget</class>
   <property><name>MyTestProperty</name>
   <value>Blah Blah</value></property>
   <property><name>MyTestProperty2</name>
   <value>98776.862187</value></property>
  </gadget>

The two properties will now be accessible inside the Gadget using the parameter access functions defined in Gadget.h:

class Gadget : public ACE_Task<ACE_MT_SYNCH>
{

//Other definitions

int get_bool_value(const char* name);
int get_int_value(const char* name);
double get_double_value(const char* name);

};

Additionally it is also possible to specify how many active threads there should be in a Gadget. This is specified with:

  <gadget>
   <name>Accumulator</name>
   <dll>gadgetroncore</dll>
   <class>AccumulatorGadget</class>
   <property><name>threads</name><value>5</value></property>
  </gadget>

Which would make the AccumulatorGadget have 5 threads.

Communication between a client and the Gadgetron follows a straightforward communication protocol. When the Gadgetron is started it will be expecting a connection on a specific port (port 9002 is the default). The communication sequence is as follows:

To make it easier to create a new client, the Gadgetron comes with a GadgetronConnector class:

class GadgetronConnector: 
  public ACE_Svc_Handler<ACE_SOCK_STREAM, ACE_MT_SYNCH> {

public:

 int open (std::string hostname, std::string port);   
 int putq  (ACE_Message_Block * mb ,  
     ACE_Time_Value *  timeout = 0);

 int register_reader(unsigned int slot, 
     GadgetMessageReader* reader);

 int register_writer(unsigned int slot, 
     GadgetMessageWriter* writer);

 int send_gadgetron_configuration_file(std::string config_xml_name);   
 int send_gadgetron_configuration_script(std::string config_xml_name);
 int send_gadgetron_parameters(std::string xml_string);
};

This class can be used to create simple clients that open a connection with the Gadgetron using the open function and then communicate with the Gadgetron through the Readers and Writers registered with the connector. See the mriclient example application (gagetron/apps/clients/mriclient in the source code archive) for a simple example of how to build a Gadgetron client.

This section provides a brief overview of the file organization in the Gadgetron installation. Once you have compiled the Gadgetron and installed it (see Section 1.3, “Compiling and Installing Gadgetron”), it will reside in its designated installation folder (GADGETRON_HOME). For the purposes of this description, we will assume that the Gadgetron was installed in /usr/local/gadgetron.

In GADGETRON_HOME you should find the following folders:

Most image processing operations involve multi-dimensional arrays. Although the Gadgetron framework does not impose any specific array structure on the user, it does come with an abstract multi-dimensional array used throughout, the NDArray. It has a specific implementation for the CPU (hoNDArray) and GPU (cuNDArray). The abstract class definition looks like (abbreviated version):

template <class T> class NDArray
{
 public:
  
  NDArray ();

  virtual ~NDArray();
  
  virtual T* create(std::vector<unsigned int> *dimensions); 

  virtual T* create(std::vector<unsigned int> *dimensions, 
                    T* data, bool delete_data_on_destruct = false);

  virtual int permute(std::vector<unsigned int> *dim_order,
                      NDArray<T> *out = 0, int shift_mode = 0);
  
  inline unsigned int get_number_of_dimensions() const {
    return dimensions_->size();
  }

  unsigned int get_size(unsigned int dimension);

  boost::shared_ptr< std::vector<unsigned int> > get_dimensions();
  
  inline T* get_data_ptr() const { 
    return data_; 
  }
  
  inline unsigned long int get_number_of_elements() const {
    return elements_;
  }

  //Other public functions....such as permute, etc.

protected:

  virtual int allocate_memory() = 0;
  virtual int deallocate_memory() = 0;

  //Other private functions
  
};

The CPU (host) definition would look like (abbreviated):

template <class T> class hoNDArray : public NDArray<T>
{

public:
   //Public functions...

protected:
   virtual int allocate_memory();
   virtual int deallocate_memory();
};

As is seen from the NDArray header file, this class has a no-argument constructor, which makes it suited for encapsulating in the GadgetContainerMessage mentioned in Section 2.1.1, “Gadgets”. The procedure for creating an array with complex float values would look something like this:

#include <hoNDArray.h>
#include <complex>

hoNDArray< std::complex<float> > myArray;

std::vector<unsigned int> dimensions;
dimensions.push_back(128);
dimensions.push_back(128);

if(!myArray.create(&dimensions)) {
   //Deal with errors
}

//process data

To create an NDArray contained in a GadgetContainerMessage would look something like this:

GadgetContainerMessage< hoNDArray< std::complex<float> > >* m = 
  new GadgetContainerMessage< hoNDArray< std::complex<float> > >();

std::vector<unsigned int> dimensions;
dimensions.push_back(128);
dimensions.push_back(128);

if(!m->getObjectPtr()->create(&dimensions)) {
   //Deal with errors
}

//Process data or pass on to other Gadget, etc. 

m->release(); //Delete the message block and containing data

As mentioned in Section 2.1.1, “Gadgets”, the GadgetContainerMessage is a specialized version of the ACE_Message_Block class from the ACE framework. Data is passed between Gadgets in the form of ACE_Message_Blocks and Gadgets have access to utility functions that allow them to test if a given ACE_Message_Block is in fact a partcular type of GagetContainerMessage.

template <class T> class cuNDArray : public NDArray<T>
{
 public:
  cuNDArray();

  cuNDArray(const cuNDArray<T>& a);

  // Constructor from hoNDArray
  cuNDArray(hoNDArray<T> *a);

  // Assignment operator
  cuNDArray& operator=(const cuNDArray<T>& rhs);
  
  virtual ~cuNDArray();

  virtual T* create(std::vector<unsigned int> *dimensions);

  virtual T* create(std::vector<unsigned int> *dimensions, 
                    int device_no);

  virtual T* create(std::vector<unsigned int> *dimensions, 
                    T* data, bool delete_data_on_destruct = false);

  virtual boost::shared_ptr< hoNDArray<T> > to_host() const;
  
  virtual int set_device(int device_no);
  inline int get_device() { return device_; }
  
 protected:
  
  int device_; 
  virtual int allocate_memory();
  virtual int deallocate_memory();
  
};

The class vector_td provides a basic representation of complex numbers as well as one-, two-, three-, or four-dimensional vectors (positions). It is templetized with the datatype T and dimensionality D. Also we provide a set of typedefs to commonly encountered instances. A subset of the definitions provided in vector_td.h is provided here:

template<class T, unsigned int D> struct vector_td
{
  T vec[D];
};

// Some typedefs for convenience

template< class REAL, unsigned int D > struct reald{
  typedef vector_td< REAL, D > Type;
};

template< unsigned int D > struct intd{
  typedef vector_td< int, D > Type;
};

template< unsigned int D > struct uintd{
  typedef vector_td< unsigned int, D > Type;
};

template< unsigned int D > struct floatd{
 typedef typename reald< float, D >::Type Type;
};

template< unsigned int D > struct doubled{
  typedef typename reald< double, D >::Type Type;
};

template< class T > struct complext{
  typedef vector_td< T, 2 > Type;
};

struct float_complext : complext<float>::Type {
  typedef complext<float>::Type Type;
  float_complext(){}
  float_complext( float r, float i ) { vec[0] = r; vec[1] = i; }
};

struct double_complext : complext<double>::Type {
  typedef complext<double>::Type Type;
  double_complext(){}
  double_complext( double r, double i ) { vec[0] = r; vec[1] = i; }
};      

A substancial subset of the C++ arithmetic operators on the vector_td are defined in vector_td_operators.h. They are convienient when performing algebra on complex numbers or spatial vectors. Similarly, the header vector_td_utilities.h wraps common math functionality for the vector_td class. Many common operations that takes one of more cuNDArray instances with element type vector_td are defined in ndarray_vector_utilities.h. We leave it for the readers to explore these utilities on their own.

The vector_td can be used in both host and device code. As an example of use it is contained in the interface of the non-Cartesian FFT described in Section 2.2.3.2, “Non-Cartesian FFT”.

A fundamental building block of most image reconstruction algorithms is the abstract matrix_operator. A range of linear imaging and regularization operators are inherited from this pure virtual base class:

template <class REAL, class ARRAY_TYPE> class matrixOperator
{
 public:

  matrixOperator() { weight_ = get_one<REAL>(); }
  virtual ~matrixOperator() {}

  inline void set_weight( REAL weight ){ weight_ = weight; }
  inline REAL get_weight(){ return weight_; }

  virtual int mult_M( ARRAY_TYPE* in, ARRAY_TYPE* out, 
                      bool accumulate = false) = 0;
  
  virtual int mult_MH( ARRAY_TYPE* in, ARRAY_TYPE* out, 
                       bool accumulate = false) = 0;
  
  virtual int mult_MH_M( ARRAY_TYPE* in, ARRAY_TYPE* out, 
                         bool accumulate = false) = 0;
  
private:
  REAL weight_;
}

The MatrixOperator is templated by two arguments: 1) the basic precision REAL (e.g. float or double) and 2) the ARRAY_TYPE (e.g. NDArray, hoNDArray, or cuNDArray) representing the expected vector format for the matrix-vector multiplication the operator implements.

Every MatrixOperator has an associated weight that is used to balance multiple matrix terms when added to a cost function (see Section 2.2.5, “Linear Solvers”).

The main functionality is provided in the three pure virtual functions, mult_M, mult_MH, and mult_MH_M, denoting multiplication with the matrix operator (M), multiplication with the adjoint (i.e. conjugate transpose) of the matrix operator (MH), and an "iteration" of the two respectively (MHM).

The MatrixOperator is used to model a linear imaging modality's encodig operation (Fourier transform for MRI, Radon transform for CT, convolution for Microscopy etc.) but also to common regularization operators such the identity matrix, the partial derivatives etc.

Here follows a list that briefly describes the matrix operators in Gadgetron that have been used for the reconstruction examples provided later in this document (Chapter 3, Gadgetron Applications, Chapter 4, Standalone Applications).

The Gadgetron 'solvers' toolbox contain both a generic conjugate gradient solver to solve linear least squares reconstruction problems (see Section 2.2.5, “Linear Solvers” ) and a two flavors of a Split Bregman solver for non-linear problems using l1-norms for regularization (see Section 2.2.6, “Non-linear Solvers”).

2.2.5.1. Conjugate Gradient Method for Linear Least Squares

The conjugate gradient solver is used to reconstruct an image posed as a minimizer to an l2-based optimization problem:

The unknown image to be reconstructed is denoted here by u and the measured data by m. E is a linear operator modelling the encoding of the imaging modality (e.g. a Fourier transform for MRI, a Radon transform for CT etc.). R is a regularization operator often required to ensure uniqueness of the solution. Lambda is a scalar weight (with a default value of one) associated to each matrix operator and used to balance the various terms in the cost function. Finally p denotes some (possibly blank) prior image in the regularization term.

The closed form solution to the optimization problem is given by the linear system of equations:

Put extremely short you set up and run a solver by 1) adding the corresponding matrix operators to the solver, 2) computing the right hand side of the equation above (probably using the mult_MH function in the matrix operators), and 3) running the solve function in the solver providing the right hand side as input argument.

An abbreviated version of the interface to the conjugate gradient solver is shown here

// Defined in solver.h

template <class ARRAY_TYPE> class solver
{
 public:

  enum solverOutputModes { OUTPUT_SILENT = 0, 
                           OUTPUT_WARNINGS = 1, 
                           OUTPUT_VERBOSE = 2, 
                           OUTPUT_MAX = 3 };

  solver( int output_mode = OUTPUT_SILENT ) { 
    output_mode_ = output_mode; }
  
  virtual ~solver() {}
  
  virtual void solver_error( std::string err ) { 
    std::cerr << err << std::endl; }

  virtual void set_output_mode( int output_mode ) {
    if( !(output_mode >= OUTPUT_MAX || output_mode < 0 )) {
      output_mode_ = output_mode;
    }
  }

  virtual boost::shared_ptr<ARRAY_TYPE> solve( ARRAY_TYPE* ) = 0;

protected:
  int output_mode_;
};

// Defined in cgSolver.h

template <class REAL, class ELEMENT_TYPE, class ARRAY_TYPE> 
class cgSolver : public solver<ARRAY_TYPE>
{
public:

  cgSolver( int output_mode = solver<ARRAY_TYPE>::OUTPUT_SILENT ) 
    : solver<ARRAY_TYPE>( output_mode ) { 
    // Initialize member variables
  }

  virtual ~cgSolver() {}

  virtual int add_matrix_operator( 
    boost::shared_ptr< matrixOperator<REAL, ARRAY_TYPE> > op ) 
    { ... }

  virtual int set_preconditioner( 
    boost::shared_ptr< cgPreconditioner<ARRAY_TYPE> > precond ) 
    { ... }

  virtual void set_limit( REAL limit ) { ... }

  virtual void set_iterations( unsigned int iterations ) 
    { ... }

  virtual bool pre_solve( ARRAY_TYPE **rhs ) { return true; }
  virtual bool post_solve( ARRAY_TYPE **rho ) { return true; }

  virtual bool solver_clear( ARRAY_TYPE* ) = 0;
  virtual bool solver_scal( ELEMENT_TYPE, ARRAY_TYPE* ) = 0;

  virtual bool solver_axpy( ELEMENT_TYPE, ARRAY_TYPE*, 
    ARRAY_TYPE* ) = 0;

  virtual bool solver_dump( ARRAY_TYPE *rho ) { return true; }

  virtual ELEMENT_TYPE solver_dot( ARRAY_TYPE*, ARRAY_TYPE* ) = 0;

  virtual boost::shared_ptr<ARRAY_TYPE> solve( ARRAY_TYPE *_rhs )
  {
     // The conjugate gradient solver code is found here...
  }

protected:
  boost::shared_ptr< std::vector< 
    boost::shared_ptr< matrixOperator<REAL, ARRAY_TYPE> > > > operators_;
  boost::shared_ptr< cgPreconditioner<ARRAY_TYPE> > precond_;
  unsigned int iterations_;
  REAL limit_;
};

// Defined in cuCGSolver.h

template <class REAL, class T> 
class cuCGSolver : public cgSolver< REAL, T, cuNDArray<T> >
{
public:

  cuCGSolver() : cgSolver< REAL, T, cuNDArray<T> >() {}
  virtual ~cuCGSolver() {}

  virtual bool pre_solve(cuNDArray<T> **rhs)
   { ... }

  virtual bool post_solve(cuNDArray<T>**)
   { ... }

  virtual void solver_error( std::string err )
   { ... }

  virtual T solver_dot( cuNDArray<T> *x, 
   cuNDArray<T> *y ){ ... }

  virtual bool solver_clear( cuNDArray<T> *x )
   { ... }

  virtual bool solver_scal( T a, 
   cuNDArray<T> *x ){ ... }

  virtual bool solver_axpy( T a, cuNDArray<T> *x, 
   cuNDArray<T> *y ){ ... }
};

The overall inheritance hierachy is modelled and implemented similarly to the matrixOperator class hierachy described above (see Section 2.2.4, “Matrix Operators”). To use the solver the user creates an instance of the solver for either the host or device (e.g. the cuCGSolver above for a GPU-based solver). The solver is configured using the functions in the cgSolver base class. Finally the solve function itself is found in the root of the hierachy; the solver.

Note that any number of terms (matrix operators) can be added to the solver (or cost function).

The following code listing provides a short example of how to define a conjugate gradient solver for GPU-based image deblurring given an image and an estimate of the point spread function that degraded the image. It uses the convolutionOperator to model the blurring and a partialDerivativeOperator in each spatial dimension for regularization. The full code can be found in $(GADGETRON_SOURCE)/apps/standalone/gpu/deblurring/2d/deblur_2d_cg.cpp.

{
  << Code that parses the command line 
     and loads the image and kernel from disk >>

  // Define the desired precision
  typedef float _real; 
  typedef complext<_real>::Type _complext;

  // Upload host data to device
  cuNDArray<_complext> data(host_data.get());
  cuNDArray<_complext> kernel(host_kernel.get());
    
  // Setup regularization operators
  boost::shared_ptr< cuPartialDerivativeOperator<_real,_complext,2> > 
    Rx( new cuPartialDerivativeOperator<_real,_complext,2>(0) ); 
  boost::shared_ptr< cuPartialDerivativeOperator<_real,_complext,2> > 
    Ry( new cuPartialDerivativeOperator<_real,_complext,2>(1) ); 

  Rx->set_weight( lambda );
  Ry->set_weight( lambda );
     
  //
  // Setup conjugate gradients solver
  //

  // Define encoding matrix
  boost::shared_ptr< cuConvolutionOperator<_real,2> > 
    E( new cuConvolutionOperator<_real,2>() );

  E->set_kernel( &kernel );
    
  // Setup conjugate gradient solver
  cuCGSolver<_real, _complext> cg;

  // encoding matrix
  cg.add_matrix_operator( E );

  // regularization matrix                   
  if( kappa>0.0 ) cg.add_matrix_operator( Rx );
  
  // regularization matrix
  if( kappa>0.0 ) cg.add_matrix_operator( Ry ); 

  cg.set_iterations( num_iterations );
  cg.set_limit( 1e-8 );
  cg.set_output_mode( cuCGSolver<_real, _complext>::OUTPUT_VERBOSE );
                
  // Form right hand side
  cuNDArray<_complext> rhs; rhs.create(data.get_dimensions().get());
  E->mult_MH( &data, &rhs );

  //
  // Conjugate gradient solver
  //
  
  boost::shared_ptr< cuNDArray<_complext> > cgresult = cg.solve(&rhs);

  // All done, write out the result
  
  boost::shared_ptr< hoNDArray<_complext> > 
    host_result = cgresult->to_host();
  
  write_nd_array<_complext>(host_result.get(), 
    (char*)parms.get_parameter('r')->get_string_value());
}

For an overview of the various standalone applications the Gadgetron provides - and instruction on how to run them - we refer to Chapter 4, Standalone Applications.

As you read on, keep in mind that the functionality of the various Gadgets presented below (Section 2.1.1, “Gadgets”) in is encompassed by toolboxes; i.e. a lot more functionality as what has been covered by the selected overview in this section (Section 2.2, “Gadgetron Toolboxes”) is contained in the available toolboxes.

One of the original motivations for creating the Gadgetron was to make a high throughput MRI reconstruction engine that could be interfaced to different MRI vendor systems. Consequently, a lot of the functionality present in the initial release toolboxes and Gadgets is focused on MRI reconstruction. In this section we review the basic data structures used to describe MRI data and list some of the MRI Gadgets that are available. These Gadgets are used in several of the example applications in Chapter 3, Gadgetron Applications.

Gadget2< GadgetMessageAcquisition, hoNDArray< std::complex<float> > >
Gadget2< GadgetMessageImage, hoNDArray< std::complex<float> > >

struct LoopCounters {
  ACE_UINT16 acquisition;
  ACE_UINT16 slice;
  ACE_UINT16 partition;
  ACE_UINT16 echo;
  ACE_UINT16 phase;
  ACE_UINT16 repetition;
  ACE_UINT16 set;
  ACE_UINT16 segment;
  ACE_UINT16 channel;
};

struct GadgetMessageAcquisition
{
  ACE_UINT32     flags;
  ACE_UINT32     meas_uid;
  ACE_UINT32     scan_counter;
  ACE_UINT32     time_stamp;
  ACE_UINT16     samples;
  ACE_UINT16     channels;
  float          position[3];
  float          quarternion[4];
  float          table_position;
  LoopCounters   idx;
  LoopCounters   min_idx;
  LoopCounters   max_idx;
};

struct GadgetMessageImage
{
  ACE_UINT32     flags;
  ACE_UINT16     matrix_size[3];
  ACE_UINT16     channels;
  float          position[3];
  float          quarternion[4];
  float          table_position;
  LoopCounters   data_idx_min;
  LoopCounters   data_idx_max;
  LoopCounters   data_idx_current;
  ACE_UINT32     time_stamp;
  ACE_UINT16     image_format;
  ACE_UINT16     image_type;
  ACE_UINT16     image_index;
  ACE_UINT16	    image_series_index;
}; 

The Gadgetron provides a mechanism to do prototype development in Python. We will again be using MRI as the example application.

The Python layer is accessed through a set of Python Gadgets that can encapsulate a Python module. This is seen in Figure 2.3, “Overview of Python Prototyping”, which illustrates a part of a Gadget chain with two Python Gadgets and one C/C++ Gadget. A Gadget chain can have any number of Python Gadgets and Python Gadgets can be mixed with C++ Gadgets.

Figure 2.3. Overview of Python Prototyping

The Python modules that are encapsulated in the Python Gadgets are expected to have certain characteristics. Specifically, the Gadgets must have at least 3 functions and these functions will be called by the Gadgetron framework at certain specific times:

The user can chose the names of these functions freely in the Python module, but the function names must be specified when the Gadget is inserted in the XML configuration:

<gadget>
 <name>AccReconPython</name>
 <dll>gadgetronpython</dll>
 <class>AcquisitionPythonGadget</class>

 <property>
  <name>python_path</name>
  <value>/home/myuser/scripts/python</value>
 </property>

 <property>
  <name>python_module</name>
  <value>accumulate_and_recon</value>
 </property>

 <property>
  <name>gadget_reference_function</name>
  <value>set_gadget_reference</value>
 </property>

 <property>
  <name>input_function</name>
  <value>recon_function</value>
 </property>

 <property>
  <name>config_function</name>
  <value>config_function</value>
 </property>
</gadget>

Notice how the 3 function names are specified through the gadget_reference_function, input_function, and config_function parameter names. Also notice that it is possible to specify a python_path to let the Python interpreter know where to search for script. By default, the gadgetron/lib is added to the search path. Multiple pathnames can be added by separating the paths with ";".

The Python script referenced in the XML configuration above could look like this:

import numpy as np
import GadgetronPythonMRI as g
import GadgetronXML
import kspaceandimage as ki

myRef = g.GadgetReference()
myBuffer = 0
myParameters = 0
myCounter = 1;
mySeries = 1;

def set_gadget_reference(gadref):
    global myLocalGadgetReference
    myLocalGadgetReference = gadref

def config_function(conf):
    global myBuffer
    global myParameters
    myParameters =  GadgetronXML.getEncodingParameters(conf)
    myBuffer = (np.zeros((myParameters["channels"], \
                myParameters["slices"], \
                myParameters["matrix_z"], \
                myParameters["matrix_y"], \
                myParameters["matrix_x"]))).astype('complex64')

def recon_function(acq, data):
    global myLocalGadgetReference
    global myBuffer
    global myParameters
    global myCounter
    global mySeries

    line_offset = (myParameters["matrix_y"] - \
                   myParameters["phase_encoding_lines"])>>1
  
    myBuffer[:,acq.idx.slice, \
             acq.idx.partition,acq.idx.line+line_offset,:] = data
    
    if (acq.flags & (1<<1)): #Is this the last scan in slice
        #FFT
        image = ki.ktoi(myBuffer,(2,3,4))

        #Create a new image header and transfer value
        img_head = g.GadgetMessageImage()
        img_head.channels = acq.channels
        img_head.data_idx_curent = acq.idx
        img_head.data_idx_max = acq.max_idx
        img_head.data_idx_min = acq.min_idx
        img_head.set_matrix_size(0,myBuffer.shape[4])
        img_head.set_matrix_size(1,myBuffer.shape[3])
        img_head.set_matrix_size(2,myBuffer.shape[2])
        img_head.set_matrix_size(3,myBuffer.shape[1])
        img_head.set_position(0,acq.get_position(0))
        img_head.set_position(1,acq.get_position(1))
        img_head.set_position(2,acq.get_position(2))
        img_head.set_quarternion(0,acq.get_quarternion(0))
        img_head.set_quarternion(1,acq.get_quarternion(1))
        img_head.set_quarternion(2,acq.get_quarternion(2))
        img_head.set_quarternion(3,acq.get_quarternion(3))
        img_head.table_position = acq.table_position
        img_head.time_stamp = acq.time_stamp
        img_head.image_index = myCounter;
        img_head.image_series_index = mySeries;


        #Return image to Gadgetron
        return myRef.return_image(img_head,image.astype('complex64'))

        #print "Returning to Gadgetron"
        return 0 #Everything OK

There is a lot going on in this script. Let us walk through the different parts and add some explanation. First look at the imports:

import numpy as np
import GadgetronPythonMRI as g
import GadgetronXML
import kspaceandimage as ki

All the Python Gadget modules must include numpy. The arrays (NDArray) are passed to the Python module as numpy arrays. The second module GadgetronPythonMRI is a Python wrapped version of some of the data structures used in the MRI part of the Gadgetron (see Section 2.3.1, “MRI Gadgets”). Specifically, the GadgetMessageAcquisition and GadgetMessageImage headers are wrapped as Python types (using Boost Python). The GadgetronPythonMRI also contains a wrapped version of the GadgetReference class:

class GadgetReference
{

 public:
  GadgetReference();
  ~GadgetReference();
  
  int set_gadget(Gadget* g)
  {
    gadget_ = g;
    return 0;
  }

  template<class T> int return_data(T header, 
          boost::python::numeric::array arr);

  int return_acquisition(GadgetMessageAcquisition acq, 
          boost::python::numeric::array arr);

  int return_image(GadgetMessageImage img, 
          boost::python::numeric::array arr);

 protected:
  Gadget* gadget_;

};

Using the return functions in this class interface, it is possible for the Python module to return data to the Gadget. GadgetronXML is a Python module provided with the Gadgetron, which contains some XML helper functions that can (it is not a requirement) be used to parse the XML parameters that the module will receive from process_config. kspaceandimage is also a python module provided with the Gadgetron, it contains some simple wrapper functions for performing Fourier transforms (to and from k-space) of MRI data. The following section contains some initialization of global variables in the Python module;

myRef = g.GadgetReference()
myBuffer = 0
myParameters = 0
myCounter = 1;
mySeries = 1;

As described above, each Python module must contain at least 3 functions corresponding to the 3 entry points from the Gadgetron framework. The first one of these functions captures the GadgetReference:

def set_gadget_reference(gadref):
    global myLocalGadgetReference
    myLocalGadgetReference = gadref

Using this reference, the Python module will be able to return images (or acquisitions) to the Gadget. The next function processes the configuration data:

def config_function(conf):
    global myBuffer
    global myParameters
    myParameters =  GadgetronXML.getEncodingParameters(conf)
    myBuffer = (np.zeros((myParameters["channels"], \
                myParameters["slices"], \
                myParameters["matrix_z"], \
                myParameters["matrix_y"], \
                myParameters["matrix_x"]))).astype('complex64')

The GadgetronXML module is used to parse some simple encoding parameters out of the XML configuration and they are stored in a convenient data structure for later reference. The config_function function also initializes a buffer for the data.

Finally the recon_function (not repeated here) simply takes the data as it comes it and stores it in a buffer. Based on the flags field in the header, it is determined when the last acquisition in each slice has arrived. As this happens the buffer is Fourier transformed, an image header is populated, and the result is returned (via the GadgetReference) to the Gadgetron where it will be processed by the next Gadget in the chain.

The Gadgetron distribution comes with a simple Python-based 2D FT MRI reconstruction. The Gadget chain configuration for this reconstruction can be found in gadgets/python/python.xml.

The easiest way to get started making a new Gadget library is to follow an example. In this example we create a new Gadget library containing a single Gadget; ThresholdGadget. Its purpose is to set all values below a certain fraction of the max value to zero.

New Gadget libraries can either be created in the Gadgetron source tree, which allows easy access to all the other files in the Gadgetron, or they can be made as external libraries that link against an installed Gadgetron system. In this example we do the latter since this creates a new library that does not "taint" the Gadgetron source tree. It is trivial to move the library inside the Gadgetron source tree at some later point in time if desired. We assume that the Gadgetron is installed on the machine that you are working on. The command line entries, etc. correspond to a Linux console. If you are using Windows you have to adjust a bit.

Start by creating a new folder for the library:

user@mycomputer:~/temp$ mkdir gadgetron_examplelib
user@mycomputer:~/temp$ cd gadgetron_examplelib

We start by creating the class ThresholdGadget. Create the following 3 files: ThresholdGadget.h, ThresholdGadget.cpp, examplelib_export.h (the last file is just to help us make sure that things work on Windows) with the following content:

//ThresholdGadget.h

#ifndef THRESHOLDGADGET_H
#define THRESHOLDGADGET_H

#include "examplelib_export.h"
#include "Gadget.h"
#include "GadgetMRIHeaders.h"
#include "hoNDArray.h"
#include <complex>

class EXPORTGADGETSEXAMPLE ThresholdGadget : 
public Gadget2<GadgetMessageImage, hoNDArray< std::complex<float> > >
{
 public:
  GADGET_DECLARE(ThresholdGadget)

 protected:
  virtual int process( GadgetContainerMessage< GadgetMessageImage>* m1,
       GadgetContainerMessage< hoNDArray< std::complex<float> > >* m2);

  virtual int process_config(ACE_Message_Block* mb);

  float threshold_level_;

};

#endif //THRESHOLDGADGET_H

//ThresholdGadget.cpp

#include "ThresholdGadget.h"

int ThresholdGadget::process_config(ACE_Message_Block* mb) 
{
  threshold_level_ = get_double_value("level");
  if (threshold_level_ == 0.0) {
    threshold_level_ = 1.0;
  }

  return GADGET_OK;
}

int ThresholdGadget::process( 
   GadgetContainerMessage< GadgetMessageImage>* m1,
   GadgetContainerMessage< hoNDArray< std::complex<float> > >* m2)
{

  std::complex<float>* d = 
    m2->getObjectPtr()->get_data_ptr();

  unsigned long int elements =  
    m2->getObjectPtr()->get_number_of_elements();

  //First find max
  float max = 0.0;
  for (unsigned long int i = 0; i < elements; i++) {
    if (abs(d[i]) > max) {
      max = abs(d[i]);
    }
  }

  //Now threshold
  for (unsigned long int i = 0; i < elements; i++) {
    if (abs(d[i]) < threshold_level_*max) {
      d[i] = std::complex<float>(0.0,0.0);
    }
  }

  //Now pass on image
  if (this->next()->putq(m1) < 0) {
     return GADGET_FAIL;
  }

  return GADGET_OK;
}

GADGET_FACTORY_DECLARE(ThresholdGadget)

//examplelib_export.h

#ifndef EXAMPLE_EXPORT_H_
#define EXAMPLE_EXPORT_H_


#if defined (WIN32)
#if defined (gadgetronexamplelib_EXPORTS)
#define EXPORTGADGETSEXAMPLE __declspec(dllexport)
#else
#define EXPORTGADGETSEXAMPLE __declspec(dllimport)
#endif
#else
#define EXPORTGADGETSEXAMPLE
#endif

#endif /* EXAMPLE_EXPORT_H_ */

Now that we have the files for the Gadget we need to set up the build environment. In the folder gadgetron_examplelib create a file called CMakeLists.txt with the following content:

cmake_minimum_required(VERSION 2.6)

project(examplelib)

if (WIN32)
ADD_DEFINITIONS(-DWIN32 -D_WIN32 -D_WINDOWS)
ADD_DEFINITIONS(-DUNICODE -D_UNICODE)
SET (CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} /EHsc")
SET (CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} /W3")
endif (WIN32)

###############################################################
#Bootstrap search for libraries 
# (We need to find cmake modules in Gadgetron)
###############################################################
find_path(GADGETRON_CMAKE_MODULES FindGadgetron.cmake HINTS
$ENV{GADGETRON_HOME}/cmake
/usr/local/gadgetron)

if (NOT GADGETRON_CMAKE_MODULES)
  MESSAGE(FATAL_ERROR "GADGETRON_CMAKE_MODULES cannot be found. 
   Try to set GADGETRON_HOME environment variable.")
endif(NOT GADGETRON_CMAKE_MODULES)

set(CMAKE_MODULE_PATH ${GADGETRON_CMAKE_MODULES})
###############################################################

find_package(Gadgetron REQUIRED)
find_package(Boost REQUIRED)
find_package(ACE REQUIRED)

set(CMAKE_INSTALL_PREFIX ${GADGETRON_HOME})

INCLUDE_DIRECTORIES(${ACE_INCLUDE_DIR} 
     ${Boost_INCLUDE_DIR}
     ${GADGETRON_INCLUDE_DIR})

LINK_DIRECTORIES(${GADGETRON_LIB_DIR})

ADD_LIBRARY(gadgetronexamplelib SHARED ThresholdGadget.cpp)

TARGET_LINK_LIBRARIES(gadgetronexamplelib 
                      hondarray 
                      optimized ${ACE_LIBRARIES} 
                      debug ${ACE_DEBUG_LIBRARY})

INSTALL (FILES	ThresholdGadget.h
         examplelib_export.h
         DESTINATION include)

INSTALL(TARGETS gadgetronexamplelib DESTINATION lib)

INSTALL(FILES threshold.xml DESTINATION config)

The last thing we need is the XML configuration file to use when running our new ThresholdGadget. In the same folder create the threshold.xml file:

<?xml version="1.0" ?>  
<gadgetron>
  <readers>
    <reader>
      <slot>1001</slot>
      <dll>gadgetroncore</dll>
      <class>MRIAcquisitionReader</class>
    </reader>
  </readers>
  
  <writers>
    <writer>
      <slot>1005</slot>
      <dll>gadgetroncore</dll>
      <class>MRIImageWriterFLOAT</class>
    </writer>
  </writers>
  
  <stream>
    <gadget>
      <name>Acc</name>
      <dll>gadgetroncore</dll>
      <class>AccumulatorGadget</class>
    </gadget>

    <gadget>
      <name>FFT</name>
      <dll>gadgetroncore</dll>
      <class>FFTGadget</class>
    </gadget>

    <gadget>
      <name>CropCombine</name>
      <dll>gadgetroncore</dll>
      <class>CropAndCombineGadget</class>
    </gadget>

    <!-- This is where we insert our new Gadget -->
    <gadget>
      <name>Threshold</name>
      <dll>gadgetronexamplelib</dll>
      <class>ThresholdGadget</class>
      <property><name>level</name><value>0.25</value></property>
    </gadget>

    <gadget>
      <name>Extract</name>
      <dll>gadgetroncore</dll>
      <class>ExtractGadget</class>
    </gadget>

    <gadget>
      <name>ImageFinishFLOAT</name>
      <dll>gadgetroncore</dll>
      <class>ImageFinishGadgetFLOAT</class>
    </gadget>

   </stream>

</gadgetron>

Check that you have 5 files in your folder:

user@mycomputer:gadgetron_examplelib$ ls
CMakeLists.txt
ThresholdGadget.cpp
ThresholdGadget.h
examplelib_export.h
threshold.xml

Next, let us create a build directory and compile:

user@mycomputer:gadgetron_examplelib$ mkdir build; cd build

In the build folder

user@mycomputer:build$ cmake ../

Assuming the cmake process was successful:

user@mycomputer:build$ make 
Scanning dependencies of target gadgetronexamplelib

[100%] Building CXX object \
    CMakeFiles/gadgetronexamplelib.dir/ThresholdGadget.cpp.o

Linking CXX shared library libgadgetronexamplelib.dylib
[100%] Built target gadgetronexamplelib

user@mycomputer:build$ make install
[100%] Built target gadgetronexamplelib
Install the project...
-- Install configuration: ""
-- Up-to-date: /usr/local/gadgetron/include/ThresholdGadget.h
-- Up-to-date: /usr/local/gadgetron/include/examplelib_export.h
-- Installing: /usr/local/gadgetron/lib/libgadgetronexamplelib.so
-- Up-to-date: /usr/local/gadgetron/config/threshold.xml

You may have to use sudo for the make install command depending on your setup.

You should now be able to run a reconstruction using your new reconstruction chain. Follow the instructions in Section 1.4, “Hello Gadgetron: Your First Image Reconstruction” if you have not yet tried to run a simple reconstruction. After having started up the Gadgetron, run the mriclient:

user@mycomputer:~/temp/test_data$ mriclient \
     -d simple_gre.gmr \
     -x simple_gre.xml \
     -c threshold.xml

Gadgetron MRI Data Sender
  -- host:            localhost
  -- port:            9002
  -- data:            simple_gre.gmr
  -- parm:            simple_gre.xml
  -- conf:            threshold.xml
  -- loop:            1
(2815|140119753992000) Connection from 127.0.0.1:9002
Image Writer writing image
Image Writer writing image
Image Writer writing image
Image Writer writing image
Image Writer writing image
Image Writer writing image
Image Writer writing image
Image Writer writing image
Image Writer writing image
Image Writer writing image
2815, 81, GadgetronConnector, Close Message received
(2815|140119714780928) Handling close...
(2815|140119714780928) svc done...
(2815|140119714780928) Handling close...

If you run it again with the level parameter set to 0.00000001 (remember to re-install the threshold.xml file in gadgetron/config by running make install):

    <gadget>
      <name>Threshold</name>
      <dll>gadgetronexamplelib</dll>
      <class>ThresholdGadget</class>
      <property><name>level</name><value>0.00000001</value></property>
    </gadget>

You should get two different results that look something like Figure 2.4, “Result from ThresholdGadget experiment”.

Figure 2.4. Result from ThresholdGadget experiment

If you create interesting Gadget libraries please consider publishing them online to the benefit of the reconstruction community. An easy way to do this is by sending them to the Gadgetron team for us to publish right away on the web and possibly include in a future release of the Gadgetron.

The purpose of this section is to maintain a list over the available clients that are included in the Gadgetron distribution. The current available clients are:

The Gadgetron distribution comes with a GadgetronConnector class, which can be used to create clients. An example main.cpp file for a client could look like:

#include "GadgetMessageInterface.h"
#include "GadgetronConnector.h"

int main(int argc, char** argv)
{

  std::string host_name("localhost");
  std::string port("9002");
  std::string config_file("threshold.xml");
  std::string xml_config;

  //Generate some XML configuration in xml_fconfig

  GadgetronConnector con;

  //Register Readers and Writers
  con.register_writer(....);
  con.register_reader(....);
  con.register_reader(....);

  //Open a connection with the gadgetron
  if (con.open(hostname, port_no) != 0) {
    //Deal with errors
  }

  //Tell Gadgetron which XML configuration to run.
  if (con.send_gadgetron_configuration_file(config_file) != 0) {
    //Deal with errors
  }

  if (con.send_gadgetron_parameters(xml_config) != 0) {
     //Deal with errors
  }


  //Send data
  while ( .... ) { //some condition
    GadgetContainerMessage<GadgetMessageIdentifier>* m1 =
      new GadgetContainerMessage<GadgetMessageIdentifier>();
      
      //Create data and add to m1

      if (con.putq(m1) == -1) {
         //Deal with errors
      }
  }

  //Put a close package on the queue

  GadgetContainerMessage<GadgetMessageIdentifier>* m1 =
    new GadgetContainerMessage<GadgetMessageIdentifier>();

  m1->getObjectPtr()->id = GADGET_MESSAGE_CLOSE;

  if (con.putq(m1) == -1) {
   //Deal with errors
  }

  con.wait(); //Wait for recon to finish

  return 0;
}

To compile this client, create a cmake file:

cmake_minimum_required(VERSION 2.6)

project(exampleclient)

if (WIN32)
ADD_DEFINITIONS(-DWIN32 -D_WIN32 -D_WINDOWS)
ADD_DEFINITIONS(-DUNICODE -D_UNICODE)
SET (CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} /EHsc")
SET (CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} /W3")
endif (WIN32)

###############################################################
#Bootstrap search for libraries 
# (We need to find cmake modules in Gadgetron)
###############################################################
find_path(GADGETRON_CMAKE_MODULES FindGadgetron.cmake HINTS
$ENV{GADGETRON_HOME}/cmake
/usr/local/gadgetron)

if (NOT GADGETRON_CMAKE_MODULES)
  MESSAGE(FATAL_ERROR "GADGETRON_CMAKE_MODULES cannot be found. 
   Try to set GADGETRON_HOME environment variable.")
endif(NOT GADGETRON_CMAKE_MODULES)

set(CMAKE_MODULE_PATH ${GADGETRON_CMAKE_MODULES})
###############################################################

find_package(Gadgetron REQUIRED)
find_package(Boost REQUIRED)
find_package(ACE REQUIRED)

set(CMAKE_INSTALL_PREFIX ${GADGETRON_HOME})

INCLUDE_DIRECTORIES(${ACE_INCLUDE_DIR} 
     ${Boost_INCLUDE_DIR}
     ${GADGETRON_INCLUDE_DIR})

LINK_DIRECTORIES(${GADGETRON_LIB_DIR})

add_executable(mygadgetronclient main.cpp)

target_link_libraries(mygadgetronclient 
      optimized gadgettools debug gadgettools${CMAKE_DEBUG_SUFFIX}
      tinyxml 
      optimized ${ACE_LIBRARIES} debug ${ACE_DEBUG_LIBRARY})

install(TARGETS mygadgetronclient DESTINATION bin)

Run cmake and follow the normal make and make install instructions (see Section 2.3.3, “Making a new Gadget Library”).