System Operation

Startup

1. Inspect T-35HD Air Compressor Filter. Rinse with soap & water as necessary.
2. Inspect T-35HD air tank for water (valve on bottom of tank). Tilt tank and drain if necessary. Ensure valve is closed prior to use. (Stem will be at its longest when closed.)
3. Inspect LFR-D-MIDI & LF-D-5M-MIDI regulators for condensate. Release condensate if it is within 10mm below filter element by opening the lower blue plugs during system operation.
4. Adjust the LFR-D-MIDI regulator. Pull blue pressure setting button upwards to unlock it (away from housing). Turn pressure setting button in the ‘-’ direction as far as possible.
5. Place T-35HD power switch in auto position. The pump will automatically maintain between 100 – 140 PSIG.
6. Turn the LFR-D-MIDI pressure setting button in the ‘+’ direction until the desired pressure is shown on its manometer. (Note: The input pressure must be at least 1 bar greater than the output pressure.)
7. Press the the LFR-D-MIDI pressure setting button down to secure it against unintentional turning.

Operation

1. To Be Completed.

Shutdown

1. Remove supply pressure. Adjust the LFR-D-MIDI regulator. Pull blue pressure setting button upwards to unlock it (away from housing). Turn pressure setting button in the ‘-’ direction as far as possible.
2. Power OFF compressor
3. Release compressor tank pressure and excess condensate via drain valve.
4. Power OFF setpoint voltage
5. Power OFF supply voltage

Schematics

Software

videoInput

• http://www.muonics.net/school/spring05/videoInput/
• a free windows video capture library

AutoCAD

• 8020 Parts

NI

• daqmx C functions
• NI DAQ A/D Interface Card Functions – Appendix D

Hardware

Festo

• http://www.festo.com/cms/en-us_us
• 1.847.759.2600
• Account #20344983
• Customer Service: 1.800.993.3786
• Our Representative: Pat Sabharwal 1.847.759.2629 (Cell: 630.487.0479)
• Tech Support: 1.866.463.3786

8020

• http://www.8020.net/
• Distributer: TECO, tecoinc.com, 800.521.3285
• Salesman: Jim Gordon, jgordon@tecoinc.com

National Instruments

• See here.

Thomas

Ultra Air-Pac T-35HD Electric Air Compressor

• 3 gallon air storage
• Pressure switch maintains 100 PSIG (689.5 KPa) -140 PSIG (861.9 KPa)
• Air Displacement: 4.5 CFM (127 LPM) @ 0 PSI
• Air Delivery: 2.9 CFM @ 50 PSI (83.5 LPM @ 345 KPa)
• 2.55 CFM @ 100 PSI (72.22 LPM @ 689.5 KPa)
• 115V, 60Hz, 10A (@ working pressure), 15A fuse
• Weight: 48 lbs
• Requires no lubrication (do not apply oil or damage may result)
• Motor is equipped with thermal overload protector. If protector trips, the user should manually turn the pump motor off and let the system cool for 5 minutes.

Relevant Papers

• (Boblan et al 2007) A Human-Like Robot Torso ZAR5 with Fluidic Muscles: Toward a Common Platform for Embodied AI
• (Boblan et al 2006) A human-like robot torso with fluidic muscles Biologically inspired engineering
• (Boblan et al 2004) A Human-Like Robot Hand and Arm with Fluidic Muscles: Biologically Inspired Construction and Functionality

Overview

Components

1. Camera (Logitech QuickCam Pro 9000)
2. Robot Arm (custom built, Lynxmotion Hi-Tec 645MG)
3. Servo Control Board (Yost Engineering, Servo Center, Full Package, USB model)
4. Software (Custom built using some open source libraries for the visual processing)

Software

The software is mostly custom built C. We make use of some open source libraries for the visual processing.

Code Block Diagram

Major Variables

• pf: parallel fiber signals – nCenters x 1
• w1,w2: synapse weights – each is nCenters x 1 – initialize to zeros
• xa,ya: actual robot arm position [units will probably be in pixels] – each is 1×1
• xd,yd: desired robot arm position – each is 1×1
• xc,yc: correction factors for robot arm position – each is 1×1
• m1,m2: servo motor angles, in radians – each is 1×1
• l1Est,l2Est: estimated lengths for robot arm segments – each is 1×1
• xErr,yErr: position error between actual and desired arm position – each is 1×1
• c: structure that contains the radial basis function description for the virtual cerebellum. It consists of:
  • c1 – pointer to x-positions of rbf centers – nCenters x 1
  • c2 – pointer to y-positions of rbf centers – nCenters x 1
  • sigma – rbf standard deviation – 1×1
  • nCenters – the number of rbf elements -1×1
  • receptiveField – receptive field size of the rbf – 1×1

• servo1, servo2: structures containing the servo calibration constants:
  • servoMin – minimum servo position (servo units)
  • servoMax – maximum servo position (servo units)
  • degreesMin – degrees corresponding to servoMin, measured according to the “Arm Calibration” diagram below
  • degreesMax – degrees corresponding to servoMax, measured according to the “Arm Calibration” diagram below

Visual Tracking Settings

• In RoboRealm:
  • Color Threshold = {170-255} for red, green, and blue
  • Center of Gravity

• In camera “settings” tab, default settings, then set manual focus to about 2/3 between left & right

• Room lights need to be off

• Black fabric might be needed to deaden remaining glare spots

Arm Calibration

servodiagram

Pixels Per cm

(camera set to 640 x 480)

Code Versions

BMI Code

v070831 Working cerebellum code with virtual arm; c & matlab

v070902 Same as ver 1 with additional code for driving the servos in open loop

v070904 Same as ver 2 with a small bugfix (write 6 bytes, not 5 in “moveservo”) – verified to work using fedora linux, serial cable, portnum 1 (/dev/ttyS0) on “Market”

v070905 Calibration code started

v070919 First fully functioning version – complete system

v070920

v070921 Includes data and video

v071010 This version includes an initial attempt to calibrate the camera/arm interaction.

v071018 Camera calibration code has been tested and added.

Computer Vision

v070905 Working vision tracking

v070917 Lightened version of vision tracking

How to Run the Code

Go to D:\Jesse\Computer_Vision

Double click on Computer_Vision.sln “solution” file

We are only considering the “Robot Arm” project

Build -> Build Robot Arm

Debug -> Start w/o Debugging

If it asks, “Never Automatically Apply RightLight Technology”

GUI Edition is not complete and will not yet drive the servos, but would not take much effort to get there. The effort in this code is to focus on making the different parts of the system modular. It was the goal of the original author to make it so that brain control, motor control and image processing would be done by separate DLL files. This would make it so that other methods could be inserted to make the system function as intended. So lets say that the current project is used to operate a single robot arm. The next project used two robotic arms. A new brain control module would be written to replace the existing one to allow the use of two arms. But this would mean that the current image processing methods would no longer be viable. So the image processing module would be rewritten to work in the system. Later in the week, someone comes and is interested in the old system. It would only than be a matter of swapping out the new DLLs for the old ones. It is highly advised that it is made certain that the project is compiled without .NET dependencies. This decreases the compatibility of the program with the operating system. This version uses VidCapture lib version to handle all of the video capturing. Another option would be to tear up the DirectInput demos that contain video capturing or do the actual process of setting up a video stream, however, these are very tedious tasks and is not advised. The brain module from the Command Line Edition could be used in the GUI Edition to drive the servos and do the correct brain processing. It would be advised to make it so the servo arm can be serviced to go to specific angular setting so that coordinates can be picked off for de-skewing.

COMMAND LINE Edition is completed and working for the most part. To prepare the software reporting to the arm to move to be fully extended and pointing towards 0 degrees, 90 degrees and 180 degrees with the LED and software running the coordinates received can be added to the brain control constructor for automatic de-skewing the images. The de-skewing algorithm can use a little extra work, but it does preform relatively well, more so than having now de-skewing. This version uses OpenCV, a DLL, which is not the most ideal video capture handler. This was used though because it produced the image in a window automatically.

How to Compile

To compile either version of the project the project should be downloaded from the SVN. To download from the SVN server the following instructions could be followed for either version.

1. Create a new folder if one does not yet exist.
2. Enter the folder and right-click the white space in the folder.
3. Select “SVN Checkout…” and make the URL of repository either one of the following links:
   • For the GUI: http://192.168.100.2/svn/robot_code/branches/gui
   • For the Command window: http://192.168.100.2/svn/robot_code/trunk
4. Press OK and wait for the SVN client to download all related files to the folder you created.

After the project is downloaded there should be a folder or solution in the top level that relates to the particular version of Visual Studio that you are using. If there is no solution or you must create one the steps for creating a new project are as follows.

1. Create a subdirectory and name it after the version of visual studio that you are using. Such as VS2008.
2. Open Visual Studio and create a new project from File->New Project and select an Empty Project
3. Give the name of RobotArm or something appropriate as the executable will be named after the Project Name. The location should be directed to the directory that was just created. The Create directory for solution option should be unchecked.
4. The Solution Explorer should be on the side of the screen or press Ctrl+Alt+L to make it appear. Right click Header Files and include in all the header files in the projects folders, Right click Source Files and include in all the source files(.cpp,.c), Right click Resource Files and include any resource type files(.rc).
5. Typical settings for the project should be set by Right clicking the project name and selecting properties.
6. Some properties of interest that should be set to avoid issues are
   • General->Character Set=Use Multi-Byte Character Set
   • C/C++->General->Additional Include Directories=Any directories that have header files that are not included with the project, this may include OpenCV if compiling the Command version of the code.
   • C/C++->Code Generation->Runtime Library=non-DLL versions of Multi-threaded because if the executable is run on a computer other than your own, there will be driver conflicts.
   • Linker->General->Additional Library Directories=Any additional lib files used, this is required for OpenCV version of Command
   • Linker->Input->Additional Dependencies may need kernel32.lib, this is missed in a few versions of Visual Studio and causes issues.
7. If using the GUI version the only requirement is that you install DirectX SDK which is on the server and should be installed if it has not been done so on the computer you are using.

Setting up the Arm

To set up the arm for usage the camera should be put into the end of the extension that sits above the robot arm. It aims down towards the arm with the pivot point of the arm at the bottom center of the screen while the camera is running.

Last time checked, the motors seemed to be suffering from degradation. This is most likely from people looking at it by manually moving the arm around whenever they come into the lab. So the motors will need to be recalibrate to achieve the proper angles specified. However, since the servo system is being swapped out for a pneumatic system this should not be an issue that needs to be dealt with.

When using the system it may be noticed that the image appears washed out or overly bright. The QuickCam tool bar that pops up on screen at the opening of a video capture session feels it knows what settings are best and locks the gain and exposure in automatic settings. It may be required of the user to go into the advanced settings, web cam settings and RightLight settings to disable automatic settings. After disabled the settings can be changed to make the table look black again and the floors not look like halogen projectors. In the best case scenario the LED could be turned on and the settings can be adjusted to make the LED the brightest object. The GUI version of the program will update with the same settings as were seen in the QuickCam setup program.

• Chou CP, Hannaford B. Study of Human Forearm Posture Maintenance With a Physiologically Based Robotic Arm and Spinal Level Neural Controller, Biol Cybern. 76(4), 1997
  • McKibben muscles have many properties in common with the human arm
  • Feedback is provided by mechanical muscle spindles (see Marbot & Hannaford 1993) and force sensors which mimic Golgi tendon organ which are attached in parallel to the antagonistic muscle pair
  • Ability to differentiate between Ia and Ib afferent feedback
    
  • Muscles exhibit hysteresis
  • Muscle attachment points are chosen to be biologically plausible
  • There is a discussion of the limitations
  • There is a discussion of why a mechanical model is superior to a computational model
  • Arm is controlled by a biologically inspired neural network programmed into a real-time DSP
  • Single DOF
  • Responses of model arm are compared to observed values for a human arm
  • Spinal cord mediated reflexs are modelled
• Hannaford B, Winters JM, Chou CP, Marbot PH. The Anthroform Biorobotic Arm: A System for the Study of Spinal Circuits, Ann Biomed Eng. 23(4), 1995
  • This paper details the development of the Biorobotic arm used in subsequent UWashington experiments
  • Interesting Introduction giving the motivation for Biorobotic research
    
  • Bones are modeled on human bones, including size, weight, and density. Built out of fiberglass
  • Elbow and shoulder joints were actual artificial joints donated by a medical device company
  • Ligaments made from knit fabrics
  • Fifteen McKibbens for all the muscles plus one 2×2 grid each for biceps and triceps
  • McKibbens are custom built – authors claim they can be easily constructed in less than 20 minutes
  • Equations are given for McKibben muscle response
  • Controlled by network of artificial neurons communicating on a 1ms timescale. Spinal cord time delays of 1ms are created in software using buffering.
  • Introduce the concept of artificial muscle spindles – I need to read up and see if there is anything more up-to-date.
• Marbot PH, Hannaford B. The mechanical spindle: a replica of the mammalian muscle spindle. IEEE Conf Eng Med Biol, 1993
• Chou CP, Hannaford B. Measurement and Modeling of Mckibben Pneumatic Artificial Muscles, IEEE Trans Robot Automat. 12(1), 1996
  • I didn’t read this one – but it basically justifies that McKibben is a reasonable model of a variety of properties of human arm and arm muscle
• Klute GK, Czerniecki JM, Hannaford B. Artificial Muscles: Actuators for Biorobotic Systems, Int J Robot Res. 21(4), 2002

University of Illinois

• Hesselroth T, Sarkar K, Vandersmagt PP, Schulten K. Neural-Network Control of a Pneumantic Robot Arm, IEEE Trans Syst Man Cybern. 24(1), 1994
  • Good description of the mechanics of constructing the McKibbens
  • 90 PSI compressed air source
  • compressed air dryer
  • 12 gallon buffer tank (to even out fluctuations)
  • regulator to reduce to 75 PSI – arm draws a max of 60 PSI, so there is some leeway built in
  • Servo Drive Units take RS-232 commands and convert into current sources – one for each of the joint’s two actuators
  • Servo Valve Units take current and produce a proportional pressure in the McKibben tube
  • Two cameras are mounted at 90 degree angles to provide visual feedback. They track a light at the arm’s tip
  • Pair of parallel antagonistic muscles with free ends connected together by a chain that passes over a sprocket mounted at the joint
  • Joints can be controlled in either Position Control Mode (PID/feedback to desired posn) or Pressure Control Mode (essentially open loop)
• van der Smagt P, Groen F, Schulten K. Analysis and Control of a Rubbertuator Arm, Biol Cybern.75(5), 1996

Cricket V5

Festo Robot Arm – Airic’s Arm Video

• Source
• Airic’s Arm Description

Assad et al 2005

Kingsley Dissertation (CWRU) 2005

Chou 1997 [Study of human forearm posture maintenance with a physiologically based robotic arm and spinal level neural controller]

I assume that Braided Pneumatic Actuators is the name of the actuator type you were talking about. Looks promising. The same guy that made the cricket is using this new type of actuator for a robot arm. By the way, the Cricket Robot V3 we saw was using double acting pneumatic cylinders.

Pneumatic Cylinders

Double Acting Pneumatic Cylinders are used to push and pull air. This means that we can control a component in both directions. A Single-Acting Pneumatic Cylinder is used when you want to push or pull air, but in the reverse direction is done by just blowing the load. The cylinders come in various diameters which help to produce a specified amount of force. Such as a 2.5mm cylinder is used to lift microchip components while 400mm diameter cylinder is used to lift a car. The force is related to the air pressure flowing into the cylinder times the cross sectional area of the cylinder.

In the application of the arm muscle is considered a rodless, single-acting actuator. The controller that connects to the arm would be in charge of how the air is delivered and received from the muscle. The company Festo provides these actuator types that can apparently be cut and fitted on site.

This link is supposed to be the pneumatic muscle provided by Festo. This is the same company that was used by the robot arm I found on the web and used by the updated version of the cricket robot.

About Air Compressors

Air Compressors

We need a specified amount of air pressure to designate the ’strength’ of the muscles of the arm. This means we need an air compressor with more pressure than is required that has some type of regulator. There are two sources we could look for air pressure. There are air lines in the lab that could be used to ‘power’ our muscle. We need to find out what the expected output of the line is by the building manager and a regulator would need to be installed on the arm to make it possible to select the amount of pressure desired. The other option is to go with a high output, quiet compressor. Such as this Huskey offered at the HomeDepot which has its own regulator and sufficient pressure.

Air Control Valves

There are multiple types of control valves and methods of control. The schema for the valves is V/P NO/NC. Where V is the number of valves offered where two is the minimum for air input and device output, three will add an exhaust or two device outputs. P is the number of positions where two is the minimum of push and release, three will add a third state which is variable depending on the type of valve purchased but is typically to hold the current air pressure. NO means normally open and NC means normally closed in the unactuated state.

The means of controlling a valve is by an actuator. This is an internal spring type which a spring will force the valve to its natural state when the external valve is open. An air pilot actuator can be used where air can be used to enable the air actuator. A solenoid actuator, also called direct acting actuator, is an actuator that is controlled electronically. These tend to be faster and is more along the lines of what we need for our project. These are meant for low pressure applications and tend to have some problems with getting gunked up if the air supply has a lot of moisture and dirt in it. In high pressure applications a direct acting actuator may not be enough and a Solenoid Pilot Valve Actuator may be needed. The problem with this type is that it has a minimum flow which would be determined by the manufacturer but is typically around 20 to 25PSI.

Manifolds

One method of distributing are from one compressor to multiple outputs is the use of an Air Manifold. These have an inlet and one of many outputs that can connect to the external systems. A compressor system would need to provide sufficient air pressure to the system to not be taxed by the multiple devices connected to the outputs. Regulators used at the outputs can be used to regulate the appropriate air pressure to each of the subsystems.

Regulators

Regulators are as their name suggest. They regulate the air pressure that passes from the input to the output. Some regulators, filtered regulators, contain a filtration system within the regulator to help eliminate problems of dirty air passing to the more delicate air control valves.

Pneumatic Retailers

• MFD Pneumatics
• Festo
• PHD, Inc.
• Rexroth Pneumatics

Micro Controllers

The PICs are cheap, like 5 bucks, for the nicest one on the market. But then you are expected to purchase a demo board or some type of board for flashing the PIC. Can you not just use the PIC stand-alone by wiring it up to a serial port or is there some crazy configuration for programming the chips?

I found that there are $25 “plug-in modules”. This suggests that you have to plug it into another board to utilize it. They say that you have to plug it into the Explorer 16 Demo board. Explorer 16 demo board is $130.

There is an option to receive samples. I will read a little more what this entails but I believe we need to contact them what we plan to do and give them a little proposal. Then we need to show them that we are linking people from our project page to their page or mentioning them several times.

The plug-in module literally plugs into a location in the center of the board. The demo board has anything you can want. LCD display, telephony cables, usb, serial… something else… a small prototyping area.

I vote we just go for the best they have to offer. It is the same price and it looks like if the need arises we can control anything else we apply to the robot arm as a controller. The main criteria for me right now with in order is PC/serial communications, ADC and expandability. Expandability will be reading, sending signals to actuators.

http://www.microchipdirect.com

DM240001 – Explorer 16 Demo Board – $130
The Explorer 16 Development Board is a low-cost modular development system for Microchips new 16-bit microcontrollers. It supports devices from the PIC24F, PIC24H, and dsPIC33 families. It is capable of interfacing with 5V peripherals and also provides basic generic functionality with the added ability to expand to vertical markets via modular expansion.

MA320001 – PIC32MX 100P QFP TO 100P PLUG IN MODULE – $25
This Plug-in Module enables PIC32 development on the Explorer 16 development board (DM240001 or DM240002) and supports the MPLAB Real ICE Trace kit (AC244006). A 72 Mhz PIC32MX360F512L with 512 KB of Flash, 32 KB of RAM, 4 channels of hardware DMA and instruction trace is installed on the plug-in module.

Development of System-Level Infrastructure for a Biofeedback Robotic Device by Jesse Krigelman

The aim of this project was to develop a simulation environment for studying adaptive learning in human motor control. A system was designed, built, and tested that was comprised of a two degree-of-freedom robot arm controlled by an adaptive feedback loop. The goal of the visually mediated feedback system was to learn to track a moving virtual target in real-time.

The arm was actuated by a pair of servo motors that were controlled by a servo control board operated remotely via PC software. Visual feedback was accomplished with a USB web camera; PC software was written to track the tip of the robot arm in real-time and to compare its position to that of the virtual target. A custom software module was also developed to deskew camera images that were not precisely parallel to the plane of the robot arm. All code was written in C++ and executed in real-time on a dual-core Dell Dimension 9200 running at 2.13GHz with 2GB of RAM. The code was written in a modular fashion to facilitate future upgrades. The system was successfully demonstrated under a variety of conditions, specifically with different virtual target trajectories, learning rates, and servo speeds. Planned future improvements include accelerometers for modeling proprioceptive feedback and pneumatic actuators to replace the servos for more biologically accurate muscle movement.

Development of system-level infrastructure for a biofeedback robotic device

Brief intro
-> 2DOF robot arm
-> Visual feedback-mediated adaptive controller
-> Goal was for robot arm to learn to track a virtual target in real-time

System-Level Design
-> Servos
-> Servo Control
-> Software Control for Servos
-> Camera
-> Interfaced camera to software
-> Algorithm for camera deskew

Results/Discussion
-> System was successfully demonstrated under a variety of conditions
-> Future work to pursue proprioceptive (accelerometer) based feedback and pneumatic actuation to replace servos.

Databases

1. SenseLab

   The SenseLab Project (hosted by Yale University) is a long term effort to build integrated, multidisciplinary models of neurons and neural systems, using the olfactory pathway as a model. This is one of a number of projects funded as part of the Human Brain Project whose aim is to develop neuroinformatics tools in support of neuroscience research. The project involves novel informatics approaches to constructing databases and database tools for collecting and analyzing neuroscience information, and providing for efficient interoperability with other neuroscience databases.

2. ModelDB

   ModelDB (a component of SenseLab) provides an accessible location for storing and efficiently retrieving computational neuroscience models. ModelDB is tightly coupled with NeuronDB. Models can be coded in any language for any environment. Model code can be viewed before downloading and browsers can be set to auto-launch the models. Click here for help on how to download and/or run models from Senselab’s Model Database.

Neural Simulators

The following guide provides a synopsis provides a summary of some known open-source neural simulators.

1. DSTOOL

   by John Guckenheimer, Cornell Univ., dynamical systems on Unix machines

2. GENESIS

   by Jim Bower, Cal. Tech., general purpose simulator for neural systems on Unix machines

3. NBC

   by Jean-Francois Vibert, Fac. de Med. St-Antoine, Paris, Network simulation and analysis on Unix and VMS machines

4. NEMOSYS

   by John Tromp, Univ. Cal., Berkeley, complex single neurons on Unix machines

5. NEUROGRAPH

   by Peter Wilke, Univ. Erlangen, Germany, Simulation of artificial neural networks on Unix, DOS, VMS machines

6. NEURON

   by Michael Hines, Duke Univ., Simulations of biologically realistic single neurons and small networks on PCs and Unix machine

   NEURON is a simulation environment for developing and exercising models of neurons and networks of neurons. It is particularly well-suited to problems where cable properties of cells play an important role, possibly including extracellular potential close to the membrane), and where cell membrane properties are complex, involving many ion-specific channels, ion accumulation, and second messengers.

   For more information NIL’s use of NEURON see our NEURON Simulator Programming Guide.

7. NEURONC

   by Rob Smith, Univ. Penn., compartmental simulations of large neural circuits on Unix machines

8. NODUS

   by Eric De Schutter, Univ. Antwerp, Belgium, simulation of small networks of neurons on Macintosh machines

9. NSL

   by Alfredo Weitzenfeld, Univ. Sou. Cal., simulation of large networks on Unix machines

10. SNNAP

    by John Byrne, Univ. Texas, Houston, Simulator for neural networks on Unix machines

11. SWIM

    by Orjan Ekeberg, Royal Inst. Tech., Stockholm, simulation of network of few compartment model neurons on Unix machines

Tool Sets

Blue Brain Project

“In July 2005, EPFL and IBM announced an exciting new research initiative – a project to create a biologically accurate, functional model of the brain using IBM’s Blue Gene supercomputer. Analogous in scope to the Genome Project, the Blue Brain will provide a huge leap in our understanding of brain function and dysfunction and help us explore solutions to intractable problems in mental health and neurological disease.”

• Markram, H. (2006). The blue brain project.. Nat Rev Neurosci, 7, 153-60.
• Migliore, M., Cannia, C., Lytton, W. W., Markram, H. & Hines, M.L. (2006). Parallel network simulations with NEURON.. J Comput Neurosci, 21, 119-29.

Neuronal Time Series Analysis (NTSA) Workbench

A Database System for Neuronal Pattern Analysis

“Biologically-detailed neural simulations of the type supported by software packages such as GENESIS (developed in Jim Bower’s laboratory at Cal Tech) and NEURON (developed by Michael Hines and John Moore at Duke) are typically used to generate time-series data of the same general form as data collected in neurophysiological experiments. The volume of time-series data produced in a typical simulation study is often comparable to, or in some cases much greater than, the amount of data yielded in physiological experiments. The uniform interface that NTSA Workbench presents to both experimental and simulated data greatly facilitates comparison of modeling results with experimental data.” The status of this project is unknown as all links to the actual project are dead.

Getting Started

NEURON may be downloaded from http://www.neuron.yale.edu/neuron/install/install.html for Linux, MSWin (95 and up), and Mac OS X 10.4. NIL is running NEURON within the Fedora Linux environment on the basis that of the available OSs, Linux requires the least computational resources and may be implemented cheaply and easily across multiple hardware systems.

NEURON is installed in Linux with the command

rpm nrn-[...].i686.rpm

or installed versions may be upgraded with

rpm -Fvh nrn-[...].i686.rpm

For those operating on a non-Linux system I recommend installing the virtual os program VirtualBox. The program allows any OS (Mac, Win, or Linux) to be run as a virtual (guest) system on any host. Virtual systems have full network and host file system access. NIL is running with both pure and virtual Linux systems.

In recent years NEURON has been modified to interact with the object-oriented language Python. The language comes installed with most flavors of Linux (including Fedora) or may be obtained through their download page. Two additional libraries, ‘numby’ and ’scipy’, are also required and may be installed through the Linux package manager.

Commands

• nrnivmodl – compiles a downloaded model. Run the command within the model’s folder.
• nrngui – executes the model. (i.e. nrngui mosinit.hoc)

Resources

• NEURON home page
• What is NEURON (by the software authors)
• NEURON Quick Reference
• NEURON Instructions and Parameters
• NEURON Tutorial
• The NEURON Forum
  
• Neural Simulator Resources (internal NIL link). Before continuing with the NEURON Simulator Programming Guide, I strongly recommend that the reader investigates the Database resources. The databases provide pre-built models for immediate gratification.
• Performance Data – Parallel network simulations with NEURON (pdf)
  

Other Projects Using NEURON

• Blue Brain Project – Massive use of mNEURON (multiprocessor version).

‘.hoc’ Template

The following document outlines the sections necessary to program a structure within the neural simulator NEURON. Where possible, the first instances of keywords have been linked to reference documents for ellaboration.

//**********************************************************************
//**********************************************************************
// TEMPLATE: Basic Neuron
// AUTHOR(s): Richard Friendlich
//   This file is a composite of various tutorials and books.
//   Primary contributors include Dr. N.T. Carnevale, Dr. M. L. Hines, Kevin E. Martin (martin@cs.unc.edu)
//   Additional authors are sited as necessary.
// PURPOSE:
// PARAMETERS: None.
// VERSION HISTORY: 20 Nov 2007
// NOTES:
//   NEURON files are saved with the .hoc extension and may be executed via the nrngui command
//**********************************************************************
//**********************************************************************

load file("nrngui.hoc")	// Loads GUI and standard runtime library

//**********************************************************************
// Procedure: celldef
// Author(s): Various
// PURPOSE: Implements modular approach to cell building
// PARAMETERS: None.
// VERSION HISTORY: 20 Nov 2007
// NOTES: 'The NEURON Book' recommends dividing the cell description into: Topology, Geometry & Biophysical Properties
//**********************************************************************
proc celldef() {
  topol()
  subsets()
  geom()
  biophys()
  geom_nseg()
}

create soma, apical, basilar, axon 

/*'create' is an nrniv command which creates a list of section names. Existing sections with the same names
are destroyed and recreated. The create statement may occur within procedures, but the names must have
been previously declared with a create statement at the command level.*/

//**********************************************************************
// Procedure: topol
// Author(s): Various
// PURPOSE: Connects basic cell structure. This represents the cells framework much like a skeleton.
// PARAMETERS: None.
// VERSION HISTORY: 20 Nov 2007
//**********************************************************************
proc topol() { local i
  connect apical(0), soma(1)
  connect basilar(0), soma(0)
  connect axon(0), soma(0)
  basic_shape()
}

//**********************************************************************
// Procedure: basic_shape
// Author(s): Various
// PURPOSE: Assigns 3d coordinates to structures defined in topol
// PARAMETERS: None.
// VERSION HISTORY: 20 Nov 2007
// NOTES: The topology works fine with out pt3dadd. Upon creating a 'shape plot' NEURON creates and arbitrary 3d structure
//   if one has not been specifically designated.
//**********************************************************************
proc basic_shape() {
  soma {pt3dclear() pt3dadd(0, 0, 0, 1) pt3dadd(15, 0, 0, 1)} // Destroys the 3d location info in the currently accessed section
  apical {pt3dclear() pt3dadd(15, 0, 0, 1) pt3dadd(120, 0, 0, 1)} // and adds the 3d location and diameter point at the end of the
  basilar {pt3dclear() pt3dadd(0, 0, 0, 1) pt3dadd(-59, 45, 0, 1)} // current pt3d list.
  axon {pt3dclear() pt3dadd(0, 0, 0, 1) pt3dadd(-104, 0, 0, 1)}
}

objref all, has_HH, no_HH
/*Object variables are labels (pointers, references) to the actual objects. Thus o1 = o2 merely states that o1 and o2 are labels
for the same object. Objects are created with the new statement. When there are no labels for an object the object is deleted.
The keywords objectvar and objref are synonyms.*/

//**********************************************************************
// Procedure: subsets
// Author(s): Various
// PURPOSE: Groups sections (made via 'create' command) into manageable units
// PARAMETERS: None.
// VERSION HISTORY: 20 Nov 2007
// NOTES: Subsets a useful shorthand for assigning morphologies to groups of sections at a time.
//**********************************************************************
proc subsets() { local i
  objref all, has_HH, no_HH
  all = new SectionList()
    soma all.append()
    apical all.append()
    basilar all.append()
    axon all.append()

  has_HH = new SectionList()
    soma has_HH.append()
    axon has_HH.append()

  no_HH = new SectionList()
    apical no_HH.append()
    basilar no_HH.append()

}

//**********************************************************************
// Procedure: geom
// Author(s): Various
// PURPOSE: Specifies the geometric properties of the sections.
// PARAMETERS: None.
// VERSION HISTORY: 20 Nov 2007
// NOTES:
//**********************************************************************
proc geom() {
  forsec all {  }
  soma {  L = 30  diam = 30  }
  apical {  L = 600  diam = 1  }
  basilar {  L = 200  diam = 2  }
  axon {  L = 1000  diam = 1  }
}

//**********************************************************************
// Procedure: geom_nseg
// Author(s): Various
// PURPOSE: Establishes required number of segments per section.
// PARAMETERS: None.
// VERSION HISTORY: 20 Nov 2007
// NOTES:
//**********************************************************************
proc geom_nseg() {
  forsec all { nseg = int((L/(0.1*lambda_f(100))+.9)/2)*2 + 1  }
}

//**********************************************************************
// Procedure: biophys
// Author(s): Various
// PURPOSE: Inserts biophysical cell properties.
// PARAMETERS: None.
// VERSION HISTORY: 20 Nov 2007
// NOTES:
//**********************************************************************
proc biophys() {
  forsec all {
    Ra = 100
    cm = 1
  }
  forsec has_HH {
    insert hh // insert is used to place distributed mechanisms (membrane properties)
      gnabar_hh = 0.12
      gkbar_hh = 0.036
      gl_hh = 0.0003
      el_hh = -54.3
  }
  forsec no_HH {
    insert pas
      g_pas = 0.0002
      e_pas = -65
  }
}
access soma

celldef()

//**********************************************************************
//**********************************************************************
/* Instrumentation
//**********************************************************************
//**********************************************************************

// Synaptic input
objref syn
soma syn = new AlphaSynapse (0.5)
syn.onset = 0.5
syn.tau = 0.1
syn.gmax = 0.5
syn.e = 0

// graphical display
objref g
g= new Graph()
g.size (0,5,-80,40)
g.addvar ("soma.v(0.5)",1,1,0.6,0.9,2)

//**********************************************************************
//**********************************************************************
/* Simulation Control
//**********************************************************************
//**********************************************************************

dt = 0.025
tstop = 5
v_init = -65

proc initialize(){
  finitialize(v_init)
  fcurrent()
}
proc integrate(){
  g.begin
  while(t
    fadvance()
    g.plot(t)
  }
  g.flush()
}
proc go(){
  initialize()
  integrate()
}
tstop()

Here are a list of resources for large scale neural simulations. I am particularly interested in knowing if there is any value in trying to get the NEURON platform to run as a massively parallel job on an FPGA or even a family of interconnected FPGAs. It would seem like simulating large populations of neurons is an ideal project for parallelization. Feel free to add your thoughts. In addition to adding links, it would be helpful to add a few brief sentences under each tab giving your thoughts or a brief summary.

Primary Resources

• A book on parallel simulation of large scale neuronal clusters.
• Research paper (2006): A Component-Based FPGA Design Framework for Neuronal Ion Channel Dynamics Simulations
• Research paper (2007): Methodology and Design Flow for Assisted Neural-Model Implementations in FPGAs

[From the Neurodudes link below] After 2007 paper [above], the first author, Randall K. Weinstein, became the co-founder and Chief Systems Architect of http://www.simatratechnologies.com/. A search over NIH awarded grants reveals grant 1R43NS057859-01 for “A low-cost, high-speed platform for neural modeling”, which this page suggests may be worth on the order of $183,163 for 2007 (or maybe that’s the amount that has already been dispensed? or the amount left?). Description of the grant:

Neural models, mathematical descriptions of neural behavior, are an invaluable tool for developing new medical treatments and understanding how the nervous system works. But as researchers discover more information about the nervous system, these models become more complicated. As a result, many modern neural models require powerful computer hardware, such as supercomputers, in order to be simulated and studied. Unfortunately, these systems are expensive and difficult to use. This project will create a low-cost, user-friendly, and computationally powerful system for neural modeling based on a technology called field-programmable gate arrays (FPGAs). This project will develop the user-friendly tools for creating neuron models on FPGAs, and the high-speed interface that will maximize the computational power of FPGAs.

• Faster neural simulations with FPGAs: Post on Neurodudes Blog
• Current performance measurements for NEURON: Migliore, M., Cannia, C., Lytton, W. W., Markram, H. & Hines, M.L. (2006). Parallel network simulations with NEURON.. J Comput Neurosci, 21, 119-29.
• Simulation of networks of spiking neurons:review of tools and strategies (Brette 2007)
• Programmable Logic Construction Kits for Hyper-Real-Time Neuronal Modeling (Guerrer 2006)
• Advancing the Boundaries of High-Connectivity Network Simulation with Distributed Computing – (Morrison 2005)

Other Relevant Resources

• Rich’s post summarizing Neural Simulators
• NIH Program Announcement for Exploratory Innovations in Biomedical Computational Science and Technology

Components

1. Camera (Logitech QuickCam Pro 9000)
2. Robot Arm (custom built, Lynxmotion Hi-Tec 645MG)
3. Servo Control Board (Yost Engineering, Servo Center, Full Package, USB model)
4. Software (Custom built using some open source libraries for the visual processing)

Software

The software is mostly custom built C. We make use of some open source libraries for the visual processing.

Code Block Diagram

Major Variables

• pf: parallel fiber signals – nCenters x 1

• w1,w2: synapse weights – each is nCenters x 1 – initialize to zeros
• xa,ya: actual robot arm position [units will probably be in pixels] – each is 1×1
• xd,yd: desired robot arm position – each is 1×1
• xc,yc: correction factors for robot arm position – each is 1×1
• m1,m2: servo motor angles, in radians – each is 1×1
• l1Est,l2Est: estimated lengths for robot arm segments – each is 1×1
• xErr,yErr: position error between actual and desired arm position – each is 1×1
• c: structure that contains the radial basis function description for the virtual cerebellum. It consists of:
  • c1 – pointer to x-positions of rbf centers – nCenters x 1
  • c2 – pointer to y-positions of rbf centers – nCenters x 1
  • sigma – rbf standard deviation – 1×1
  • nCenters – the number of rbf elements -1×1
  • receptiveField – receptive field size of the rbf – 1×1

• servo1, servo2: structures containing the servo calibration constants:
  • servoMin – minimum servo position (servo units)
  • servoMax – maximum servo position (servo units)
  • degreesMin – degrees corresponding to servoMin, measured according to the “Arm Calibration” diagram below
  • degreesMax – degrees corresponding to servoMax, measured according to the “Arm Calibration” diagram below

Visual Tracking Settings

• In RoboRealm:
  • Color Threshold = {170-255} for red, green, and blue
  • Center of Gravity

• In camera “settings” tab, default settings, then set manual focus to about 2/3 between left & right

• Room lights need to be off

• Black fabric might be needed to deaden remaining glare spots

Arm Calibration

Pixels Per cm

(camera set to 640 x 480)

Code Versions

BMI Code

v070831: Working cerebellum code with virtual arm; c & matlab

v070902: Same as ver 1 with additional code for driving the servos in open loop

v070904: Same as ver 2 with a small bugfix (write 6 bytes, not 5 in “moveservo”) – verified to work using fedora linux, serial cable, portnum 1 (/dev/ttyS0) on “Market”

v070905: Calibration code started

v070919 First fully functioning version – complete system

v070920

v070921 Includes data and video

v071010 This version includes an initial attempt to calibrate the camera/arm interaction.

v071018 Camera calibration code has been tested and added.

Computer Vision

v070905: Working vision tracking

v070917: Lightened version of vision tracking

How to Run the Code

Go to D:\Jesse\Computer_Vision

Double click on Computer_Vision.sln “solution” file

We are only considering the “Robot Arm” project

Build -> Build Robot Arm

Debug -> Start w/o Debugging

If it asks, “Never Automatically Apply RightLight Technology”