Monday, December 12, 2011

Creating an Eclipse Dynamic War project from a Maven project is easy

The Maven Eclipse plugin is a great tool that lets you create an Eclipse project from a Maven project with just the mvn eclipse:eclipse command.  It always seems to work great on my big projects,probably  because someone has already done the configuration required to get the right information generated into .project and .classpath.  I've always struggled to make it work in a small demonstration or test project.  The maven constructed Eclipse project is never configured right and I can't run the war on my local Servlet container.

Eclipse supports web files through the Eclipse Web Tools Project (WTP) project and through the Dynamic Web Facet when facets are enabled.  The Dynamic Web Facet is either missing or mis-configured after the mvn eclipse:eclipse command.

You must tell the Maven Eclipse Plugin (not to be confused with m2e) to use WTP when building the Eclipse project. The simplest command is:

      mvn -Dwtpversion=2.0 eclipse:eclipse

You must use the correct wtpversion for your eclipse installation. The latest version, as of December 2011, is 2.0. You can find more information about the maven eclipse plugin configuration at http://maven.apache.org/plugins/maven-eclipse-plugin/wtp.html


Thursday, November 17, 2011

Accessing the RabbitMQ Console in Cloud Foundry

Cloud Foundry - PaaS

Each Cloud Foundry instance is a self contained environment (cloud) that is generally intended to be use as a black box application container. It is implemented in a virtualized environment that hides the actual virtual/physical topology from the deployer.  Service configuration, provisioning, log viewing and application deployment are done from outside the cloud through external tools.  Cloud services may use any ports or other resources and use unique usernames and passwords for each instance. Applications rendezvous with services, at run time, via declared service names. Cloud services are not visible outside the container.  All system monitoring must be done through log files, through code deployed in the application or through tooling provided by Cloud Foundry.

Service tunneling, where you can bind to the data port of a provisioned service, will be added to Cloud Foundry in a post 1.0 release.  This blog article describes how it will work and how you can use it today.

Cloud Foundry supports a broad set of services that can be discovered through the service catalog.  Those services are  provisioned, making them available to applications, via the vmc command line tool or through the STS Cloud Foundry plug-in .  Multiple instances of any service may be provisioned in a single cloud/container.  Service instances start as soon as they are provisioned. This screenshot shows the output of the vmc services command run against a (Micro) Cloud Foundry instance. There are 5 services types available for provisioning and one RabbitMQ instance started.

Applications can be made up of multiple custom code components each of which can be bound to some subset of the provisioned services.  Individual application components can communicate with each other or share data through services that they share.

Example:  A coffee shop workflow involving 5 people.  One places the order, one takes the order, one person processes hot drinks, one cold drinks and one delivers the order.  Those 5 components (people) might be implemented as separate processes that could be independently deployed.  The 5 processes could communicate via a previously provisioned message broker that is service bound to each application.





Micro Cloud Foundry is a self contained, Virtual Machine (VM), version of Cloud Foundry running on a Linux Operating System (OS).  It comes pre-loaded and configured to enable the various services without any installation hassle.  Micro Cloud Foundry does not implement all of the isolation levels of the public cloud. We will use this feature to gain access to the RabbitMQ management console.

RabbitMQ in Cloud Foundry

Cloud Foundry supports several different services including RabbitMQ.   You can see available services via the vmc services command line tool described above.  A service like RabbitMQ can be provisioned (started) via vmc or SpringToolSuite (STS).  That services is then bound a deployed application when the application is deployed and started.  The application finds the service at run-time with a service name resolution process.  Java/Spring users can see these as spring beans whose ids match the service name. The following screenshot shows a single RabbitMQ service instance that is called loanshark-rabbitmq.  The instance is bound to the cafe-create-order-web application.
Each entry in the "Services" tab represents a different service instance with its own name, resource pool, ports and user ids.  The application in the "Applications" tab shows that "cafe-create-order-web" is selected.  The information on the right is tied to that application. You can see that "loanshark-rabbitmq" is an application service for this application.

The RabbitMQ Management Console

RabbitMQ supports extension via plugins.  The Management Console plug-in provides information and control functions for it's associated broker.  Users can view queue depth, create exchanges and queues and set up security functions.  Queued messages can be de-queued, viewed or deleted.  This picture shows two queues each with 200 undelivered messages.  Port 55672 is the standard admin port.  

The Management Console plugin is configured and enabled in Cloud Foundry but is not publicly available.  Cloud Foundry assigns random ports so to each RabbitMQ instance and management server.  This lets it support multiple brokers on a single hosts and removes consumers assumptions about the environment.

Accessing the Management Console in Micro Cloud Foundry

Micro Cloud Foundry is designed to do local development and small deployments.  It is not as "locked down" as the public or hosted Cloud Foundry environments letting us log in and access internal web services. RabbitMQ in the MCF runs with the management console enabled.  The problem is finding the port number and username/password for running RabbitMQ instances. 

This section assumes that you have a working Micro Cloud Foundry VM version 1.1.  You should know the host name of that instance and the password you gave it at time of configuration.  The host name will be something like <your cluster name>.cloudfoundry.me that is set by CloudFoundry.com when you got your token. You can see the host name in the VM console at the top of the screen.
  1. Connect to the Micro Cloud Foundry instance with vmc. You may have run a set of commands to connect to the right cloud foundry instance, using vmc target, and register a user id with vmc register to create a useable user id.
  2. Verify that rabbitmq is an available service with vmc services .
  3. Provision a rabbitmq instance using either STS or vmc.  The vmc command would be something like vmc create-service rabbitmq <instance name>  where <instance name> is the name the running instance is registered under.  STS users should use the server configuration screen that you get by double clicking on your cloud foundry server in the Servers tab.
  4. ssh into the MCF virtual machine as root ssh root@<your cluster name>.cloudfoundry.me using the vcap password you entered when you started up the MCF virtual machine.
  5. Find the running instance of Rabbit.  All instances are located in /var/vcap/store/rabbitmq_srs/agent/instance/ There should be only one instance unless you provisioned rabbit more than once.
  6. Look in the rabbitmq.log file for the instance at /var/vcap/store/rabbitmq_srs/agent/instance/<rabbit_instance_id>/log/rabbitmq.log. Look for a line that gives the name and port.  You'll need that to connect with browser.
  7. Look in the configuring_users log file to see what user id and password the cloud gave to your service instance. It should be a line that includes rabbitmqctl add_user.  The full path to the file should be /var/vcap/store/rabbitmq_srs/agent/instance/<rabbit_instance_id>/log/configuring_users.log They will look similar to encrypted passwords.
  8.  Connect to the management server with your web browser.  Use the URL http://<cluster_name>.cloudfoundry.me Authenticate with the user name and password you got from the log file. You should now have full access to the management web interface. The user interface should initially show now exchanges or queues.
  9. Deploy an application bound to rabbit. You should see traffic in the management console you just connected to.

Using Spring to Simplify Cloud Migration

The opaque/proprietary nature of cloud deployments often means a developer has to code to the cloud.  They end up with one version that uses the local deployment APIs and one that version that uses the Cloud APIs with the associated problems of determining which version to run at any given time. Spring's Dependency Injection (DI) architecture makes it easy for Java developers to flip between Non-Cloud and Cloud  configurations by decoupling the module connection process. The following XML config shows how an AMQP connection bean could be configured Cloud Foundry and for a standard environment. The consuming code that is injected with or looks up this bean has no idea when version it is getting.

    <!-- Obtain a connection to the RabbitMQ via cloudfoundry-runtime: -->
    <beans profile="cloud">
        <cloud:rabbit-connection-factory
            id="rabbitConnectionFactory" />
    </beans>

    <!-- connect to the local broker using the default user name and password -->
    <beans profile="dev">
        <bean id="rabbitConnectionFactory"
            class="org.springframework.amqp.rabbit.connection.CachingConnectionFactory">
            <constructor-arg value="localhost" />
            <property name="username" value="guest" />
            <property name="password" value="guest" />
        </bean>
    </beans>

The XML uses Spring Profiles to pick the correct bean definition at initialization time.  The originating example runs as a java program in development and and in a Cloud Foundry environment for deployment testing.  All Java code in Cloud Foundry is deployed in a container so we use a web application listener to do spring initialization in the cloud and a traditional Main() when running on a local machine. These two entry points provide a simple place to set the profile without having to create separate config files.

This code shows how the profile is used for the stand alone program:
    public static void main(String[] args) {
        AbstractApplicationContext context =
            CafeDemoAppUtilities.loadProfileContext(
                    "/META-INF/spring/integration/amqp/cafeDemo-amqp-xml.xml",
                    CafeDemoAppAmqp.class,"dev");
                   ...
        context.close();
    }

This code shows how the profile is used in the web application listener:
    public void contextInitialized(ServletContextEvent arg0) {
        AbstractApplicationContext context =
                CafeDemoAppUtilities.loadProfileContext(
                        "/META-INF/spring/integration/amqp/cafeDemo-amqp-xml.xml",
                        CafeDemoAppAmqp.class,"cloud");
        ...
   }


Conclusion

Micro Cloud Foundry provides a simple way of deploying and testing applications along with their associated services without the headaches of manually installing all the components. It provides more open access than an external cloud making it possible to accesses tools and monitors that would not otherwise be available.

Tuesday, November 8, 2011

Running RabbitMQ 2.6 and 2.7 on Mac OS/X

AMQP is an open protocol for Asynchronous message queueing.  The number one implementation for AMQP is RabbitMQ.
 
I normally run RabbitMQ on my Mac in a Linux/Ubuntu VM  but I recently got a new Macbook Air that's limited to 4GB.  I wanted to run RabbitMQ native on the Macbook Air instead of the Linux virtual machine to save space and give me better performance when running tcServer, RabbitMQ and STS on this slightly smaller machine. 

Installation
RabbitMQ is written in Erlang.  The Mac doesn't come with Erlang so you have to install it. You can use Macports or Homebrew per the Rabbit installation instructions.  I'm lazy so I used Rudix  Erlang-Solutions which provides Erlang in neat Mac installation packages that put the files in /usr/local /usr/bin. (The rudix version of Erlang went away over the weekend of 11/12/2011). 
  1. Download the img/package mentioned above and install it like any other .img file.
  2. Download RabbitMQ and unpack it into some working directory.
  3. cd into the RabbitMQ sbin directory.
  4. Start the server with sudo ./rabbitmq-server
  5. Use the sudo sbin/rabbitmqctl status command to see the status of the broker.
You now have a running RabbitMQ broker with only administrative exchanges and no queues configured.  You should now stop that server if you want to install the optional Web Management console or any other plugin.

Web Management Console
The web management console makes troubleshooting and experimentation easier. The steps for configuring plugins differ based on the version. 

Users access the web management console using URL http://localhost:55672/mgmt/. The default username and password are guest/guest. The console uses JavaScript so make sure you have JavaScript enabled for that web site.

Rabbit 2.6 
The plugins must be manually downloaded and installed in Rabbit 2.6
  1. Download the RabbitMQ management console components from the plugins page.  Download the plugins and copy them into RabbitMQ the plugins directory. Each of the plugins has a .ez file extension. The console plug-in requires 6 components at the time of this blog posting.
  2. Restart the RabbitMQ server. You should see the plugins load in the console output.
  3. Connect to the management console with a web browser and log in.
Rabbit 2.7 
The plugins are installed but not enabled for RabbitMQ 2.7.  You manually enable plugins with the new sbin/rabbitmq-plugins command.  First bring down your RabbitMQ server you started above.
  1. Create the location where the config file will be generated sudo mkdir /etc/rabbitmq
  2. cd into <rabbitmq_home>/sbin
  3. run sudo ./rabbitmq-plugins enable rabbitmq_management to create the config file and add this plugin. Note that no plug-in version numbers are specified. You should see that it adds all of the necessary dependencies.  A core dump probably means you forgot to use "sudo"
  4. Start the RabbitMQ server as you did in the previous steps.
  5. Connect to the management console with a web browser and log in.

Conclusion
You should now have a running server with the optional web console configured and loaded.  Use the previous section as a guide if you want to add additional plug-ins.  See the Rabbit tutorials for examples on how you can use the RabbitMQ server configured above.



Wednesday, May 4, 2011

SeeedStudio 4x4x4 LED cube and Rainboduino

Overview



The Seeed Studio Rainbow Cube Kit is a 4x4x4 RGB LED kit with 64 tri-color LEDs for a total 192 connections. The kit comes with professionally designed components. A connector on the bottom of the kit is designed to mate up with the Seeed Studio Rainboduino which contains all of the drivers and connections needed to control an 8x8 tri-color LED array.  The LED cube comes nicely packaged with no installation instructions, schematics or hints.   The LED Cube does have it's own wiki page, which is probably the worst wiki page on the planet with no schematics, theory of operation or description of what kind of line driving is required by controller board. The LED cube wiki does contain a link to a slick Rainbowduino driver program. Assembly instructions are available on the Seeed Studio site. Robotshop has done an instructable and has the assembly instructions on their site.





The Rainboduino is an Arduino clone specifically designed for LED control.  It was  designed for an 8x8, 64 tri-color LED with 192 connections the same as a used for the 4x4x4 cube.  It comes with no schematics, instructions or internet references.



The Red, Green and Blue squares identify the RGB drivers.  The two 4 pin connectors on either side of Green are the 8 selector lines. The black connector on the left hand side contains inbound I2C and serial pins. You will use the serial pins to program the board.  The connector on the right hand side  comes with the I2C header installed by default.



You can see a SeeedStudio BluetoothBee installed in the upper portion of the board.  This cannot be used without soldering an additional connector on the Rainbowduino.



Assembly

The on line assembly guide is pretty good about telling you how to assemble the cube to be controlled over I2C or daisy chained together.  Assembly took me about 4 hours.  You need to solder everything to the base plate before attaching the LED cube risers to the bottom plate.   It is pretty hard to solder connections on the top side of the plate once all the cross pieces are in place.  You will need an external USB/serial adapter or an arduino to program the device. I used the UartsB since it came in RobotShop  kit.





Ignore this if you are not interested in the on-board XBee socket  The kit comes with parts to solder an XBee socket with 3.3V/5V level converters on the underside of the main board. The on-line guide describes component placement for the socket. There is no description of how to use the socket and how to connect the GND, RX and TX lines from the soldered in socket. It turns out the Rainbowduino pins of the downstream (right hand side in the picture above) are directly above a row of header mounting points on the base of the cube.  You can connect the serial lines from the XBee to the Rainbowduino by soldering a 4 pin connector in the position shown on the right.  You should not make the connector any longer, going up, because it will conflict with the I2C pins already mounted on the Rainbowduino.



The earlier picture of the bottom of the cube has been annotated as to which pins should have connector installed to mate with the one installed on the motherboard. This picture shows a serial connector mounted to the Rainbowduino board that will mate up with the 4 pin connector previously installed on the cube base.



Firmware

The simplest way to program the device is to get the UartsB USB/serial converter. It plugs into one edge of the Rainbowduino board. (Remove this if communicating over I2C or Bluetooth)  Remember to remove any bluetooth adapters when programming. They share the same lines as the USB/serial adapter and they will conflict.



The firmware only knows about the 8x8 mapping so PC software must map from 8x8x3 Rainbowduino programming model to the 4x4x4x3 LED cub physical model.  It is possible to write different Rainboduino firmware that intrinsically understands the cube. You can find links to firmware that does this in this blog. The Rainbowduino firmware supports 12 bit (4,4,4) color using a 96 byte command payload. The default Rainboduino software accepts i2c commands to set the colors on the lights and supports the daisy chaining of controllers to build up strings of 8x8 panels.  There is no information as to what version of the firmware comes preloaded on the Rainbowduino. This blog contains one of the better first steps overview. There is a Rainboduino wiki page that should scare away all but the most motivated.  It contains links to at least three different sets of firmware.  All require an I2C master to talk to the Rainbowduino once it is flashed.

  1. The stock firmware. There appears to be a version that may talk over a serial port possibly negating the need for an additional arduino.
  2. Alternative firmware
  3. Neorainbow firmware  This appears to have the most active development and the biggest feature set.
  4. There is a custom version of the plasma demo that you can load into the Rainboduino that automatically cycles through colors. 
  5. Firmware and library for the processing lauage.
Directly Supporting 3D

I have updated the stock firmware to create serial port firmware to add 3D drawing methods.  They handle the mapping from the 2D 8x8 internal representation to the 3D 4x4x4 cube drawing space. The code uses this array to map from x,y,z to row,col that the actual frame buffer uses. The firmware is available on GitHub

unsigned int cubeToRowCol[4][4][4] PROGMEM = {
  // x 0
  {
    {0x0007,0x0006,0x0005,0x0004}, // y=0
    {0x0107,0x0106,0x0105,0x0104}, // y=1
    {0x0207,0x0206,0x0205,0x0204}, // y=2
    {0x0307,0x0306,0x0305,0x0304}, // y=3
  },
  // x 1
  {
    {0x0000,0x0001,0x0002,0x0003,}, // y=0
    {0x0100,0x0101,0x0102,0x0103,}, // y=1
    {0x0200,0x0201,0x0202,0x0203,}, // y=2
    {0x0300,0x0301,0x0302,0x0303,}, // y=3
  },
  // x 2
  {
    {0x0407,0x0406,0x0405,0x0404,}, // y=0
    {0x0507,0x0506,0x0505,0x0504,}, // y=1
    {0x0607,0x0606,0x0605,0x0604,}, // y=2
    {0x0707,0x0706,0x0705,0x0704,}, // y=3
  },
  // x 3
  {
    {0x0400,0x0401,0x0402,0x0403,}, // y=0
    {0x0500,0x0501,0x0502,0x0503,}, // y=1
    {0x0600,0x0601,0x0602,0x0603,}, // y=2
    {0x0700,0x0701,0x0702,0x0703,}, // y=3
  }
};
The standard 7 byte command structure is too short for 3D drawing with RGB values so I don't see any way to add 3D drawing commands to the stock firmware without breaking backwards compatibility.  The stock firmware is limited to 7 bytes 'R',cmd_byte,5 data bytes which doesn't work that well for x,y,z,r,g,b so the 3D drawing command packs x,y,z in the first data byte with two bits per pixel. A sample perl script is included in the zip package that shows how to use this new command.

$DRAW_DOT_CMD = 5;
sub sendPoint
{
    my $x = $_[0];
    my $y = $_[1];
    my $z = $_[2];
    my $coord = ($x << 4) | ($y <<2) | $z;
    my $r = $_[3];
    my $g = $_[4];
    my $b = $_[5];
    my $stringToSend = sprintf("R%c%c%c%c%c%c",$DRAW_DOT_CMD,$coord,$r,$g,$b,0);
    if ($DEBUG){
        my $commandHex = sprintf("R%d-%x-%x-%x-%x-%x",$DRAW_DOT_CMD,$coord,$r,$g,$b,0);
        my $debugString = sprintf "Command is %s - %i characters\n",$commandHex,length($stringToSend);
        print($debugString);
    }
    $port->write("$stringToSend");
}


 Robotshop.com sells a full kit as a single SKU.  They also have a self-produced video that shows how to assemble the cube and attach the Rainbowduino.  They have a complete diy package that includes the the LED cube, Rainboduino and a UartsB that lets you directly re-program the Rainboduino without going through an Arduino.  The UartsB is actually a USB to serial bridge making the Rainboduino appear as a virtual serial port.  The stock version of the firmware is designed to control a series of cubes via I2C.  The examples show how to do that using an Arduino .  I thought about using a Chumby as my I2C master but it runs at 3.3v and the Rainboduino/Arduino run at 5V so I instead loaded an alternate firmware that accepts commands via the UartsB allowing me to reprogram and control using the same port.



UartsBee (optional) Bluetooth Control

I normally used the Sparkfun BlueSmirf Bluetooth adapters but purchased the SeeedStudio Bluetooth Bee because it fits on the UartsB and because it could be installed in the on-board socket so that the UartsB USB/serial adapter can be removed.  See the section above on what hardware needs to be installed. The UartsBee runs at a default speed of 38400bps.  You should add something like the following code to the Rainbowduino initialization sequence after setting the data rate to 38400.

            delay(1000);
    Serial.print("\r\n+STWMOD=0\r\n");     // slave
    Serial.print("\r\n+STNA=LEDCube\r\n"); // device name
    Serial.print("\r\n+STAUTO=0\r\n");     // auto connect to last device 0=no,1=yes
    Serial.print("\r\n+STOAUT=1\r\n");     // permit paired device to connect 0=no,1=yes
    Serial.print("\r\n +STPIN=1234\r\n");  // set pin to 1234
    delay(2000); // This delay is required.
    Serial.print("\r\n+INQ=1\r\n");        // Slave/Enable being inquired
    delay(2000); // This delay is required.


 

Troubleshooting Tips

The Rainbowduino looks like an Arduino Duemilanove.  You need to set the board type in the IDE in order to upload firmware.



The cube is divided in halves  A and B. Rainbowduino is designed to drive an 8x8 display with 8 select lines.  Each A/B post contains RGB lines and a single select line.  Each select line drives 8 LEDs on a single level, two rows of 4 closest to that side's posts. The instructions tell you to short 4 jumpers on each the Side A and Side B supports, one on each level. (Do not connect all 16 jumpers on each side.) These jumpers connect the + select line from the base of that support to the bank of 8 (2x4) LEDs  on the level next to the jumper.  The jumpers actually connect to the post/level solder pad closest to the jumper.  A dark set  lights either means the jumper isn't connected or that the join at that corner isn't soldered together.  I had to solder a jumper form the pads to the corner solder joint for a couple of mine, probably because I had played with the jumpers a couple times while troubleshooting.  All of the lights on one side always changing together may mean you soldered too many jumpers.  You only need 4 per side.



This LED cube used as a CI build light.  Each level represents a different build. The top build is blinking (in progress). The second level down shows a broken build.  The bottom two builds were both successful in their last iteration. The cube is driven over bluetooth by a java program using rxtx to talk to the virtual serial port. The foam plate  is used as a reflector to increase contrast in an office environment.

Tuesday, April 26, 2011

Extreme Feedback aka Status Lights (Das Blinkenlights)

Monitoring Automated Builds
The Agile concept of continuous integration is useful no matter what the actual software development methodology.  This is especially true as teams grow, projects branch and activity increases.  We initially set up Hudson for our MAIN and FUTURE branches with one build per branch. Hudson notifies everyone about build failures via email.

That works fine but it developers eventually suffer e-mail overload so we needed a more noticeable way of telling folks when builds are broken.  (For us, broken means either compilation or unit tests fail).   We purchased two Delcom USB powered lights and mounted them on the wall where everyone can see them. A Groovy script reads the Hudson build status for our two main branches using the CruiseControl (cctray.xml) URL and then sends commands to the build lights. The cctray standard is limited and doesn't show how far along a build and doesn't differentiate between compilation and unit tests.

The Delcom lights that support RGB and blinking via commands over USB.  We currently use the older Generation 1 lights connected to one of the developer's Windows development machines. Newer 2nd Generation versions of these lights act as HID devices which means they should work on any system in a driverless fashion.   Each light shows two types of information using the color and blink/no-blink.

Color
  • Green: previous build compilation and tests succeeded
  • Red: previous build compilation or tests failed
  • Yellow:  can't reach the CI server
Blink
  • Blinking: a build is in progress.
  • Solid: no buld in progress
The Delcom lights should work for most teams and I highly recommend them.  We wanted lots of status lights with a lower cost.

More Build Lights
Our development team has 35 developers operating on 3 branches across 650,00 lines of Java code plus about the same amount of Javascript, rule engine, xhtml and xml code.  We have Hudson continuous integration set up on all branches for compilation and unit test and have integration, deployment and automated UI builds.  Our full build takes about 40 minutes with unit tests.  The continuous integration builds whenever changes are made and runs all the unit tests and then deploys the main applications to verify deployment still works.  Integration test builds and Sonar metrics builds run once per day. There are over 20+ Hudson build jobs configured in the system. when you take into account all the branches and the types of builds done.

Rolling Our Own
This means we needed more lights so I prototyped a dual build light using an Arduio Uno and two RGB  Sparkfun LED breakout kits along with a wood box and two candle holders from Michael's crafts. The first picture shows the completed system with the USB cord coming out of the front in the lower left.


This picture shows the inside of the box.  You can see the two candle holders hot glued into the top of the box.  The two LED units are each in their own compartment in attempt to reduce bleeding from one light to another. The blue board in the bottom left is the Arduino processor.

The Arduio is a USB powered micro-controller board with a nice IDE and lots of pins to drive LEDs.  It emulates a serial port under Linux and MS Windows so almost any language can talk to it once you write the firmware for the board. The USB/Serial port defaults to 9600 baud. The Arduino has 6 PWM ports that make it simple to programatically set the brightness of the LEDs.  This lets us drive 2 RGB units or 3 RG units while still staying within the limits of the USB port. The Sparkfun LED breakout kits provide the buffering and amplification required to connect high current drain LEDs to micro-controllers without having to design any circuits or source any parts.  Each LED uses about 80mA.  The Arduino and LED kits are currently stocked at the local Microcenter, if you're lucky enough to have one nearby.


Each light requires 3 (RGB) signal connections, 5V power and Ground.  This picture shows the two RGB connections on the left side and the two power/ground connections on the right side for a total of 10 connections. I used some jumper cables I had from another project but you could solder them, make jumper cables or used headers and sockets to do the connections.


The GitHub firmware assumes the first RGB is on pins 9,10,11 and the second RGB is on 3,5,6

More RGB Lights with the PCA9635

I wanted to collapse down our 3 build light devices into a single unit so I built a lunch box build light with 4 indicators on it. This once again uses the Spark Fun LED breakout kits.  The covers are actually the central tube to a roll of large format plotter paper. I cut the tube in half, cut holes in the lunch box and then hot glued the tubes in.  The ends of the tubes are cut-up paper roll end protectors that came with the printer paper.



You can add more PWM ports using I2C expander chips like the PCA9635 which provides 16PWM outputs. The Arduino has a pretty straightforward I2C control library makes it simple to talk with the PCA9635. I used the Jee Labs Dimmer Plug which is essentially a breakout board for the PCA9635. Here is an Arduino prototyping shield with the dimmer installed on it.

Each of the lights needs a cable but the pinout on the dimmer plug means the you can't just plug 4 cables straight in. JeeArduino.



The GitHub firmware assumes the LEDs are configured on the plugs in the following RGB sets.
    {{0,1,2},{5,6,7},{8,9,10},{11,12,13},{3,4,14}}


Build Light Firmware
The Arduino is a stand alone computer that requires firmware to do what you want.  There are lots of LED examples floating around the Internet that can be used as a basis for this project. I wrote firmware that provides a tiny command interpreter that lets you set the colors, brightness and blink rate of individual LEDs. The code ramps the LED brightness to the desired levels instead of just going immediately there. This provides a smoother on and off transitions.  All commands follow this format
  1.  The '~' character
  2. A single command character
  3. The RGB unit number, 0 or 1
  4. The data for the command
The following commands are supported
  • ~c#RGB;  set color    #=triplet, RGB is 0-F
  • ~b#lllddd; set blink    #=triplet, lll is # # 1/2 secs lit and ddd is # of 1/2 secs dark
  • ~q#;           query status #=triplet, results 3 LED x4 attrib
  • ~h;             this help string
Example: 
  • ~b1666111; blinks RGB LED 1 with 3 seconds on and 1/2 second off.
  • ~b1600111; blinks R LED 1 with 3 seconds on and 1/2 second off

The firmware uses a PCA9635 if available and automatically falls back to the two RGB lights directly connected to the Arduino PWM ports. You can download this code from GitHub, load  it into the arduino IDE, compile and then upload to the Arduino.

From Hudson to Status Lights via Java
Here is Java program that drives these lights with data from Cruise Control or Hudson This program reads status of any system producing cc.xml file output and sends the status to various lights.  I've included drivers for the Ambient Orb, the Seeed Studio  LED Cube and an Arduino project I built.  It is pretty easy to add any Serial port based device.  The program uses rxtx as it's library.  I've only included the windows dll but it should work fine for unix machines also. The program includes the maven build and execution pom.xml.

Future Directions
We have 20+ CI builds so we'd really like to have 20+ CI lights.  Scrolling LED displays aren't exactly what we're looking for and aren't visible from all angles so were are thinking of using addressable RGB LED strips.

USB tethering in the project listed above is nice because it provides power and communications but restricts placement.  We're looking at converting to bluetooth communication.  The lights could then be AC powered and we could communicate using one of the Serial port emulating Bluetooth modules. This would make it possible to put the lights anywhere within Bluetooth range instead of being restricted by the USB cable lengths and power capabilities of the USB ports.

Wednesday, February 2, 2011

Adding an I2C device to a Chumby

Chumby I2C

The Chumby is a 3.3V device.  This means that you have to use 3.3V signal compatible devices.  Several of the JeeLabs I2C devices are 3.3V signal compatible. It is possible for some of the 3.3V I2C devices to control higher voltages either through their intrinsic design or through the use of opto-isolators or relays.

I2C connection pins are available on the Hacker Port on most models. There are some internal devices at certain address so you have to make sure that you don't add additional devices that conflict.  I cut one end off of a JeesLabs extension cables and soldered it to the connector.  We don't use the IRQ line so it can be removed or taped off to avoid shorts.  This is pretty straightforward.



I2C uses 7 bit addressing on a bus that does 8 bit transfers.  The thrifty I2C folks make use of the extra bit as a R/W bit.  They standardized on the addresses in the upper 7 bits and the R/W bit as the least significant bit.  This means that the address is shifted up one bit when calculating addresses in programs. Chumby forum postings indicated that devices exist at:

Address  Device
0x3AAccelerometer
0x56FM Radio
0xA2coproessor EEPROM
0xA8DCID EEPROM
0xAADCID EEPROM
0xACDCID EEPROM
0xAEDCID EEPROM

Device addresses are programmed as x2 addresses. A JeeLabs dimmer at 0x40 (64) is programmed as address 0x80 (128).  The programming libraries could have abstracted this away but it don't.  Everyone should just get used to the idea of this one bit shift.  You will see this in the programming examples below.

I2C controllers normally require a 4 wire connection, power, ground, SDA and SCL.  Some I2C devices may provide power to devices on their ports. This means some controllers have an extra pin for alternate voltages.  This is also useful when attempting to isolate the external device power bus from the I2C control bus power.

JeeLabs I2C Devices

JeeLabs has their own pin naming convention that they use for I2C and non-I2C devices so you have to map them to normal I2C pin names when connecting to the Chumby.

JeeLabs pinChumby pinJeeLabs color
PWR3.3v or 5vRED
DIOSDAYELLOW
GNDGndGREEN (new) BLACK (old)
+3V3.3vBLUE (new) GREEN (old)
AIOSCLWHITE
IRQN/CBLACK (new) BLUE (old)
Note: I received the "old" cables so the wires in the pictures match the "old" color scheme in the picture above.  The JeeLabs PWR and +3V pins can be independent of each other.  The +3V is used to power the chip and the PWR pin is used to provide power to the devices being controlled.

Programming I2C

The folks at ladyada have a great wiki on the Chumby hacker board which is a Falconwing design very similar to the Infocast 3.5" used for this project.  They have a sample program similar to regutil that lets you interrogate and update any i2c device from the command line.  Source and compilation instructions for i2c.c are available at http://www.ladyada.net/learn/chumby/i2c.html  




JeeLabs Dimmer Plug

The JeeLabs dimmer board is a 3.3v compatible I2C device that has 16 independently controllable PWM ports through the use of an NXP PCA9635 chip. This is a really nice board with pass through connectors for daisy chaining boards and with changeable address jumpers to support multiple boards on the bus.  JeeLabs provides an Aruino compatible ports library to simplify talking to their devices. I used their demo program as a basis for my example shell script.

The demo setup ran an LED under dimmer control. The LED was connected from the PWR pin to one of the PWM pins. I connected PWR to a 5V pin on the Chumby to provide higher voltages to my LED strings.  We can always use PWM to dim them down later without wasting juice on big resistors.


Example Program for Dimmer Plug

I wrote a shell script that demonstrates how to use that i2c program to exercise the JeesLabs dimmer unit. The following script configures the device and flashes an LED connected to PWM0.  An LED is connected to the PWR pin and PWM0 pin with the assumption that PWM0 will sink current from the PWR pin. You can download DimmerDemo.bash

You can create custom C code that provides a device specific UI based on the i2c example program. The folks at ladyada.net did that for the Chumby-One/Hacker boards that include an accelerometer. acceld_i2c.c in the Chumby source tree is also a good code example on how to talk I2C from a C program.

JeeLabs Output Plug
The JeeLabs output board is a 3.3v compatible I2C device that has 8 independently controllable output GPIO each of which has a Darlington transistor pair on it to increase the current limit. JeeLabs docs state that each pin can sink 500ma. I/O is controlled by a Microchip MCP23008T chip. This is a really nice board with pass through connectors for daisy chaining boards and with changeable address jumpers to support multiple boards on the bus.  JeeLabs provides an Aruino compatible ports library to simplify talking to their devices. I used their demo program as a basis for my example shell script.

The demo setup ran an LED under output plug control. The LED was connected from the 3.3v pin to one of the IO pins. Real world applications would either use the PWR connection or an external power source. The "common" pin was tied to 3.3v. It acts as the reference voltage for a diode in the driver chip.

Example Program for Output Plug
I wrote a shell script that demonstrates how to use that i2c program to exercise the JeesLabs Output Plug. The following script configures the device and toggles all of the putput pins flashing any connected LEDs.  An LED was connected from a power source (3.3v) to one of the I/O pins.  Download OutputPlugDemo.bash to see this example

JeeLabs Expander Plug
The JeeLabs Expander board is a 3.3v compatible I2C device that has 8 independently controllable  GPIO I/O is controlled by the same Microchip MCP23008T used on the I/O board . This is a really nice board with pass through connectors for daisy chaining boards and with changeable address jumpers to support multiple boards on the bus.  JeeLabs provides an Aruino compatible ports library to simplify talking to their devices. I used their demo program as a basis for my example shell script.

The demo setup reads the input levels one time per second for 10 seconds. I connected a couple pins to ground and turned on the pull-ups so that i could get a mix of low and high values

Example Program for Expander Plug
I wrote a shell script that demonstrates how to use that i2c program to exercise the JeesLabs Output Plug. Download ExpanderPlugDemo.bash .  It configures the device and reads the values from the GPIO pins.

Conclusion

It is really easy to add I2C devices to a Chumby.

Saturday, January 15, 2011

Installing a serial port connector on the Chumby/Infocast

Overview
Both the Infocast 8" NS-DP8CH and the Infocast 3.5" NS-DP3CH have 3.3v TTL level serial ports available on connector ports on the motherboards. The serial ports come with transmit (TX), receive (rx) and ground connections and are enabled by default by the Chumby operating system as the boot log on startup and as a serial based terminal after boot.  Traditional computer serial ports operate at different voltages and modern computers often have no serial ports at all.  Developers and users can connect to the Chumby through the use of a USB to 3.3v serial device adapter cables.  Pre-made ftdi brand cables are available on the internet but a lot of folks hack up cheap or surplus USB-cell phone cables like the Nokia. The Chumby is a 3.3v system so don't use a 5v cable. Search on the internet for appropriate cables. USB to serial cables appear to operating systems as (virtual) COM ports.  Mine shows up as COM4 on my Windows 7 machine.  Many terminal emulators communicate with the device over the virtual serial port as if it was a true serial port simplifying configuration when communicating with the device. I use Putty on my Windows 7 machine.

I like to bring the serial port out of the case with a stereo socket connector.  This lets us communicate with the device through the jack without opening up the Chumby. Stereo jacks have 3 wires which matches up with our TX, RX and Ground needs.  FTDI sells a USB to stereo jack cable but I make my own following their pinout. TX is on the tip, followed by RX on the 2nd ring and ground closest to  the plug.   The wires are TX<-->RX RX<-->TX and ground<-->ground.  So the stereo socket mounted on the side of the box will connect the Chumby's RX to the tip, TX to the middle ring and ground to the ground ring.  The basic steps are.

Infocast 8"
  1. Prepare or modify your USB-to-ttl cable. It should be a bus powered cable where the power comes fro the PCGo to radio shack and buy a stereo socket.  I shredded an old ISA card for its audio out jacks.
  2. Open up the Infocast.  On the 8", there are 4 screws on the bottom under the rubber feet.  There are also two locks in the center of the base on the long sides. I just pried until they popped off but there may be some special way to take off the bottom.  The 3.5" exposes 4 screws when looking at the device from the back. Unscrew the 4 motherboard screws and cautiously remove all the connectors.  Note that you will have to remove the LCD cable from its' connector on the motherboard. Pry the grey bar up to release the cable. This picture shows the motherboard with a detached LCD cable


  3. Find the serial connector on the motherboard at the bottom left as pictured below. Note that the heat shield leaves all the interesting ports exposed. The following pictures shows an open Infocast 8" from the bottom.  The unit is face down on the table.

  4. Solder wires to the TX, RX, GND connectors on the serial port connection points.  I like colored cables because it makes easier to figure out what I'm doing I put a little dab of hot clue on my to holder it to the board as a strain relief.
  5. Solder the TX, RX and GND wires stereo jack making sure that you put the Infocast on the TX on the middle ring and the RX to the tip. The picture below shows the 3 wire connection to the serial port connector (right under the flash glare)  This board also has two additional cables attached.  The grey cable is connected to the ground and the 4 GPIO pins and has a 5 pin female header socket at the end.  The colored wires are connected to the two i2c lines, ground and the 3.3v power pin and have a 4 pin female header socket at the end.  This cable's ground line is tied to an unused USB grounding point so that I didn't have to try and solder two wires to the single ground point on the 8 pin port.
     
  6. Mount the stereo jack in the side of the Infocast (shown facing down). The empty space next to the motherboard provides ample room for one or more stereo connectors.  Youi might be able to barely pick out the grey GPIO cable mentioned above. It and the i2c cables are just left with the connectors on when the unit is sealed up. That lets me use those lines later without having to take the whole unit apart and start soldering

Here's the finished product.


Infocast 3.5"
The 3.5" infocast comes with two unpopulated connectors. The serial. port is in the corner right next to the power button. The following picture shows a right angle connector. The idea here was that a small slit could be cut in the back of the unit next to the power button and that an external connector could be pushed into the unit and on to the pins. 

An alternative is to use a straight through header and then create a short stereo jack to header adapter cable. The jack can be mounted anywhere there is room.






The following picture shows the 4 pin serial port connector in the bottom left corner.  The 3.3v power pin has been cut because it was not needed and it simplifies figuring out the orientation of  the header style connector from a USB/TTL serial cable.


Here you can see that the connector has been replaced with a straight connector.  I made up a 3 pin cable with a 4 pin header connector.  Just the TX, RX and Ground pins have been connected. The gray plastic under the RYB cable is the rotary encoder used for volume control.  We will use the empty space behind that for the stereo jack that will be mounted to the case.


The three wires are connected to the serial headphone jack purchased at Radio Shack. We connect the RX pin on the tip, the TX pin on the middle and the ground on the ring closest to the base.  This will connect the USB serial cable's TX (on the tip) to the Chumby RX and the USB cable's RX (2nd ring) to the Chumby TX.  You could mount the connector on the back of the unit but liked putting it on the side of the box far enough back that it will not interfere with the rotary encoder on that side.

 

Here's the 3.5" unit all buttoned up with the console port installed and labeled.


Wrap Up
Exposing the internal serial port provides a way of communicating with the Infocast machines without enabling SSH.  You just plug in your USB/TTL serial cable and fire up your terminal emulator.  The TX and RX pins can be converted to PWM lines with a few regutil commands. You can then use the installed stereo connector as a way of bring out those pwm signals. 

The Chumby will also bring up a terminal on the device if you plug in a USB keyboard. The downside of this approach is that it ties up the chumby screen while you are exploring.

Thursday, January 6, 2011

Hints on installing a qt webkit browser on the Chumby/Infocast

This Chumby Forum posting describes how to install a webkit browser on a Silvermoon/Infocast 8" device and on the Falconwing/ChumbyOne device. 


Silvermoom Infocast 8" Setup 
Download the pre-built browser usb image that is designed to run on a usb thumb drive (/mnt/usb).  Unpack the image to the root directory of the thumb drive.  Stick the thumb drive into the device and restart the device.  It should come up with the web browser on the www.chumby.com page.  This relies on the fact that the Chumby system will run  the debugchumby in the root of any thumb drive that is inserted into a usb port.  The debugchumby script can be customized with an editor to drive the web browser to any url that you wish. The system also supports a USB keyboard for URL entry if you are running the browser version with the URL bar.

Webkit browser components can be built to operate in various ways.  The one from the package above runs as a window. The actual browser executable is in the demos/browser folder. You can download alternative full screen browser or a kiosk style browser components that are used with the image mentioned above.  Download the appropriate executable and put into the demos/browser folder.  You can overwrite the existing demos/browser/browser executable on the thumb drive or you can add it to the folder with a different name and edit the debugchumby script to use that browser.  

The best part of a thumb drive installation is that you can restore he Chumby to it's original configuration just by unplugging the thumb drive.  Some folks will want to "modify" their devices to run the browser without the need for a USB drive.  This is still reversible but requires a little more software skills.  Most Chumby devices enough space to install this browser directly to the microsd card in the /mnt/storage directory.  I took the following steps.

  1. SSH into the device using instructions available on the chumby wiki or my other blog postings.
  2. Make a directory on /mnt/storage.  I used /mnt/storage/browser.
  3. cd into the directory
  4. Download the zip package to /mnt/storage/browser using wget <http_path_tofilename>
  5. Unzip the package in the browser directory using unzip <filename.zip>
  6. The file permissions may be wrong for execution.  Turn on the execute bits of the appropriate files with chmod +x <filename>.  While sitting in /mnt/storage/browser I did chmod +x debugchumby bin/* demos/browser/browser
  7. Rename debugchumby to <some_name>.sh The .sh suffix isn't required but  makes it easier to see that it's a script. Some web server cgi-bin configurations require the .sh suffix to recognize the file as a shell script. Of course that only matters if you start switch to "browser mode" via cgi-bin
  8. Change the script to point at the browser in the new (non thumb drive) location.  Here is my script. You can see that I'm playing around with different browser styles and default web sites.
#!/bin/sh

CHUMBY_BROWSER_ROOT=/mnt/storage/browser
export LD_LIBRARY_PATH=$CHUMBY_BROWSER_ROOT/lib:$LD_LIBRARY_PATH
export QT_QWS_FONTDIR=$CHUMBY_BROWSER_ROOT/lib/fonts

stop_control_panel
rmmod silvermoon_tsb
insmod $CHUMBY_BROWSER_ROOT/silvermoon-tsb.ko scaled_touchscreen=1
switch_fb.sh 0

if [ ! -e $CHUMBY_BROWSER_ROOT/browser_etc ]; then
cp -rP /etc $CHUMBY_BROWSER_ROOT/browser_etc
fi
mount -t loop -o bind $CHUMBY_BROWSER_ROOT/browser_etc /etc

mkdir -p $CHUMBY_BROWSER_ROOT/root
mount -t loop -o bind $CHUMBY_BROWSER_ROOT/root /root

export TSLIB_CONFFILE=$CHUMBY_BROWSER_ROOT/etc/ts.conf
export TSLIB_CALIBFILE=/etc/pointercal
export TSLIB_TSDEVICE=/dev/input/by-id/soc-noserial-event-ts
export TSLIB_PLUGINDIR=$CHUMBY_BROWSER_ROOT/lib/ts
export QWS_MOUSE_PROTO=Tslib

if [ ! -e /etc/pointercal ]; then
$CHUMBY_BROWSER_ROOT/bin/ts_calibrate
sync
fi

while [ 1 ] ; do
# regular browser
# $CHUMBY_BROWSER_ROOT/demos/browser/browser -qws 'www.chumby.com'
# fulscreen browser
# $CHUMBY_BROWSER_ROOT/demos/browser/browser_qt-4.6.3-fullscreen -qws 'http://www.yahoo.com'
# no URL browser
$CHUMBY_BROWSER_ROOT/demos/browser/wv -qws 'http://www.yahoo.com'
done

Silvermoom Infocast 8" Execution 

The simplest way to test the browser installation is to just run your start script that you created above.  with sh <name of script>.  The control panel should disappear and the touch screen calibration screen should show up.  Run through the calibration and the browser will startup automatically.  Touchscreen calibration only happens the first time you run the browser.

It is also possible to start the browser programmaticly or remotely.  Possibilities include:
  1. ssh into the box and run the start script.  This is great for debugging but can be cumbersome in any type of real world situation.
  2. Fall back to the default startup method.  Copy the start script to debugchumby on a thumb drive and insert the thumb drive.
  3. Use one of the standard startup hooks.  Create the directory /mnt/storage/psp/rfs1/ and copy the start script into the file userhook1 . I already have a userhook1 there to start my web server so now my file starts both a web server and a web browser.
  4. You could put the start script in the cgi-bin directory of the built in web server. Then you could flip the Chumby into browser mode remotely by requesting that URL with a remote browser.  This lets the device run in "standard" mode until some network tool decides they need to be in browser mode.  This might be useful when maintaining a network of devices.
  5. Do a "no coding" hack on the standard Chumby control panel.   The control panel screen availble from the Pi menu.  It contains a menu of functions where each button actually runs an .sh script located in /usr/chumby/scripts. You can replace one of those scripts with the browser start script. Possible candidates include fb_cgi.sh

You can only return to normal chumby mode by cycle power. There is a good chance that a browser will become standard on the larger screen devices. This blog entry will become obsolete at that time.


Chumby One / Falconwing
Setting up a browser on a small screen Chumby/Infocast is similar to the 8 inch. The main Falconwing browser download contains the entire package. There is an additional web view download that is available for those folks that want to run a browser full screen with no URL entry area.You should download it after installing the full package and put the wv file in the <root_dir>/demos/browser.  I put the whole package in /mnt/storage/browser.  The general steps are:
  1. ssh into the device
  2. mkdir /mnt/storage/browser
  3. cd  browser
  4. wget <falconwing browser package>
  5. unzip <falconwing browser package>
  6. cd demos/browser
  7. wget <the web view single file>
  8. move back to /mnt/storage/browser with cd /mnt/storage/browser
  9. Turn on the execute bits to make shell script execution easier easier  with
    chmod +x debugchumby bin/* demos/browser/browser demos/browser/wv_falconwing
  10. Patch debugchumby. First make a bakcup copy.
  11. Copy debugchumby to the root of a USB thumb drive or mkdir /psp/rfs1 and copy the script to /psp/rfs1/debugchumby
Here is a sample startup script that does the appropriate setup for the browser sitting in /mnt/storage/browser:

#!/bin/sh

CHUMBY_BROWSER_ROOT=/mnt/storage/browser
export LD_LIBRARY_PATH=$CHUMBY_BROWSER_ROOT/lib:$LD_LIBRARY_PATH
export QT_QWS_FONTDIR=$CHUMBY_BROWSER_ROOT/lib/fonts

stop_control_panel
switch_fb.sh 0
/usr/bin/switch_output -l

if [ ! -e $CHUMBY_BROWSER_ROOT/browser_etc ]; then
    cp -rP /etc $CHUMBY_BROWSER_ROOT/browser_etc
fi
mount -t loop -o bind $CHUMBY_BROWSER_ROOT/browser_etc /etc

mkdir -p $CHUMBY_BROWSER_ROOT/root
mount -t loop -o bind $CHUMBY_BROWSER_ROOT/root /root

export TSLIB_CONFFILE=$CHUMBY_BROWSER_ROOT/etc/ts.conf
export TSLIB_CALIBFILE=/etc/pointercal
export TSLIB_TSDEVICE=/dev/input/by-id/soc-noserial-event-ts
export TSLIB_PLUGINDIR=$CHUMBY_BROWSER_ROOT/lib/ts
export QWS_MOUSE_PROTO=Tslib

if [ ! -e /etc/pointercal ]; then
    $CHUMBY_BROWSER_ROOT/bin/ts_calibrate
    sync
fi

while [ 1 ] ; do
#    $CHUMBY_BROWSER_ROOT/demos/browser/browser -qws 'www.chumby.com'
#   Note that webkit views need the fully qualified URL including protocol
    $CHUMBY_BROWSER_ROOT/demos/browser/wv_falconwing -qws 'http://www.chumby.com'
done



Usability Issues 


If you are hosting pages on your web site for this then you might use something like this floating javascript keyboard in your pages
http://www.greywyvern.com/code/javascript/keyboard