High Speed Barcode Scanning on A WAN

Design, Use Cases, and Software Implementation & Adaptability
(Technical description of XD₂) 

XD₂ Barcode Scanning, Box & Shipment Tracking Design Summary

The XD2 design represents many man-years of development at Quantum. It is based on a modular code base for mobile terminals that Quantum Associates has used for a number of very different projects without needing to re-code the infrastructure. It is the intention here to document the functionality and structure of the design with an eye to adaptation for those who are in need of bare code scanning/tracking.

The technology choices we made were based on availability and cost-effectiveness. Our choices have turned out to be very effective. In this case efficiency, flexibility, and scanning performance were given a high priority. The architecture is very efficient while still being highly object-oriented. The ability to concurrently support e.g. 30 terminals running interactive wireless telnet sessions with short latency from a low cost Windows 2000 PC is made possible by measures such as having written our own multi-threaded socket level Telnet server in C++.

The use of early binding between objects was also chosen to optimize performance. XD2 is clearly a Windows NT/2000 based solution, but while there are areas where the implementation makes use of Microsoft technologies, wherever practical we avoided using single vendor components to avoid lock-in. The Telnet server for instance uses standard Socket calls, and could be recompiled under Unix with the COM calls replaced by CORBA or EJB interfaces without a major re-write.

The use of a thin Telnet client in the RF LAN scenario eliminates the need to maintain and update code on the handhelds themselves. 

All inter-module connections were designed for expandability by using a messaging paradigm, rather than function calls. This allows the back end to be appropriately distributed, as will be discussed below. For instance, while COM/DCOM was chosen as a connection method, rather than rely on a wide range of remote methods, and becoming dependent on COM features, we used a simple method carrying a changing text message payload. This makes it easy to swap out the COM link and replace it with a CORBA, Session Bean or EJB connection.  

It is also possible to pass the same message to a local method. There is an option switch in the code to compile the whole application, including a Business Rules back end, to run locally on a handheld. This option comes with a fully developed back-end interface for RF communications that passes only changes in data, to use in a WAN scenario.  

We feel that the modularity of the design lends itself extremely well to integration with other systems. The specific expansion/connection points are discussed below. 

The specific screens that were created for the XD2 application sit on top of this generalized software layer. Because this infrastructure covers many contingencies, it is a wider subject than can be thoroughly covered in a paper of this sort.

Use Cases

The XD2 application tracks shipments from warehouse, through distribution centers, to delivery vehicles. The fundamental use cases for the XD2 handheld terminal user are exceedingly simple and shown on the next page. The basic concept is that Items are placed on Locations by scanning barcodes labels on both the Item and the Location. The main worker interface is presented on wireless terminals and provides screens for scanning Items when they are handled by the dock workers. The system as a whole keeps track of what Items are on what Locations, and when a move occurred. It can determine what Items have been handled, and which ones are lost in transit. A set of separate query tools running on a workstation provides a high level view into the resulting state of the system.

An example of an Item includes a Box, a Pallet, a Container, or a Truck.

An example of a Location includes a Pallet, a Truck, a Dock, or a Conveyor.

The role of Item and Location is the role of contained and container. An Item is what is at a Location.

A Pallet acts as a Location, for a Box that is placed on it, but as an Item when the Pallet is placed in a Truck or on a Conveyor. A Truck is a Location for a Pallet that is placed in it, but becomes an Item when the Truck is scanned to a Dock location.

This relationship allows queries such as “where is Pallet Number XYZ?”, “when was Box number NNN last scanned”, “which Boxes were lost between Dock AAA and Delivery Truck DDD?”.

The concept of planned and unplanned moves allows the system to also notify the user if he is making the wrong move. For instance, a Box that is to be delivered in Boston is to be placed on Locations (e.g. Trucks) that are heading in that direction, not towards Atlanta. An unplanned move is an exception that requires an override.

Locations have destinations. This is part of detecting invalid moves. These destinations are directly or indirectly inferred by the system. Direct inference comes from using a labeling system that identifies destinations. For instance, Pallets are labeled with adhesive barcode labels that identify their source and intended destination. Other Locations may have to be tagged as to destination by a supervisor. For instance the first time a Truck is scanned, the system may complain that its destination is unknown. A screen is available to tack one or more destinations onto a Location.

It may be reasonable for a truck heading from San Francisco to Boston to stop in Atlanta. In that case the Truck will have multiple destinations attached to it. In more complex routing scenarios, intermediate destinations graphs can be built into the system by using the Destinations table and related relations.

The handheld user deals with 7 fundamental cases, of which only 5 are significant:

Use Cases for Handheld Users

The workstation user, usually a supervisor, deals with a separate set of tools that happened to be largely coded in C++ for XD2 (and in VB6 for XD1). A prototype web site was designed that provides the same sort of query capability through web pages.

This is presentation layer material, stuff that could be re-coded as needed in any available language or infrastructure, and we will not delve into the specifics here. Although these modules represent significant value, from an interface standpoint they rely on standard mechanisms. Some screen shots of the workstation tools are included in the Appendix.

The XD₂ Handheld Screens

Not counting Alerts and Confirmation Dialogs, 8 screens are used to cover the Handheld Users Use Cases.

Basically a user scans a Location, followed by one or more Items, or scans an Item, followed by a Location. If the Move is detected as invalid or unplanned, exception screens notify the user, and prevent further scanning until the situation is resolved or overridden. The detailed use scenarios are discussed in the Appendix.

Barcode Scanning Software Implementation

The software consists of 6 layers as shown below. This is the most important point of interface between XD2 and any new systems, and this is where the object-oriented design provides good connection options. The 6 layers can be stretched across a network, or compressed into a single machine. 

1. Client Handheld session

2. Telnet Screen Handler

3. Screen definitions

4. Business Rules

5. Database

6. Message Queue Interface

  The interaction between layers is shown sequenced below:

1. The handheld itself runs a standard ANSI terminal session over the wireless link to a custom Telnet Server. This works very well for local and continuous RF connection.

2. The Telnet Server listens on a specific port and provides screen handling for the terminal client, and a messaging interface to the Business Rules layer. The screen handler provides cursor movement, scanning input and keyboard entry on screens as compiled into the Server.

3. The screens are defined in C++  #include files for the Telnet Server. A module to read the screen definitions from the database itself was under development. The screen definitions make use of predefined elements and types so that they are easily modified by editing these #include files. New screens can be added in the same manner. A typical screen definition is shown below:

Void ClientSession :: SetupXD2Screen3 (void)
{
        new Screen     (3, "3",         this, 0 );
        new Text       (3, 1, "X-DOCK MENU       03",1, 1,  this );
        new Input      (3, 2, "LoggedIn", 0, 20, READ_ONLY,20,2,1,NONE, this );
        new Button     (3, 3, "B1", "1 SELECT LOCATION", 4, 2, 1, TRANS, 4, 1, this );
        new Button     (3, 4, "B2", "2 SCAN   ONE ITEM", 5, 2, 2, TRANS, 8, 1, this );
        new Button     (3, 5, "B3", "3 LOG OUT        ", 6, 2, 3, GOTO, 2, 7,    this );
}

The details don’t matter, but this initialization lays out the position of the different fields of Screen 3 on an X-Y coordinate plane. For instance Button B1 starts in the 2^nd column of row 4. The violet labels, such as B1, are the field names. These will be passed as a name-value pair along with the field contents in the message that is pumped to the Business Rules software layer. Input areas can be READ_ONLY, SCAN_ONLY, NORMAL, etc.

Button areas invoke an action. The most important are GOTO, which jump to another screen without further processing, and TRANS, that invoke a Transaction from the Business Rules layer. The various field types available are outlined in the Appendix.

When the user activates a TRANS by keying ENTER or scanning into a TRANS input field, the contents of the screen is packaged up in a text message as a set of Name-Value pairs prefaced with the screen ID and the ID of the Activating Field. This message is referred to as a “scrape” as it contains the screen contents. At the time XD2 was designed, XML had not been fully defined, but it would be a very simple modification to now bracket the name-value pairs with XML tags.

4. The Business Rules layer accepts and parses the scrape message, processes the fields and applies them to the Database through Stored Procedures. It also asynchronously kicks off messages into the Message Queue interface. Upon receiving back a Dataset from the database, the Business Rules layer formats the fields into a return message for the Telnet Server, which will insert the fields into a new screenfull in accordance with the layout of the screen whose ID is indicated in the header.

5. The SQL database is the persistent storage for the system. XD2 natively uses Microsoft SQL Server but requires minimal changes to talk to a DB2, Informix, Sybase or Oracle back end. The database is also used to hold sets of SQL statements that implement the transactions that implement the actual Business Rules methods. These stored procedures are used to encapsulate the methods and make them easier to modify than if one had to each time recompile the Business Rules object.

6. The Message Queue interface is used to notify an Upstream System of changes that are noteworthy. It is possible to trigger a message upon any change in location for an item, but the Business Rules level normally aggregates many micro-events before it sends an update.

For instance when a Truck is closed, all the contents of the (truck) Location are sent at once. Contents are defined as 1^st, 2^nd and higher degree depending on how deeply they are nested in Containers. A Pallet sits on a Truck as 1^st degree content, and the Boxes on the Pallet are considered 2^nd degree contents of the Truck. This repeating container-contained relationship has made it simple to add relatively complex features on request without any changes to the framework. At Quantum’s original deployment at an international Office supply firm, initially the arrival of trucks at a dock was not supposed to trigger any notification. By treating the Truck as an item scanned to a (dock) location, it was easy to add this capability.

WAN Barcode Scanning Interface and Expansion points

The XD2 system was designed to be scalable and expandable. All of the software layers thus can run locally on the same machine or be separated over the network. In the most extreme case the division takes the following form, where each circle indicates a separate machine, with the protocols involved shown between them.

Because of this layered design, to integrate XD2 into a wider system is relatively simple, although there are degrees to the task. Generally, practice, one machine runs the Telnet Server and Business Rules, and calls on separate Database Servers. MQ itself can be run in split Client/Server mode or with server installations on each Telnet Server machine. Bandwidth between nodes is a factor in deciding on the optimal split under the circumstances.

In any case XD2 as a whole is a functioning module that represents significant functionality, is ready to use as a building block, and relatively easy to modify.

The big Black Box

The simplest way to integrate it is to consider all of XD2 as a black box that takes IBM MQ messages that define items such as Boxes, including their delivery information, and passes back IBM MQ messages when Locations, “containers”, such as trucks, are closed, or arrive at a new dock. This update message includes a list of all the Items on that Location and is sufficient to feed a presentation layer such as a FedEx Shipment Tracker on e.g. a Web Server. An XD2 domain can cover a single dock, a set of warehouses and docks, or an entire enterprise.

All of the handheld terminals, scanning, and synchronized multiple replicated databases, or a single powerful DB Server for the domain, are handled by this black box. There are of course specifics that must be set up for an enterprise, such as the design of barcode labels for the various cases, the entry of routing tables, etc, but the entire framework is done and you only need to “fill in the blanks”.

What this black box functionally handles is perhaps best illustrated by looking at the gory detail of a multi-site installation:

If a finer reporting granularity is desired, it is simple to modify the Business Rules layer to trigger messages under other conditions, so as to create additional interface points. For instance every scan that changes a location of an item could trigger an update message.

This change involves merely altering a few lines of VB code in the Business Rules object. VB was, incidentally, chosen for this layer because of its high level integration for coding COM/DCOM objects with surprisingly fast and solid multi-threaded performance. This code is OOP-clean and will compile with the new generation of Microsoft .NET tools. All of the MQ messaging code uses standard IBM MQ libraries, and MQ in turn was chosen for its maturity and cross-platform availability.

Extra Message hook interfaces

There are also asynchronous queuing messaging hooks that were developed for use on a LAN without the need to license IBM MQ. These interfaces supplement or replace the MQ message calls, are based on named pipes, and pass text messages like the MQ ones. They are optionally enabled or disabled by changing a single compilation switch. To adapt them to another protocol stack such as SOAP/XML would be quite simple.

One important thing is that these interfaces include their own decoupling/queuing mechanism to get away from classical COM/CORBA/RPC blocking calls and thus can be invoked without holding up a handheld user while a message is passed to an upstream system. This is one of the key problems in a multi-user app such as this, and it has been amply addressed. To replace IBM MQ and instead send to JMS, or MSMQ, or directly to a J2EE App Server, either synchronously, or asynchronously, would therefore be quite doable.

Incoming Data

[XD2MQDBTables]

Standalone Handheld Black Box

Finally, the Telnet screen handler was also designed to implement a standalone system on RIM Handhelds. We did not want to have to keep re-doing the infrastructure, so we designed a code base that would handle these very different scenarios identically. There are compilation switches in the include files that will compile the exact same code as a 16 bit or 32 bit standalone application where the Business Rules layer is replaced by local storage on the Handheld.  A different back-end messaging interface is then enabled for use on a WAN where the handheld might not be continuously in range, and character by character interaction is out of the question.

This is part of the Quantum Dispatch offering, known as Mobi/Quant, but it is in fact the same code base that is compiled for the RIM handhelds as is used for XD1 and XD2. The COM messages that talk to the Business Rules object in XD2 call local functions in the standalone case. The same messaging is used, but all within the standalone handheld. What differs is the screen setup, and which compilation switches are set. In this case the black box looks more like this:

The standalone handheld is a tool used for the delivery portion of the shipping cycle for a region, in conjunction with a central dispatching server. The application presents a list of stops for the driver, with specific shipments to be dropped off and picked up. The RF data messages in and out of this handheld are not concerned with screens, as these are now handled entirely within the handheld. Instead they represent “deltas”, synchronizing changes in the database between the user’s local copy in the handheld, and the master copy on the home base server. A serial connection module allows the local database snapshot to be inexpensively uploaded over a cable into the handheld before the driver leaves the base.  

The standalone mode intrinsically supports the RIM Inter@ctive pagers, but could be modified to support any character-oriented handheld based on a 16 or 32 bit Intel compatible processor, such as the Symbol and LXE models. With a little more change, the code could be compiled for other processors as well.

Barcode Scanning Application Conclusions

We hope to have provided enough information to show how, aside from providing intrinsic value as complete product, the Quantum Associates shared code base offers a good opportunity for quick integration of different wireless handheld subsystems into next generation designs. We believe that this offers a quick jump-start into a wide range of mobile terminal applications with little up-front programming.

Appendix - Details

Scanned Data Feedback: Workstation Tools

A wide range of XD2 tools is available on attached workstations, ranging from control interfaces for various processes like the MQ Listener, to simple query viewers and specialized reports, for instance tracking Boxes that never made it past specific points in the flow.

The tool shown above allows a user to locate a Box. In this case the Box was scanned onto a specific Pallet, then the Pallet with this Box was scanned onto a specific Line Truck. The bottom window shows that this Line Truck, containing this Pallet, was closed at its departing dock, PLN. The relationship of 1^st, 2^nd and 3^rd degree contents is illustrated, allowing a user to track items throughout the system.

Specialized tools exist for editing destination paths, printing barcode labels, and so on. There are some 20 different tools and reporting applications included.

Queries and views were also developed for the proof-of-concept web browser interface.

Recently the release of Borland Kylix on the Linux platform has opened up more RAD development options in that environment. A year ago a Linux port of XD2, including both its Telnet Server and its rich GUI worsktation tools, would have been hard to contemplate, for lack of good RAD tools under Linux. Today that is actually a plausible task.

Barcode Box Tracking Database Structure

It is interesting that a very simple set of tables manages to cover a wide range of situations. The Boxes table represents the lowest granularity in the current design, and it includes many fields passed from the upstream mainframe. The Container-Containee relationship is created by specific events in the Events table as well as triggers that populate the Contents table. The details are beyond the immediate concerns of this document, but the main tables are shown below:

Screen Elements

A handheld terminal typically has limited screen area and requires crafty layout to present functionality. A set of screen attributes is used to create all the screens that the handheld user interacts with. These screens are described in other sections. These attributes include (along with their respective color code as used in illustrations):

BLACK on WHITE for Static areas, the unchanging text.

BLACK on YELLOW for alphanumeric input fields

BLACK on VIOLET for Scan-Only fields

WHITE on BLUE for password fields (only echo asterisks)

Black on LIME for Numeric input fields

BLACK on GREY for Read-Only (program-updated) fields

BLACK on CYAN for Select lists (similar to dropdowns)

WHITE on SALMON for Buttons – ENTER executes function

BLACK on ORANGE for Execute Fields – input field, but ENTER executes now

Scan areas are generally treated like normal input, accepting either keyboard or Scanner entry, though a SCAN_ONLY field type also exists.

he screens are built up from these elements. Like on a 5250 or 3270 screen, the display has input fields and static text areas. 

There are also areas called “buttons”, which are displayed in reverse video on the device. These are like GUI buttons, in that the idea is to put the cursor on it, and “click” it to execute a certain function

There are other resemblances to a GUI interface, in that there is Text input areas, Select Lists (which resemble a GUI dropdown ListBox), as well as Buttons. Since there is no mouse to press the button with, a Button is engaged by moving the cursor to that position, and pressing ENTER.

The cursor is moved around the screen by pressing the TAB key, using the up and down arrow keys, or by pressing ENTER in each input field in succession. Once the cursor is in an EXECUTE field, or on a button, pressing the ENTER key triggers a transaction. The result is that the ENTER key can be used on input screens to step through the fields, entering data as needed, and finally landing on the default first Button, whereupon keying ENTER performs the transaction. A scan in an Execute area normally executes a function as well.

The Buttons can also be directly executed by pressing the Function key whose number matches the number of the button.

ESCape is a very important key. It acts like the BACK button on a web browser, and takes the user back up the tree. It is common to enter a screen, perform some action that does not necessarily change the database, and then key ESCape to back out and do something else. In some of the XD2 screens, this contributes to the way that the same screen can be used for different purposes.

On some keyboard layouts the CLEAR key is mapped to the ESCape function. This is the case for the 1040 Wearables, where there is actually no ESCape key at all.

The 1040 Wearables have a dedicated MENU key, and we use this key if possible to always return to the Top Level Menu.

Barcode Scanner Use Scenarios

Scenario 1: “Forward Barcode Scanning”

User is moving boxes onto pallets. This is the common mode where a user first specifies the Location, then scans the Items onto that Location.

User selects menu item 1 and reaches screen 04, the Select Location screen.

User scans a Target Location, i.e. a Pallet, a Line-haul Truck, a Dock location, a conveyor, a delivery Truck.

In the case below the user has scanned a Putnam to North Reading pallet with a pre-sort destination 01. The pallet happens to be using the 12^th pre-printed license plate of this type, and previously had 2 boxes on board. 

o this point no data has been changed. This screen can thus be used to check the number of pieces a location has on-board.

(The screen can also be used to close out a location by selecting Menu 2. This updates the database to indicate the loading of that location is complete, and takes one to the top menu.)

The user will now normally enter a fast-scan mode. Selecting Menu 1 (Work) places the user in the Scan Item screen:

This screen will accept repetitive scanning. Upon scanning an item, the user can ignore the display and continue scanning more items. If the user pauses, he sees information about the last item scanned.

Here we see the item, e.g. Box Number, how many times it has been scanned at this location, the type of the item (Box, Pallet, etc), the currently assigned location, and the previously assigned location.

In this case there is no previous location, this box is being loaded onto a pre-sort Pallet at the DC.

The user would normally not pause to even look at this information, although he/she could.

If a business rule violation is detected, such as attempting to place a box destined for another Hub, the unit beeps, throws up an Alert and prevents continuing until the user decides whether to override the alert, and bind the item to this loc in spite of warnings, or to set the item aside and continue loading other items.

Menu 2 (Undo) provides the opportunity to undo the last scan and (in the database) puts the item back at its Previous Location.

Menu 1 (Done) goes to the previous screen, the location select screen. This is one way to move to another task. Furthermore, the ESC key always takes one back one screen, so hitting ESC on the Scan Item screen would take the user back to the Select Location screen, where he can Close out the Location using Menu 2, e.g. when he has finished scanning the Boxes onto this Pallet.

After the Close, the fields are cleared, and he is ready to scan another location, or use ESC to back out to the top Menu.

The Location can be closed as described above, or conceivably the user could work another set and not close out the Location until later.

Because the Close is so important in reference to Trucks and unloading docks, to tell the system to send a report to e-Mobile, the default behavior is to alert the user if he does not close out a location before starting on the next one.

Scenario 2, “Reverse Scanning”

User is scanning boxes, then putting them somewhere. This is the operation whereby a user first selects an item, and then specifies its location. This is used when unloading a conveyor belt for instance.

There are several difficulties with reverse scanning multiple items, not the least of which is the inability to alert the user if an invalid move is being made, because the Location will not be known until after the move has been made. This would require keeping a buffer of items scanned, and the ability to step back, both on the display and in the pile of boxes, to identify the wrongly moved item once the Location is specified and the error discovered. This is logistically unattractive.

XD₂ implements Reverse Scanning with a buffer depth of one: one can scan one item, then a location.

The user selects the Scan One Item menu option

This will take him to the simplified Scan Item screen, to allow scanning.