M3 Picking Lists in Google Glass

Here is the first tangible result of M3 Picking lists in Google Glass. This is a continuation of my previous posts in the series: Getting and processing an M3 picking list and Hello Google Glass from Infor Process Automation.

Goal

As a reminder, I’m developing a proof-of-concept to show rich picking lists from Infor M3 in Google Glass with item number, item description, quantity, stock location, image of the item from Infor Document Archive, and walking directions in a warehouse plan. Also, for interactivity, the picker will be able to tap Glass to confirm the picking. Also, the picker will be able to take a picture of a box at packing.

The result will be useful to showcase the integration capabilities of M3, it’s a first implementation of wearable computing for M3, and it sets a precedent for future Augmented Reality experiments with M3. And the advantages for a picker in a warehouse would be more efficiency and new capabilities. And for me it’s a way to keep my skills up-to-date, and it’s an outlet for my creativity. It’s a lot of software development work, and I’m progressing slowly but steadily on evenings and on week-ends. If you would like to participate please let me know.

Why it matters

According to Gartner, by 2016, wearables will emerge as a $10 billion industry [1].

According to Forbes, “Smart glasses with augmented reality (AR) and head-mounted cameras can increase the efficiency of technicians, engineers and other workers in field service, maintenance, healthcare and manufacturing roles” [2].

According to MarketsandMarkets, the Augmented Reality and Virtual Reality market is expected to grow and reach $1.06 billion by 2018 [3].

Basics of Google Glass

Google Glass is one of the first wearables for the mass market. It is a notification device on the face, an eyewear with all the capabilities of an Android device including camera, head-mounted display, touchpad, network connectivity, voice recognition, location and motion sensors.

It works by displaying a timeline of cards we can swipe back and forth to show past and present events.

To write applications for Google Glass we can use the Mirror API, the Glass Development Kit (GDK), or standard Android development. I will use the Mirror API for simplicity.

I will display an M3 picking list as a series of detailed cards on the timeline that pickers can swipe and interact with like a to-do list as they are progressing in their picking.

Card template

For the template of the timeline card, I will use SIMPLEEVENT from the Google Mirror API Playground per picking list line to easily identify the four pieces of information the picker will need to read per picking list line: item quantity, item number, item description, and stock location:
template

Bundling

I will use Bundling with bundleId and isBundleCover to group the cards together by picking list:
bundling

Picking list lines

I will get the picking list lines with the SQL of my previous post, and I will sort them in descending order because Glass will display them reversely last in, first out, thus they will appear naturally in-order to the user.

SELECT H6WHSL, H6ITNO, H6ITDS, H6ALQT
FROM MVXJDTA.MHPICD
WHERE H6CONO=<!CONO> AND H6DLIX=<!DLIX> AND H6PLSX=<!PLSX>
ORDER BY H6WHSL DESC

HTML card

I will use the SIMPLEEVENT template’s HTML fragment and replace the sample values with item quantity ALQT, item number ITNO, item description ITDS, and stock location WHSL:

<article>
  <section>
    <div class="text-auto-size">
      <p class="yellow"><!SQL_H6ALQT><sub><!SQL_H6ITNO></sub></p>
      <p><!SQL_H6ITDS></p>
    </div>
  </section>
  <footer>
    <div><!SQL_H6WHSL></div>
  </footer>
</article>

JSON

I will embed the HTML fragment in the JSON payload for the Mirror API:

{
	"html": html,
	"bundleId": DLIX
}

Bundle cover

The bundle cover will be:

{
	"text": "Picking list <!DLIX>",
	"bundleId": <!DLIX>,
	"isBundleCover": true
}

Infor Process File

My process file in Infor Process Designer is illustrated in this GIF animation (you can open it in Gimp and see the layers in detail):
flow

Problems

I had to solve the following two trivial problems:

  • I had problems parsing new line characters in the MsgBuilder activity node with JavaScript in an Assign activity node. According to the ECMAScript Language Specification – ECMA-262 Edition 5.1 on String Literals the character for new line is simply \n (line feed <LF>). Then Samar explained to me that the MsgBuilder activity node in IPA uses both characters \r\n (carriage return <CR> and line feed <LF>).
  • JSON is not implemented in IPA for JavaScript in the Assign activity node. So I had to manually add it to IPA. I used Douglas Crockford’s json2.js, and I appended it in the two files <IPDesigner>\IPD\pflow.js and <IPALandmark>\system\LPS\pflow.js.

I still have the following problem:

  • The subscription I used in my previous post, M3:MHPICL:U, seems to occur too early in some of my tests, and that is a blocking problem because when my flow runs the SQL to get the picking list lines only gets the first line – which is the only line that exists in the database at that point in time – while the other lines haven’t yet been created in the database at that time and the flow misses them. I must find a solution to this problem. I haven’t been able to reproduce it.

Result

From my previous post, I had the following picking list:

H6WHSL H6ITNO H6ITDS H6ALQT
T0101 TLSITEM01 Item 01 11
T0102 TLSITEM02 Item 02 13
T0301 TLSITEM03 Item 03 17
T0302 TLSITEM04 Item 04 19

When I re-run the scenario, here are the resulting timeline cards in my Google Glass where black pixels are see-through pixels:
2.1 2.2 2.3 2.4 2.5

And here is a video capture of the result (I used Android screencast which captures at a slow frame rate):

And here is a picture of what it would look like to the user in Glass with see-through pixels:
result

Future work

Next, I will implement the following:

  • Programmatically get the OAuth 2.0 token instead of copy/pasting it manually in the WebRun activity nodes.
  • Show the item image from Infor Document Archive.
  • Show walking directions on a warehouse plan.
  • Tap to confirm the picking.
  • Take a picture of the box at packing.

 

That’s it! Check out my previous posts on the project. Like this post. Tell me what you think in the comments section below. Share with your colleagues, customers, partners. Click the Follow button to subscribe to this blog. Be an author and publish your own ideas. And enjoy.

OptiMap_V2

Here is the second version of the OptiMap script for Smart Office that integrates the Delivery Toolbox – MWS410/B with OptiMap – Fastest Roundtrip Solver to calculate and show on Google Maps the fastest roundtrip for the selected Routes. This extends the first version of the script.

In this second version I added the possibility to set the starting address (for example the Warehouse) as the Script argument. See lines 52-54.

import System;
import System.Web;
import System.Windows;
import Mango.UI.Services.Lists;
import MForms;

/*

	OptiMap_V2 for M3
	Thibaud Lopez Schneider, Infor, October 19, 2012 (rev.2)

	This script illustrates how to integrate the Smart Office Delivery Toolbox - MWS410/B with OptiMap - Fastest Roundtrip Solver, http://www.optimap.net/
	to calculate and show on Google Maps the fastest roundtrip for the selected Routes; it's an application of the Traveling Salesman Problem (TSP) to M3.
	This is interesting for a company to reduce overall driving time and cost, and for a driver to optimize its truck load according to the order of delivery.

	To install this script:
	1) Deploy this script in the mne\jscript\ folder of your Smart Office server
	2) Create a Shortcut in MWS410/B to run this script; for that go to MWS410/B > Tools > Personalize > Shortcuts > Advanced > Script Shortcut, set the Name to OptiMap, and set the Script name to OptiMap
	3) Optionally, set the starting address (for example the Warehouse) as the Script argument; the address must be recognized by Google Maps
	4) Create a View (PAVR) in MWS410/B that shows the address columns ADR1, ADR2, ADR3

	To use this script:
	1) Start MWS410/B
	2) Switch to the View (PAVR) that shows the address columns ADR1, ADR2, ADR3
	3) Select multiple Routes in the list (press CTRL to select multiple rows)
	4) Click the OptiMap Shortcut
	5) The Shortcut will run the script, the script will launch OptiMap in a browser and pass the selected addresses as locN parameters in the URL, and OptiMap will optimize the roundtrip

	For more information and screenshots refer to:
	https://thibaudatwork.wordpress.com/2012/10/04/route-optimization-for-mws410-with-optimap/
	https://thibaudatwork.wordpress.com/2013/03/08/optimap_v2/

*/
package MForms.JScript {
	class OptiMap_V2 {
		public function Init(element: Object, args: Object, controller : Object, debug : Object) {
			try {
				// get a reference to the list
				var listControl: MForms.ListControl = controller.RenderEngine.ListControl;
				var listView: System.Windows.Controls.ListView = controller.RenderEngine.ListViewControl;
				if (listControl == null || listView == null) { MessageBox.Show('Error: Couldn\'t find the list.'); return; }
				// get the selected rows
				var rows = listView.SelectedItems; // System.Windows.Controls.SelectedItemCollection
				if (rows == null || rows.Count == 0) { MessageBox.Show('Error: Select multiple routes in the list (press CTRL to select multiple rows).'); return; }
				// get the address columns ADR1, ADR2, ADR3
				var column1: int = listControl.GetColumnIndexByName('ADR1');
				var column2: int = listControl.GetColumnIndexByName('ADR2');
				var column3: int = listControl.GetColumnIndexByName('ADR3');
				if (column1 == -1 || column2 == -1 || column3 == -1) { MessageBox.Show('Error: Couldn\'t find the address columns ADR1, ADR2, ADR3.'); return; }
				// construct the URL
				var query: String = 'http://www.optimap.net/?';
				// set the optional starting address
				var offset: int = 0;
				if (!String.IsNullOrEmpty(args)) { offset=1; query += 'loc0=' + HttpUtility.UrlEncode(args) + '&'; }
				// add the selected addresses
				for (var i: int = 0; i < rows.Count; i++) {
					var row: ListRow = rows[i];
					var ADR1: String = row[column1];
					var ADR2: String = row[column2];
					var ADR3: String = row[column3];
					var address: String = ADR1 + ',' + ADR2 + ',' + ADR3;
					query += 'loc' + (i+offset) + '=' + HttpUtility.UrlEncode(address) + '&';
				}
				// launch OptiMap in a browser
				ScriptUtil.Launch(new Uri(query));
			} catch (ex: Exception) {
				MessageBox.Show(ex);
			}
		}
	}
}

Here is a screenshot of how to set the address as an Argument of the script:

8_

That’s it!

(Oh, and I finally learned how to post source code to WordPress. Duh!)

Related Articles

Google Glass

I just applied to get a pair of Google Glass.

Google Glass is an anticipated product from Google X for bringing Augmented Reality to the masses in a sports fashion pair of glasses containing a video camera, a Heads-Up Display, a processing unit running Android, Wifi connectivity, and a battery (c.f. the patent).

I was at Google I/O 2012 were they accepted pre-orders for Glass Explorer Edition but I made the regretful decision to not apply. Google is now offering a second chance: What would you do if you had Glass? Answer with #ifihadglass.

If I had Glass I would improve the workers job in a warehouse: I would show walking directions to the picking location, I would display information about the item, and I would keep track of the picking list. I’m an enthusiast I/On working on AR in the enterprise.

This would be a continuation of my previous implementation of Augmented Reality for M3.

Here are my three concepts pictures for Google Glass:

1

2

3

Here’s my application:

4

Wish me luck, and see you at Google I/O 2013.

 

M3 + Augmented Reality

In this article I introduce the first implementation that I know of Augmented Reality for Infor M3. Augmented Reality is the ability to superpose digital information on top of real world objects. This is achieved by locating the user’s head in space, by determining the user’s point of view, by registering real world objects, and by projecting virtual 3D objects accordingly. Implementing it has been a deer dream of mine. In this example I use fiducial markers and data coming from Item Master – MMS001.

Applications

Augmented Reality for M3 could be used for many applications. For example, it could help a worker find an Item in the warehouse by showing optimized walking directions and distance to possible picking locations. Also, it could help a worker show contextual information at a glance.

I believe Augmented Reality to be a disruptive technology and one of the next big revolutions in the software industry, with positive impacts similar to those of the Internet and mobile devices, that will reshape entire industries in the next 10 years.

Timeline & motivation

In 1998 I got a summer job in a warehouse for a company that sold car brakes. Every few minutes a printer spit out a picking list of items that I had to collect. As a temporary worker unfamiliar with the place, I spent most of my time wandering through the warehouse, searching for the items, and asking the more seasoned workers for help; I found that inefficient and I wished the computer gave me a map with directions of where to go. Also, the picking lists were un-ordered and I often had to go back to a previous location I had just visited; I found that inefficient and I wished the computer optimized the picking lists. Also, once I found the location, I often discovered the boxes were empty and I had to ask a forklift driver to replenish the stock location from a box of a higher shelf; I found that inefficient and I wished the computer planned replenishment ahead of time. That was in 1998 and nowadays ERP and Warehouse management systems are more common. Yet, I kept my wish to make better systems.

Then, In 2001 I read about Professor Steven Feiner’s Augmented Reality KARMA project from 1992 at Columbia University. The system fit in a backpack and had portable computer, batteries, GPS, compass, and head-mounted display. It would give detailed instructions to a user on how to repair a printer. That was my first exposure to Augmented Reality and ever since I have been wanting to implement it.

In 2007 Apple introduced the iPhone, with a stunning user interface, graphics, and processing power, blowing everybody’s mind about mobility and redefining an industry. And in 2009 Apple added a camera to the iPhone 3GS. The hardware technology for Augmented Reality started becoming accessible to the masses.

In 2009 I met with Brad Neuberg of Google at the Google I/O conference and I started working on a client-side search engine for M3 source code. That was my first exposure to HTML5.

In 2010 I implemented my first Warehouse 3D demo using Google Earth, with real data fed from the ERP, and I projected the result on a large touch screen for an immersive experience. That was my first step towards implemented Augmented Reality for M3.

In 2011 I proposed an idea for an internal project for M3 + Augmented Reality on mobile devices.

In parallel, WHATWG and W3C have been working hard to standardize HTML5 with the ability to use the webcam in JavaScript with WebRTC, to access pixel data, to paint on the canvas, and to use WebGL for 3D rendering. The software technology for Augmented Reality is becoming accessible to the masses.

More recently I started working on geo-locating Stock Locations in M3. This opens the door to new applications for geo-coded data in M3.

Then, at the Google I/O conference this year, I met with Ilmari Heikkinen whom pointed me to his article in HTML5 Rocks on Writing Augmented Reality Applications using JSARToolKit. That was the last push I needed to implement actual Augmented Reality for M3. So I did.

Implementation

I used Ilmari’s source code and I added a few lines of code to call an M3 API using REST in JavaScript when a marker is detected. In this example, the marker is mapped to an Item number (ITNO), but it could also be mapped to a Stock Location (WHSL) for example. Then, for that Item number I call the M3 API MMS200MI.GetItmBasic and I display the Name (ITDS), Description (FUDS), Basic unit of measure (UNMS), Volume (VOL3), Net weight (NEWE), Gross weight (GRWE).

Result

Here is a video of the result. Note the section below the canvas that shows M3 data coming from MMS200MI.GetItmBasic for the detected marker. We can see an activity indicator flickering as the markers are detected. For best viewing, watch the video in YouTube, in HD, and in full screen.

Source code

I provide the result for download at http://ibrix.info/ar/demo.zip with HTML and JavaScript source code, sample fiducial markers, and sample images.

Future work

With the simple example I introduced in this article I illustrate that hardware and software technology for Augmented Reality have have already become accessible for the masses. The technology is still maturing. There are on-going projects to provide registration without the use of markers. Also, sensors are becoming better for indoor location.

That’s it for now.

Please click ‘Follow’ to subscribe to my blog.

Route optimization for MWS410 with OptiMap

Here is a script for Lawson Smart Office that integrates the Delivery Toolbox – MWS410/B with OptiMap – Fastest Roundtrip Solver to calculate and show on Google Maps the fastest roundtrip for the selected Routes. It’s a solution to the Travelling salesman problem (TSP) for M3 Routes.

This is interesting for a company to reduce overall driving time and cost, and it’s interesting for a driver to optimize its truck load according to the order of delivery.

To solve the TSP on Google Maps, the authors of OptiMap implemented several algorithms, including brute force, nearest-neighbor, and the Ant Colony Optimization, and released the code for the TSP Solver for Google Maps API as open source with an MIT License. Because the TSP is an NP-complete problem the solution only works well for up to 10 cities on current desktop computers. But OptiMap can apparently solve more than 15 cities. You can read more about OptiMap at Behind the Scenes of OptiMap and OptiMap version 4 is here.

To integrate OptiMap with Smart Office I wrote a simple Personalized Script that gets the addresses of the selected Routes in MWS410/B, and that opens OptiMap in a web browser with the delivery addresses in the URL. The GET parameters are explained in OptiMap’s Optimize Your Trips.

Setup

To install and use the script:

  1. Deploy this script in the mne\jscript\ folder in your Smart Office server:
  2. Create a Shortcut in MWS410/B to run this script; for that go to MWS410/B > Tools > Personalize > Shortcuts > Advanced, expand Script Shortcut, set the Name to OptiMap, set the Script name to OptiMap, click Add, and click Save:
  3. Create a View (PAVR) in MWS410/B that shows the address columns ADR1, ADR2, ADR3:
  4. Select multiple Routes in the list (press CTRL to select multiple rows), and click the OptiMap Shortcut to run the script:

Result

The script will launch OptiMap for the selected Routes, and OptiMap will optimize the order of delivery:

Source code
Here is the complete source code for the script:

import System;
import System.Web;
import System.Windows;
import Mango.UI.Services.Lists;
import MForms;

/*
Integrates the Smart Office Delivery Toolbox - MWS410/B with OptiMap - Fastest Roundtrip Solver, http://www.optimap.net/
to calculate and show on Google Maps the fastest roundtrip for the selected Routes.
This is interesting to reduce driving time and cost, and for a driver to optimize its truck load according to the order of delivery.
1) Deploy this script in the mne\jscript\ folder in your Smart Office server
2) Create a Shortcut in MWS410/B to run this script; for that go to MWS410/B > Tools > Personalize > Shortcuts > Advanced > Script Shortcut, set the Name to OptiMap, and set the Script name to OptiMap
3) Create a View (PAVR) in MWS410/B that shows the address columns ADR1, ADR2, ADR3
4) Select multiple Routes in the list (press CTRL to select multiple rows)
5) Click the OptiMap Shortcut to run this script and launch OptiMap for the selected Routes
For more information and screenshots refer to https://thibaudatwork.wordpress.com/2012/10/04/route-optimizer/
Thibaud Lopez Schneider, Infor, October 4, 2012 (rev.2)
*/
package MForms.JScript {
    class OptiMap {
        public function Init(element: Object, args: Object, controller : Object, debug : Object) {
            try {
                // get the list
                var listControl: MForms.ListControl = controller.RenderEngine.ListControl;
                var listView: System.Windows.Controls.ListView = controller.RenderEngine.ListViewControl;
                if (listControl == null || listView == null) { MessageBox.Show('Error: Couldn\'t find the list.'); return; }
                // get the selected rows
                var rows = listView.SelectedItems; // System.Windows.Controls.SelectedItemCollection
                if (rows == null || rows.Count == 0) { MessageBox.Show('Error: No rows selected.'); return; }
                // get the address columns ADR1, ADR2, ADR3
                var column1: int = listControl.GetColumnIndexByName('ADR1');
                var column2: int = listControl.GetColumnIndexByName('ADR2');
                var column3: int = listControl.GetColumnIndexByName('ADR3');
                if (column1 == -1 || column2 == -1 || column3 == -1) { MessageBox.Show('Error: Couldn\'t find the address columns ADR1, ADR2, ADR3.'); return; }
                // construct the URL
                var query: String = '';
                for (var i: int = 0; i < rows.Count; i++) {
                    var row: ListRow = rows[i];
                    var ADR1: String = row[column1];
                    var ADR2: String = row[column2];
                    var ADR3: String = row[column3];
                    var address: String = ADR1 + ',' + ADR2 + ',' + ADR3;
                    query += 'loc' + i + '=' + HttpUtility.UrlEncode(address) + '&';
                }
                var uri: Uri = new Uri('http://www.optimap.net/?' + query);
                // launch OptiMap
                ScriptUtil.Launch(uri);
            } catch (ex: Exception) {
                MessageBox.Show(ex);
            }
        }
    }
}

That’s it!

Related posts

See also version OptiMap_V2.

Geocoding of Stock Locations in MMS010

Here is a video that illustrates the process to set the Geo Codes XYZ of Stock Locations in MMS010 in Smart Office, i.e. to set the latitude, longitude, and altitude of Stock Locations, a.k.a. geocoding. In my example I determined the coordinates based on an 3D model built in Google SketchUp and geo-located in Google Earth; a GPS receiver with good indoor accuracy would work as well. With geocoded information, we can present data from the Warehouse Management System in a graphical way. This is important for applications such as showing Stock Locations on a map, or finding the shortest path for a picking list.

Demo video

How to proceed

These are the steps I followed in the video to geolocate the Stock Locations in MMS010:

  1. I used this SketchUp model of a 3D warehouse that I had previously geo-located:
  2. I also used this other SketchUp model of the Stock Locations that I had previously uniquely identified:
  3. Then, I used this Ruby script to get the geocoding of the floor plan:
  4. Then, I used this other Ruby script to get the geocoding of each Stock Location:
  5. The result is this CSV file of the floor plan’s geocodes and each Stock Location’s geocodes:
  6. Then, I used this Lawson Web Service of type Display Program to set the values for the fields Geo Code X (GEOX), Geo Code Y (GEOY), and Geo Code Z (GEOZ) in MMS010/F for a specified Warehouse (WHLO) and Stock Location (WHSL):
  7. Then, I used a Visual Basic macro for Microsoft Excel to call the Web Service for all Stock Locations:
  8. Finally, I used this script to display the Geo Codes XYZ in MMS010/B1:

Result

The result is the list of Stock Locations in MMS010/B1 displaying all the Geo Codes XYZ:

Resources

  • Download the SketchUp model of the geo-located 3D warehouse.
  • Download the SketchUp model of the uniquely identified Stock Locations.
  • Download the Ruby script to get the geocoding of the floor plan.
  • Download the Ruby script to get the geocoding of each Stock Location.
  • Download the resulting CSV file of all Stock Locations and their Geo Codes.
  • Download the Lawson Web Service to set the Geo Codes XYZ of a Stock Location.
  • Download the script to display the Geo Codes XYZ in MMS010/B1.
  • Watch the video of the entire process.

Related articles

UPDATE

2012-09-28: I had a bug in the Ruby script that miscalculated the Y and Z geocodes for the Stock Locations. I corrected the script and the resulting CSV file and I updated the links above.

Warehouse 3D demo

I implemented a Warehouse 3D demo that demonstrates the integration capabilities of M3 with cool stuff from the software industry.

The Warehouse 3D demo displays racks and boxes with live data coming from Stock Location – MMS010, and Balance Identity – MMS060. The Location Aisle, Rack, and Level of MMS010 is written dynamically on each box. The Status Balance ID of MMS060 is rendered as the color of the box: 1=yellow, 2=green, 3=red, else brown. And the Item Number is generated dynamically as a real bar code that can be scanned on the front face of the box.

Here is a screenshot:

The demo uses the Google Earth plugin to render a 3D model that was modeled with Google SketchUp, Ruby scripts to geocode the boxes and identify the front faces of the boxes, PHP to make the 3D Collada model dynamic, SOAP-based Lawson Web Services that calls M3 API, and the PEAR and NuSOAP open source PHP libraries.

The result is useful for sales demos, and as a seed for customers interested in implementing such a solution.

Try for yourself

http://ibrix.info/warehouse3d/

You can try the demo for yourself with your own M3 environment. For that, you will need several things. You will need to install the Google Earth plugin in your browser. You will also need to deploy the Lawson Web Service for MMS060MI provided here; note that your LWS server must be in a DMZ so that the http://www.ibrix.info web server can make the SOAP call over HTTP. Also, you will need to follow the Settings wizard to setup your own M3 environment, user, password, CONO, WHLO, etc. The result is a long URL that is specific to your settings.

Constructing the 3D model

I built a 3D model with Google SketchUp.

Here is the video of the 3D model being built in Google SketchUp:

You can download my resulting SketchUp model here.

Identifying Aisles, Racks, Levels

Then, I set the Aisle, Rack, and Level of each box as in MMS010 using a custom Ruby script for Google SketchUp.

Here is a video that shows the script in action:

You can download this Ruby script here.

You can download the resulting SketchUp model here.

Identifying front faces

Then, I identified each front face of each box so as to dynamically overlay information, such as the Item Number, Item Name, etc. For that, I implemented another Ruby script.

Here is a video of that process:

You can also download this Ruby script here.

Exporting the Collada model

The original model is a SKP filetype, which is binary. I exported the model to a Collada DAE filetype, which is XML. The file is very big, 30.000 lines of XML.

The Collada file contains this:

  • Components (racks, boxes, walls, etc.)
  • Homogenous coordinates (X, Y, Z, H) relative to the model
  • Absolute coordinates (latitude, longitude)
  • Orientation (azimut, etc.)
  • Scale
  • Effects (surface, diffusion, textures, etc.)
  • Colors in RGBA

From the top of my head, the Collada hierarchy in XML is something like this:

Node Instance
	Node Definition
		Instance Geometry
			Instance Material
				Material
					Instance Effect
						Color
						Surface
							Image

Making the model dynamic

The goal is to set the color of each box dynamically, based on the Location of the box, and based on the Inventory Status in MMS060.

Unfortunately, Google Earth doesn’t have an API to change the color of a component dynamically. So, I decided to change the XML dynamically on the server. There are certainly better solutions but that’s the one I chose at the time. And I chose PHP because that’s what I had available on my server ibrix.info; otherwise any dynamic web language (ASP, JSP, etc.) would have been suitable.

In the XML, I found the mapping between the box (nodeDefinition) and its color (material). So, I changed the mapping from hard-coded to dynamic with a PHP function getColor() that determines the color based on the Location and based on the result of the web service call.

The color is determined by the Balance ID: 1=yellow, 2=green, 3=red, else brown. The Balance ID is stored in the SOAP Response of the web service.

Lawson Web Service

I created a SOAP-based Lawson Web Service for MMS060MI. I invoke the SOAP Web Service at the top of the PHP script, and store the Response in a global variable. To call SOAP Web Services, I use NuSOAP, an open source PHP library.

Generating front faces

I dynamically generate a texture for each each front face as a PNG image with the Item Number, Item Description, Quantity, and the bar code. I set the True Type Font, the size, the XY coordinates, and the background color.

Bar code

I generate an image of the bar code based on the Item Number using PEAR, an open source PHP library.

Settings wizard

I made a Settings wizard to assist the user in setting up a demo with their own M3 environment, user, password, CONO, WHLO, etc.

Applications

This Warehouse 3D demo illustrates possible applications such as:

  • Monitoring a warehouse
  • Locating a box for item picking
  • Implementing Augmented Reality to overlay relevant data on top of the boxes

Demo

Finally, I made a demo video using the back projection screen at the Lawson Schaumburg office, and using Johny Lee’s Low-Cost Multi-point Interactive Whiteboards Using the Wiimote and my home made IR pens to convert the back projection screen into a big touch screen. The 3D model in the demo has 10 Aisles, 6 Racks per Aisle (except the first aisle which only has 4 racks), and 4 Levels per Rack. That’s 224 boxes. There is also a floor plan that illustrates that structure.

Limitations

The main limitation of this demo is performance. When programming with Google Earth we do not have the capability of dynamically changing a 3D model. I would have liked to dynamically set the color of a box, and dynamically overlay text on the face of a box. Because that capability is lacking – there’s no such API in the Google Earth API – I chose to generate the XML of the 3D model dynamically on the server. As a result, the server has to send 30k lines of XML to the web browser over HTTP, it has to generate 224 PNG images and transfer them over the network, and the Google Earth plugin has to render it all. As a consequence, it takes between one and four minutes to fully download and render the demo. This design turns out to be inadequate for this type of application. Worse, it is not scalable nor improvable. I would have to re-think the design from scratch to get a more performant result.

Future Work

If I had to continue working on this project (which is not planned), I would implement the following:

  • Ideally, we would generate boxes, colors, and text dynamically on the client-side, with JavaScript and WebGL for example. Google Earth doesn’t support that, and generating the model on the server-side turns out to be a bad design. So we need a different technique.
  • Also, we could use a better 3D client, like O3D.
  • Also, we would need to implement easy keyboard navigation, like the First Person Camera demo, and like the Monster Milktruck demo.
  • Also, we would need to implement hit detection, so as to click on a box and display more M3 data in a pop-up for example. Google Earth supports even listeners but doesn’t yet support hit detection.
  • Finally, we would need to improve performance by an order of magnitude.

Thanks

Special thanks to Gunilla A for sponsoring this project and making it possible.

Resources

  • Download Ruby script to set the Aisle, Rack, and Level of each box as in MMS010
  • Download Ruby script to identify each front face of each box so as to dynamically overlay information
  • Download SketchUp model with floor plan, geo-location, racks, and walls
  • Download SketchUp model of boxes identified by Stock Location
  • Watch video of the 3D model being built in Google SketchUp
  • Watch video of the process of setting the Aisle, Rack, and Level of each box as in MMS010
  • Watch video of the process of identifying each front face of each box
  • Watch video of the demo on the large touch screen
  • Download Lawson Web Service for MMS060MI

Related articles

UPDATE

2012-09-27: I added the SketchUp models and Ruby scripts for download.