Here is a Hello World illustration of Augmented Reality (AR) for Infor M3 using Metaio Creator and the Junaio Browser on my iPad. The demo shows a 3D warehouse with aisles, racks, and levels, where I highlighted one of the boxes in red. This new result complements my previous demo of AR for M3 which was implemented programmatically in JavaScript. This time I am using Metaio Creator.
Why it matters
The idea is to highlight the stock location of the the next item to pick in a picking list so the picker can quickly identify where to go in the warehouse. This scenario is specially useful for temporary workers that are hired for campaigns on short notice and are not yet familiar with the warehouse thus saving costs in training and picking time.
Also, Augmented Reality is predicted to be one of the next multi-billion dollar industries in five years from now, so this is one of the learning steps I am taking in that direction.
Preview the demo
To preview the demo on your device (PC, Mac, iPad, Android) follow these instructions:
Print the following satellite picture in full page or bigger, and place it on a flat surface; that will be the trackable AR marker:
Install the Junaio Augmented Reality Browser app on your device (from junaio.com for PC/Mac, from the App Store for iPad, or from the Google Play Store for Android).
Open the app and click Scan.
Scan the following QR code; Junaio Browser will identify the QR code, and will download the resources from my channel ThibaudWarehouse3D:
Point your device’s camera towards the printed satellite picture. Junaio Browser will track the satellite picture and will register the warehouse 3D accordingly. Here is a screenshot of the result:
How I built it
The creation process is simple.
I used my previous 3D model of a warehouse with racks, aisles, levels, and boxes that I had created in SketchUp for a demo three years ago. I removed the walls and roof. I removed unnecessary 3D elements that slow down the 3D rendering pipeline on iPad. And I hard-coded an arbitrary box in red.
Here is a screenshot of the trackable and 3D model in Metaio Creator:
Here is a screenshot of the channel creation:
Here is a video of the entire creation process and preview:
Summary
That was how to create a simple Hello World demo of Augmented Reality for M3 using a 3D warehouse and Metaio Creator to highlight the stock location of the next item to pick in a picking list to save training time and picking time.
Future version
In a future version, I will un-hard-code the red box, and I will highlight it programmatically using Metaio SDK.
“University of Bremen student Hannes Baumann demonstrates using a wearable computer for order picking. 750,000 warehouses worldwide distribute approximately $1 US trillion in goods, and order picking accounts for about 60% of the total operational costs of a warehouse. Using a head-up display to guide order picking virtually eliminates errors and is significantly faster than paper or audio based methods. HUD-based order picking is also less expensive and more flexible than many other automated picking techniques. Hannes’s thesis work was supported by the SiWear project and advised by Professor Thad Starner and Professor Michael Lawo.” — Thad Starner, Published on Sep 7, 2012
Last week I attended the Augmented World Expo (AWE) 2014 [1] in Santa Clara, one of the world conferences on Augmented Reality, Virtual Reality, Augmented Virtuality [2], and smart glasses [3]. There, I saw Steve Feiner, pioneer of Augmented Reality in the 1990s [4][5], Professor of computer science and director of the Computer Graphics and User Interfaces Lab at Columbia University, and adviser for Space Glasses at Meta [6]. I also saw Mark Billinghurst, director of the HITLab in New Zealand [7] whom created the AR Toolkit which I later used (JavaScript port) for my prototype M3 + Augmented Reality. I didn’t see Steve Mann, also adviser for Meta, and one of the pioneers of the Wearable Computing group in the Media Lab in the 1980s [8]; Thad Starner was in that group and later went on to design Google Glass for Sergey Brin [9]. I got inspiration from their work when I was younger, and I was excited to see them.
I went to the conference to learn more about the future. I’m currently working on a personal project to develop an app to display picking lists in Google Glass with data from Infor M3.
Here are some pictures of me at the conference, dreaming my vision of future picking lists 😉
In this article I introduce the first implementation that I know of Augmented Reality for Infor M3. Augmented Reality is the ability to superpose digital information on top of real world objects. This is achieved by locating the user’s head in space, by determining the user’s point of view, by registering real world objects, and by projecting virtual 3D objects accordingly. Implementing it has been a deer dream of mine. In this example I use fiducial markers and data coming from Item Master – MMS001.
Applications
Augmented Reality for M3 could be used for many applications. For example, it could help a worker find an Item in the warehouse by showing optimized walking directions and distance to possible picking locations. Also, it could help a worker show contextual information at a glance.
I believe Augmented Reality to be a disruptive technology and one of the next big revolutions in the software industry, with positive impacts similar to those of the Internet and mobile devices, that will reshape entire industries in the next 10 years.
Timeline & motivation
In 1998 I got a summer job in a warehouse for a company that sold car brakes. Every few minutes a printer spit out a picking list of items that I had to collect. As a temporary worker unfamiliar with the place, I spent most of my time wandering through the warehouse, searching for the items, and asking the more seasoned workers for help; I found that inefficient and I wished the computer gave me a map with directions of where to go. Also, the picking lists were un-ordered and I often had to go back to a previous location I had just visited; I found that inefficient and I wished the computer optimized the picking lists. Also, once I found the location, I often discovered the boxes were empty and I had to ask a forklift driver to replenish the stock location from a box of a higher shelf; I found that inefficient and I wished the computer planned replenishment ahead of time. That was in 1998 and nowadays ERP and Warehouse management systems are more common. Yet, I kept my wish to make better systems.
Then, In 2001 I read about Professor Steven Feiner’s Augmented Reality KARMA project from 1992 at Columbia University. The system fit in a backpack and had portable computer, batteries, GPS, compass, and head-mounted display. It would give detailed instructions to a user on how to repair a printer. That was my first exposure to Augmented Reality and ever since I have been wanting to implement it.
In 2007 Apple introduced the iPhone, with a stunning user interface, graphics, and processing power, blowing everybody’s mind about mobility and redefining an industry. And in 2009 Apple added a camera to the iPhone 3GS. The hardware technology for Augmented Reality started becoming accessible to the masses.
In 2009 I met with Brad Neuberg of Google at the Google I/O conference and I started working on a client-side search engine for M3 source code. That was my first exposure to HTML5.
In 2010 I implemented my first Warehouse 3D demo using Google Earth, with real data fed from the ERP, and I projected the result on a large touch screen for an immersive experience. That was my first step towards implemented Augmented Reality for M3.
In 2011 I proposed an idea for an internal project for M3 + Augmented Reality on mobile devices.
In parallel, WHATWG and W3C have been working hard to standardize HTML5 with the ability to use the webcam in JavaScript with WebRTC, to access pixel data, to paint on the canvas, and to use WebGL for 3D rendering. The software technology for Augmented Reality is becoming accessible to the masses.
More recently I started working on geo-locating Stock Locations in M3. This opens the door to new applications for geo-coded data in M3.
I used Ilmari’s source code and I added a few lines of code to call an M3 API using REST in JavaScript when a marker is detected. In this example, the marker is mapped to an Item number (ITNO), but it could also be mapped to a Stock Location (WHSL) for example. Then, for that Item number I call the M3 API MMS200MI.GetItmBasic and I display the Name (ITDS), Description (FUDS), Basic unit of measure (UNMS), Volume (VOL3), Net weight (NEWE), Gross weight (GRWE).
Result
Here is a video of the result. Note the section below the canvas that shows M3 data coming from MMS200MI.GetItmBasic for the detected marker. We can see an activity indicator flickering as the markers are detected. For best viewing, watch the video in YouTube, in HD, and in full screen.
Source code
I provide the result for download at http://ibrix.info/ar/demo.zip with HTML and JavaScript source code, sample fiducial markers, and sample images.
Future work
With the simple example I introduced in this article I illustrate that hardware and software technology for Augmented Reality have have already become accessible for the masses. The technology is still maturing. There are on-going projects to provide registration without the use of markers. Also, sensors are becoming better for indoor location.
Here is an idea that would be great to implement: M3 + Augmented Reality. I believe AR to be one of the next big revolutions in the software industry, and the technology is available today. We have mobile phones with cameras, GPS, compass, and millimetric indoor radio positioning, fast CPU for feature registration, localization and mapping, REST Web Services, etc. It went from being mostly reserved to research labs, to being the hype of emerging start ups. Get ready for the future 🙂
M3 ideas 