Tag Archives: M3

HTTP channels in MEC (part 4)

Here is the fourth part of the guide on how to setup HTTP channels in Infor Enterprise Collaborator (MEC), this time with how to setup and use the HTTPOut channel such that MEC acts as a client making HTTP requests to servers; that is the opposite of the HTTPIn and HTTPSyncIn and HTTPSyncOut channels where MEC acts as a server.

Documentation

The M3 Enterprise Collaborator Partner Admin Tool User Guide does not mention anything about the HTTPOut channel (no matches found in search), but if we read the pages attentively there is a chapter Setting Up Send Channels with a section on HTTP. After some studying of the HTTPOut Java code and testing in PartnerAdmin I confirm that section is effectively referring to the HTTPOut channel. Unfortunately it is just one sentence with no substantial information, and it is incorrect in that “the content type returned” should be “the content type of the request” and in that there is no such thing as “content type web browser”.

doc

Java code

By reading the partially recovered Java source code of class HTTPOut in package com.intentia.ec.communication we determine it is indeed an HTTP client (not a server, and not a response like the word out in HTTPSyncOut suggests). It uses a java.net.HttpURLConnection to make the request to the server defined in PartnerAdmin with host, port number and path.

Java

It supports connection keep alive, Basic authentication, no-cache directives, and multi-part content types.

Unfortunately, there are some drawbacks. The scheme is hard-coded to “http” so there is no HTTP-Secure possible; whereas it is possible with javax.net.ssl.HttpsURLConnection; although there is an HTTPSOut channel provided in the sample chapters of the documentation. The user and password transit in clear text (trivial Base64-decode). It is a departure from the benefits of Java NIO; see part 1 of the guide. The response is ignored (at best it is logged in the debug files) so I do not think there is a possibility for MEC to process the response. And it does not have support for a proxy.

Activity diagram

The HTTPOut channel is a Send process in a partner agreement, in other words it is a message sender, or an outgoing channel. More precisely it is an HTTP client making a request to a server. For the HTTPOut channel to run, the partner agreement must be triggered by an incoming channel such as DiskIn, Event Hub subscriptions, HTTPIn, and others.

Here is a simple activity diagram of the HTTPOut channel:

activity

Again, the word pair in/out does not reflect the word pairs client/server and request/response, and that can cause confusion.

How to setup

To setup an HTTPOut channel in MEC:

  1. Start the PartnerAdmin.
  2. In Communications > Receive, setup any incoming channel such that we can simply trigger the partner agreement for the purposes of illustration; the protocol is not important right now; for instance I have a simple DiskIn channel that has status RUNNING in my Infor Grid, and I have a partner agreement with it in a channel detection:
    1.11.4_3.1_
  3. In Communications > Send, add a new HTTPOut channel by selecting Protocol HTTP:
    2.1__
  4. Enter the host, port number, and path of the server; that will construct the URL http://host:port/path:
    2.2
    Note 1: The Content type is optional as HTTPOut will attempt to guess it based on the contents of the file using the Java method URLConnection.guessContentTypeFromName.
    Note 2: The checkbox Keep connection alive is to keep the TCP connection persistent and not close it, probably if there are multiple Send processes to eliminate the three-way handshake during connection establishment and reduce latency, although that is only useful for millisecond requirements, or to reduce network congestion (fewer TCP connections) although that is only useful for requirements with tens of thousands of concurrent connections; for the rest of us we can leave unchecked; I have not tested this.
    Note 3: The checkbox Post as multi-part is for sending multiple messages into one request; I have not tested this. By reading the Java code of the HTTPOut class, I can see the boundary is hard-coded to —————————, i.e. simply 27 repetitions of the dash character, which means if your file contains the same string (plausible) there could be unexpected results, like the request will fail or the message will be truncated.
    Note 4: The user/password is for HTTP basic authentication (Base64 encoded), but because this is for HTTP only (not HTTPS) the user/password transit in clear text (trivial Base64 decoding), i.e. it can be viewed by eavesdroppers, and it has no encryption and no message verification, i.e. it can be tampered by man-in-the-middle, so do not use this over unsecure networks.
  5. You can click Send test message for the HTTPOut channel to send a sample HTTP POST request to your server; I have Fiddler running on default port 8888 and it has a neat Fiddler Echo Service that I use as a simple HTTP server:
    2.3_
    Quirk: I notice the HTTPOut channel (or something else in MEC) generated two content type headers, they do not have the same case, they have different values from each other, and neither value is text/plain as I had specified in the PartnerAdmin; that again causes an HTTP protocol violation:
    2.3__
  6. Go to the Partner Agreement > Processes, add a Process Send:
    3.2
  7. Select the HTTPOut channel created earlier:
    3.2_
    Note 1: I suppose the encoding is for the binary encoding of the message in the HTTP response body; I have not tested this.
    Note 2: I suppose the Zip-output checkbox is for compressing the message in the HTTP response body; I have not tested this.
    Note 3: The Zip entry name is probably referring to java.util.zip.ZipEntry to compress the message and give it a file name; I did not know HTTP compression needed a file name.
    Note 4: I do not know what Routing Run On Host means.
  8. You can choose actions for Error Handling; in my case I leave it blank to keep it simple here:
    3.3
  9. The HTTPOut channel is now ready to use.

How to test

To test the new HTTPOut channel:

  1. Prepare a sample trigger for the partner agreement; in my case I have a simple message.txt file with content Hello, World! that I drop in the folder to trigger my DiskIn channel detection.
  2. Prepare the HTTP server on the other end to log the request so we can see the result; in my case I have Fiddler AutoResponder that will return the built-in HTTP 200 with a sample HTML page:
    4.2_
  3. Trigger the partner agreement to execute the Send Process; in my case I drop the message.txt file in the folder.
  4. The HTTPOut channel makes the request to the server, and we get the result; in my case in Fiddler:
    4.2__
  5. We can see the Event in the MEC Management Pages in the Infor Grid “Message successfully sent using protocol HTTP”:
    4.3_
    Note: I see no trace of the HTTP response in MEC; as suspected, MEC seems to ignore the response body.
  6. Also, for testing purposes, I set my Fiddler AutoResponder to respond with HTTP 404 Not Found, and thankfully the MEC Events show “error processing”, ERROR, and a Java stack trace with java.io.FileNotFoundException which is correct:
    HTTP404
  7. Also, for testing purposes, I set my Fiddler AutoResponder to respond with HTTP 502 Unreachable Server, and likewise the MEC Events show “error processing”, ERROR, and a Java stack trace with “java.io.IOException: Server returned HTTP response code: 502 for URL” which is correct:
    HTTP502
  8. You can now use your third party applications to receive HTTP requests from MEC.

Conclusion

That was how to setup the HTTPOut channel for MEC to act as a client and make requests to servers. For that, we setup the HTTPOut channel in PartnerAdmin, we setup the partner agreement with the Send process, and we tested with Fiddler. The HTTPOut channel is almost undocumented, not a state-of-the art HTTP client, and has drawbacks, but it suffices for most simple cases. In the next part of this guide I will illustrate how to setup a secure incoming HTTP channel for MEC, i.e. HTTP over SSL/TLS for MEC.

Related articles

UPDATES

2015-03-31: I removed my incorrect note about the boundary as in fact there is a random string appended to the boundary.

2015-03-19: I updated the documentation section about the content type, proxy, and HTTPSOut, and updated the quirk section about the content type.

Tagged ,

HTTP channels in MEC (part 3)

Continuing the guide on how to setup HTTP channels in Infor M3 Enterprise Collaborator (MEC), here is the third part with how to setup and test the HTTPSyncIn and HTTPSyncOut channels, and how to use the MEC Mapper to process the incoming message and return a customized response.

HTTPSyncIn + HTTPSyncOut

The HTTPSyncIn and HTTPSyncOut channels are used in concomitance to accomplish both HTTP request and response. The HTTPSyncIn channel creates a simple HTTP server to accept requests, and the HTTPSyncOut channel returns custom responses, in the same connection. This is unlike the HTTPIn channel which only responds with a hard-coded acknowledgment of receipt.

Here is a simple activity diagram of both channels jointly in action:
activity

And here is an excerpt of the decompiled Java classes HTTPSyncIn and HTTPSyncOut of package com.intentia.ec.communication:
HTTPSyncIn HTTPSyncOut

Both Java classes HTTPSyncIn and HTTPSyncOut live in separate threads, and the keyword sync refers to thread synchronization for the threads to communicate with each other and access the socket channel in shared memory; see class SyncComPool. I do not think the keyword sync refers to synchronous and asynchronous, as in JavaScript XMLHttpRequests, as there are no event handlers in the client here; for the client it is all the same connection.

That means the entire processing of the message in MEC must be completed before the client times out. MEC usually responds within milliseconds, and the client usually times out in tens of seconds, so that leaves plenty of time.

How to setup

We need to setup the following:

  • Messages
  • MEC mapping
  • Receive channel
  • Send channel
  • Detection
  • Processes
  • Test

Prepare the messages

Let’s prepare two messages: one for the request and one for the response. I will keep it simple for the purposes of illustration, so for the request I will simply send a name, and for the response I will return “hello” + name; it is like an echo request (“ping”) and an echo reply (“pong”) containing the exact data received in the request message.

I have to choose my message format. Remember, MEC is XML-centric. If I choose flat file format I have to create a flat file definition which is also XML. I might as well just do XML from the beginning.

My incoming sample message request.xml will be:

<?xml version="1.0" encoding="UTF-8"?>
<name>Thibaud</name>

And my outgoing sample message response.xml will be:

<?xml version="1.0" encoding="UTF-8"?>
<message>Hello Thibaud</message>

Then, we have to create an XML Schema (*.xsd) for each XML file. For that, I will use the XML Schema Definition Tool (Xsd.exe) from the Microsoft .NET Framework with the command:

C:\Program Files\Microsoft SDKs\Windows\v7.1\Bin\
xsd.exe request.xml
xsd.exe response.xml

2.1_ 2.2_ 2.3_

The tool creates the files request.xsd and response.xsd.

There is a quirk. The files are encoded in UTF-8 and contain a Byte order mark (BOM) like most Unicode files should contain. But when we generate the mapping the MEC Mapper will trip on the BOM and will throw:

com.ctc.wstx.exc.WstxUnexpectedCharException: Unexpected character '?' (code 63) in prolog; expected '<'
 at [row,col {unknown-source}]: [1,1]
Generate failed

3.10_

So we have to remove the BOM, for example with Notepad++ select Encode without BOM and save the file:
2.4

XML

Setup the mapping

Now, let’s create a mapping in MEC Mapper to take the input message and create the customized response:

  1. Start the MEC Mapper.
  2. Create a new Project and a new Mapping, browse to the request.xsd and to the response.xsd, and click Finish:
    3.1_
  3. MEC mapper will display the XML elements of the request on the left and those of the response on the right. We have to do the mapping in between.
  4. From the palette, add a Java function to the mapping, then add an Input parameter and an Output parameter to the function, and connect the elements together from left to right:
    3.4__
  5. Then, right-click on the Java function, select Edit Java code, enter the following code in function(), save the file, and close the tab:
    oParameter = "Hello " + iParameter;

    3.5_

  6. Right-click in the mapping, select Save to Database, select the mapping database location, and click Finish:
    3.7_ 3.7__ 3.9_
  7. Right-click in the mapping again, select Generate, select the server location, and click Finish:
    3.10b 3.10c 3.11_
  8. Right-click in the mapping one last time, select Publish, select the server location, and click Finish:
    3.12_ 3.13 3.14
  9. Go to the Infor Grid Management Pages for MEC > Server > Mappings, locate the mapping, click Activate for the state to become Active:
    4.1_
  10. Go to the Server tab, click Reload Server Control, and click Yes to confirm:
    4.2__
  11. The mapping is now ready to be used in Partner Admin.

Setup in Partner Admin

Now let’s setup the HTTPSyncIn and HTTPSyncOut channels in MEC:

  1. Start the Partner Admin.
  2. Go to ManageCommunication.
  3. In the Receive tab, create a new inbound channel with protocol HTTPSyncIn and a port number, and set the checkbox Enabled:
    1.1
  4. In the Send tab, create a new outbound channel with protocol HTTPSync (it means HTTPSyncOut):
    1.2_ 1.2___
  5. Create a new Agreement.
  6. In the Detection tab, select the HTTPSyncIn channel just created:
    1.5_
  7. In the Processes tab, add process XML Transform, and select the Mapping created earlier:
    1.6_ 1.7
  8. Then add a process Send and select the HTTPSyncOut channel created earlier:
    1.8_ 1.9_
  9. Stop and start the MEC application in the Infor Grid, and the HTTPSyncIn channel will be in status RUNNING:
    4.2__
  10. Now the HTTPSyncIn and HTTPSynOut channels and agreement are ready to be used.

Test

Let’s test the new HTTPSyncIn and HTTPSyncOut channels:

  1. Prepare a sample HTTP request with header and body:
    POST http://localhost:8085/ HTTP/1.0
    Content-Type: application/xml
    Content-Length: 62
    Host: localhost:8085
    
    <?xml version="1.0" encoding="UTF-8"?>
    <name>Thibaud</name>
    
  2. Use an HTTP client like Fiddler Composer to send the request to MEC, and in return we get the customized response:
    5a 5b
  3. You can now use your third-party applications to communicate with MEC via HTTP.

Conclusion

That was how to setup and use the HTTPSyncIn and HTTPSyncOut joint channels for a client to send a request to MEC via HTTP and receive a custom response. For that, we setup the XML and XSD files for the request and for the response, we created the mapping in MEC Mapper, we setup the agreement in Partner Admin with the receive channel, the detection, the processes, and the send channel, and finally we tested with Fiddler Composer. It is admittedly quite a lot of work for in the end it is just a dynamic HTTP server, but the power of MEC lies in that it is an EAI product tailored to Infor M3 wherein HTTP is just one of the parts of the bigger puzzle. In the next part of the guide I will illustrate the HTTPOut channel for MEC in turn to become an HTTP client.

Related articles

Tagged ,

HTTP channels in MEC (part 2)

Continuing the guide on how to setup HTTP channels in Infor M3 Enterprise Collaborator (MEC), here is the second part with how to setup and test an HTTPIn channel.

HTTPIn channel

The HTTPIn channel is the easiest to use of the incoming HTTP channels. It creates a simple HTTP server for MEC.

By reading the Java source code of method HTTPIn.runChannel(), we can tell the channel will open a non-blocking Java NIO ServerSocketChannel on the port number that we setup in Partner Admin, and it will wait for incoming connections. Upon receiving a request, the channel will start a thread worker HTTPInSubThreadWorker, then method runWork() will call class HTTPRequest to parse the HTTP request (remember the MEC HTTP channels are not fully HTTP/1.1 compliant so only specific HTTP requests will work), then it will extract the message, it will create a manifest, it will persist the message, it will add the message to the MEC queue, it will return a hard-coded HTTP response with status code 200 OK “The request was successfully processed” regardless of the incoming message (it is also called an acknowledgment received response), then it will close the socket connection, and then the MEC agreement will do the detection and processing of the message.

In EAI, this pattern is called “fire and forget”. There is no possibility of changing the HTTP response, it is a fixed response. I suppose this pattern was designed for high throughput by MEC, to process incoming requests as fast as possible to be able to accept the next requests without bottleneck. This reminds me of the Snd transactions of MI programs that accept requests and do not return a response.

Here is a simple activity diagram of the HTTPIn channel:

HTTPIn

Here is an excerpt of the decompiled Java class HTTPIn of package com.intentia.ec.communication:
HTTPIn_

How to setup

To setup an HTTPIn channel in MEC in an agreement with incoming channel, detection and process:

  1. Start the Partner Administrator.
  2. Select Manage > Communications:
    3.2
  3. In the Receive tab, click New:
    3.3
  4. Enter a Name, and select Protocol HTTPIn:
    2.3
  5. Find an available port number on the MEC server, for example use the following command to see which port numbers are already in use:
    netstat -an | findstr LISTEN

    2.4

  6. Enter that port number in the Property Port, and click OK:
    2.5
  7. Check the box Enabled:
    2.6
  8. Leave the other tabs – Send, M3 API, Databases, DAF – to their default values and click Close.
  9. In the Agreement View, right-click and select Insert group, enter a name for the group, then right-click the group and select Insert agreement, enter a name for the agreement, then in the Basic tab enter a Name:
    3.1_
  10. In the Detection tab, select either XML Detection, Channel detection, or Flat detection. I select Channel detection, Channel Group, and the HTTPIn channel I created earlier:
    3.3_
  11. When we click Save, the Partner Admin will show the warning message “No Body XPath is defined. Are you sure you want to continue?”, it is a false positive because we are not using XML Detection, so we can click Yes to ignore and continue:
    3.3__
  12. In the Process tab, select a process – in order to keep the illustration simple I choose to only Archive the message and not process it – and click Save:
    3.5_
  13. Now Stop and then Start the MEC application in the Infor Grid so that MEC starts our new channel:
    4.14.1_
  14. In the MEC Management Pages, in the Communication tab, verify that the channel is in State RUNNING:
    4.2
  15. The HTTPIn channel is now ready to use.

How to test

To test the new HTTPIn channel:

  1. Prepare a sample HTTP request with header and body:
    POST http://localhost:8082/ HTTP/1.1
    Host: localhost:8082
    Content-Type: text/plain
    Content-Length: 12
    
    Hello World!
    

    Quirk 1: We send data in the request body using method POST but somehow MEC will also accept method GET even if it is a protocol violation.
    Quirk 2: The HTTP request header Content-Type is mandatory, and without it MEC will throw a java.lang.NullPointerException without further details.

  2. Use an HTTP client like Fiddler Composer to send the request:
    Fiddler
    Quirk 3: Somehow when I use a telnet client to send a request to MEC I get a java.nio.BufferUnderflowException on the first byte whereas the telnet client otherwise works correctly with other HTTP servers.
  3. In return, MEC sends the hard-coded response “HTTP 200 OK […] e-Collaborator HTTP Reply […] The request was successfully processed”:
    Test2
  4. Back in the Infor Grid Management Pages, go to the Message tab and click show on your message:
    6.2_
  5. Click on the icon to open the Archived .rcv file:
    6.3_
  6. That will open the archive file of the message we sent:
    6.4_
  7. The archives files are stored in the file system somewhere at D:\Infor\MECDEV\archive\doc\f\:
    6.5_
  8. You can now use your favorite programming language or application to make the HTTP request.

Conclusion

That was an illustration of how to setup an HTTPIn channel in MEC using the Partner Administrator and Infor Grid so that clients can send HTTP requests to MEC in a “fire and forget” style. For that, we setup the agreement with the incoming channel, detection, and process, then we tested with Fiddler Composer, then we received the hard-coded acknowledgement of receipt, and finally we confirmed in the Infor Grid. In order to keep the illustration simple, I just archived the message and did not process it. In the next part of the guide I will illustrate the HTTPSyncIn and HTTPSyncOut channels and the MEC Mapper to process the message and return customized responses.

Related articles

Tagged ,

HTTP channels in MEC (part 1)

Here is a multi-part guide on how to setup HTTP channels in Infor M3 Enterprise Collaborator (MEC) using the HTTPIn, HTTPOut, HTTPSyncIn, HTTPSyncOut channels such that MEC can communicate with other applications via HTTP.

About this guide

In this first part of the guide I will give an overview of MEC and the tools we will need. In upcoming parts of the guide I will illustrate each HTTP channel: HTTPIn, HTTPOut, HTTPSyncIn, and HTTPSyncOut. And at the end of the guide I will illustrate how to setup MEC for HTTP-Secure (HTTPS), i.e. HTTP over SSL/TLS.

The latest version of MEC that is available for download on Infor Xtreme is MEC version 11.4.2.0, but I do not have that version available to play with, so I will use the previous MEC version 11.4.1.0 from 2013 for which I have a server to play with and to learn.

I do not know the actual design decisions of MEC, and I have not been to a MEC training. In addition, it is difficult to learn MEC from its documentation because it only has partial explanations about the HTTP channels. Therefore, I will decompile the MEC server Java libraries to discover how MEC works internally, and I will ask my colleague Sheryll Limmex for help. As a result, what I write on this guide about MEC may or may not be accurate.

HTTP overview

HTTP is a stateless application protocol for communication between clients and servers usually over TCP/IP. There is an HTTP client and an HTTP server. The client makes an HTTP request to the server with a GET or POST method and a message, and the server responds with an HTTP response to the client with a status code and a message; in brief. The request and response each have a header and a body separated by a blank line. To maintain a session across requests the client and server must use a session identifier usually as a unique id in cookie or a parameter in the URL.

Here is a simple activity diagram between HTTP client and server:
0

Here is a sample HTTP request:

POST http://www.example.com/ HTTP/1.1
Host: www.example.com
Content-Length: 12

Hello World!

Here is a sample HTTP response:

HTTP/1.1 200 OK
Content-Type: text/html
Content-Length: 81

<!doctype html>
<html>
    <body>
        Example Domain
    </body>
</html>

I like to use a telnet client or Fiddler Composer to simulate a simple HTTP client and forge HTTP requests. And I like to use Netcat or Fiddler AutoResponder to simulate a simple HTTP server and forge HTTP responses.

Here is a sample HTTP request/response using the Windows telnet client:
y

Here is a sample HTTP request/response using Fiddler Composer:
0_ 0__

Documentation

To download the MEC documentation, go to Infor Xtreme > Downloads > Products > Product Search, search for MEC Server and download the documentation:
1.2

The M3 Enterprise Collaborator Partner Admin Tool User Guide trickles a few sentences on the HTTPIn and HTTPSyncIn channels, but nothing on the HTTPOut or HTTPSyncOut channels, and nothing on how to secure the communication either:
1.3

And the training material does not provide any information on the HTTP channels either:
Course

Java libraries

The most interesting Java library to learn about MEC channels is ec-core-11.4.1.0.0.jar. You can find it in the product download by unzipping products\MEC_11.4.1.0\components\MEC_11410.gar and continuing to MecServer\lib\ . You can also find it in the LifeCycle Manager folder of the MEC server at D:\Infor\LifeCycle\<host>\grid\<grid>\grids\<grid>\applications\<MEC>\MecServer\lib\ :
2

Client + server

MEC can act either as a server listening to incoming requests from clients (HTTPIn and HTTPSyncIn channels) and returning a response (HTTPSyncOut), either as a client making requests to servers (HTTPOut). The suffixes in/out in the channel names can lead to confusion as they do not exactly reflect the pairs of words client/server nor request/response.

Java NIO

By looking at the Java source code for the HTTP channels in MEC, I can tell they use Java Non-blocking I/O (NIO) and buffers, I suppose it is for high scalability and high throughput of its HTTP channels, for good reason, and it does not use the former and unfavorable Java I/O with its serialization, one-thread per connection and other bottlenecks; that is wondrous as I suppose that allows MEC to handle Gigabytes of data received, and tens of thousands of messages or connections simultaneously; to be confirmed.

Not HTTP/1.1

Unfortunately, when I look at the source code, I can tell the HTTP channels in MEC do not fully implement the HTTP 1.1 specification. They are just in-house subsets of the specification for the purposes of MEC; that is wary if we need specific parts of HTTP 1.1.

Partner agreements

MEC works with partner agreements. They are a combination of:

  • Incoming channels of various sorts to accept incoming messages
  • Detections to detect the type of incoming messages
  • Processes to take action on the messages, for example to decode the message
  • Mappings to act on the message, for example to call M3 API, SQL statements or Java code
  • Message senders to send a response back to the client

Here is a sample activity diagram:

agreement

Partner Admin

We will use the Partner Administrator tool to setup channels and agreements:
3.1

The Communication’s Receive tab reflects the Java classes that extend com.intentia.ec.shared.IncommingChannel:
Receive_ IncommingChannel

The Communication’s Send tab reflects the Java classes that implement com.intentia.ec.shared.IMessageSender:
Send_ MessageSender
The Agreement’s Process tab reflects the Java classes that extend com.intentia.ec.server.process.AbstractProcess:
3.5_ 3.5___

Mapping development

We will use the MEC mapper (plugin for Eclipse) to create the mappings that will process the incoming requests (and eventually transform XML, call M3 API, SQL statements, and Java code) and send customized responses.

Mapper Mapper

Grid management

We will administer the HTTP channels and messages from the Infor Grid Applications and the MEC Management Pages:
app Grid

Conclusion

That was a quick introduction of the HTTP channels and tools in MEC that we will use in this guide to setup channels and agreements to communicate with MEC via HTTP.

Related articles

Tagged ,

Getting printouts/reports from Movex / M3 directly to Excel

There are several ways of getting printouts from M3 (Movex) to Excel.  You can create queries, use StreamServe XML etc but all of these options lack the flexibility of an easy way to design the layout with the correct formatting and graphics, they also lack the ability for the user to print exactly as they are used to.

The software M3 Excel Generator solves this problem by utilizing the existing the output and keywords coming from M3.
It is a 30 Minute setup and premade templates for all the M3 standard reports exists.

The process is extremely easy, you define a new Service ID and route the users who want their printouts to Excel instead of PDF or print in MNS204.
The M3 Excel Generator will automatically handle if the printout should be sent via mail or file using the settings in MNS205. It will also take care of getting the correct language and report type on the document.

In the properties you can define how you would like to handle each printout. You can change attachment names using values from the stream, changing separators to be used and  create new sheets in Excel for each process of data etc.

properties 2

The big advantage with the M3 Excel Generator is that all design of the actual printout is done directly in standard Microsoft Excel with no additional software needed.

Users and admins and consultants can do changes to printouts directly with basically no training at all.
A typical design can look like this:

CAS531 ars511

In the template you can add any font, picture,  formatting settings as you would like.

The design of the Movex / M3 printouts are usually based on old spoolfiles and contains several rows for headings etc which makes the document hard to read and since the standard is PDF you can´t do anything with the data.
Below is a comparison for the same printout in PDF and in Excel

CAS5313CAS5311

 

Contact Consebo at info (at) consebo.com for a free demo/trial version or go to http://www.consebo.com for more information.

Tagged , , , , ,

Java Debugging an Infor Grid application at runtime

Here is an illustrated example of how to use the Java Debugger in Eclipse to debug an existing Infor Grid application at runtime for which we don’t have the source code; for example I will debug the M3 Web Services (MWS) server, line by line, while I make a SOAP request.

For that, I will use the technique I learned in my previous post, Hacking Infor Grid application development (part 6), where I learned how to use the Java Debugger for my own Grid application for which I had the source code, and I will extend that technique to an existing Infor Grid application for which I don’t have the source code but for which I have the JAR files.

I had to learn this technique in order to troubleshoot a timeout issue we are currently having in MWS when making a SOAP request. This helped me pinpoint the modules, packages, classes, methods and lines of code that were at cause and report the results to Infor Support so they can report them to Infor Product Development for help. To that effect, please join the campaign and sign the petition asking Infor Product Development to make their source code available so we can better troubleshoot issues and learn. Also, this week is Open Access Week and it’s a great opportunity to ask Infor to make more of their documentation available.

Logging and DEBUG level

In order to narrow down the search space for the issue I was troubleshooting, I increased the log level of the application’s node to DEBUG level. In my case, I selected ALL modules: SYSTEM and MWS. I had also selected TRACE but it didn’t produce any log.
2

I then opened the log file and found interesting information about the bottleneck issue, that helps me narrow down which modules, packages, and classes to debug. In my case they are: MWS com.lawson.webservices.m3.program.M3Session, and SYSTEM DistributedLock:
2_

Remember to revert the log level changes after you’re done to avoid clogging the disk space with increasing log.

Setup the Java Project in Eclipse

Let’s setup a new Java Project in Eclipse with the JAR files of the Infor Grid application (e.g. MWS in my case). For that:

  1. Create a new Java Project in Eclipse, give it a name, and select Add External JARs:
    1.6_
  2. Select the JAR files of the Infor Grid application in LifeCycle Manager (e.g. MWS\lib\*.jar in my case):
    1.7
  3. Optionally, you can also add the JAR files of the Grid runtime (grid-core.x.y.z.jar, etc.).
  4. Open the desired Java class (double-click), Eclipse will open that class in the Class File Editor, and for now it will say “Source not found”, we will fix that later:
    1.11

Start debugging

Now let’s debug the application, even without the source code:

  1. Toggle the breakpoints at the desired methods:
    2.0
  2. Prepare the Infor Grid application for debugging as detailed in my previous post:3
  3. Start debugging in Eclipse:
    2.1_
  4. Make the application go through your breakpoint (in my case I make a SOAP request, the application will call M3Session.logon, and Eclipse will suspend at my breakpoint).
  5. At this point, even without the source code, you can use the debugger as usual, to execute line by line, inspect variables, evaluate expressions, etc.; each class will show “Source not found” for now:
    2.6

Add the Java decompiler

Now let’s use a Java decompiler for Eclipse to recover the source code (not the original source code though) and attach it to the classes.

  1. I installed the JD-Eclipse plugin by Emmanuel Dupuy:
    0
  2. JD-Eclipse will change the file association in Eclipse to open class files that don’t have source, decompile them, and attach the decompiled source back to the class:
    1.9
  3. Also, JD-Eclipse will realign the line numbers of the recovered source code so they match with the debug information of the class, otherwise the debugger will show the wrong lines of source code and it will be very confusing:
    1.10
  4. Now open a class file (double-click) and JD-Eclipse will show the recovered source code with line numbers realigned:
    2.1
  5. Now debug again with the breakpoints. This time the debugger will show the source code. This is the ideal scenario for debugging line by line:
    3.3

Problem

I realized that as I’m debugging, JD-Eclipse will not decompile classes which it hasn’t previously decompiled. So if the debugger steps through a class that hasn’t been previously decompiled, the debugger will show the usual “Source not found”.

The workaround is to open all the classes we need the source code for, prior to debugging, one by one, so JD-Eclipse can decompile them, and attach the source code back to the class, and then we can try debugging again.

UPDATE 2014-10-29: That is not true anymore, and I found the solution: we have to change the default file association for the *.class files in order to use Java Decompiler’s Class File Editor, and we must do so every time we start Eclipse because Eclipse will revert to the default Class File Viewer when we exit Eclipse:
0

Future work: batch decompilation with realignment

I tried the plugin Realignment for JD-Eclipse by Alex Kosinsky that does batch decompilation, but it doesn’t realign the line numbers, so that doesn’t work. I also tried the plugin JDEclipse-Realign by Martin “Mchr3k” Robertson, it does decompile, and it does realign the line numbers, but I couldn’t get the source code to show while debugging, even after having previously decompiled the class, and I don’t know if it does batch decompilation.

UPDATE 2014-10-29: Also, I tried Java Decompiler’s > Save All Sources, but it doesn’t realign the line numbers, even though JD-Core’s change log for version 0.7.0 says “Added an algorithm to realign the decompiled source code to the original line numbers”:
y

That’s future work to solve.

Summary

That was an illustrative guide of how to do Java debugging on an Infor Grid application, at runtime, without having the source code of the application, having just the JAR files from LifeCycle Manager. This technique helped my troubleshoot an issue with MWS. I hope it will help you too.

That’s it! If you learned something, please click Like below, click Follow to receive notifications about new blog posts, share around you, sign the petition, and write the next blog post with me, you are great. Thank you.

Tagged , ,

Hacking Infor Grid application development (part 6)

Here is the sixth article in the series on Hacking Infor Grid application development, and today I will illustrate how to debug the Grid application at runtime. This is useful for development and troubleshooting purposes. For that, I will use the standard Java Platform Debugger Architecture (JPDA) that is included in Java.

I will do the illustration gradually: I will start by debugging a simple HelloWorrrld class from the command line, then I will debug it remotely, then I will debug it from Eclipse, and finally I will debug the Infor Grid application.

Remember the intention is to hack ethically, to learn and to push the boundaries of what is possible. Also, remember to join the campaign and sign the petition asking Infor Product Development to make their source code available so we can do better work. Also, next week is Open Access Week, and it is a great time to ask Infor Product Development to make more of their documentation available.

Debugging from the command line

Here is a quick illustration of how to use jdb, the Java Debugger, to debug a simple HelloWorrrld class from the command line.

Here is the simple HelloWorrrld class:

public class HelloWorrrld {
    public static void main(String[] args) {
        int i = 11;
        int j = 13;
        int k = i * j;
    }
}

Suppose we want to debug the class at runtime and set a breakpoint at line 6 to see the value of variable k.

For that, we compile the class with the -g command line parameter to generate all debugging information, including local variables; by default, only line number and source file information is generated:

javac -g HelloWorrrld.java

Then, we have jdb launch a new Java Virtual Machine (VM) with the main class of the application to be debugged by substituting the command jdb for java in the command line:

jdb HelloWorrrld

Then, we set the breakpoint at line 6:

stop in HelloWorrrld:6

We could have also stopped execution at the beginning of the main method instead of at a specific line number:

stop in HelloWorrrld.main

Then, we start the execution of the debugged application:

run

The debugger will execute up to the breakpoint, at which point we can display the value of k:

print k

It will display k = 143.

Then, we advance execution to the next line and so on until the end:

step

Here is a screenshot of the steps:
1

Debugging remotely

Now I will illustrate how to debug the same class remotely by attaching jdb to a Java VM that is already running.

First, we find a port number that is available on the host where we will run the class, for example I use netstat, and for example I will try port number 1234:

netstat -an | findstr :1234

If netstat returns LISTENING then that port number is already in use and unavailable; otherwise you can use it.

Then, we start the Java VM (the debuggee) with the following options to load in-process debugging libraries and specify the kind of connection to be made and allow jdb to connect to it at a later time:

java -agentlib:jdwp=transport=dt_socket,address=127.0.0.1:1234,server=y,suspend=y HelloWorrrld

-agentlib:jdwp loads the native agent library for the Java Debug Wire Protocol (JDWP).
suspend=y makes the JVM wait on startup for a debugger to attach before running.
transport=dt_socket sets the transport specification to socket transport using a single stream TCP/IP connection between the debugger application and the target VM.
address=1234 is the port number to which the debugger should attach to the JVM. This number should normally be above 1024. Verify that it is available.
server=y indicates that the JVM is accepting connections from debuggers.

Then, on the remote host (the debugger), we start jdb and attach it to the remote Java VM (the debuggee), and we specify the path of the source files:

jdb -sourcepath ./src/ -connect com.sun.jdi.SocketAttach:hostname=host,port=1234

Here is a screenshot of the steps:
2.1
2.2

Debugging from Eclipse

Now I will illustrate how to debug the same class remotely from Eclipse.

First, create a new Java Project in Eclipse, and import the source code of the Java class, and set the desired breakpoints:
3.1

Then, select Run > Debug Configurations.

Then, click New launch configuration, select the Project, Connection Type Standard (Socket Attach), Host, Port, Source, and click Debug:
3.3

Open the Debug perspective, and debug as usual with threads, stepping, immediate console, local variables, etc.:
3.4

Debugging Infor Grid application

Finally, I will illustrate how to debug our Infor Grid application remotely at runtime. We will need the source code of that Grid application, for example I will use the source code I wrote in part 4 of this series.

As a reminder, an Infor Grid application runs in a Grid Node, and a Grid Node is a Java VM. We will set the JDWP debugging options on that Java VM (debuggee). For that, we must change the Java command line options. Fortunately, there is a convenient node property Debug Port in the Node properties that takes care of it.

Select Infor ION Grid Management Pages > Applications > HelloWorld > Configuration:
4.0

Select Edit Properties:
4.1

Select Debug Port:
4.2

Set the application’s debug port number (specify a zero port to assign any free ephemeral port), and save the configuration changes:
4.3

Then, go to the Node Properties (Node > Advanced > Properties), ensure the debugging options were set, and take note of the port number:
4.4

Now, stop the Node, and wait until it restarts automatically, for the Java VM debug options to take effect:
4.4

Now, open the Java project with the source code in Eclipse, and set the desired breakpoints:
5.1

When you compiled the source code, verify that the Compiler Preferences included the debugging options:
5.2

Now, go to the Debug Configuration, set the port number, and click Debug:
5.2

The debugger will attach to the remote Java VM, and will show the current threads:
5.3

Now, reload the Module so we reach our breakpoints:
5.3

Finally, the debugger will reach our breakpoints, and now we can introspect the variables, step through the code, and so on:
5.4

Conclusion

That was an illustration of how to remotely debug an Infor Grid application at runtime with the Java Debugger and Eclipse. This is useful for learning, development and troubleshooting.

Note that when the Grid application threads are suspended the Node will show as status NOK_STALE, and unexpected consequences might happen in the Grid, such as timeouts or attempts to recover.

Also, remember this technique is not official and untested as we are still learning how to deal with Infor Grid applications.

 

That’s it! Please thumbs up if you like this, share around you, follow this blog, sign the petition, and author the next article.

Tagged , ,

Walking directions in a warehouse (part 2)

Today I will illustrate how I started implementing my proof-of-concept for walking directions in a warehouse, and I will provide the source code. The goal is to show the shortest path a picker would have to walk in a warehouse to complete a picking list and calculate the distance for it. This is most relevant for big warehouses and for temporary staff that are not yet familiar with a warehouse. The business benefit is to minimize picking time, reduce labor costs, increase throughput, and gather performance metrics. I used this for my demo of M3 picking lists in Google Glass.

A* search algorithm

I used Will Thimbleby’s wonderful illustration of A* shortest path in Javascript. We can drag and drop the start and stock locations to move them around and recalculate the path, and we can draw walls. I made the map bigger, and I put a warehouse image as a background.

Implementation

Here are the steps I performed:

  1. Double the map’s width/height
  2. Un-hard-code the map width/height
  3. Set the cell size and calculate the canvas width/height
  4. Un-hard-code the cell size
  5. Make a warehouse image in an image editor (I used Gimp)
  6. Add the warehouse image as background of the map
  7. Hide the heat map (search scores)
  8. Patiently draw the map, the walls, the doors, and the stock locations
  9. Save the drawing by serializing the map to JavaScript source code
  10. Replace startMap with the saved drawing
  11. Thicken the path’s stroke
  12. Hide the grid lines
  13. Hide the map
  14. Use diagonals
  15. Emphasize the path length

Here is a video of the making process (watch it in full-screen, HD, and 2x speed):

Result

You can test the result for yourself on my website here.

Here is an animated GIF of the result:
result1

Here is a video of the result for a small warehouse:

Here is a video of the result for a big warehouse:

Source code

I put the resulting HTML, JavaScript source code and images in my GitHub repository for you to download and participate.

Future work

Some of the future work includes:

  • Convert the path length into meters or feet
  • Project the geocoded stock location coordinates to the map’s coordinates
  • Set the start and end locations as input parameters
  • Automatically generate a screenshot of the path for printing alongside the picking list
  • Show the shortest path for an entire picking list using a Traveling Salesman Problem (TSP) algorithm
  • Improve performance for big maps
  • Provide a map editor to more accurately align the warehouse image with the map

Also, a much better implementation would be to use Google Maps Indoors.

 

That’s it. If you liked this, please thumbs up, leave a comment in the section below, share around you, and come author the next post with me.

Tagged , ,

M3 picking lists in Google Glass @ Inforum

I am very pleased to announce that after months of working here and there in the evenings voluntarily after work hours, I finally completed and presented both my demos of M3 picking lists in Google Glass and Augmented Reality at Inforum. They were a success. I showed the demos to about 100 persons per day during six days flawlessly with very positive reception. The goal was to show proof of concepts of wearable computers and augmented reality applied to Infor M3. My feet hurt.

Features

This is my second Glass app after the one for Khan Academy.

This Glass app has the following features:

  • It displays a picking list from Infor M3 as soon as it’s created in M3.
  • For each pick list line it shows the quantity (ALQT), item number (ITNO), item description (ITDS), and stock location (WHSL) as aisle/rack/level.
  • It displays the pick list lines as a bundle for easy grouping and finding.
  • It shows walking directions in the warehouse.
  • It has a custom menu action for the picker to mark an item as picked and to change the status of that pick list line in M3.
  • It uses the built-in text-to-speech capability of Glass to illustrate hands-free picking.
  • It’s bi-directional: from M3 to Google’s servers to push the picking list to Glass, and from Google’s servers to M3 when the picker confirms a line.
  • The images come from Infor Document Management (formerly Document Archive).
  • I developed the app in Java as an Infor Grid application.
  • I created a custom subscriber and added a subscription to Event Analytics to M3:MHPICL:U.
  • It uses the Google Mirror API for simplicity to illustrate the proof-of-concept.

I have been making the resulting source code free and open source on my GitHub repository, and I have been writing the details on this blog. I will soon post the remaining details.

Acknowledgements

I want to specially thanks Peter A Johansson of Infor GDE Demo Services for always believing in my idea, his manager Robert MacCrate for providing the servers on Infor CloudSuite, Philip Cancino formerly of Infor for helping with the functional understanding of picking lists in M3, Marie-Pascale Authié of Infor Pre-Sales for helping me setup and create picking lists in M3 and for also doing the demo at Inforum, Zack Makris of Infor Labs for providing technical support, Jonathan Amiran of Intentia Israel for helping me write the Grid application, and some people of Infor Product Development that chose to remain anonymous for helping me write a Java application for Event Hub and Document Archive. I also want to specially thank all the participants of Inforum whom saw the demo and provided feedback, and all of you readers for supporting me. And I probably missed some important contributors, thank you too. And thanks to Google X (specially Sergey Brin and Thad Starner) for believing in wearable computers and for accelerating the eyewear market.

Screenshots

Here below are the screenshots from androidcast. They show the bundle cover, the three pick list lines with the items to pick, the Confirm custom menu action, the Read aloud action, and the walking directions in the warehouse:

result0_ result1_ result2_ result3_ result3c_ result3r result4_

Vignettes

Here below are three vignettes of what the result would look like to a picker:

1 2 3

 

Inforum

Here are some photos at Inforum:

In the Manufacturing area:
20140917_162028_998

 

 

In front of the SN sign:
10704143_10152726242211873_7070689785986702196_n

 

Holding my Augmented Reality demo:
IMG_0007

Playing around with picking lists in virtual reality (Google Cardboard, Photo Spheres, and SketchFab):
bild 3

Playing around with picking lists in Android Wear (Moto 360):
20140915_110737_664_

 

That’s it! If you liked this, please thumbs up, leave a comment, subscribe to this blog, share around you, and come help me write the next blog post, I need you. Thank you!

Tagged , , , ,

Walking directions in a warehouse

Two years ago I had implemented a proof-of-concept that showed walking directions in a warehouse to complete a picking list. The idea is to visually represent what a picker has to do, and where they have to go, while calculating the minimum distance they have to walk. It will make it easier to get the job done for pickers that are not familiar with a warehouse, like temporary staff. And the business benefit is to minimize picking time, to make savings in labor costs, and to increase throughput. Also, we can save performance data on the ground, derive the gap between goals versus results, and use that as a feedback loop for continuous improvement of internal processes.

I had used my previous work on geocoding of stock locations in M3, and I had used Will Thimbleby‘s JavaScript implementation of the A* search algorithm to calculate and show the shortest path between two stock locations. I yet have to integrate the Traveling Sales Problem (TSP) algorithm for the entire picking list similar to my previous work on delivery route optimization for M3.

And there is more work to be done. For instance, the calculation responds quickly on my laptop for about 11 picking list lines, perhaps 13 with the ant colony optimization, but beyond that the calculation time grows exponentially beyond useful. Also, the calculation does not take into account bottleneck or collision avoidance for forklifts. Currently, this project is a great proof-of-concept to be further explored.

When Google Maps launched their outstanding Indoor Maps I had shelved my small project. But Google Indoor Maps requires the maps to be public which our M3 customers are reluctant to do. So for Inforum this week, I un-boxed my project and integrated it in my Google Glass demo. Here below are some screenshots of the project. I will be writing a series of posts soon on how to implement it. And I would like your feedback.

In future work, I will implement the same proof-of-concept using Google Indoor Maps.

planL

 

x y

Tagged , ,
Follow

Get every new post delivered to your Inbox.

Join 166 other followers