April 3, 2015 10:52 am
Continuing the guide on how to setup HTTP channels in Infor M3 Enterprise Collaborator (MEC), I will illustrate how to setup the HTTPSOut channel for MEC to make requests using HTTP Secure (HTTPS), i.e. HTTP over SSL/TLS; I will investigate the channel’s features and drawbacks, and I will verify if it is safe to use (it is not).
It is important to use SSL/TLS in partner agreements that need to transfer via HTTP sensitive information such as names, addresses, bank account numbers, purchase orders, credit card numbers, financial transactions, health records, user names, passwords, etc. More generally, it is important to accelerate the adoption of cryptography.
SSL/TLS is a security protocol that provides privacy for a client and a server to communicate over insecure networks. It is a communication layer that sits between a transport layer (usually TCP) and an application layer (for example HTTP). Secure Sockets Layer (SSL) is the original protocol and is not considered secure anymore. Transport Layer Security (TLS) is the successor and everybody should upgrade to its latest version 1.2. https is the scheme token in the URI that indicates we are using a dedicated secure channel often on port 443.
SSL/TLS provides the following properties of secure communication:
It can also provide non-repudiation, using digital signatures to assert the sender cannot deny having sent the message, provided the sender’s private key is not compromised or changed.
And depending on the key agreement protocol, it can also provide perfect forward secrecy, using ephemeral key exchanges for each session, to ensure that a message cannot be compromised if another message is compromised in the future.
During the SSL/TLS handshake, the client and server exchange digital certificates and establish cipher suite settings. The connection bootstraps with asymmetric cryptography using public and private keys (which eliminates the problems of distributing shared keys), and then switches to symmetric cryptography using a temporary shared key for the session (which is several orders of magnitude faster).
A digital certificate is a public key and the owner’s identity that have been verified and digitally signed by a certificate authority (CA). Public key infrastructure (PKI) is used to create, distribute, and revoke certificates. If an entity trusts a CA, then by transitive relation it will trust that any certificate the CA issues authenticates the owner of the certificate. There can be any number of intermediate CAs in the chain of certificates, and root CAs use self-signed certificates.
The client must verify the server certificate, and optionally the server may verify the client certificate:
Despite all that, validating certificates is difficult, and the CA model is broken.
You can test HTTP over SSL/TLS and see the chain of certificates,
with cURL:
curl http://curl.haxx.se/ca/cacert.pem > cacert.pem curl --verbose --cacert cacert.pem https://www.example.com/
with OpenSSL:
openssl s_client -connect www.example.com:443 -showcerts GET / HTTP/1.1 Host: www.example.com:443
A web proxy is a program that makes requests on behalf of a client. There are different types of web proxies, for example tunneling proxies, forward proxies, and reverse proxies. And there are different implementations, for example, explicit proxies and transparent proxies. And they have different purposes, for example caching, load balancing, securing, monitoring, filtering, bypassing, and anonymizing. Proxies may require authentication.
An explicit web proxy is for example when you set your browser to use a proxy at a certain host and port number. With that proxy, the client makes an HTTP request to the proxy using the CONNECT method, then the proxy establishes a TCP connection to the destination server, and then the proxy acts as a blind tunnel that is forwarding TCP between client and server, seeing only the SSL/TLS handshake and then encrypted traffic.
You can test HTTP over SSL/TLS via an explicit proxy,
with cURL and Fiddler:
curl --verbose --cacert cacert.pem --proxy 127.0.0.1:8888 https://www.example.com/
with proxy authentication:
curl --verbose --cacert cacert.pem --proxy http://username:password@proxy:port/ https://www.example.com/
with GNU Wget and Fiddler:
export https_proxy=localhost:8888 wget --debug --https-only https://www.example.com:443/
You will see this request and response between client and proxy, then the SSL/TLS handshake, then the HTTP traffic:
CONNECT www.example.com:443 HTTP/1.1 Host: www.example.com:443 HTTP/1.1 200 Connection Established
On the other hand, a transparent proxy intercepts the traffic at the lower network level without requiring any client configuration. It acts as the SSL termination for the client, and establishes a second encryption channel with the destination server thereby being able to monitor and filter the SSL/TLS traffic in transit; in that case the client is not aware of the presence of the proxy but must trust the proxy’s certificate, and the proxy must trust the server’s certificate.
In our case, we want MEC to communicate with partners over a secure channel with confidentiality, integrity, and authentication to protect the data being exchanged, optionally via proxy.
MEC does not provide secure channels for HTTP out of the box. For instance, there are no HTTPSIn, HTTPSOut, HTTPSSyncIn, or HTTPSSyncOut channels. There is an HTTPSOut channel, but it is only a sample in an appendix of the documentation (does that mean it is safe to use?), and it does not come out of the box in the Partner Admin, it must be added manually (although it is available in the Java library). Also, there is a WebServiceSyncIn channels that uses WS-Security specifically for XML over SOAP, but in my case I am interested in HTTPS in general, for example for flat files, I am not interested in SOAP. Ideally, I would prefer to use the Infor Grid which already communicates via HTTPS, but unfortunately it does not have a native connection handler for MEC.
None of this means MEC is insecure, it just means you have to add the security yourself.
The MEC Partner Admin Tool User Guide contains the necessary source code and some explanation for the HTTPSOut channel and user interface:
However, the documentation references HTTPS Communication, HttpServer.xml, and HTTPSOut at the same time, yet it does not clearly explain the relationship between them, so it is confusing:
I had to find the Communication Plug-in Development User Guide of an old version of MEC to find the answer. It turns out the HttpServer.xml was a file that existed in old versions of MEC to configure the web user interface. I think the sample in the documentation used that server to test the HTTPSOut channel; I do not know. Anyway, that file does not exist anymore after MEC was ported to the Infor Grid, so that chapter is irrelevant, confusing, and unrelated to HTTPSOut. We can ignore it, and Infor should remove it from the Partner Admin guide.
The MEC Java library ec-core-x.y.z.jar includes the Java classes sample.HTTPSOut and sample.HTTPSOutPanel:
Looking at the source code, the HTTPSOut channel is similar to the HTTPOut channel: they both make a POST HTTP request to a destination server, at a host, port number, and path defined in the Partner Admin, with the message sent in the request body.
HTTPSOut uses javax.net.ssl.HttpsURLConnection which in turn uses javax.net.ssl.SSLSocket to establish a secure socket with the destination server. From the HTTPSOut point of view, it just reads/writes plain HTTP to the socket, and the socket underneath takes care of the handshake and encryption/decryption.
HTTPSOut has an advantage over HTTPOut. Given the entire HTTP traffic is encrypted over SSL/TLS, we can safely set a username and password for HTTP Basic authentication, unlike with HTTPOut where we should not set a username and password because they transit in clear text (trivial Base64 decode).
HTTPOut and HTTPSOut share the same drawbacks: they do not use Java NIO, and there are no proxy settings.
HTTPSOut has additional drawbacks: there are no settings for Content-type, multi-part, and keep alive, unlike HTTPOut. Worse, the response is completely ignored unlike HTTPOut where the response is at least added to the debug log.
You can test the HTTPSOut class with the following code and with a message.txt file:
javac -cp log4j-1.2.17.jar;ec-core-11.4.1.0.0.jar;. Test.java java -cp log4j-1.2.17.jar;ec-core-11.4.1.0.0.jar;. Test < message.txt
import java.util.Properties;
import sample.HTTPSOut;
public class Test {
public static void main(String[] args) throws Exception {
Properties props = new Properties();
props.setProperty(HTTPSOut.HOST, "www.example.com");
props.setProperty(HTTPSOut.PORT, "443");
props.setProperty(HTTPSOut.PATH, "/");
props.setProperty(HTTPSOut.USER, "username");
props.setProperty(HTTPSOut.PASSWORD, "*****");
props.setProperty(HTTPSOut.TRUST_STORE, "C:\\somewhere\\keystore");
props.setProperty(HTTPSOut.TRUST_STORE_PWD, "*****");
HTTPSOut client = new HTTPSOut();
client.send(System.in, null, props);
}
}
HTTPSOut will send the following request over SSL/TLS:
POST /Hello HTTP/1.1 Content-Type: multipart/form-data; boundary=----------------------------afatudbu6sb5-875oo9dmn52fzfypuig0x1kv Authorization: Basic: dXNlcm5hbWU6cGFzc3dvcmQ= User-Agent: M3 Enterprise Collaborator v11.4.1.0 Cache-Control: no-cache Pragma: no-cache Host: www.example.com:443 Accept: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2 Connection: keep-alive Content-Length: 256 ------------------------------afatudbu6sb5-875oo9dmn52fzfypuig0x1kv Content-Disposition: form-data; name="name"; filename="output.xml" Content-Type: application/xml Hello, World! ------------------------------afatudbu6sb5-875oo9dmn52fzfypuig0x1kv--
Here is a sample activity diagram of the HTTPSOut channel:
There is a bug in the HTTPSOut class regarding the HTTP Basic authentication header that will cause the server to return HTTP/1.1 401 Unauthorized, and that is easy to fix:
// incorrect
urlConn.setRequestProperty("Authorization: Basic", Base64.encode (to64, "ISO-8859-1"));
// correct
urlConn.setRequestProperty("Authorization", "Basic " + Base64.encode (to64, "ISO-8859-1"));
The bug is located in both the HTTPSOut source code in the Partner Admin guide, and in the HTTPSOut class in the JAR file. And the JAR file is located in both the Partner Admin tool and in the MEC server in the Infor Grid.
That code is only executed if you need HTTP Basic authentication (i.e. if you specify a username and password); you can ignore it otherwise.
If you need to use a proxy, add the following code to the HTTPSOut class and recompile it:
import java.net.Authenticator;
import java.net.InetSocketAddress;
import java.net.PasswordAuthentication;
import java.net.Proxy;
[...]
// get the proxy properties
String proxyHost = props.getProperty(PROXY_HOST);
int proxyPort = Integer.parseInt(props.getProperty(PROXY_PORT));
String proxyUser = props.getProperty(PROXY_USER);
String proxyPassword = props.getProperty(PROXY_PASSWORD);
// connect via proxy
InetSocketAddress proxyInet = new InetSocketAddress(proxyHost, proxyPort);
Proxy proxy = new Proxy(Proxy.Type.HTTP, proxyInet);
HttpsURLConnection con = (HttpsURLConnection) url.openConnection(proxy);
// authenticate to proxy
Authenticator.setDefault(new Authenticator() {
protected PasswordAuthentication getPasswordAuthentication() {
return new PasswordAuthentication(proxyUser, proxyPassword.toCharArray());
}
});
//con.setRequestProperty("Proxy-Authorization", "Basic " + Base64.encode(proxyUser + ":" + proxyPassword, "ISO-8859-1")); // this did not work for me
You will need to add some if-then-else so the user can choose to use a proxy or not, and add some exception handling around parseInt.
Also, for the user interface, you will need to add a group widget, a checkbox, four labels and four text boxes to the HTTPSOutPanel class and recompile it:
Unfortunately, the HTTPSOut class does not verify the certificate and as such the connection is susceptible to man-in-the-middle attack.
For instance, it does not verify the hostname of the certificate, it just returns true for any hostname:
HostnameVerifier hv = new HostnameVerifier() {
public boolean verify(String s, SSLSession sslSession) {
return true;
}
};
More so, HTTPSOut does not verify the certificate’s dates, revocation, basic constraint, extensions, etc.
Thus, HTTPSOut is not safe to use out of the box. You have to implement certificate validation yourself. That is probably why it is only a sample.
To compile the HTTPSOut and HTTPSOutPanel classes:
Before we can use HTTPSOut, we have to prepare the terrain for SSL/TLS.
As a reminder, the HTTPSOut channel is an HTTP client that will make an HTTP request to a destination HTTP server over SSL/TLS, e.g. to https://www.example.com/.
The short instructions to setup server authentication are:
openssl s_client -connect www.example.com:443 > example.cer keytool.exe -import -file example.cer -keystore example.ks -storepass changeit
The long instructions are the following.
We need to setup server authentication, i.e. for HTTPSOut to affirm the identity of the server it is connecting to; it will rely on SSLSocket to verify that the server certificate is signed by a certificate authority that is stored in a trusted keystore.
Internet Explorer, Google Chrome, Fiddler and other programs that use WinINet ultimately rely on the Windows certificate manager, certmgr.msc, but Firefox and Java each use their own key management. The JRE has its default keystore at %JAVA_HOME%\lib\security\cacerts that SSLSocket uses. HTTPSOut ignores the default keystore and requires the user to specify one. To create a Java keystore we use keytool.exe and import the server’s certificate in it.
The steps are:
keytool.exe -list -keystore cacerts -storepass changeit | findstr /i DigiCert keytool.exe -list -keystore cacerts -storepass changeit -v -alias digicerthighassuranceevrootca
keytool.exe -import -file example.cer -keystore example.ks -storepass changeit
As for client authentication, the HTTPSOut channel does not support that, i.e. it does not have a public/private keys and a digital certificate to present to the server if the server requests it, so we have to forget about that. If the server needs to authenticate the client, we can use Basic authentication (provided we fix the authentication bug).
Now let’s setup the HTTPSOut channel in MEC.
The steps will be:
To test the HTTPSOut channel, follow the same instructions as in the test of Part 4.
If you are doing the HTTPS requests to a server out on the Internet you should consult with your security team. I have MEC in a LAN behind a NAT, firewall, content based filter, security appliance, and transparent proxy.
In case HTTPSOut does not work as expected, here are some troubleshooting tips.
The MEC server logs are useful for troubleshooting problems. For example, in one case I got the following exception:
sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
That means the server certificate validation failed, either because we put the incorrect server certificate in the keystore that we defined in PartnerAdmin, either because a man-in-the-middle server presented a different certificate.
Some network and security administrators setup firewalls, proxies, and gateways that interfere with SSL/TLS. In my case, I will ensure that my host can access the destination server over SSL/TLS. For that:
openssl s_client -connect www.example.com:443 -showcerts
// javac Test.java && java -cp . Test
import java.io.OutputStream;
import java.net.URL;
import java.security.cert.Certificate;
import javax.net.ssl.HttpsURLConnection;
public class Test {
public static void main(String[] args) throws Exception {
System.setProperty("javax.net.ssl.trustStore", "D:\\java\\jre\\lib\\security\\cacerts");
System.setProperty("javax.net.ssl.trustStorePassword", "changeit");
URL url = new URL("https://www.example.com/");
HttpsURLConnection con = (HttpsURLConnection) url.openConnection();
con.setDoOutput(true);
OutputStream out = con.getOutputStream();
Certificate[] scerts = con.getServerCertificates();
for (Certificate scert: scerts) {
System.out.println(scert.toString());
}
con.disconnect();
}
}
See the section at the top to test the HTTPSOut class.
That was an illustration of how to setup the HTTPSOut channel in MEC so that MEC can act as a client making secure HTTP requests to servers over SSL/TLS. For that, we reviewed the basics of HTTPS, we reviewed the features and drawbacks of HTTPSOut, we learned how to fix the authentication bug, how to add proxy settings, we identified security holes, we learned how to recompile the classes, how to setup the channel in PartnerAdmin, how to test, and how to troubleshoot.
HTTPSOut lacks features, it has a bug, and it has security holes; that is probably why it is just considered a sample channel that is unsafe to use in a production environment. Take it as a starting point from which you can build your own safe channel with production quality.
Hopefully, Infor will remove the deprecated documentation from the Partner Admin guide, will enhance the HTTPSOut channel from unsafe sample to safe channel with production quality, and will eventually provide a native connection handler in the Infor Grid for MEC.
In the next part of this guide, I will illustrate how to setup HTTPS for inbound messages.
(Congratulations if you made it this far! Let me know what you think in the comments below.)
Related articles
Posted by thibaudatwork
Categories: M3 Enterprise Collaborator (MEC), Security
Tags: Enterprise Collaborator, M3, Security
Mobile Site | Full Site
Get a free blog at WordPress.com Theme: WordPress Mobile Edition by Alex King.
NOTE: To be more specific, a digital certificate must be verified twice:
1) Verify the certificate before you store it in the keystore for the first time. This should be done out-of-band, i.e. not using the same communication channel for SSL/TLS but using another communication channel, like phone or email or snail mail; it’s a public key so it doesn’t matter if it’s seen, as long as it’s not tampered with. Windows, browsers, Firefox and the JRE come pre-loaded with certificates they trust. If you are using a certificate that’s not in those key stores, for example if you install the certificate of a partner, then you should verify out-of-band what your SSL/TLS client receives.
2) Verify the certificate programmatically in your Java code. That’s what the sample.HTTPSOut class does not do and that you should add.
That’s all standard for digital certificates. Nothing specific to MEC.
LikeLike
By thibaudatwork on April 6, 2015 at 4:14 pm
[…] Here is how to securely receive messages in MEC from partners over the Internet, in this sixth part of the guide on HTTP channels for Infor M3 Enterprise Collaborator (MEC). I will illustrate two goals: how to setup an HTTPIn or HTTPSyncIn channels in MEC over SSL/TLS, and how to expose them securely over the Internet. Previously, for the HTTPIn channel, refer to part 2; for the HTTPSyncIn channel refer to part 3; and for MEC over HTTP Secure (HTTPS) refer to part 5. […]
LikeLike
By HTTP channels in MEC (part 6) | M3 ideas on April 10, 2015 at 4:02 pm
UPDATE: The folder for the Java classes may not be MecServer\lib\ folder but MecServer\custom\ . To be verified.
LikeLike
By thibaudatwork on September 4, 2015 at 1:57 pm
NOTES: some examples of certificate validation:
CWE-295: Improper Certificate Validation
http://cwe.mitre.org/data/definitions/295.html
CWE-297: Improper Validation of Certificate with Host Mismatch
http://cwe.mitre.org/data/definitions/297.html
LikeLike
By thibaudatwork on August 30, 2016 at 8:41 am
UPDATE: Unless you have specific requirements to use non-blocking I/O with java.nio (unusual; needed mostly for gigabyte file transfers), for sending HTTPS from MEC, I would use the Web Service process which despite its name is not SOAP exclusive but byte-based and can send anything such as XML, text, etc. That’s built-in MEC. I wouldn’t mess with the HTTPSOut above. See this post for more details https://m3ideas.org/2016/08/10/calling-soap-web-services-from-mec/
LikeLike
By thibaudatwork on September 26, 2016 at 3:01 pm
UPDATE: Here is a new post replacing this: https://m3ideas.org/2016/10/13/http-channels-in-mec-part-5-bis/
LikeLike
By thibaudatwork on October 13, 2016 at 5:07 pm
[…] I will re-visit the HTTPSOut communication channel of Infor M3 Enterprise Collaborator (MEC) as a follow-up of my previous post part 5. […]
LikeLike
By HTTP channels in MEC (part 5 bis) – M3 ideas on October 13, 2016 at 5:02 pm